Bug 345926 - "Inconsistent classfile encountered" for projects that reference jars compiled by scala
Summary: "Inconsistent classfile encountered" for projects that reference jars compile...
Status: VERIFIED NOT_ECLIPSE
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.7   Edit
Hardware: Macintosh Mac OS X - Carbon (unsup.)
: P3 normal (vote)
Target Milestone: 3.8 M3   Edit
Assignee: Srikanth Sankaran CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-05-16 08:23 EDT by Michael Busch CLA
Modified: 2011-10-24 19:34 EDT (History)
3 users (show)

See Also:


Attachments
Compiled finagle core 1.4.1 (692.94 KB, application/java-archive)
2011-05-16 19:12 EDT, Michael Busch CLA
no flags Details
Java program that can't be compiled in eclipse (337 bytes, application/octet-stream)
2011-05-16 19:31 EDT, Michael Busch CLA
no flags Details
New version of the program that shows the problem (351 bytes, text/plain)
2011-07-21 17:20 EDT, Michael Busch CLA
no flags Details
New finagle version 1.7.4 (830.47 KB, application/java-archive)
2011-07-21 17:21 EDT, Michael Busch CLA
no flags Details
scala libraries 2.8.1 (6.20 MB, application/java-archive)
2011-07-21 17:23 EDT, Michael Busch CLA
no flags Details
twitter util core 1.9.2 (473.89 KB, application/java-archive)
2011-07-21 17:24 EDT, Michael Busch CLA
no flags Details
finagle-core compiled with scala 2.8.2 RC1 (904.42 KB, application/java-archive)
2011-08-30 12:22 EDT, Michael Busch CLA
no flags Details
util-core compiled with scala 2.8.2 RC1 (494.24 KB, application/java-archive)
2011-08-30 12:23 EDT, Michael Busch CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Busch CLA 2011-05-16 08:23:46 EDT
Build Identifier: 20100917-0705

When using Twitter's finagle scala project in a Java program eclipse stops building the project as soon as it tries to compile a class that depends on the Service class from finagle. 
https://github.com/twitter/finagle

This is the error message eclipse shows:
Inconsistent classfile encountered: The undefined type parameter Req1 is referenced from within 
 Service<Req,Rep>.$anon$2

This is the scala signature of the Service class:
abstract class Service[-Req, +Rep] extends (Req => Future[Rep]) {

Reproducible: Always

Steps to Reproduce:
1. Make your project depend on the finagle core jar
2. Instantiate an instance of com.twitter.finagle.Service in a java file
3. Build your project
Comment 1 Michael Busch CLA 2011-05-16 08:24:35 EDT
This happens for me with eclipse 3.6, 3.7 and 4.1 M7.
Comment 2 Olivier Thomann CLA 2011-05-16 09:14:54 EDT
You should report this against scala compiler.
Comment 3 Michael Busch CLA 2011-05-16 09:18:50 EDT
(In reply to comment #2)
> You should report this against scala compiler.

Well that was my first thought too, but javac (version 1.6.0_24) compiles the exact same program just fine.I'm really curious why eclipse behaves differently from javac.

We have also talked to the scala folks who think that their compiled code conforms to the java spec and eclipse is the problem here.
Comment 4 Olivier Thomann CLA 2011-05-16 09:23:20 EDT
Then please provide a test case that is only using JDT and not scala.
Comment 5 Michael Busch CLA 2011-05-16 09:47:50 EDT
(In reply to comment #4)
> Then please provide a test case that is only using JDT and not scala.

I wasn't able so far to reproduce this with Java only.

I'm not sure how to proceed. Fyi: IntelliJ also compiles the project using the finagle jar fine, just as javac does. My solution right now is to transition to IntelliJ - and every other developer at Twitter who uses eclipse currently will have to do the same.

Eclipse must be doing something differently from javac/IntelliJ. So if you could give me a hint what exactly the problem with the compiled code is and what Eclipse's error message means then I could ask the scala guys to fix it if the problem turns out to be at their end.
Comment 6 Michael Busch CLA 2011-05-16 09:50:06 EDT
(In reply to comment #5)
> (In reply to comment #4)
> > Then please provide a test case that is only using JDT and not scala.
> 
> I wasn't able so far to reproduce this with Java only.
> 


And btw: I'm not using the scala plugin for Eclipse. The finagle jar I'm using is compiled with scalac.
Comment 7 Olivier Thomann CLA 2011-05-16 10:02:06 EDT
I am not saying we don't have a problem. I am just asking if you can provide a test case that reproduces this issue.
I checked the link for finagle, but this is only sources. Can you provide the finagle core jar ?
Does IntelliJ have its own compiler ?
Comment 8 Olivier Thomann CLA 2011-05-16 10:19:31 EDT
You can send me the jar file directly if it is too big to be attached here.
Comment 9 Olivier Thomann CLA 2011-05-16 15:35:43 EDT
If you can't provide the jar, how do I build it ? I want to make sure that I am getting exactly the same jar.
Comment 10 Michael Busch CLA 2011-05-16 19:12:01 EDT
Created attachment 195798 [details]
Compiled finagle core 1.4.1
Comment 11 Michael Busch CLA 2011-05-16 19:12:41 EDT
(In reply to comment #9)
> If you can't provide the jar, how do I build it ? I want to make sure that I am
> getting exactly the same jar.

Sorry for the delay. Just uploaded the jar.
Comment 12 Michael Busch CLA 2011-05-16 19:31:51 EDT
Created attachment 195800 [details]
Java program that can't be compiled in eclipse

Multiple markers at this line
	- The type com.twitter.util.Future cannot be resolved. It is indirectly referenced from required .class 
	 files
	- The type com.twitter.util.Future cannot be resolved. It is indirectly referenced from required .class 
	 files
	- Inconsistent classfile encountered: The undefined type parameter Req1 is referenced from within 
	 Service<Req,Rep>.$anon$2
Comment 13 Olivier Thomann CLA 2011-05-17 08:45:40 EDT
Reproduced.
[total compilation time: 1903]
----------
1. ERROR in C:\tests_sources\X.java (at line 1)
	import com.twitter.finagle.Service;
	^
The type com.twitter.util.Future cannot be resolved. It is indirectly referenced from required .class files
----------
2. ERROR in C:\tests_sources\X.java (at line 1)
	import com.twitter.finagle.Service;
	^
Inconsistent classfile encountered: The undefined type parameter Req1 is referenced from within Service<Req,Rep>.$anon$2
----------
3. ERROR in C:\tests_sources\X.java (at line 2)
	import com.twitter.util.Future;
	       ^^^^^^^^^^^^^^^^^^^^^^^
The import com.twitter.util.Future cannot be resolved
----------
3 problems (3 errors)

I am investigating.
Comment 14 Olivier Thomann CLA 2011-05-17 09:12:42 EDT
The signature of the method apply in type com.twitter.finagle.Service$$anon$2 is:
 (TReq1;)Lcom/twitter/util/Future<TRep;>;

There is no type variable argument to "resolve" the Req1 type variable.

When the binary type binding is built for the type com.twitter.finagle.Service$$anon$2, all method signatures are validated.

When this method is resolved, the only type arguments that are known are the type arguments of the enclosing type com.twitter.finagle.Service: [<Req extends java.lang.Object>, <Rep extends java.lang.Object>]

Req1 is not one of them so an error is returned.
I believe the signature of this method is invalid. Once this is fixed, we should see if the two other issues are still reported.

Do you have a bug report opened against the scala compiler for this issue ?

The definition of this type is:
abstract class Service[-Req, +Rep] extends (Req => Future[Rep]) {
  def map[Req1](f: Req1 => Req) = new Service[Req1, Rep] {
    def apply(req1: Req1) = Service.this.apply(f(req1))
    override def release() = Service.this.release()
  }

  /**
* This is the method to override/implement to create your own Service.
*/
  def apply(request: Req): Future[Rep]

  /**
* Relinquishes the use of this service instance. Behavior is
* undefined is apply() is called after resources are relinquished.
*/
  def release() = ()

  /**
* Determines whether this service is available (can accept requests
* with a reasonable likelihood of success).
*/
  def isAvailable: Boolean = true
}

Maybe javac doesn't validate the signature of methods. And this could explain why javac is compiling the code ok.
Comment 15 Olivier Thomann CLA 2011-05-17 09:24:27 EDT
Using javac 1.6 or 1.7, I get:
X.java:2: package com.twitter.util does not exist
import com.twitter.util.Future;
                       ^
X.java:8: cannot find symbol
symbol: class Future
      public Future<Object> apply(Object o) {
             ^
X.java:6: <anonymous X$1> is not abstract and does not override abstract method <A>andThen$mcDD$sp(scala.Function1<java.lang.Object,A>) in scala.Function1
    Service<Object, Object> service = new Service<Object, Object>() {
                                                                    ^
3 errors

Am I missing something on the classpath ?
I have your jar and the scala library 2.9.0.
Comment 16 Olivier Thomann CLA 2011-05-24 09:28:27 EDT
Are you sure you got it working with javac ? The example you sent doesn't compile for me.
I'd like to clarify that. Also did you get feedback from Scala's people for the comment 14 ?
Comment 17 Olivier Thomann CLA 2011-06-08 14:09:33 EDT
Would it be possible to get some feedback on this issue ? With what I have so far, there is nothing I can do to get rid of the problem.
Comment 18 Michael Busch CLA 2011-07-21 17:20:58 EDT
Created attachment 200127 [details]
New version of the program that shows the problem

Sorry about the delay. This program now compiles fine with javac 1.6.0_24 but not in eclipse.

Eclipse still shows:
Inconsistent classfile encountered: The undefined type parameter Req1 is referenced from within Service<Req,Rep>.$anon$2	InconsistentClassfile.java

I will upload the three jar dependencies that you need to compile it too.
Comment 19 Michael Busch CLA 2011-07-21 17:21:48 EDT
Created attachment 200128 [details]
New finagle version 1.7.4
Comment 20 Michael Busch CLA 2011-07-21 17:23:24 EDT
Created attachment 200129 [details]
scala libraries 2.8.1
Comment 21 Michael Busch CLA 2011-07-21 17:24:02 EDT
Created attachment 200130 [details]
twitter util core 1.9.2
Comment 22 Olivier Thomann CLA 2011-07-25 13:16:28 EDT
Srikanth, do we have a way to do lazy initialization of type variables ?
I believe the scala library contains a bogus signature, but it looks like javac can still compile the code.

Michael, what I reported in comment 14 still stands. I believe the signature is invalid. It would be good to get someone that works on scala involved in this issue.
Comment 23 Olivier Thomann CLA 2011-07-25 14:13:59 EDT
Would it be possible to get the finagle library recompiled using either scala 2.9.0 or scala 2.8.2-SNAPSHOT ?
It looks like the bug in the scala compiler has been fixed in one of these versions.
Comment 24 Michael Busch CLA 2011-08-30 12:22:41 EDT
Created attachment 202434 [details]
finagle-core compiled with scala 2.8.2 RC1
Comment 25 Michael Busch CLA 2011-08-30 12:23:28 EDT
Created attachment 202435 [details]
util-core compiled with scala 2.8.2 RC1
Comment 26 Michael Busch CLA 2011-08-30 12:26:32 EDT
> Would it be possible to get the finagle library recompiled 
> using either scala 2.9.0 or scala 2.8.2-SNAPSHOT ?

I just attached new jars compiled with 2.8.2 RC1. You can find the scala binaries here: 
http://www.scala-lang.org/downloads/distrib/files/scala-2.8.2.RC1.tgz

The same problem seems to happen (eclipse can't compile the project, while javac can)

Sorry about the delay, had some trouble upgrading to the new scala version.
Comment 27 Olivier Thomann CLA 2011-08-30 12:34:55 EDT
Did you report the issue to the scala team ? Maybe 2.8.2RC1 doesn't contain the fix. What about 2.9.0 ?
Comment 28 Michael Busch CLA 2011-08-30 13:18:26 EDT
(In reply to comment #27)
> Did you report the issue to the scala team ? Maybe 2.8.2RC1 doesn't contain the
> fix. What about 2.9.0 ?

We don't have more time to spend on this right now. Upgrading to 2.9.0 needs more work.

I can try to talk more to the scala team, but I guess the solution for now is to use other IDEs unfortunately. I haven't seen this behavior from javac or IDEs like IntelliJ, eclipse is the only one that can't handle these signatures.
Comment 29 Olivier Thomann CLA 2011-08-30 13:24:13 EDT
(In reply to comment #28)
> I can try to talk more to the scala team, but I guess the solution for now is
> to use other IDEs unfortunately. I haven't seen this behavior from javac or
> IDEs like IntelliJ, eclipse is the only one that can't handle these signatures.
I can investigate if we can resolve these signatures later, but this remains invalid signatures.
Srikanth, do you think we have a way to prevent early initialization of the signatures ?
Comment 30 Michael Busch CLA 2011-08-30 13:57:58 EDT
> Did you report the issue to the scala team ? 

I opened this jira issue with scala:
https://issues.scala-lang.org/browse/SI-4952
Comment 31 Srikanth Sankaran CLA 2011-10-24 01:03:13 EDT
(In reply to comment #30)
> > Did you report the issue to the scala team ? 
> 
> I opened this jira issue with scala:
> https://issues.scala-lang.org/browse/SI-4952

Where it is acknowledged to be a scala issue
(" This is fixed in 2.9.0, but unfortunately is missing from the maintenance release 2.8.2.").

I intend to close this as NOT_ECLIPSE.
Comment 32 Michael Busch CLA 2011-10-24 01:17:05 EDT
(In reply to comment #31)
> (In reply to comment #30)
> > > Did you report the issue to the scala team ? 
> > 
> > I opened this jira issue with scala:
> > https://issues.scala-lang.org/browse/SI-4952
> 
> Where it is acknowledged to be a scala issue
> (" This is fixed in 2.9.0, but unfortunately is missing from the maintenance
> release 2.8.2.").
> 
> I intend to close this as NOT_ECLIPSE.


We weren't able to upgrade to 2.9.0 yet, so I can't confirm yet if the problem is really fixed.
But I think closing this issue like you suggest is the right thing to do.

Thank you for bearing with me - I appreciate your help!
Comment 33 Srikanth Sankaran CLA 2011-10-24 01:22:21 EDT
(In reply to comment #32)
> But I think closing this issue like you suggest is the right thing to do.

Thanks, if upon upgrade you see some issues, you can either reopen this
bug or open a fresh one as the situation demands.
Comment 34 Srikanth Sankaran CLA 2011-10-24 19:34:47 EDT
Verified by Scala project folks at: https://issues.scala-lang.org/browse/SI-4952