Bug 36397 - Compiling source which indirectly references unavailable classes
Summary: Compiling source which indirectly references unavailable classes
Status: RESOLVED WONTFIX
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 2.1   Edit
Hardware: PC Windows 2000
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: JDT-Core-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2003-04-11 06:03 EDT by Philipe Mulet CLA
Modified: 2009-08-30 02:16 EDT (History)
1 user (show)

See Also:


Attachments
Project that shows the build issue (2.07 KB, application/octet-stream)
2004-07-19 22:08 EDT, Damien Mascord CLA
no flags Details
Cut down test case to show behaviour (15.44 KB, application/x-zip-compressed)
2004-09-27 01:54 EDT, Damien Mascord CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Philipe Mulet CLA 2003-04-11 06:03:54 EDT
From mail discussion

>
>Hi Philippe,
>
>I am running Build id: 200303071024, though I will try out the Integration 
>Build 2.1 I20030317 once it has downloaded...
>
>I am seeing the same two errors within I20030317:
>
>Kind    Status  Priority        Description     Resource        In 
>Folder       Location
>Error                   The project was not built since its classpath is 
>incomplete. Cannot find the class file for org.tusker.test1.ExternalOne. 
>Fix the classpath then try rebuilding this project.   Test3
>
>
>Kind    Status  Priority        Description     Resource        In 
>Folder       Location
>Error                   This compilation unit indirectly references the 
>missing type org.tusker.test1.ExternalOne (typically some required class 
>file is referencing a type outside the 
>classpath)      TestCompile.java        Test3/org/tusker/test3  line 0
>
>I will zip up my workspace once again, and include it, and see if it can 
>be reproduced on your side :)
>
>Damien
>
>At 11:22 AM 18/03/2003 +0100, you wrote:
>
>>Damien,
>>
>>I am only getting compilation errors reported against Test2 (which are
>>expected since ExternalOne is referenced from sources).
>>However, Test3 has no compilation error. So I cannot reproduce what you are
>>seeing exactly. Which build are you using ? Did you try on latest
>>integration build 2.1 ?
>>
>>- Philippe
>>
>>
>>
>>|---------+---------------------------->
>>|         |           Damien Mascord   |
>>|         |           <tusker@tusker.or|
>>|         |           g>               |
>>|         |                            |
>>|         |           03/18/2003 03:00 |
>>|         |           AM               |
>>|         |                            |
>>|---------+---------------------------->
>> 
>>  >---------------------------------------------------------------------------
-------------------------------------|
>>   | 
>>                                            |
>>   |       To:       Philippe P 
>> Mulet/France/IBM@IBMFR 
>> |
>>   |       cc: 
>>                                            |
>>   |       Subject:  Re: Compiling source which indirectly references 
>> unavailable   classes                         |
>>   | 
>>                                            |
>> 
>>  >---------------------------------------------------------------------------
-------------------------------------|
>>
>>
>>
>>
>>Hey Philippe,
>>
>>Sorry for the really long delay, I was really busy with work.
>>
>>Here are the entries that I did to compile the source on the command line:
>>
>>C:\eclipse\workspace\Test3>cd ..\Test2
>>C:\eclipse\workspace\Test2>%JAVA_HOME%\bin\jar -cvf Test2.jar
>>org\tusker\test2\Frub.class
>>added manifest
>>adding: org/tusker/test2/Frub.class(in = 856) (out= 480)(deflated 43%)
>>C:\eclipse\workspace\Test2>cd ..\Test3
>>C:\eclipse\workspace\Test3>del org\tusker\test3\TestCompile.class
>>C:\eclipse\workspace\Test3>%JAVA_HOME%\bin\javac -classpath
>>../Test2/Test2.jar org\tusker\test3\TestCompile.java
>>C:\eclipse\workspace\Test3>
>>
>>In javac command line, it compiles correctly.
>>
>>In Eclipse, when you have Test2.jar included in the classpath, it fails to
>>compile because it can't discover ExternalOne.  ExternalOne is never used
>>by TestCompile.java, and therefore TestCompile.java compile
>>correctly.   Eclipse is overzealous at finding external dependencies which
>>are not needed for compiling, and therefore should be declared a warning
>>only.
>>
>>Attached is the example source again if you do not have a reference to it.
>>
>>Damien
>>
>>At 11:17 AM 31/01/2003 +0100, you wrote:
>>
>> >If you compile Test2 without test1.jar, and without telling Test2 it can
>> >point at Test1, how do you expect it to magically discover ExternalOne ?
>> >I suspect that the classpath you pass to javac provides access to Test
>> >sources.
>> >
>> >You can achieve the same by adding project Test on the classpath of
>>project
>> >Test2.
>> >
>> >In Eclipse, each project can have its own custom classpath.
>> >
>> >
>> >
>> >
>> >
>> >                       Damien
>> > Mascord
>> >
>> >                       <tusker@tusker.or        To:       Philippe P
>> > Mulet/France/IBM@IBMFR
>> >
>> >                       g>                       cc:
>> >
>> >                                                Subject:  Re: Compiling
>> > source which indirectly references unavailable  classes
>> >                       01/31/2003
>> > 03:08
>> >
>> >                       AM
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >Heya Philippe,
>> >
>> >Any findings from my examples?
>> >
>> >Damien
>> >
>> >At 11:02 AM 13/01/2003 +0100, you wrote:
>> >
>> > >Thanks, we will investigate.
>> > >
>> > >
>> > >Damien Mascord <tusker@tusker.org> on 01/13/2003 04:52:49 AM
>> > >
>> > >To:    Philippe P Mulet/France/IBM@IBMFR
>> > >cc:
>> > >Subject:    Re: Compiling source which indirectly references unavailable
>> > >        classes
>> > >
>> > >
>> > >
>> > >Hey Philippe,
>> > >
>> > >I have managed to create a testcase which compiles in javac and no in
>> > >eclipse.
>> > >
>> > >Attached is the source and workspaces for the test case.
>> > >
>> > >It seems as though the compile breaks if the super class imports a class
>> > >that is currently not in the classpath.
>> > >
>> > >To test the above:
>> > >
>> > >Create jars for test1 and test2.
>> > >
>> > >Compile all three projects with test1.jar and test2.jar in the
>> >classpath...
>> > >this will work correctly.
>> > >
>> > >Then remove the test1.jar, and you will see eclipse complaining yet
>>javac
>> > >correctly compiles.
>> > >
>> > >Hopefully this is useful,
>> > >
>> > >Damien Mascord
>> > >
>> > >At 01:39 PM 8/01/2003 +0100, you wrote:
>> > >
>> > > >I would need a precise example to do anything. Our current assessment
>> > > >(which could be wrong) is that we don't need more files than javac to
>> > > >compile.
>> > > >There are portions of a missing required file you could need to verify
>> >the
>> > > >consistency of your files... I suspect this is such a scenario where
>>we
>> > > >differ from javac.
>> > > >
>> > > >
>> > > >Damien Mascord <tusker@tusker.org> on 01/07/2003 07:14:07 AM
>> > > >
>> > > >To:    Philippe_Mulet@oti.com
>> > > >cc:
>> > > >Subject:    Compiling source which indirectly references unavailable
>> > > >        classes
>> > > >
>> > > >
>> > > >Heya Philippe,
>> > > >
>> > > >Sorry to bother you via your private email address, but I am currently
>> > > >experiencing the same bug as in 6984 for Eclipse.  The projects that I
>> >am
>> > > >working on are not available to be released as they are under I.P.
>> > > >
>> > > >What I am experiencing is that certain projects will not compile
>>because
>> > > >the jars that are referenced, refer to other classes that aren't
>> >currently
>> > > >in the build path.
>> > > >
>> > > >Example as follows:
>> > > >
>> > > >TestClass.java refers to TestClassInJar, and uses various methods in
>> >that
>> > > >class.  Inside the TestClassInJar, there is a reference to a class
>> >outside
>> > > >of the build path.   This reference is not used within TestClass.java
>>in
>> > > >any way, but for some reason Eclipse requires this class to be present
>> > > >because of it's very strict referential integrity.
>> > > >
>> > > >javac + ant + jbuilder + netbeans all compile these projects as
>> >expected,
>> > > >but Eclipse requires every class that is ever references to be in the
>> > >build
>> > > >
>> > > >path.
>> > > >
>> > > >If you need any further information, please feel free to ask.
>> > > >
>> > > >Damien
Comment 1 Philipe Mulet CLA 2003-04-11 06:04:54 EDT
I can see the problem in your workspace, but using R2.1, as soon as I rebuild 
Test3, the problem goes away.
Can you reproduce in R2.1 ?
Comment 2 Philipe Mulet CLA 2003-04-11 06:22:39 EDT
Kent - this could well be addressed with your recent changes post 2.1, but what 
puzzles me is that I can't reproduce with R2.1 (nor could I before).
Comment 3 Philipe Mulet CLA 2003-04-11 06:29:16 EDT
Even more weird. When opening project Test (which was closed in workspace) and 
opened editor for Frub.java, some errors got reported against it since unable 
to resolve "import org.tusker.test1.ExternalOne;".

Forced a build, it worked fine. There seems to be something weird going on with 
first time actions prior to building.

We did force refresh on build actions, but the first time it opens and 
populates the model, it should always be correct.
Comment 4 Philipe Mulet CLA 2003-04-11 06:39:16 EDT
Actually, my version of C:\Test2.jar did contain Frub.class with only 
#otherMethod, but not #testMethod.

This explains why the offending signature wasn't causing me grief. When 
replacing the JAR with Test2/Test2.jar, the problem is visible again.
Comment 5 Philipe Mulet CLA 2003-04-11 06:48:40 EDT
This is actually solved with recent changes, also see bug 21661.


*** This bug has been marked as a duplicate of 21661 ***
Comment 6 Damien Mascord CLA 2004-07-15 05:44:04 EDT
Hi,

This issue seems to have resurfaced with the 3.0 release.  I'm not exactly sure 
when the problem came about, though the project I have started working on 
compiles correctly in 2.1.3 and with the sun compiler.

Did the fixes that were commited to the 2.x branch (for this particular issue) 
get left out in the 3.0 branch?

Damien
Comment 7 Damien Mascord CLA 2004-07-16 02:06:05 EDT
Ok, upon further investigation, the case is as follows:

Test.java uses Blah.class, which is in a included resource, say Blah.jar.

Blah.class extends class.in.classpath and implements class.not.in.classpath.

Both ant and javac don't mind this condition at all, though Eclipse isn't very 
happy about this :)


Even compiling with ant inside Eclipse works correctly.
Comment 8 Kent Johnson CLA 2004-07-19 10:25:45 EDT
Then I suspect your classpath is incomplete.

Can you please double check that your source project includes ALL the 
necessary jar files.
Comment 9 Damien Mascord CLA 2004-07-19 11:21:41 EDT
I can confirm that the "class.not.in.classpath" is not in any of the three 
classpaths, ie ant, suns javac or eclipse.

Interestingly enough, a different class that is not found is shown on Eclipse 
2.1.3.  The reason for this one is because of an unused method within an 
included class contains a method signature that references a class that isn't 
in the classpath.

Since suns javac and ant both compile these two cases "correctly" in terms of 
our project, would it be possible for Eclipse to handle the case the same?

I have not tried to use IBMs JDK since it is quite hard to download without 
downloading the full WSDK (any pointers on this btw?)


Comment 10 Kent Johnson CLA 2004-07-19 11:46:59 EDT
"correctly" is your interpretation. ;)

Can you paste in stubs for Test & Blah so we can try to duplicate this?

I assume Blah looks like:

public class Blah extends KnownSuperclass implements NotIncludedInterface {}

But what does Test look like?
Comment 11 Kent Johnson CLA 2004-07-19 13:08:43 EDT
Can you also explain why you cannot add the jar file for the missing interface 
to your classpath?
Comment 12 Damien Mascord CLA 2004-07-19 22:08:08 EDT
Created attachment 13436 [details]
Project that shows the build issue

If you leave the project as it is, (ie, just the known.jar in the classpath),
it fails to compile.

If you place the unknown.jar in the classpath, it compiles.

%JAVA_HOME%\bin\javac -classpath known.jar Test.java

The following command line builds the class Test.class with no warnings or
errors.

The verbose version of javac looks quite normal too.

%JAVA_HOME%\bin\javac -verbose -classpath known.jar Test.java

[parsing started Test.java]
[parsing completed 40ms]
[checking Test]
[loading C:\jdk1.3.1_12\jre\lib\rt.jar(java/lang/Object.class)]
[loading known.jar(test/Blah.class)]
[loading known.jar(test/KnownSuperclass.class)]
[wrote Test.class]
[total 161ms]

Even if you include unknown.jar on the classpath, javac doesn't even touch it.
Comment 13 Damien Mascord CLA 2004-07-19 22:15:40 EDT
In terms of adding the missing jar to the classpath...

There may be a case where the jar is not so easy to locate.  Perhaps it's a
licensed jar that a portion of our core product uses, and our site doesn't have
a license for that particular binary.
Comment 14 Kent Johnson CLA 2004-07-20 10:53:10 EDT
Thanks for the testcase... I'll let you know this week whether we can do 
anything.

But you should know that if you do much more with the instance of Blah, which 
causes a search up its hierarchy, then you'll be right back in the same spot 
(even with javac).
Comment 15 Damien Mascord CLA 2004-07-20 12:12:56 EDT
Sure,

I understand that javac will bork with certain usages of the class.

What would be ideal is that the Eclipse environment compiles the source using 
the same "strictness" that ant or javac itself uses, with any disrepancies 
pushed to a notice or warning status rather than error.  (And yes, I've tried 
the flag within the preferences, and the behaviour doesn't change).

Anyway, say we have an engineer running with JBuilder, or Netbeans or vi or 
something, and a particular classpath or environment enables you to build a 
project.

If that person were to migrate over to Eclipse and find his project no longer 
compiles, at least having a warning would point him to the fact that he is in a 
borderline case.  Having an error makes it a bit difficult :)

Thanks for taking the time to look at it.

Comment 16 Kent Johnson CLA 2004-07-20 12:47:28 EDT
My main point is that you know you're working with an incomplete classpath & 
the obvious solution is to make it complete.

I accept there may be cases when that's not possible, but what happens when a 
relatively new team member makes a small change that now requires the missing 
type? Do you think he'll understand why? Or know how to fix it?

It doesn't really matter what environment he is using. The project's classpath 
is incomplete & it should be fixed.
Comment 17 Damien Mascord CLA 2004-07-20 21:32:16 EDT
*nods* Yeah, agree with you :)
Comment 18 Kent Johnson CLA 2004-08-12 13:07:51 EDT
Released changes into the first milestone build for 3.1.

Give them a try if you have time.
Comment 19 Kent Johnson CLA 2004-08-26 11:29:02 EDT
Marking as fixed.

Changes were actually in M1 but waited for any negative feedback.
Comment 20 Damien Mascord CLA 2004-08-27 05:06:27 EDT
Hey Kent,

Sorry I haven't had the time to respond to your query about whether it was fixed
in 3.1M1.  I have tried 3.1M1, but it doesn't seem to have fixed this.  Do I
need to change any flags within the settings?

Damien
Comment 21 Kent Johnson CLA 2004-08-27 10:27:37 EDT
I tried your testcase again & it worked fine.

Can you double check with last night's build to see if another fix from the 
last few weeks is also necessary?
Comment 22 Frederic Fusier CLA 2004-09-23 12:07:46 EDT
Verified for 3.1 M2 with build I200409230010.
Note that it is already fixed in 3.1 M1...
Comment 23 Damien Mascord CLA 2004-09-27 01:30:23 EDT
Hi,

I am unable to create a test case for this issue anymore (so some things have 
been resolved, thanks!), but I will continue to try.

Is there any debugging within Eclipse that I can turn on to determine exactly 
why the error is showing?
Comment 24 Damien Mascord CLA 2004-09-27 01:54:32 EDT
Created attachment 14786 [details]
Cut down test case to show behaviour

There is a compile.bat which uses javac to compile this class.

The classes contained in this zipfile are Copyright Datalex (and Sun), and are
only present to show the behaviour for Eclipse.  It is not intended to be used
for any other purpose, and is not licensed for any other purpose.
Comment 25 Kent Johnson CLA 2004-09-27 11:41:35 EDT
This is the class definition for HASEventManager

public class HASEventManager extends java.rmi.server.UnicastRemoteObject 
implements com.sun.jini.lease.landlord.Landlord

So since you have a source reference to HASEventManager:
   errorEventManager = new HASEventManager();

We need to resolve its hierarchy when you send messages to errorEventManager.
Comment 26 Damien Mascord CLA 2004-09-27 12:08:54 EDT
Since it compiles with javac, would it be possible to have this as a warning, 
rather than an error?
Comment 27 Kent Johnson CLA 2004-09-27 13:40:13 EDT
Sorry we cannot make this a warning and I don't see how javac doesn't have a 
problem.

This line
   errorEventManager.notifyListeners(event);
causes us to look up the hierarchy of HASEventManager.
Comment 28 Philipe Mulet CLA 2004-09-27 17:20:33 EDT
Kent - why do we resolve entire hierarchy, and not only up to the point we 
find the target method ?
HASEventManager defines #notifyListeners(...)
Comment 29 Philipe Mulet CLA 2004-09-27 17:30:19 EDT
I figured the method lookup needs to walk superclass since there is no exact 
match. It needs to find in superclass whether a better match would exist. 
Apparently javac treats the missing superclass as if no better match would 
exist up in the hierarchy. This seems to be a wrong decision, as at runtime, 
if the missing class is added and would define a better match, the wrong 
method would get invoked. Our compiler identifies this issue at compile-time.
Comment 30 Damien Mascord CLA 2004-09-27 21:47:39 EDT
I agree that a more accurate method signature is the ideal goal, but as an 
implementor of an interface, I don't see how you will get any more accurate 
methods checking the interface itself...  

Checking up the super-class tree for the method would useful, and necessary, 
though since the class implements the interface, all method signatures would be 
in the class itself, no ?
Comment 31 Philipe Mulet CLA 2004-09-28 04:09:23 EDT
Damien: Look at the following example:
Even though a match #foo(Object) exists on type X (and implements interface), 
the compiler will check superclass to find #foo(String) which is a better 
match.

public class X extends Supc implements Intf {
	public void foo(Object o) {
		System.out.println("foo(Object)");
	}
	public static void main(String[] args) {
		new X().foo("hello");
	}
}
interface Intf {
	void foo(Object o);
}
class Supc {
	void foo(String s) {
		System.out.println("foo(String)");
	}
}

Now imagine Supc was missing on the classpath at compile time, if we did 
ignore the missing superclass, then we would bind to #foo(Object). Then when 
running the code later on, you wouldn't invoke #foo(String) since Java uses 
static linking (method signatures contain statically determined argument 
types). I believe javac is wrong in tolerating this mode.
Comment 32 Kent Johnson CLA 2004-09-28 10:23:48 EDT
But its not a missing superclass, it is a missing superinterface.

The method resolution is as follows:

- try to find an exact match to notifyListeners(ConnectionErrorEvent) starting 
in HASEventManager & then up thru its superclass chain. An exact match is not 
found since the declared arg type is HASEvent.

- so then we try to find the best match which is the method notifyListeners
(HASEvent)

- then we MUST check for a better matching default abstract method, which 
means we walk the superinterface chain, since its possible that you don't 
implement all of the methods defined by a superinterface.
Comment 33 Damien Mascord CLA 2004-09-29 03:49:11 EDT
Ok, full understand.  Should we raise this to Sun regarding their compiler?  If 
so, what would be the best way to go about this?
Comment 34 Philipe Mulet CLA 2004-09-29 04:56:04 EDT
The JLS deliberatly does not address dealing with incomplete libraries. It is 
left up to compiler implementations. So we are on a border case here, and 
cannot really complain. 

Note that further thinking made us realize that we could improve support for 
missing superinterface scenario. The reason why we need to walk 
superinterfaces (and thus notice its missing in your scenario) is to find 
possible unimplemented abstract methods. 

This could be avoided on a non-abstract (receiver) type, since per contract it 
should implement these abstract methods. However, since we support compiling 
in degraded mode (i.e. still dump problem classfiles out for further compiling 
dependents), we walk superinterfaces in case there would be some unimplemented 
abstract method (diagnosed elsewhere, but not outputed to classfiles). We 
believe that if we made our compiler still dump artifacts for missing abstract 
method implementations, it would allow us to tune our lookup semantics in a 
way which would make your scenario work.

The superinterfaces walk would then only occur on abstract receiver types 
(class or interface). This likely would be a performance improvement for our 
lookup algorithm, in addition to make us more resilient to these corner 
situations.

On your end, we still believe you should fix up your classpath to be on the 
safe side. Any subtle change in your code could require these interfaces to be 
loaded, so you are living dangerously.
Comment 35 Philipe Mulet CLA 2004-09-29 04:57:21 EDT
Reopening for further looking at lookup optimization.
Comment 36 Philipe Mulet CLA 2005-04-07 09:26:18 EDT
Deferring post 3.1
Comment 37 Denis Roy CLA 2009-08-30 02:16:55 EDT
As of now 'LATER' and 'REMIND' resolutions are no longer supported.
Please reopen this bug if it is still valid for you.