Bug 210746 - Class folder in build path does not refresh properly
Summary: Class folder in build path does not refresh properly
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.3.1   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.3.2   Edit
Assignee: Jerome Lanneluc CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-11-23 06:20 EST by Christian Holm CLA
Modified: 2008-01-24 04:46 EST (History)
3 users (show)

See Also:
philippe_mulet: pmc_approved+


Attachments
Screenshot of the class folder change problem (191.73 KB, image/png)
2007-11-23 06:20 EST, Christian Holm CLA
no flags Details
Class change partial discovery (138.90 KB, image/png)
2007-11-28 05:47 EST, Christian Holm CLA
no flags Details
Latest problem illustration (737.74 KB, application/zip)
2007-12-18 11:10 EST, Christian Holm CLA
no flags Details
A new case with debug log (695.38 KB, application/zip)
2007-12-19 08:57 EST, Christian Holm CLA
no flags Details
Proposed fix and regression test (3.70 KB, patch)
2008-01-16 10:27 EST, Jerome Lanneluc CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Christian Holm CLA 2007-11-23 06:20:09 EST
Created attachment 83620 [details]
Screenshot of the class folder change problem

Build ID: M20071023-1652

We are developing a very large project divided into multiple projects. One of the projects is the main server-side component based on EJB. This project contains around 5000 classes. When we build this project, around half of these classes are packaged in a jar-file for client-side interface use. 

After the build process (using ANT) is complete in the server-project, we can refresh the project, and Eclipse will start to recompile the dependent project, seeing that the jar-file changed. This works fine, but requires most of the dependent projects to be rebuilt from scratch which takes a very long time.

To remedy this, we switched to referencing the ant class output folder instead of the jar file in the build path. This way Eclipse can see exactly which class-files changed, and recompile only a small subset, which dramatically reduces the recompile time.

However, for whatever reason, sometimes part of Eclipse does not recognize changes in the class-files. Sometimes only the actual editors will not see the changes, while other times the package explorer will not see changes.

If I browse the build path of the dependant projects in the package explorer (under Referenced Libraries), I can see that the class explorer has not detected the change, but if you double click the class file, the class viewer will show the change. I have attached a screenshot of this case.

I have tried cleaing both the dependant projects and the server-project. I have also tried restarting Eclipse, but nothing helps. The only way to get Eclipse to see that the class-file changed, is to delete all the class-files in the server output folder, and rebuild the server-project from scratch (using ant). This however negates all the benefits of a partial rebuild.

As I said in the beginning our entire workspace is very big (around 1 gigabyte in total), so I can not in anyway upload that for reproducing. If you can't reproduce the problem, I can try to setup a small workspace with two projects, that exhibits the problem.

More information:
Comment 1 Christian Holm CLA 2007-11-28 05:47:11 EST
Created attachment 83961 [details]
Class change partial discovery

Added another attachment showing a related problem, with what I guess is the same underlying bug.

Here you can see that part of JDT detected a change I did to the ClientFacade-class. Before the refresh after the rebuild of the server, the tab for ClientChangeCenterController displayed that the class file had en error, as well as a line-marker in the editor. After the refresh the error decoration for the tab disappeared (as it should due to the discovery of the updated ClientFacade.class). The error also disappears from the "Problems"-view, but the error line-marker in the editor does not go away. Also the class-insight has not discovered the new method.
Comment 2 Jerome Lanneluc CLA 2007-11-28 07:11:18 EST
When this happens, does selecting the Ant output folder in the Package Explorer and refreshing (File > Refresh) help?
Comment 3 Christian Holm CLA 2007-11-28 07:36:30 EST
I usually refresh entire project (which also contains the ANT output folder). I assume this should also refresh all sub folders?

I will try to refresh only the ANT output folder next time I see the problem, and report back here.
Comment 4 Christian Holm CLA 2007-12-13 08:55:42 EST
This is still happening daily on a both Windows XP, Windows Vista and MacOSX on a multitude of different machines.

Can you please supply details on what you need to try to fix this problem?
Comment 5 Jerome Lanneluc CLA 2007-12-13 09:12:09 EST
(In reply to comment #3)
> I will try to refresh only the ANT output folder next time I see the problem,
> and report back here.
> 
So that didn't help?

> I can try to setup a small workspace with two projects, that exhibits the
> problem.
That would be the best thing.
Comment 6 Christian Holm CLA 2007-12-14 05:36:28 EST
(In reply to comment #5)
> (In reply to comment #3)
> > I will try to refresh only the ANT output folder next time I see the problem,
> > and report back here.
> > 
> So that didn't help?

No, I have tried that multiple times, it makes no difference.

> > I can try to setup a small workspace with two projects, that exhibits the
> > problem.
> That would be the best thing.

I have created such a workspace, but have been unable to reproduce the problem using it. Do you have any pointers to what I could do to reproduce it (looking at the screenshots and seeing the state that the cache is in)?

Comment 7 Jerome Lanneluc CLA 2007-12-14 10:40:45 EST
Sorry since I have no idea of what is wrong, I cannot give any steps to reproduce.
However, from comment 0, I see that restarting Eclipse doesn't help and the .class file in the Package Explorer still shows only 1 method with name getAvailableBookingPrivilegeGroups. In this case, it means that the .class file on disk contains only 1 such method. And it means that Ant didn't generate the correct .class file. Can you please verify that?
Comment 8 Christian Holm CLA 2007-12-18 11:10:53 EST
Created attachment 85468 [details]
Latest problem illustration

This is the latest example I had of this problem.

In the base-project, I modified a class called SQLQuery. I had added a new interface, and changed the two methods "executeSingle" and "executeList" to use this new interface instead of old ones.

I then rebuild the base-project using ANT, and did a refresh of both the base-project (which contains the ANT output folder), and the ANT output folder directly. The first refresh results in a quick rebuild in Eclipse, while the second refresh seems to have no effect (which is what I would expect as the folder was included in the first refresh).

I then go to the sub-project, where the ANT output folder is referenced as a class-folder in the build path (shown in screenshot classfolder7.png). Under "Referenced Libraries" I can find the class-folder and in there the class-file. If I double click to view it, I can see the changed methods in the main editor-view, but not in Outline. This is shown in the screenshot classfolder3.png.

If I open the class-file from the ANT output folder directly, I can also see that the methods have changed. This is shown in screenshot classfolder6.png.

I therefore find no reason to believe that our ANT scripts should be faulty. 

I then change the files in the sub-project that uses these updated methods, to use the new interface. When the change is done, part of Eclipse complains that there is an error (it thinks I should use the old interfaces). This is shown in classfolder4.png. Also, the code-insight have not seen the updated methods either, as shown in classfolder5.png.

I hope this illustrates the problem clearly.

The problem went away after a restart this time. I have not been able to figure out exactly what makes the problem go away, a restart is not always enough. Maybe there is simply a timeout on the cache?
Comment 9 Jerome Lanneluc CLA 2007-12-19 07:47:42 EST
> Under "Referenced Libraries" I can find the class-folder and in there the
> class-file. If I double click to view it, I can see the changed methods in the
> main editor-view, but not in Outline. This is shown in the screenshot 
> classfolder3.png.
This is the root of the problem. The Outline not showing your change is the symptom we should concentrate on.

> The problem went away after a restart this time. I have not been able to figure
> out exactly what makes the problem go away, a restart is not always enough.
> Maybe there is simply a timeout on the cache?
A restart will always clear the cache (it is not persisted between sessions). So I'm not sure how the Outline could not show your change after a restart.

To investigate further, can you please enable some tracing as follows?
1. Add a .options file in the same directory as eclipsec.exe with the following content:
org.eclipse.jdt.core/debug=true
org.eclipse.jdt.core/debug/javadelta=true
org.eclipse.jdt.core/debug/javamodel/cache=true
2. Open a DOS console
3. Change its Screen Buffer size to 2000x9999
4. cd to the directory that contains eclipsec.exe
5. eclipsec.exe -debug -data <path to your workspace>
   This should put some tracing in the DOS console.
6. Run Ant and go to the Outline
7. If your change was not reflected in the Outline, copy/paste the whole trace from the DOS console into a text file
8. Attach this file to this bug
Comment 10 Christian Holm CLA 2007-12-19 08:57:41 EST
Created attachment 85555 [details]
A new case with debug log

This is another example of the problem, although not completely the same.

I added a new method, 
boolean runBatchJob(Class<? extends BatchJob> batchJobClass, String customEntity)
to an interface called BatchRunner in the parent project. There is already an existing method in that interface with the same name, but different parameters.
After this I compiled using ANT, and refreshed the parent project containing both the interface and the ANT output folder. This resulted in a short rebuild by Eclipse. After this I dumped the output from the console into "console.log".

Looking at the class-file in ANT output folder in the parent project, I can see that the new method is there. See screenshot "screenshot2.png".

Looking at the class-file under Referenced Libraries in the child-project, the new  method does not show up. See screenshot "screenshot1.png".

Using code-insight when trying to use an instance of the interface, does not show the method either. See screenshot "screenshot3.png".

Trying to implement the interface and adding the @Override annotation, results in an error marker in the editor, but not under the "Problems"-view and there are no error-decorations either. See screenshot "screenshot4.png".

This case is not exactly the same as the one I described yesterday, as Eclipse does not see the updated class-file under "Referenced Libraries". However the rest of the symptons are the same.
Comment 11 Christian Holm CLA 2007-12-19 08:59:28 EST
After a restart, everything was as it should be.
Comment 12 Christian Holm CLA 2008-01-14 09:46:07 EST
Do you need any additional information or logs from me to continue work on fixing this bug?
Comment 13 Jerome Lanneluc CLA 2008-01-16 06:21:25 EST
Sorry it took so long. I finally had a chance to look at the log from comment 10.

It looks like the Java element for BatchRunner.class is not removed from the cache before the Java delta is created:

-----------------------------------------------------------------------------------------------------------------------
FIRING POST_CHANGE Delta [Thread[ModalContext,6,main]]:
Java Model[*]: {CHILDREN}
        dataconversion[*]: {CHILDREN}
                /eclub2-common/classes/server[*]: {CHILDREN}
                        dk.procard.eclub.jobs[*]: {CHILDREN}
                                BatchExecution.class[*]: {CONTENT}
                                BatchRunner.class[*]: {CONTENT}
        eclub2-srv-admin[*]: {CHILDREN}
                /eclub2-common/out/eclub2-common-server.jar[*]: {CONTENT | ARCHIVE CONTENT CHANGED}
        eclub2-common[*]: {CONTENT}
                ResourceDelta(/eclub2-common/classes)[*]
                ResourceDelta(/eclub2-common/out)[*]

-----------------------------------------------------------------------------------------------------------------------

However I cannot see how this can happen. Looking at the code, we force the closing of the element just before we create the delta.

To make sure that we are looking at the same code, could you please:
1. Select Help > About Eclipse SDK
2. Press the Plugin Details button
3. Find the Plug-in Id org.eclipse.jdt.core
4. Copy its Version in this bug
Comment 14 Christian Holm CLA 2008-01-16 06:28:39 EST
I'm just glad that you have time to look at this problem at all.

The version of org.eclipse.jdt.core is 3.3.1.v_780_R33X
Comment 15 Jerome Lanneluc CLA 2008-01-16 08:04:05 EST
I finally was able to reproduce following those steps:
1. Start a new workspace
2. Create a Java project 'P1' with 1 source folder 'src' and an output 'bin'
3. Add a new class 'P1/src/p/X.java'
4. Create a Java project 'P2' with 1 class folder '/P1/bin' and attach '/P1/src' to this class folder
5. Create a Java project 'P3' with 1 source folder 'src' and 1 class folder '/P1/bin' and attach '/P1/src' to this class folder
6. Add a new class 'P3/src/q/Y.java':
package q;
import p.X;
public class Y {
	void foo(X x) {
	}
}
7. Go to X.java and change the source to:
package p;
public class X {
	void bar() {
	}
}
8. Save X.java
9. Go to Y.java and change the source (without saving) to:
package q;
import p.X;
public class Y {
	void foo(X x) {
		x.bar();
	}
}
Observe: bar() is said to be unknown
10. Save
Observe: The Problem view doesn't contain the same problem
Comment 16 Jerome Lanneluc CLA 2008-01-16 10:27:15 EST
Created attachment 87052 [details]
Proposed fix and regression test

The corresponding .jar can be found at: http://www.eclipse.org/jdt/core/patches/org.eclipse.jdt.core_3.3.2.z20080116-1553.jar

Christian, can you please copy this jar in your .\eclipse\plugins directory, restart Eclipse and let me know if it fixes your problem?
Comment 17 Christian Holm CLA 2008-01-16 11:00:38 EST
I just did a few tests with the updated plugin, and everything seems to be working like it should. I will keep testing, but I think its fair to say that the problem is fixed.

Thanks a lot for your work on this bug. It is much appreciated.

PS. Gotta love those bugs that takes a very long time to track down, and a single line of code to fix :)
Comment 18 Jerome Lanneluc CLA 2008-01-16 11:56:20 EST
Thanks for the quick feedback. Indeed this one was hard to nail down.
Comment 19 Jerome Lanneluc CLA 2008-01-16 12:03:07 EST
Fix and test released for 3.4M5 in HEAD.

Asking PMC approval to backport to 3.3.2 since:
- when a user encounters this bug, s/he cannot rely on reported errors in the Java editor any longer
- there is no workaround
- the fix is a one-line change
- the fix is safe
Comment 20 Philipe Mulet CLA 2008-01-16 13:16:43 EST
+1 for 3.3.2
Comment 21 Jerome Lanneluc CLA 2008-01-17 04:53:48 EST
Fix and test released for 3.3.2 in R3_3_maintenance
Comment 22 Eric Jodet CLA 2008-01-24 04:46:07 EST
Verified for 3.3.2 using M20080123-0800 build