Bug 161631 - Refresh jar broken for content assist on external modification
Summary: Refresh jar broken for content assist on external modification
Status: RESOLVED DUPLICATE of bug 162621
Alias: None
Product: WTP Java EE Tools
Classification: WebTools
Component: jst.j2ee (show other bugs)
Version: 1.0.1   Edit
Hardware: PC Windows All
: P3 major (vote)
Target Milestone: 1.5.3 M153   Edit
Assignee: Konstantin Komissarchik CLA
QA Contact: Chuck Bridgham CLA
URL:
Whiteboard:
Keywords:
: 158779 161632 (view as bug list)
Depends on:
Blocks:
 
Reported: 2006-10-19 15:22 EDT by mailmur CLA
Modified: 2007-01-08 12:24 EST (History)
5 users (show)

See Also:


Attachments
Options file to activate Java Model trace (484 bytes, text/plain)
2006-11-01 07:33 EST, Frederic Fusier CLA
no flags Details
Complete example for stale content assist (6.00 KB, application/octet-stream)
2006-11-06 09:12 EST, Markus Keller CLA
no flags Details
Test case showing that refresh fails only with Dynamic Web projects (16.55 KB, application/octet-stream)
2006-11-08 14:00 EST, Frederic Fusier CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description mailmur CLA 2006-10-19 15:22:00 EDT
Have ./lib/my.jar resources in a build path. Jar file is updated in an another project, file copy&pasted to the project, refresh (F5) should reload jar changes but package explorer does not see changes.

So writing a code "MyJarClass.doXX" code insight does not display changed methods, new methods, new parameters. 

A horrible kludge to avoid restarting Eclipse, but this will lose all src+javadoc links and is tedious to do each time.
* 2nd mouse button on "my.jar" entry
* Build path/Remove from build path
* select "my.jar" file again in package explorer
* 2nd mouse buton on "my.jar" entry
* Build path/Add to build path 

See here links about the issue.
http://www.eclipsezone.com/forums/thread.jspa?messageID=92040529&#92040529
http://bugs.eclipse.org/bugs/show_bug.cgi?id=16091
https://bugs.eclipse.org/bugs/show_bug.cgi?id=19071
https://bugs.eclipse.org/bugs/show_bug.cgi?id=24019
https://bugs.eclipse.org/bugs/show_bug.cgi?id=38607
Comment 1 Olivier Thomann CLA 2006-10-19 19:05:44 EDT
*** Bug 161632 has been marked as a duplicate of this bug. ***
Comment 2 Frederic Fusier CLA 2006-10-24 13:45:19 EDT
It works for me using either 3.2.1 and 3.2 using copy/paste or drag&drop
+
Bugs you referred to were closed as WORKSFORME since initial bug 16091 has been fixed
=> also close it as WORKSFORME.

Note that refresh is necessary only if you pasted the jar file outside your eclipse session, otherwise jar content is refreshed automatically.

Note also that F5 is not refresh (unless you changed Keys preferences...), you need to select the project and "Refresh..." item in popup-menu.

Please reopen with a more detailed scenario if you're still able to reproduce, thanks
Comment 3 mailmur CLA 2006-10-26 14:22:55 EDT
Tested environments:
  WinXP & Win2K, Eclipse 3.2.1 M20060921-0945+Callisto webtool, JRE1.5.0_07

Files:
  src/fi.test/Test.java
  web/index.jsp
  web/WEB-INF/lib/myjar.jar

myjar.jar has ExProperties class and the following method.
  public int getInt(int x) { return 0; }

1) use class in myjar.jar library
I open index.jsp page and write code.
  ExProperties props = new ExProperties();
  props.get<and press CTRL+SPACE here>

Same in Test.java file and codesinsight works fine first time in source files (.jsp and .java). I use package explorer to see inside myjar.jar and it lists all methods as should do.

2) edit library (outside of this eclipse process)
Edit myjar sources and rebuild jar. This is done outside eclipse project.

3) copy new jar 
copypaste myjar.jar in Windows Explorer to project's lib folder. Real project has ant buildscript that does file copy to this project lib folder. But I created this short test project to avoid extra files.

4) now try codeinsight in .jsp and .java files
writing "props.get<press CTRL+SPACE>" should display changed method arguments and new methods. But it does not see any changes.

It does not helpe using Refresh command or closing a project. I have two choices:
restart eclipse or use "removeFromBuildPath-addToBuildPath" action.

4.1) try again copypaste in Win explorer
I update myjar.jar library, copypaste to lib folder many times and no luck.

So this goies just like another user verified in a javalobby forum. I have now tested this in two machines and both failed to see changes in myjar.jar package.
Comment 4 Frederic Fusier CLA 2006-10-31 12:40:16 EST
Does it work if at step 3) you paste the jar file in your eclipse workspace directly?
Comment 5 mailmur CLA 2006-11-01 06:47:37 EST
Step 3) copypasting .jar in Eclipse IDE instead of Win file explorer
Copypasted a new jar in Eclipse IDE, "do you want to override" msg I choose yes. But Eclipse does not see changes in a jar files. Doing Refresh command it does not help.

I have to close eclipse or do removeFromBuildPath/addToBuildPath tweak to see changes in .jar file.

Clearly there is a combination when Eclipse does not dump old jar cache and reload changes. Is it Callisto webtools plugin that broke this, its only plugin package I have installed on top of the plain E3.2.1?
Comment 6 Frederic Fusier CLA 2006-11-01 07:29:58 EST
(In reply to comment #5)
> Step 3) copypasting .jar in Eclipse IDE instead of Win file explorer
> Copypasted a new jar in Eclipse IDE, "do you want to override" msg I choose
> yes. But Eclipse does not see changes in a jar files. Doing Refresh command it
> does not help.
> 
> I have to close eclipse or do removeFromBuildPath/addToBuildPath tweak to see
> changes in .jar file.
> 
OK, thanks

> Clearly there is a combination when Eclipse does not dump old jar cache and
> reload changes. Is it Callisto webtools plugin that broke this, its only
> plugin package I have installed on top of the plain E3.2.1?
> 
I assume it could be an explanation as I never noticed such a behavior using only eclipse SDK...

Perhaps may you activate some trace to try to figure out what happen there. I'll attach an .options file to put in your <install dir>/eclipse directory (or add lines of it if you already use one) which will activate Java Model all traces.
Comment 7 Frederic Fusier CLA 2006-11-01 07:33:02 EST
Created attachment 53054 [details]
Options file to activate Java Model trace

You also need to use -consoleLog while starting your eclipse session. Then, after having reproduced your problem, copy outputs displayed in the console, paste them in a file and attach this file to this bug, thanks
Comment 8 mailmur CLA 2006-11-03 05:07:33 EST
I tried this, but some problems to grap output. Is there a way to output logfile?

Dos console's line buffer is not enough to keep all debug info. I even put buffer size to maximum but still its not even close enough.

I had to start eclipse with the following command to see anything in a console window, without "-debug" option it was blank.
  start eclipse.exe -debug -consoleLog

Problem is Eclipse always use a new console window, so using " > log.txt" directive does not help in .bat script.

I took some screenshots to describe problem. It should show how things go wrong.
http://koti.mbnet.fi/akini/java/eclipse/
Comment 9 Frederic Fusier CLA 2006-11-03 05:32:01 EST
(In reply to comment #8)
> I tried this, but some problems to grap output. Is there a way to output
> logfile?
> 
I'm afraid not.

> Dos console's line buffer is not enough to keep all debug info. I even put
> buffer size to maximum but still its not even close enough.
> 
Perhaps can you make several tries activating only part of the trace:
1) only zipaccess:
    # Turn on debug tracing for org.eclipse.jdt.core plugin
    org.eclipse.jdt.core/debug=true

    # Reports access to zip and jar files through the Java model
    org.eclipse.jdt.core/debug/zipaccess=true

2) only java deltas (not verbose):
    # Turn on debug tracing for org.eclipse.jdt.core plugin
    org.eclipse.jdt.core/debug=true

    # Print notified Java element deltas
    org.eclipse.jdt.core/debug/javadelta=true

3) only java model:
    # Turn on debug tracing for org.eclipse.jdt.core plugin
    org.eclipse.jdt.core/debug=true

    # Reports Java model elements opening/closing
    org.eclipse.jdt.core/debug/javamodel=true

> I had to start eclipse with the following command to see anything in a console
> window, without "-debug" option it was blank.
>   start eclipse.exe -debug -consoleLog
> 
You're right, sorry I forgot this point...

> Problem is Eclipse always use a new console window, so using " > log.txt"
> directive does not help in .bat script.
> 
Unfortunately no... :-(

> I took some screenshots to describe problem. It should show how things go
> wrong.
> http://koti.mbnet.fi/akini/java/eclipse/
> 
I see that and it's a real problem. Another question: do you have any other plugins installed on top of eclipse SDK?
Comment 10 mailmur CLA 2006-11-03 05:36:24 EST
Managed to redirect debugs to logfile. This is what I did starting eclipse
without a launch wrapper. Now its uses the parent console and direct works as
expected:

[ eclipseDebug.bat]
java.exe -Xms40m -Xmx256m -jar startup.jar -os win32 -ws win32 -arch x86
-showsplash 600 -consoleLog -debug > log.txt

Added logA.txt link to my webpage: http://koti.mbnet.fi/akini/java/eclipse/
(try refreshing page if does not see link)
Comment 11 Frederic Fusier CLA 2006-11-03 06:15:04 EST
(In reply to comment #10)
> Managed to redirect debugs to logfile. This is what I did starting eclipse
> without a launch wrapper. Now its uses the parent console and direct works as
> expected:
> 
> [ eclipseDebug.bat]
> java.exe -Xms40m -Xmx256m -jar startup.jar -os win32 -ws win32 -arch x86
> -showsplash 600 -consoleLog -debug > log.txt
> 
> Added logA.txt link to my webpage: http://koti.mbnet.fi/akini/java/eclipse/
> (try refreshing page if does not see link)
> 
OK, I got it thanks.

Looking at this trace it appears that java delta corresponding to jar changes is well fired:
FIRING POST_CHANGE Delta [Thread[ModalContext,6,main]]:
Java Model[*]: {CHILDREN}
	CalendarTest[*]: {CHILDREN | CONTENT}
		lib/restest.jar[*]: {CONTENT | ARCHIVE CONTENT CHANGED}
		ResourceDelta(/CalendarTest/lib)[*]


So, I would say that DeltaProcessor is doing its job correctly...

May you answer the question about additional plugins? (I guess yes as I see a web icon in your toolbar which is not in basic eclipse SDK...)

If this assumption is correct, would it be possible to make another try using only eclipse SDK on a workspace with same projects?
Comment 12 mailmur CLA 2006-11-03 07:09:12 EST
Ok, took the long path.
* downloaded fresh "eclipse-SDK-3.2.1-win32.zip" release
* downloaded fresh "eclipse-SDK-3.3M2-win32.zip" release

Updated site to show 4th screenshot from v3.3M2:
http://koti.mbnet.fi/akini/java/eclipse/

Same problems in both versions without any optional plugins. I created a new workspace and own project source folders both revions to make sure it's a fresh start for both.

Only version what works fine is old Eclipse3.1.2 release.
Comment 13 Frederic Fusier CLA 2006-11-03 07:36:38 EST
OK, thanks again for the time you took to do all these tries.
Comment 14 Frederic Fusier CLA 2006-11-03 07:40:43 EST
JDT/Core seems to be aware of the jar change (delta is well fired, indexes have been updated - seen in log file -, code assist see new method - jar has been refreshed on classpath -) => everything looks ok from our point of view...
Move to JDT/UI to investigate further why the change is not displayed in package explorer...
Comment 15 Markus Keller CLA 2006-11-03 13:55:56 EST
I could not reproduce with 3.2.1, 3.3M2, nor 3.3M3. Here are my exact steps:
- new workspace
- File > Import > Plug-ins and Fragments > Next > choose org.junit & org.junit4
- remove JUnit jars from build paths and rename junit-4.1.jar to junit.jar
- create new Java project MyTest, create a source folder
- copy org.junit/junit.jar into MyTest and add to build path
-> Open Type, Package Explorer, Content Assist shows correct contents (e.g. only one class ComparisonFailure)

- on file system, copy org.junit4/junit.jar and paste into MyTest (replacing)
- in Package Explorer, select My Test and choose File > Refresh
-> jar contents updated in Package Explorer, Open Type, Content Assist, etc..

mailmur, can you try if this scenario fails for you?
Frederic, could you reproduce the non-updating Package Explorer or is comment 14 just based on the screenshots?
Comment 16 Frederic Fusier CLA 2006-11-04 07:11:17 EST
Just based on screenshots showing this issue...
Comment 17 mailmur CLA 2006-11-05 05:18:44 EST
Mentioned JUnit test pattern worked fine, but I think it's not a real life test how project development goes (two projects, change 'lib' project, build and copy jar, use new jar in 'main' project after refresh, this cycle goes over and over again during the development).

This is weird problem, sometimes is jar-refresh working fine. But then it stops working and current Eclipse session does not see any changes without restart or removeFromBuild/addToBuildPath tweak.

I have been trying to find an exact pattern to reproduce the problem. Updated web page:
http://koti.mbnet.fi/akini/java/eclipse/
- added image5
- added "example projects" hyperlink I used to track problem

image 5: Shows how TestUtil2 is not refreshed properly within EclipseTest1 project. I edited EclipsteTest2/TestUtil2.java class several times by modifying "doit2.." method signatures. Refreshing jar worked fine when I was just modifying existing methods.

Then I added new method "doit2C(int)", refreshed EclipseTest1 but refresh does not show changes anymore and must restart Eclipse to "refresh" it.

Usecase:
* EclipseTest1/build.xml: build and copies MyLib1.jar to EclipseTest2
* EclipseTest2/build.xml: build and copies MyLib2.jar to EclipseTest1
* modify EclipseTest2/TestUtil2.java, build, refresh EclipseTest1
* modify EclipseTest1/TestUtil1.java, build, refresh EclipseTest2
* use testclasses in Main1.java and Main2.java files

Screenshot displayes a situation after:
* added new method to EclipseTest2/TestUtil2.java, run EclipseTest2/build.xml, refresh EclipseTest1, show codeassist in EclipseTest1/Main1.java
-> EclipseTest1 does not see any changes in MyLib2.jar anymore until restart.
Comment 18 Markus Keller CLA 2006-11-06 09:12:34 EST
Created attachment 53292 [details]
Complete example for stale content assist

This is not a UI problem. So far we have not seen concrete steps where the Package Explorer is showing wrong contents after the workspace has been refreshed. However, there's a problem with content assist:

- new workspace
- Import > General > Existing Projects into Workspace, select attached archive
- open EclipseTest1/src/com.test/Main1.java by doubleclicking in Package Exp.:
  - do content assist after test.
    -> doit2A(int, int) and doit2B(int) are first 2 proposals
- open EclipseTest2/src/com.test/TestUtil2.java by doubleclicking in Pack. Exp.:
   - uncomment the declaration of method doit2C(int),
   - save editor
- run EclipseTest2\build.xml as Ant Build
- select project EclipseTest1, choose File > Refresh
- in Main1, do content assist after test.
  -> still only doit2A(int, int) and doit2B(int), but no doit2C(int)
Comment 19 Markus Keller CLA 2006-11-06 09:26:59 EST
Moving back to core and adjusting title. In my tests, I found that only content assist and the compiler are affected. However, sometimes deeply expanding the jar in the Package Explorer helped resynchronizing content assist. Open Type was also OK in all my trials.

mailmur, you may want to use one of these ways for automatic refresh:
- if you run build.xml from Eclipse: Run As > Ant Build..., go to Refresh tab, select EclipseTest1 as "Specific resources"
- or the brute force: Preferences > General > Workspace > Refresh automatically
Comment 20 mailmur CLA 2006-11-07 16:13:49 EST
thx for autorefresh hint. I have moved all projects back to E3.1.2 release due to this refresh bug in E3.2/E3.3+ releases. I will happily test anything you might need.

About "So far we have not seen concrete steps where the
Package Explorer is showing wrong contents after the workspace has been
refreshed." comment:
As my webpage image5 indicates EclipseTest1/MyLib2.jar explorer displayes wrong method signatures. Valid signatures are seen in EclipseTest2/src/* leafs.

I have found out its a rather random how Eclipse lost itself and internal .jar cache(?) is not properly refreshed. I must modify doitXXX method sources back and fort (add method, comment it, add and remove int arguments) and then suddenly Eclipse stops refreshing the jar file.
Comment 21 Markus Keller CLA 2006-11-08 04:58:38 EST
> I must modify doitXXX method sources back
> and fort (add method, comment it, add and remove int arguments) and then
> suddenly Eclipse stops refreshing the jar file.

That's what I found as well -- and every time I checked the situation more closely then, I realized that I've forgotten to run the build script or to refresh ;-)
I will not investigate further in the Package Explorer issue unless someone finds reproducable steps.
Comment 22 mailmur CLA 2006-11-08 10:51:40 EST
What is your development system (JVM, exact eclipse revision, windows)?

http://koti.mbnet.fi/akini/java/eclipse/
Its not about forgetting build.xml, as screenshots indicates it clearly. And this never happens in E3.1.2 in a same machine with same test projects.

image1+2: correct method is "doitA(int, int)". tooltip says "doitA(int, int)". Package explorer and codeassist say "doitA(int)". jar was refreshed somehow just explorer+codeassist does not see it. Only tooltip error msg was able to display valid signatures.

image4: correct is "doitA(int)". But E marks both doitA(int) and doitA(int,int) as an invalid syntax.

image5: correct method seen in TestUtil2.java expand. Running build or d&d file did not help any further. It was built and copied to another project.

What I have found tooltip error message usually displayes a valid signatures, only other Eclipse parts go wild randomly.

If I doubleclick EclipseTest1/TestUtil2.class file to see "Class file editor" screen, it prints valid method signatures. But explorer+codeassist dont within the very same session state.

I begin to think its some finetuned timing issues somewhere what breaks this in Eclipse3.2+ releases. Eclipse thinks it has a valid snapshot in the internal method cache, but its not the case. Doubleclicking .class file always reads the class binary from the disk so can print valid singatures.
Comment 23 mailmur CLA 2006-11-08 11:05:17 EST
And I updated my second test project so display what parts work fine (ClassFileEditor, tooltip text), what parts failed (Packageexplorer+codeassist).
http://koti.mbnet.fi/akini/java/eclipse/
http://koti.mbnet.fi/akini/java/eclipse/jar-codeinsight-conflictF.png
Comment 24 Frederic Fusier CLA 2006-11-08 13:41:14 EST
(In reply to comment #22)
> What is your development system (JVM, exact eclipse revision, windows)?
>
Everything works well for me importing the projects setup you gave in your page (http://koti.mbnet.fi/akini/java/eclipse/EclipseTest1and2.zip). When my workspace refresh automatically is ON, I can see the content of the MyLib2.jar updated after a few seconds in EclipseTest1 project.

I verified it works using JVM 1.4.2 and 1.5.0, with eclipse 3.2, 3.2.1 and WTP (get from Callisto Update site). My windows is Windows XP SP2.
Comment 25 Frederic Fusier CLA 2006-11-08 13:49:50 EST
I noticed one point: there were neither .project nor .classpath files in your zip file. Furthermore looking again at your example in comment 3, I saw that your jar file was in web/WEB-INF directory...

So, I also made a final try creating Dynamic Web projects instead of Java ones and then I was able to reproduce the problem with WTP!

=> Move to Web Tools for further investigation as we never succeeded to reproduce while using Java projects...
Comment 26 Frederic Fusier CLA 2006-11-08 14:00:05 EST
Created attachment 53480 [details]
Test case showing that refresh fails only with Dynamic Web projects

Test case containing 4 projects:
 - b161631_1 and b161631_2: Java projects
 - b161631_W1 and b161631_W2: Dynamic Web projects

Scenario using WTP:
1) Unzip this file and import these 4 projects in a brand new workspace
2) Set refresh automatic ON in the workspace
3) Run build_2.xml in b161631_2 project
   => observe after a few seconds that MyLib2.jar in Web App Libraries
      of project b161631_1 is well refreshed
4) Run build_2.xml in b161631_W2 project
   => observe that MyLib2.jar in Web App Libraries of project b161631_W1
      is _never_ refreshed
Comment 27 Frederic Fusier CLA 2006-11-08 14:03:39 EST
Sorry, in previous comment, step 3 should read:
3) Run build_2.xml in b161631_2 project
   => observe after a few seconds that MyLib2.jar in project b161631_1 is well
      refreshed

(there's no Web App Libraries in the Java projects...)
Comment 28 Markus Keller CLA 2006-11-09 05:28:38 EST
(In reply to comment #25)
Frederic, I was able to reproduce at least parts of the problem with a pure SDK and a fresh workspace. I've spawned bug 163935 for my comment 18.
Comment 29 Frederic Fusier CLA 2006-12-19 03:41:53 EST
*** Bug 158779 has been marked as a duplicate of this bug. ***
Comment 30 Jason Sholl CLA 2007-01-08 11:22:46 EST
Sending this your way, Kosta.  Seems like there is something is missing from the WEB-INF/libs classpath container update.
Comment 31 Konstantin Komissarchik CLA 2007-01-08 12:24:46 EST
This looks like it might be a dupe of JDT's 162621.

*** This bug has been marked as a duplicate of bug 162621 ***