Community
Participate
Working Groups
I20070220-1330 When I start up my dev workspace where all binary plug-ins are imported and no PDE projects are selected with a newly installed build then a full build happens. Switched from I20070213-0907 to I20070220-1330.
Created attachment 59458 [details] Full debug log The debug log shows many INCREMENTAL bug also some FULL builds.
This might be related to the correction of bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=172444
Note that the FULL build: Successfully read state for org.apache.lucene.analysis Clearing last state : State for org.apache.lucene.analysis (#0 @ Tue Feb 20 18:07:05 CET 2007) FULL build Recording new state : State for org.apache.lucene.analysis (#0 @ Wed Feb 21 10:28:22 CET 2007) Finished build of org.apache.lucene.analysis @ Wed Feb 21 10:28:22 CET 2007 happens on an imported BINRARY plug-in.
(In reply to comment #2) after further investigation, bug 172444 seems unrelated
Using I20070313-1051 it got harder to reproduce. In the normal exit/start scenario I couldn't make it happen anymore BUT: when I did these steps: 1. start workspace 2. import binary plug-ins 3. wait until building done 4. exit 5. start ==> full build
Created attachment 61116 [details] Debug info for the session where I imported and exited
Created attachment 61117 [details] Debug info from the restart
Created attachment 61141 [details] Another trace with a full build OK, this really seems to be the pattern. Here's another debug trace. And yes - I waited until the workbench was calm before I exited and restarted ;-)
Thanks Dani. It looks like the order of access rules is being changed between exit and restart, thus causing the full build. before exit: [...] pattern=org/eclipse/core/resources/team/* (ACCESSIBLE) pattern=org/eclipse/core/internal/resources/refresh/win32/* (DISCOURAGED | IGNORE IF BETTER) pattern=org/eclipse/core/internal/indexing/* (DISCOURAGED | IGNORE IF BETTER) [...] after restart: [...] pattern=org/eclipse/core/resources/team/* (ACCESSIBLE) pattern=org/eclipse/core/internal/indexing/* (DISCOURAGED | IGNORE IF BETTER) pattern=org/eclipse/core/internal/resources/refresh/win32/* (DISCOURAGED | IGNORE IF BETTER) [...] Investigating who is changing this order...
Dani is using -Dosgi.clean=true to launch his workspace, Wassim could that be the problem (i.e. are you ensuring the same order for access rules is used when this option is used) ?
-Dosgi.clean=true does not matter since we store our state in the workspace metadata. The order of access rules is computed based on the order that the runtime gives us. In this case though, I suspect the fact that many new plug-ins were added to the sdk between the two builds may have something to do with the difference in the classpaths from one week to the next.
The title might say otherwise, but I believe that Dani is restarting the workbench on the same SDK, so no new plug-ins are added. Am I right Dani ?
>In this case though, I suspect the fact that many new plug-ins were added to >the sdk between the two builds may have something to do with the difference in >the classpaths from one week to the next. NOTE: the problem happens after I started with the new buid (see comment 5 for the scenario). After exiting I would expect that my workspace is in a stable state again.
oh wait. this bug was reported on a relatively ancient (Feb) build. As for the present, what plug-ins can we import as binary as per comment 5 to reproduce the issue?
>oh wait. this bug was reported on a relatively ancient (Feb) build. Nope. Latest logs are from I20070313-1051 I have almost all plug-ins imported as binary (all that are required by JDT UI minus the platform-text cvs module).
Moving to PDE/UI as it seems that access rules are changing between shutdown and restart.
yes, we need to investigate on the pde side.
Wassim ? Are you fixing it for M6 ? I'd like M6 to include all our fixes for classpath issues (either JDT or PDE). On JDT front, all our available fixes are in.
I plan to take a look/fix for M6. It came into our bucket a bit late though.
Brian, upon shutting down and restarting, the access rules and their order returned should be exactly the same since the workspace/target has not changed. can you verify that the rules are being unnecessarily reordered? Thanks.
Hi Brian, here are the easy reproducible steps: 1. start fresh workspace using I20070321-1800 2. import ALL plug-ins and fragments as binary projects 3. wait until workspace has been built 4. exit 5. restart ==> full build Here's how I launch my workspace from the command line in case that matters: C:\JavaSDKs\jdk1.5.0_10\bin\java -showversion -Xms50M -Xmx350M -Dosgi.clean=true -jar plugins\org.eclipse.equinox.launcher_*.jar -update -debug -keyring c:\eclipse\.keyring -application org.eclipse.ui.ide.workbench -showlocation -data c:\eclipse\workspaces\tmpx 1>log.txt 2>&1
I saw this bug again today, self-hosting on 3.3M6. Looks like the target of the bug should be moved to M7.
After several attempts following steps in comment #21, I was not able to reproduce the full build problem. Dani, when you follow steps in comment #21 (assuming you are running on a plain 3.3M6), do you see the full build problem on every attempt ?
The problem is apparently still there as you can see from comment 22. I will try again later today. Did you try with the command line args I provided? NOTE: I do not start eclipse.exe.
I agree that the problem is still there. I'm just trying to find reliable steps to reproduce the problem so that Brian can debug his code. Yes, I used the same command line as you provided.
I can reproduce it with fresh 3.3 M6. HOWEVER, I found out something important: the problem goes away if I disable my other (linked) locations that come in via 'links' folder. So, try this: 1. install fresh M6 into c:\eclipse\drops ==> c:\eclipse\drops 2. rename c:\eclipse\drops\eclipse to c:\eclipse\drops\3.3_M6 3. download and unzip the attached bug.zip to c:\ (verify that your install gets a 'links' directory 4. continue with comment 21 Philippe, are you also using additional locations / 'links' folder?
Created attachment 62107 [details] ZIP over M6
Re: comment 26. I wasn't using other locations/links folder.
My impression was that the behavior was VM specific (I often switched between VMs), but this could be a red herring.
Could anyone now reproduce the problem with my latest steps?
(In reply to comment #30) > Could anyone now reproduce the problem with my latest steps? I just tried it with your links folder but I was not able to reproduce the problem.
That's really strange. Can you verify the setup by checking the PDE Target location and see whether the additional location has been recognized?
I already did :-) I yes, the additional location was there.
Jerome, do you have a T60 (I have a T43)? It might be a timing issue: Markus can also not reproduce on his T60. Brian can you give it a shot?
I have a T41p.
Sorry about not replying earlier. I tried it yesterday with the linked content provided by Dani and unfortunately was not able to reproduce the problem. For the record I have a T42p and was running with the IBM 1.5 JVM.
Can you advise on where to set breakpoints to track down the problem on my side?
I can now also reproduce on a different machine but with the complete contents of my custom location. I've sent Brian, Jerome and Wassim that via e-mail as it contains information that I cannot share here. Please try to reproduce.
Dani, I was unable to reproduce, but I am a bit confused about this scenario. If the entire workspace is made up of binary plug-ins, what does a full build do?
This is was just to find the small scenario. The full build also happens on my dev workspace with 100s of source plug-ins - which is very bad.
I was able to reproduce with the custom location that Dani sent.
I'm so happy :-)
ok, I think I may have been able to reproduce in a runtime workbench, which is good. Jerome, what are some minimal jdt/core debug flags that would show me the before-after classpaths that would cause a rebuild?
Running with the classpath resolution tracing on (org.eclipse.jdt.core/debug/cpresolution=true), I see several instances of "missbehaving container" where the access rules on restart are in a different order than the access rules given during the first session. E.g. while initializing "org.eclipse.pde.core.requiredPlugins" for "org.eclipse.ltk.ui.refactoring", the classpath entry for "/org.eclipse.core.resources" has the following rules in different order: - pattern=org/eclipse/core/internal/resources/refresh/win32/* (DISCOURAGED | IGNORE IF BETTER) - pattern=org/eclipse/core/internal/indexing/* (DISCOURAGED | IGNORE IF BETTER) Is it because org/eclipse/core/internal/resources/refresh/win32 is a fragment ?
(In reply to comment #43) > ok, I think I may have been able to reproduce in a runtime workbench, which is > good. > > Jerome, what are some minimal jdt/core debug flags that would show me the > before-after classpaths that would cause a rebuild? > Enabling this flag should show you the before-after classpaths: org.eclipse.jdt.core/debug/cpresolution=true
An update: State#getVisiblePackages() is returning a different order of packages coming from the two fragments of core.resources. We need to figure out now if PDE is adding/removing fragments from the state to cause the runtime to return a different order, or if the order coming out of the runtime state is inconsistent.
Created attachment 62699 [details] patch This patch does an insertion sort for the unresolved bundles when in devmode. As long as the bundle ids stay the same for the BundleDescriptions in PDE this should help give a consistent ordering for fragment resolution. The theory is that fragments are getting resolved in different orders by PDE. This is causing getVisiblePackages to return the packages exported by two or more fragments attached to the same host in different orders. Performance: This is probably not the fastest way to sort the unresolved bundles list, but it was the easiest to implement to test out the possible fix. Need to keep an eye on performance with large target platform sets.
Tom, I verified that a full build occurs without the patch, and no false build occurs with the patch.
Moving this to Equinox->Framework. I'm not sure there is a consistent way for PDE to fix this. I think it needs to be done in the resolver.
Created attachment 62774 [details] patch New patch that optimizes the case where bundles are added with incrementing bundle ids (the typical scenario).
Tom, would you like me to work with the patch before you release it?
Yes, please. I want to release the patch on in the next 2 days. I have been running with the patch myself. I would be useful if others on the bug report could try it out in their environments as well.
>I would be useful if others on the bug report >could try it out in their environments as well. Just send me the patched plug-in(s).
I verified with the patched plug-in that the fix does NOT work. See attached debug log.
Created attachment 62934 [details] Debug info from using I20070403-1110 + patched osgi plug-in from Tom
I still see the error when using Dani's large test case. What I see is that The BundleDescription objects that PDE creates for the initial load use different Bundle IDs (long) than on the subsequent restarts. In the resolver we sort the BundleDescriptions according to there Bundle ID values. On a restart it seems that the IDs use a different order than on the initial workspace load. For example, we have two fragments to org.eclipse.core.resources: org.eclipse.core.resources.compatibility org.eclipse.core.resources.win32 On the initial workspace load I see these fragments get the following IDs org.eclipse.core.resources.compatibility -> 181 org.eclipse.core.resources.win32 -> 182 On ever restart after that I see these fragments get the following IDs org.eclipse.core.resources.compatibility -> 289 org.eclipse.core.resources.win32 -> 268 Notice that the id order is different from the initial load to subsequent restarts. This change in id orders makes the patch in the resolver worthless because it still changes the order in which fragments will be attached to their hosts which will lead to exports from the fragments being in a different order.
Created attachment 62986 [details] patch Discussed this with Wassim. Seems that sorting by bundle id is not good enough because PDE-UI may use different long IDs. This patch sorts by Bundle-SymbolicName instead. This patch works for me on Dani's large testcase.
I verified that the latest patch does not cause a rebulid on dani's workspace. Now I have to try it without the patch. This scenario is so long.
ok, the patch is good. You should go ahead and release it.
Fixed released to HEAD.
Verified in I20070410-1043. Thanks guys!
