Community
Participate
Working Groups
Build: 3.2 I20051025-0800 I have a number of PDE projects that have cyclical dependencies. The projects have explicitly exported/imported packages and use the "Eclipse-JREBundle: true" manifest entry. The bundles do not have an associated JRE JAR. When setting workspace preferences to allow cycles the 'lowest' level bundle compiles successfully (with a cycle detected warning). Other bundles fail to compile with an "Access restriction" due to "a restriction on the required project" on a number of types that are explicitly exported and visible to downstream plug-ins. Changing the forbidden reference level down to 'warning' allowed all the projects to build successfully. I expected the projects to compile with the forbidden reference level at 'error'.
I should point out that I *failed* to reproduce this on a simple testcase of two inter-dependent projects, with similar import/export settings. The projects both built successfully. However, when trying various permutations of the two test projects, I have now got into a situation where the compiler error on all except the 'lowest' project, is "The project cannot be built until its prerequisite is built." Cleaning and rebuilding all projects does not get me out of this situation. Even the simple projects have this prerequisite deadlock problem now.
I can't reproduce this on 3.2M3; so I'll close this bug (and reopen it if it recurs).
I am now seeing problems in a similar setup to Tim. I also have a number of PDE projects with cyclical dependencies, which have all had the "Eclipse- JREBundle: true" line added to their manifests. Similarly the bundles do not have an associated JRE JAR. I am running 3.2M3. When I build all, rather than the incremental builder making multiple passes over the source to resolve the dependencies, it only makes a single pass before completing, leaving me with errors stating "The project was not built since its build path is incomplete" and "The import cannot be resolved". All manifests contain appropriate package import and export lines.
Haven't figured out yet why some workspaces containing the same projects do this cyclical building and others don't.
I discovered I needed to add a '<classpathentry kind="con" path="org.eclipse. pde.core.requiredPlugins"/>' line into my .classpaths for each of my projects (even though they were converted to plugin projects). This temporarily solved some of my problems (unresolved dependencies in other projects) but I am still not getting multiple passes over the code by the incremental builder, leaving classes with cyclic dependencies only partially compiled and full of errors. However, after a few further cleans and compiles, Eclipse has stopped computing the dependencies (I just get an empty Plug-in Dependencies twisty under each project) and Im back to the same errors I had originally. My .classpath files still have the plugin container specified, and my manifests have all the correct imports and exports.
ok - went back and double checked all my dependencies and manifest. It appears that one of my projects was importing a package in its manifest that did not exist in my workspace, and this was causing my build to fail completely (all classes complain that they cannot find java.lang.Object). It seems there are two issues that have been uncovered here: 1) When converting a Java project to a plug-in project (by right clicking on the project and choosing PDE->Convert to plug-in project), a <classpathentry kind="con" path="org.eclipse.pde.core.requiredPlugins"/> is _not_ added to the project's .classpath file. 2) If an import in a project's manifest does not exist in the workspace, the build fails completely. No dependencies are generated for the projects, and no classes are compiled, even in projects whose classes and manifest are completely unrelated to the project with the erroneous manifest entry. It would be useful if the build continued as far as possible (marking classes that fail to build with errors), and if the manifest entry were highlighted to indicate itself as a cause for the failure.
Oliver - any word on the builder trace?
Kent - Since I reported the problems above, Ive been working to get my workspace into a useful state. With great help from Wassim, Ive now got a workspace without errors, but can no longer create the problem described in (2) above. Now when I add an import that does not exist, I no longer lose all my dependencies. However, if I take my working workspace and add a package import of foo.bar to any manifest, then that project does not get built, even though none of its classes depend on the foo.bar package. The manifest line will be highlighted with a warning telling me that the import package does not exist in my workspace. If eclipse is aware that this import does not exist, shouldnt it just ignore the import and try to compile the classes? ie Shouldnt my workspace still build if I add in a non-existent package import if that import is then unused?
What does the builder trace tell you when you add the missing import statement to the manifest file? Does a build start and then fail? Can you paste the trace into the PR...
Created attachment 30530 [details] Output from debug tracing Here's the trace from my workspace with all cyclical projects loaded. I did a clean and rebuild all projects. The workspace options allow build cycles, but in this case all projects fail with prereq errors.
Are you sure you changed the cycle error to a warning? Take a look at JavaBuilder.isWorthBuilding(). The message: 'Aborted build because prereq project Harmony_LUNI was not built' is printed only after we check to see if JavaCore.CORE_CIRCULAR_CLASSPATH is a warning.
Guaranteed -- double checked and the workspace setting is Page : "Java -> Compiler -> Building" Section : "Build Path Problems" Setting: "Circular Dependencies" = "Warning" There are no project specific settings on any project. I tried switching the workspace setting back to Error (and applying it) then back to Warning and that had no effect.
FYI The contents of: C:\Eclipse\workspace-3.2\.metadata\.plugins\org.eclipse.core.runtime\.settings\org.eclipse.jdt.core.prefs is as follows #Thu Nov 24 21:09:40 GMT 2005 org.eclipse.jdt.core.classpathVariable.JRE_SRCROOT= org.eclipse.jdt.core.compiler.problem.missingSerialVersion=ignore org.eclipse.jdt.core.classpathVariable.JRE_LIB=C\:/java/Sidecar 1.4.2/GA/jre/lib/core.jar org.eclipse.jdt.core.compiler.problem.forbiddenReference=error org.eclipse.jdt.core.codeComplete.visibilityCheck=enabled org.eclipse.jdt.core.classpathVariable.ECLIPSE_HOME=c\:/Eclipse/3.2-I20051031-2000/eclipse org.eclipse.jdt.core.builder.resourceCopyExclusionFilter=*.launch org.eclipse.jdt.core.classpathVariable.JUNIT_HOME=c\:/Eclipse/3.2-I20051031-2000/eclipse/plugins/org.junit_3.8.1 eclipse.preferences.version=1 org.eclipse.jdt.core.classpathVariable.JUNIT_SRC_HOME=c\:/Eclipse/3.2-I20051031-2000/eclipse/plugins/org.eclipse.jdt.source_3.1.0/src/org.junit_3.8.1 org.eclipse.jdt.core.compiler.codegen.inlineJsrBytecode=disabled org.eclipse.jdt.core.circularClasspath=warning org.eclipse.jdt.core.classpathVariable.JRE_SRC=
It would appear that we're not picking up that option then. What do you have set for: Preferences->Java->Compiler->Building->'Abort build when build path errors occur' Preferences->Java->Compiler->Building->'Incomplete build path'
> What do you have set for: > > Preferences->Java->Compiler->Building->'Abort build > when build path errors occur' That is checked - i.e. 'true' > Preferences->Java->Compiler->Building->'Incomplete build path' That is 'Error'. --- This may be a migration problem. My workspace was created by a 3.2 version a while ago (don't know exactly when, around October 11th looking at the metadata file dates). If I create a brand new workspace on this I-build and checkout all the projects again then I _don't_ get the same cycle failures.
Do you know exactly what build you're using now? And a good guess at which one you used to build the workspace? Oliver/Tim, so just to recap: with a freshly built workspace, you are not seeing any cycle problems?
> Do you know exactly what build you're using now? Build 3.2-I20051122-0800 > And a good guess at which one you used to build the workspace? It's only a guess, but since I don't take any nightly builds it would be 3.2M2 or possibly an integration build soon thereafter. > Oliver/Tim, so just to recap: with a freshly built workspace, > you are not seeing any cycle problems? Right. Using the same Eclipse version (I20051122-0800), JRE, etc; and pointing to different workspaces (using -data) I get a cycle build failure on the 'old' and not on the 'new' workspace even though both have the compiler build options set identically.
Frederic - do you know if we have changed the JDT preference story at all in the last 4-6 weeks?
I didn't change anything in JDT preferences behavior since 3.1. However, some default values may have been changed since then... From my hand, I worked on bug 110964 which changed default values for some javadoc compiler options, but it seems unrelated to this problem... I don't remember having done any other changes in this area.
I'm going to close this PR since building cycles works as expected, unless someone (DJ?) wants to investigate what happened with Tim's preferences between builds in October & now.
Further problems of this nature still exist and are reproducible in a workspace I can make available to the compiler team.
Got Tim's workspace. Indeed the cycles can cause grief. Problem is with Harmony class libs, where base classes are partionned in different projects, all parts of a cycle. Object is not in first built project, and amongst the few classes in its project, Object doesn't even get built first. Some other type (Throwable) gets compiled first, which refers to String, defined elsewhere, indirectly through prereq projects' binaries, and thus abort even before dumping Object. Further projects keep failing to built as unable to find Object etc... We should handle missing binaries more gracefully, and avoid aborting compilation. This is however a bigger change than what we can afford for 3.2. As a workaround, Harmony base classes should be partitionned differently, to ensure we can bootstrap the build process.
Tim, what happens if you manually change the build order so that the project containing Object is the first to build?
I don't think it would change anything, as it doesn't even get to build Object. It dies on a previous type which has a reference to prereq binary which looks inconsistent. BTW, in Harmony, the build order is controlled by PDE depencencies, which are computed automatically (i.e. no easy way to make luni-kernel stand first), unless going manual.
Reproduced with HEAD. Added MultiProjectTests#_testCycle6 (fails) and MultiProjectTests#testCycle7.
I think there are several issues. 1. wrappering of MissingClassFile exception which should occur to abort silently from compilation stage (avoid extra pb reporting, trace in log etc...) 2. avoid aborting when missing required type from classfile, which should improve overal behavior in Harmony case, since each build loop would make incremental progress, allowing it to finally succeed building the dispersed class lib. 3. are there issues with build order computing ? I cannot recall these... Kent is looking at (1) and (2).
The change for #1 is released in HEAD
(In reply to comment #26) > 3. are there issues with build order computing ? I cannot recall these... Not that I am aware of -- though since the compilation failed early I cannot say for sure. Providing pre-built versions of some key types (as a workaround) allowed the cyclical projects to be built successfully.
Created attachment 50377 [details] Proposed patch for Proxy Binary Type Bindings
I like the direction of the patch. Now there are more senders of #isClasspathCorrect(...). All senders should likely be caching proxies as well...
Created attachment 51842 [details] Better patch Evolution from Kent's proposal, which takes care of more senders, and also passes more information about problem reference context.
Latter patch is for 3.2 maintenance stream.
Created attachment 51847 [details] Even better patch This patch also handles the settings for aborting if classpath error, and severity of invalid classpath markers. i.e. - if willing to build in spite of errors, other problems are still accumulated. - if turning invalid CP problem to warning, no abort occurs, accumulation of problems still occur, and extra problem on project is made warning (as stated in options) - if aborting on build errors, and invalid CP is set to error, then as in the past, all other problems (but next one) are discarded to make issue very proeminent.
Created attachment 51851 [details] "Le" patch Small improvement for error location (now passes MultiProjectTests#testMissingRequiredBinaries)
Latter patch isn't great, since it breaks some GenericTypeTest. Reverting to previous patch. Need to tune error reporting to preserve MultiProjectTests current behavior (some context information was stamped during exception propagation, which no longer occurs).
Created attachment 51944 [details] "Ze" patch This patch passes all our tests.
Created attachment 51955 [details] Variation on "Ze" patch for setting Object as superclass of proxy type This patch sets proxy superclass to be Object. This is easing the job of codeassist.
This fix may change the behavior other related issues: - bug 160550 - bug 142023 - bug 73957 - bug 107931 Maxime - pls check them all and add comments here (and there). Olivier - pls hunt for more similar issues hanging in the bug database, even considering closed ones. Basically we want to find all possible/imaginable test cases where this extra resilience may improve overall behavior.
Created attachment 51996 [details] Same patch as previous (51955), including more testcases Additional tests come from https://bugs.eclipse.org/bugs/attachment.cgi?id=51947 attached to bug 160550
Bug 107931 seems unaffected by the patch.
Bug 160550 would be fixed by the patch.
The patch somewhat worsens the behavior described in bug 142023. Using the data provided in attachment https://bugs.eclipse.org/bugs/attachment.cgi?id=41605, the following scenario behaves differently depending on the patch being applied or not: - install data in a workspace and build; expect one error into c/src/ql/XmlHandler.java; - open b/src/xmlimport/ImportTaskDefinition.java; - rename the first occurrence of importData to importData2; - rebuild; expect XmlHandler to compile OK. Without the patch, the expected behavior happens. With the patch, the latest expectation is deceived as XmlHandler is not rebuilt. Touching XmlHandler and rebuilding gets rid of the error though.
Bug 73957 seems unaffected by the patch.
The patch essentially fixes bug 156339 (leaving some room for further improvement though).
Should also check consequences on bug 36397
Need a bit more work there, and consequences of the changes could ripple through all tools. Need more testing before releasing. Early M4.
Created attachment 53615 [details] Current version of the patch in 3.3 stream
Philippe - a couple quick questions: There are currently 4 spots in Scope that catch AbortCompilation & call updateContext() to fix up the source positions for the missing .class file error. There are also 4 spots in various ASTNodes that do the same thing. In some cases we could pass along the invocationSite so that the error can be created with the correct positions (instead of the start of the file), but I'm not sure if its worth the change in all cases. Are the source positions of the error worth it? Should availableFields() & availableMethods() on a BinaryTypeBinding remove ones that contain proxies ? These methods are used by code assist.
Bug 166449 is a little bit related; it has to do with whether build continues in the presence of no-longer-valid classpath markers (i.e., when a compilation participant's aboutToBuild() method has fixed a missing source directory). I've not looked to see whether this patch would fix it.
Created attachment 55276 [details] Current version of the patch against HEAD
Re: comment 48. Error positions are often key to track a problem, this is why we try to remember the last source type reference being resolved in lookupEnv. Now if slowing down everything for the exceptional case... For #availableFields/Methods() methods, my initial take would be to leave them all, so as to be consistent with compiler.
Created attachment 55339 [details] Proposed patch against HEAD for M4 We need to perform a full build whenever a project with a missing class file error is incrementally built, otherwise the project still has a buildpath error attached.
Deferring until early M5, since latest patch is not yet addressing Harmony scenario.
Added DependencyTests testMissingClassFile Enabled MultiProjectTests testCycle6 Received confirmation from Tim that his workspace builds successfully This fix also takes care of the infinite build scenario as reported in bug 160550 - by doing incremental builds of all the source files in a project, we eliminate any delta if the generated .class files are identical. As a result, projects with missing .class file errors that are involved in cycles no longer cause unnecessary deltas & infinite builds. Released for 3.3 M5 in HEAD stream Released for 3.2.2 in R3_2_maintenance stream
Created attachment 56096 [details] Final patch applied against HEAD
Created attachment 56097 [details] Final patch released to 3.2.2 stream
Verified for 3.2.2 using build M20070112-1200. Note that our Harmony workspace still fails to compile but there are more classes without errors. Currently we still got around 3700 compiler errors. This will be investigate later...
Wrong status. Need to keep the bug open for 3.3 M5 verification...
.
Verified for 3.3M5 with I20070205-0009.
