Community
Participate
Working Groups
I'm using external tools (Ant or Maven) to do a clean builds on the same source code as Eclipse is working with (it is verified on 3.1 N20050609-0010 on Windows XP with auto refresh rurned off). During an external build Maven removes several source directories and creates them again using XDoclet to generate these sources from scratch. When build is done I go back to eclipse and manually run refresh on this project and when refresh is done I see several compilation errors (usually not on regenerated sources) and they disappear only after I run Project -> Clean... It seems that this is a very old issue, because I've filled request for a workaround almost a year ago. https://bugs.eclipse.org/bugs/show_bug.cgi?id=60905 Please let me know if you need more details or testcases.
Moving to Platform/Resources. The refresh should trigger a recompilation of the changed resources.
It does trigger some recompilation, but recompilation is incomplete after that.
If a build is occurring but the errors don't go away, it sounds like a problem with the Java builder. I think this is a duplicate bug but I can't find the original right now - Kent, I think it was one of yours?
It is hapening on every external build for me. Please, please fix this! My build deletes some source dirs that are in Eclipse's .claspath and create them from scratch (using XDoclet) and I noticed that if build does not delete directories compilation is ok. But when it deletes (autorefresh is off), compilation always incomplete and require explicit "Clean" action.
Can you please provide us a reproduceable testcase? What specifically are the errors that do not go away? What happens if you refresh the projects individually & in order (instead of all together), do the errors still happen?
Kent, it is a _single_ Eclipse project with multiple source directories managed with Maven, XDoclet and Cactus. No dependencies on other projects. I've uploaded a bundle at http://w8.platonoff.com/11/commonj.zip Before importing this project into Eclipse you'll have to run Maven to get dependencies. Download and install Maven 1.0.2 from http://maven.apache.org/ Add properties to ~/build.properties (on Windows C:/Documents and Settings/<user name>/build.properties --- commonj.home=c:/dev/xcommonj/lib --- Run following commands: maven plugin:download -DgroupId=cactus -DartifactId=cactus-maven -Dversion=1.7 maven plugin:download -DgroupId=xdoclet -DartifactId=maven-xdoclet-plugin -Dversion=1.2.3 maven default Now you need to add MAVEN_REPO classpath variable pointing to local Maven's repository. Eg. C:/Documents and Settings/<user name>/.maven/repository After that you can import Eclipse project into workbench. Make sure that it can be compiled (refresh, clean). Then from a command line run maven clean default Go back to Eclipse and refresh project. You'll see build errors. I wasn't able to figure out any patters about what classes are failing. Those errors will go away after "Clean..." By the way, new Clean... dialog is really suck in a large workspace.
the command 'maven default' failed. It said: 'Goal "default" does not exist in this project.' I tried to continue but none of the jars in the maven repo match the expected ones in your project, so the build fails with 15 missing jars & 3 missing source folders. BUT just to clarify: 1. All your compile errors go away when you do a Clean? If so then this is not a JavaBuilder issue but a refresh problem in Core. 2. What happens if you turn off 'Project->Build Automatically' & then perform the refresh. Do the deleted/recreated source folders appear in the workspace? If they do not, can you create them & do a build (hit the build button, not Clean). Do you get errors in this case?
(In reply to comment #7) > the command 'maven default' failed. It said: 'Goal "default" does not exist in > this project.' Strange. It is defined in maven.xml. Did you run "maven default" from a commonj2 dir? I'm not using external tools here, just a command line on windows. > I tried to continue but none of the jars in the maven repo match the expected > ones in your project, so the build fails with 15 missing jars & 3 missing > source folders. This is very strange. All jars are on ibiblio.com/maven repository. > BUT just to clarify: > 1. All your compile errors go away when you do a Clean? If so then this is not > a JavaBuilder issue but a refresh problem in Core. Yes. They all go away on clean. > 2. What happens if you turn off 'Project->Build Automatically' & then perform > the refresh. Do the deleted/recreated source folders appear in the workspace? > If they do not, can you create them & do a build (hit the build button, not > Clean). Do you get errors in this case? ------- Disabled build Run Project -> build project No errors From command line run "maven clean default" Switch to Eclipse and run refresh on this project. All folders and files are there. No build errors Run Project -> Build project. Errors appears! Run Project -> Clean... Errors cleaned up. -------- It seems there is no difference if autobuild is on or off. One more thing. Eclipse build output folder is different from the folders used by external builds. See .classpath
How many errors are there, and what are they? Also could you please turn on builder tracing so we can see what is happening: 1. Create a .options file with the following content: # Turn on debug tracing for org.eclipse.jdt.core plugin org.eclipse.jdt.core/debug=true # Reports incremental builder activity : nature of build... org.eclipse.jdt.core/debug/builder=true # Turn on debugging for the org.eclipse.core.resources plugin. org.eclipse.core.resources/debug=true # Reports the start and end of all builder invocations org.eclipse.core.resources/build/invoking=true 2. Start Eclipse with -debug <path to the .options file> -vm <path to jre>\bin\java.exe 3. Ensure that the DOS console that is opened has a buffer size set to 9999 4. Copy/paste the trace in a .txt file and attach it to this bug report thanks
Created attachment 23353 [details] Build errors after refresh Here is the list of errors I'm getting. It looks like I'm getting the same errors on every run.
Created attachment 23354 [details] Trace file with suggested .options enabled
I'm starting to wonder if this just may be a bug/problem with your setup. You have the same package in several source folders (which is supposed to work) and it also appears that the output folder is in the same folder as your generated files. Could you please move the output folder to /bin instead of /target/classes & see if things are better. Its very strange that you would have errors like: "Locator cannot be resolved" from XWorkManager.java since the 2 types are in the same package in the same source folder. It would also be very helpful to get a list of .class files in the output folder before AND after maven has generated files. I suspect some .class files (such as Locator.class) are being deleted.
For the list of .class files, please get them from Windows (not Eclipse itself, which may not know that they have been deleted). Given your errors, I'm really interested when Locator.class was deleted.
Hmm. It looks like Maven's clean goal is deleting the entire target dir in a project root. So, all classes compiled by eclipse are deleted. Also note that the following source folders also deleted on clean: <classpathentry kind="src" path="commonj-ejb/target/xdoclet/ejb"/> <classpathentry kind="src" path="commonj-it1/target/xdoclet/ejb"/> <classpathentry kind="src" path="commonj-it2/target/xdoclet/ejb"/> I can see that Eclipse is recreating target/classes directory on refresh and even see new classes in there (including Locator.class). Note that Maven build is not compiling into /target/classes, but put all compiled classes into the following dirs (primarily because javac target is different from the one used by Eclipse's builder): /commonj-ejb/target/classes /commonj-ejb/target/test-classes /commonj-ejb/target/test-cactus-classes /commonj-it1/target/classes ...
Well we don't consider this to be our bug. I strongly suggest to keep your eclipse source files & .class files away from anything created/modified by maven.
Kent, please consider to fix this. It is very annoying when building/running integration tests for in-container testing to do refresh, clean, select projects for cleaning. 5..6 operations just to pickup external changes is way too much for frequent activity. Generated source files can't be kept away from Eclipse classpath because they are used to compile annotated classes (currently annotated with XDoclet). Keeping away compiled classes probably possible, but I consider it risky because I don't feel that Eclipse's incremental compiler can completely track down all the dependencies from the modified sources.
Sorry but how is this our bug when an external builder is deleting our generated .class files? You can easily setup your project so that the eclipse files are not in the same folder structure as the maven files. Just have 2 folders at the root: eclipse vs. maven. It does not prevent eclipse source types from referencing maven types, but it does keep maven from deleting our .class files.
Kent, first of all I consider that deleting Eclipse's output folder is a way to force Eclipse to rebuild everything. To me it makes sense if half part of the sources were regenerated. Also note that currently, there is no way to force "Clean" from external tool configuration, so you can only do refresh of selected folders. So, I assume that Eclipse _must_ nicely recover from the situation when its classes had been changed externally.
Well that is where we disagree. Eclipse should handle external changes to its source files, but the generated .class files are just that - generated .class files. Why have Eclipse build them, only to have an external builder delete them? If Eclipse was deleting the maven generated .class files, do you think they would notice & rebuild them? Not a change - they would tell you to report it to us as our bug. Please change your setup. It will take you a few minutes & you won't experience any more problems.
(In reply to comment #19) > Well that is where we disagree. > > Eclipse should handle external changes to its source files, but the > generated .class files are just that - generated .class files. Why have > Eclipse build them, only to have an external builder delete them? Let's be reasonable. Recovering from the situation when external tools deletes classes should be just natural and I look at it as a feature that improve user experience and simplify integration with external tools. > If Eclipse was deleting the maven generated .class files, do you think they > would notice & rebuild them? > > Not a change - they would tell you to report it to us as our bug. I probably agree with you if automatic updates are turned on. However when thay are off, it is reasonable to assume that eclipse build and external build does not overlap and developers may choose to automate a cleanup process using external tools, so when they switching back into eclipse it should recover. Actually Eclipse does recreate output folder, but does not complain or rebuild all the classes. I believe that if any of the output folder has to be recreated, Eclipse should run clean automatically. > Please change your setup. It will take you a few minutes & you won't > experience any more problems. Unfortunately it is not that easy. We have over 50 modules (in a different project, then I've sent to you) configured to share output folders beteen Eclipse and Ant build. There are advantages to share classes folders between eclipse and external tools, such as Ant or Maven: -- Disk space -- Improve Ant build time (it can pickup classes already built by Eclipse) -- Ensure that remotely deployed classes are the same as the one used by debugger without using manual launch configuration -- Easy to run clean build between Eclipse and Ant (this one apparently does not work now) From other hand, there is also chances that configuration of the output folders is managed by custom classpath container (eg. maven project.xml or even 3rd part IDE configs) and you have to use the same output folders as declared in external config and don't use any Eclipse-managed output folders at all. So, I believe this is very important and actually very natural to recover for the situation when output folders are deleted.
Consider this simple experiment: 1. Eclipse 3.1 RC2 on Windows (any release/version of Eclipse will suffice) 2. Create a new Java project using the 'New Java Project' wizard 2a. Name the project 'TryIt' 2b. Check 'Create separate source and output folders' so that 'bin' and 'src' folders are created 3. Click 'Finish' 4. Create some Java source file in the default package, say, 'Main.java'. Its contents don't really matter. 5. Now here's the fun part: let's go behind Eclipse's back and open a Command Prompt and delete the 'bin' directory: C:\workspace\TryIt>rmdir /q/s bin As a developer, I would intuitively *think* that this would be a good way to manually clean the project... 6. Now go back to Eclipse and do a 'Refresh' of the Java project. Note that nothing really happens. The 'bin' directory is *not* recreated. Neither are the .class files. I would have expected Eclipse to discover that 'bin' and its class files have suddenly disappeared and that the 'bin' directory needs to be recreated and .class files need to be regenerated. So, ya, I've run into similar situations as Eugene has run into, though not as complicated (i.e., does not involve Maven). In my situation, I have an external build.xml ANT script whose 'clean' target simply deletes the entire 'bin' directory (including the actual directory itself), and I always run into the above problem, whereby I clean externally via ANT and then zoom back to Eclipse and do a Project 'Refresh', but nothing happens.. though the problem is solved by doing an explicitProject->Clean from within Eclipse...
Comment on attachment 23353 [details] Build errors after refresh nailed the issue
Comment on attachment 23354 [details] Trace file with suggested .options enabled nailed the issue
I think I'm seeing this same problem when trying to do Mondrian development (mondrian.sourceforge.net) in Eclipse. The main Mondrian developer doesn't use Eclipse so he has everything building with ant files. Whenever I update the code from source control I have to go through this dance that takes me about an hour, cleaning the project in Eclipse, cleaning it with ant, rebuilding with one or the other or both, turning off automatic rebuilds, repeat, turn builds on, have tests fail because something didn't get rebuilt, etc. etc.. I couldn't tell you what order of things works because I just keep messing around until it runs again. One of the things his ant script does is generate some java files and then compile them.
(In reply to comment #24) Ed, that's exactly right! I too have a hard time explaining other developers how to sort out all the errors in Eclipse builder after picking up major changes that require some cleanup and source regeneration.
Yes be reasonable! There is no Java Builder bug here. 1. Eclipse allows you to setup your project in numerous ways & the root of your problem is YOUR project setup. 2. Why share the output folder with an external builder? You can have multiple .class folders on your path. 3. The Java Builder will NEVER do a refresh for you since it does not know when you have run an external builder. You must always initiate the refresh, but not every refresh requires a Clean... Spend your time convincing Core that bug 60905 is a useful feature that will save you time, but there is no Eclipse builder bug here. It is totally unacceptable to slow down every incremental build for every user of Eclipse to look for non-source deletions because an external builder has bugs & may have changed the project.
(In reply to comment #26) > Yes be reasonable! There is no Java Builder bug here. > > 1. Eclipse allows you to setup your project in numerous ways & the root of > your problem is YOUR project setup. Unfortunately this luxury is not always awailable. I can give you number of examples when project configuration can't be changed. Please read my previous responses, I have several complicated cases explained up there. By the way, if you look at JUnit project by Erich Gamma, it is even use project root folder as a source dir, so there is no other place to put classes in order to keep eclipse build separate from external one. I'm not saying this is the best way to configure project, but it is a representative example of real case. > 2. Why share the output folder with an external builder? You can have > multiple .class folders on your path. This is making project configuration more complicated and I don't really see any reasons (except this issue with Eclipse) why to duplicate class directories. In some cases I have to use 2 or 3 differwent IDEs and several external tools on the same project, so why should I spearate their class files? > 3. The Java Builder will NEVER do a refresh for you since it does not know > when you have run an external builder. This in _NOT_ about builder. This is about runnin authomatic clean when Eclipse tetect that target folder disappear. > You must always initiate the refresh, but not every refresh requires a Clean... I agree with that. However "Clear..." action is completely NOT intuitive and really annoying feature in Eclipse. Refresh should be smart enough to pickup external changes. > Spend your time convincing Core that bug 60905 is a useful feature that will > save you time, but there is no Eclipse builder bug here. What could help but does not resolve actual problem cause. The very same issue happens when I run external tool from eclipse where you can only specify refresh for selected resources, but not Clean! > It is totally unacceptable to slow down every incremental build for every user > of Eclipse to look for non-source deletions because an external builder has > bugs & may have changed the project. I believe that it is acceptable and not performance critical to verify if target directories _WITHIN_ manual refresh scope had been deleted (don't you schan trough this dir already?) and then run full build in this case!
No we do not scan the generated .class files. Why should we? For the large majority of eclipse users, their .class files remain unchanged until the Java Builder recreates them. Our goal for incremental builds with auto-build enabled is 50-250 ms... we do not have spare cycles to waste looking for changes from out of the blue. "so why should I spearate their class files?" To prevent the exact problem you are having! Maven has is corrupting the Eclipse build state. Why are you not asking Maven to fix their problem? "This in _NOT_ about builder. This is about runnin authomatic clean when Eclipse detect that target folder disappear." Then please continue the discussion in your other bug, because JDT/Core does not define the semantics of Refresh or Clean.
(In reply to comment #28) Kent, I have feeling that JDT/Core component could be a wrong asignee for this bug. Please reassign it to JDT inbox or to more appropriate component. > No we do not scan the generated .class files. Why should we? I wasn't talking about class files. I was talking about directories that contain class files. > For the large majority of eclipse users, their .class files remain unchanged > until the Java Builder recreates them. I wonder how do you measure that? I see that my coworkers have to run refresh and then clean. I know that some people are using "export to jar" feature and never using ant or other command line tools. So what? It should be an excure to do not fix this issue. :-) > Our goal for incremental builds with auto-build enabled is 50-250 ms... we do > not have spare cycles to waste looking for changes from out of the blue. Before incremental build is kicked of, I guess there is another process that is scanning trough project directories on "refresh" action. Am I right? If so, this process should decide to run complete build when it detected that output folders had been deleted (or maybe even externally modified). > "so why should I spearate their class files?" > > To prevent the exact problem you are having! Maven has is corrupting the > Eclipse build state. Why are you not asking Maven to fix their problem? If Eclipse would run "Clean" when detected that target directory had been deleted it wouldn't be an issue. Consideting that Maven is not the only tool, and I can continue list of such tools... it would make sense to fix just one other tool that is toooooo sensitive about its own directories. I'm talking about Eclipse here. :-) > "This in _NOT_ about builder. This is about runnin authomatic clean when > Eclipse detect that target folder disappear." > > Then please continue the discussion in your other bug, because JDT/Core does > not define the semantics of Refresh or Clean. I have feeling that this bug is more appropriate, comparing with refresh option in Clean dialog, which is quite hidden then btw. From an end user prospective it should significantly improve user experience with picking up external changes. It is not about incremental builds, but about very specific cases that should be possible to detect without performance issues in incremental builder.
Even if it's not a builder bug there's some kind of issue here with people working in a mixed environment where some developers use Eclipse and some don't. It would be nice to support this in some way without changing the project structure. For example on Mondrian I'm not the main developer, and the main developer doesn't use Eclipse, and he's not going to change anything to make the Eclipse users happier. So although I'm not going to reopen the bug, and don't know exactly where it belongs, it's annoying enough and common enough that I think it should be considered for addressing somehow in a future release. Maybe through a smarter Eclipse<->Ant integration (without losing the Eclipse integrated incremental compiliation) or something, I don't know. Thx.
Eugene, I have been very polite and patient while trying to explain to you that the Java Builder is one of many builders. JDT does not define refresh or clean. We do not know when a refresh has happened before an incremental build is invoked. We will not slow down every incremental build trying to recover from an external builder's bug. If you want the option to force a clean after a refresh, then please add your comments to bug 60905. The only bug you have found is with Maven. Please be reasonable and recognize that fact.
Ed, for some users who build portions of their project with external builders, they could use an integrated refresh/clean command. But the Java Builder, as one of many builders in Eclipse, does not know when a refresh has been started by the user. The incremental build which follows (assuming source changes are found) is the first time the Java Builder is invoked by the Build Manager. But as the user you do know & can force the Clean & full build to happen. Its not reasonable to expect an incremental build to detect/recover from any number of external changes to non-source files - that's what a Clean/full build is for...
(In reply to comment #32) > Ed, for some users who build portions of their project with external builders, > they could use an integrated refresh/clean command. The thing is that they don't really know when to run just refresh and when refresh + clean. I have difficult time to explain how it suppose to work from Eclipse. So, developers are ending up with using command-line tool to compile, or even swith to other IDEs that have explicit build command. > But the Java Builder, as one of many builders in Eclipse, does not know when a > refresh has been started by the user. The incremental build which follows > (assuming source changes are found) is the first time the Java Builder is > invoked by the Build Manager. As you said, it should happens before incremental builder. So, build manager should be able to discover that target/output directories had been changed or deleted and it should kick clean (or full rebuild) authomatically. > But as the user you do know & can force the Clean & full build to happen. Its > not reasonable to expect an incremental build to detect/recover from any > number of external changes to non-source files - that's what a Clean/full > build is for... Again, lets leave both your incremental build and our tools (Maven, Ant, make, and all others) aside of this issue. The point is that it could happend that external tool may need to clean output folders. Please accept this as a fact. I don't really see any reasons why only Eclipse has to have problem with this if none other tools have any issues with this and can share the same output folder.
> But as the user you do know & can force the Clean & full build to happen. Its > not reasonable to expect an incremental build to detect/recover from any > number of external changes to non-source files - that's what a Clean/full > build is for... it looks like the functionality is not belong to incremental builder, but to something else (external thread), that will trigger full build...
The proposed solution "clean project when output folder disappears" is probably quite misleading... (it is to me) I doubt the solution is to throw more code at it... The solution might be do just fix a small bug in the jdt builder? Here's my theory: I think the problem/solution is much easier/simpler to explain... I'll give a shot (also, note that I might be completely wrong!) I would assume that when a user invokes "Project Refresh" that JDT builder is notified of changes in *source* files (not generated output files)... fine. When the JDT builder is notified of changes in, say, .java source files, I would assume that it runs off and checks the date/time stamp of the corresponding generated .class file wrt the .java source file to see if it needs to be rebuilt? (Note- if this is false then my theory is blown out of the water!) Well, perhaps the problem is that everything works fine when the output directory where the generated .class file lives in (e.g., 'bin') exists... And the problem is that this bug surfaces when the output directory where the the generated .class file lives in does *NOT* exist... Perhaps there's a bug somewhere whereby this timestamp comparison trips up when 'bin' does not exist that causes the resources/refresh jdt/builder to trip up in the case..?!? This would certainly consistent with my previous comment "Consider this simple experiment..." (Note- this is just a theory: I have *not* looked at any JDT source to attempt to verify this theory...)
"Note- if this is false then my theory is blown out of the water!" Its false. Individual builders in Eclipse do not scan files/folders looking for changes. The Build Manager computes the deltas & informs each builder attached to the changed project to incrementally build IF it is affected by the deltas. The Java Builder records build state information about its generated .class files & projects it depends on. We must do full builds when this information cannot be trusted (such as external builders deleting .class files). So how many projects are being rebuilt by Ant or some other external builder AND then by Eclipse... over and over? If your project is built by Ant, then why build it again with Eclipse? Why is the Ant output not acceptable? If you have added all the source into Eclipse, why run the external builder (if it does NOT generate more source files), only to build it with Eclipse afterwards? If it does generate source, then why compile it with the external builder if Eclipse will build it anyway?
JavaBuilder.hasClasspathChanged() perhaps this could be changed to detected the physical non-presence of any output folders? dunno... or perhaps JavaBuilder's IBuild.build(...) could do the actual check for the presence/absence of any output folders and invoke a buildAll() if they are physically missing from the filesystem (is this also applicable for source folders too?!? dunno...)
(In reply to comment #35) > The proposed solution "clean project when output folder disappears" is probably > quite misleading... (it is to me) ... What called "Clean..." in JDT is actually full rebuild.
>>> We must do full builds when this information cannot be trusted (such as external builders deleting .class files). How about instead of: (such as external builders deleting .class files) How about: (such as external builders deleting folders containing .class files) I think this is the root cause... I think this is the case that is not taken care of?
(In reply to comment #35) > So how many projects are being rebuilt by Ant or some other external builder > AND then by Eclipse... over and over? > > If your project is built by Ant, then why build it again with Eclipse? Why is > the Ant output not acceptable? > > If you have added all the source into Eclipse, why run the external builder > (if it does NOT generate more source files), only to build it with Eclipse > afterwards? If it does generate source, then why compile it with the external > builder if Eclipse will build it anyway? Major reason, of course is that Eclipse has an incremental compiler which is incredible fast. However if project is using some kind of code generation (javacc, xdoclet, etc) it may be necessary to run clean when generator metainfo had been significantly changed, so incremental compilation won't work. I case of XDoclet this happens every time when for example EJB interfaces are changed. It is also a good practice to have as few dirs in a project root as possible (this is very well supported by Maven and it is also in a Ant best practices), so ideally you'd have all build output under one directory (eg. target - target/classes, target/test-classes, etc) and common implementation of Ant's clean target is just deleting entire target directory and it is also the fastest way to clean everything. So, this dir can't be used for your proposed workaround to put eclipse classes aside and that forces developers to croud project root directory, which leads to adding this additional dir to cvs/svn exclude, removing it from worksing sets, etc...
>>How about: (such as external builders deleting folders containing .class files) No that is just one of the many things an external builder can do. An external builder could delete the entire contents of the output folder and then compile half of the source files in the project. Every package could be added back to the output folder, but 10 .class files could be missing. We cannot trust our build state to perform an incremental build & must perform a Clean, followed by a full build. As the user, you know when you have run your external builder & when a Refresh + Clean is the appropriate action. Added your comments to bug 60905 that you want Clean to have a Refresh flag attached. Then it will be a single operation.
(In reply to comment #41) > >>How about: (such as external builders deleting folders containing .class > files) > > No that is just one of the many things an external builder can do. And this is the only thing that makes a huge difference when refresh is kicked by the user or explicitly configured in "External tools configuration" in Eclipse. > An external builder could delete the entire contents of the output folder and > then compile half of the source files in the project. Every package could be > added back to the output folder, but 10 .class files could be missing. > > We cannot trust our build state to perform an incremental build & must perform > a Clean, followed by a full build. For that case it would be acceptable to let user to explicitly run "clean" > As the user, you know when you have run your external builder & when a Refresh > + Clean is the appropriate action. > > Added your comments to bug 60905 that you want Clean to have a Refresh flag > attached. Then it will be a single operation. As I already mentioned, this won't help with refresh kicked by Eclipse's "external tools". I have feeling that it could work for me to always run Clean on manual refresh (F5 on any of the project dirs).
I'm not sure why, and I can't reproduce it, but once in a while the manual refresh and Project > Clean after doing an Ant build outside Eclipse doesn't work for me. Just a few days ago I had this problem and ended up having to clean several times both with Eclipse and with Ant and eventually delete and recreate the generated Java files to make them rebuild properly. When this happens it seems to help to turn off automatic builds in Eclipse for a while and not turn them back on until everything is rebuilt and running correctly. It's only happened about a half dozen times in the past year so I can live with it but it's kind of annoying when it happens. Note I don't have the automatic-detect-external refresh option turned on because it tends to be flakey, so by refresh I mean F5.
Ed, what is the source file overlap between the Ant build & Eclipse? Is it all the files or just some? Does the Ant build generate any source files?
In the case I just described, the Ant script generates two .java files from .xml files. Most of the time I'm ok using the Eclipse compiler, but if the .xml files change when I fetch the latest from the Perforce repository, I start to get compile errors (or runtime errors sometimes) because of code that is expecting those changes to be there. That's when I have to go back and use the Ant build, to do all the things Eclipse doesn't know about. Except for a couple of weird cases like this, all the source overlaps between Eclipse and Ant builds, and all the class files overlap too.
Kent- Thanks for the reply... Indeed, my assumptions were incorrect. I guess that's what I get for attempting to solve this problem without looking at the source code! =) After further research (i.e., *looking at the source code*), some interesting findings: Going back above to my "Consider this simple experiment" comments, the sample 'TryIt' project... One does *not* have to use a command-prompt to 'delete things behind Eclipse's back'... One can simply use the Resources Navigator view (much easier to demonstrate!): [*without* my change below] a. If one deletes Main.class from the Resources Navigator, one can note that Main.class will *not* get regenerated. b. interestingly, though, if one deletes the 'bin' folder from the Resources Navigator, the 'bin' directory *does* reappear (the Java builder notices that it has disappeared and re-creates it), but Main.class does not re-appear. This seems inconsistent to me. I would think that Folders(packages) and Files(.class files) should be treated roughly the same in this respect. If they are part of the output and they are deleted behind Eclipse's back (more specifically, behind JDT's back) then Java builder should re-create them. Currently, java builder recreates folders, but not files. So, I had a peek at jdt source and made the following change in "IncrementalImageBuilder.java" (added two (2) lines, prefixed with [randy])... You'll note that by applying this fix and running the above 'Consider this simple experiment' (comment#21), if you delete Main.class via Resources Navigator it'll just magically reappear! (automatically rebuilt by java builder) Kent- were you able to reproduce the experiment in comment#21? If so, do you agree with my findings? [disclaimer: this is my first look at JDT source, so, I'm not necessarily saying that these two lines are ready to checkin! I'm really not familiar w/JDT and can't say that I really know the ramifications of these two lines... however, running simple tests, it *seems* to be a good fix! =)] --- snip (lotsa context) --- public boolean build(SimpleLookupTable deltas) { // initialize builder // walk this project's deltas, find changed source files // walk prereq projects' deltas, find changed class files & add affected source files // use the build state # to skip the deltas for certain prereq projects // ignore changed zip/jar files since they caused a full build // compile the source files & acceptResult() // compare the produced class files against the existing ones on disk // recompile all dependent source files of any type with structural changes or new/removed secondary type // keep a loop counter to abort & perform a full build if (JavaBuilder.DEBUG) System.out.println("INCREMENTAL build"); //$NON-NLS-1$ try { resetCollections(); notifier.subTask(Messages.build_analyzingDeltas); IResourceDelta sourceDelta = (IResourceDelta) deltas.get(javaBuilder.currentProject); if (sourceDelta != null) if (!findSourceFiles(sourceDelta)) return false; [randy] if (sourceDelta != null) [randy] if (!findAffectedSourceFiles(sourceDelta, nameEnvironment.binaryLocations, javaBuilder.currentProject)) return false; notifier.updateProgressDelta(0.10f); Object[] keyTable = deltas.keyTable; Object[] valueTable = deltas.valueTable; for (int i = 0, l = valueTable.length; i < l; i++) { IResourceDelta delta = (IResourceDelta) valueTable[i]; if (delta != null) { IProject p = (IProject) keyTable[i]; ClasspathLocation[] classFoldersAndJars = (ClasspathLocation[]) javaBuilder.binaryLocationsPerProject.get(p); if (classFoldersAndJars != null) if (!findAffectedSourceFiles(delta, classFoldersAndJars, p)) return false; } }
Randy, the only folder that is recreated is the output folder... and that is a side affect of how we look it up. Its pointless exploring hacks to the incremental builder given the numerous different ways projects can be set up + the different combinations of external modifications (adding/changing/deleting folders & .class files). As I have said several times already: When the build state can no longer be trusted, we MUST do a Clean followed by a full build.
Ed, do you have any projects that depend on the project with the generated source files? If so, then I think this is what is happening: Say I have Project A & inside is X.java, which is generated whenever I run an Ant script... it also builds all the source so project A is consistent (if it only builds some, then A is also inconsistent) but Project B is not since it depends on Project A. After a refresh is run, an incremental build finds X.java as a source change & compiles it again inside Eclipse, but no structural change is found because Ant has already replaced the X.class. Project B is told that A was rebuilt but no structural changes were found, so B does not rebuild dependents. Make sense? The refresh launching an incremental build cannot possibly be guaranteed to produce a consistent result in all cases. Instead refresh should launch a full build to be sure that all changes are propagated correctly, and if not, then at least tell builders that a refresh caused the incremental build, so a full build can happen instead.
I don't have any projects that depend on the project with the generated source files.
Kent- >>> [Kent] As I have said several times already: When the build state can no longer be trusted, we MUST do a Clean followed by a full build. agreed 100% My change above improves on this by doing 2 things: (a) *increases* the number of cases whereby the build state *can* be trusted thereby making the Java builder even better, i.e,. there are more cases where the Java Builder can *confidently* perform a fast/correct incremental build rather than default to the user having to know to do a manual clean/rebuild all. (b) eliminates the case whereby Eclipse can 'shoot itself in the foot' by allowing a user to delete a .class file (Main.class) via Resources Navigator and trip up JDT. If JDT builder is notified of a .class file deletion then JDT builder should rebuild it. Currently this is not the case. The above change makes it so. I guess that is the crucial statement: "If JDT builder is notified that a .class file has changed (in this case, deleted) then JDT builder should rebuild it from its corresponding .java source.". More generally, if JDT is notified of any changes to .class files (and Platform/Resources *is* indeed correctly reporting this), then JDT should perform an incremental build of those .java files that correspond to (affected by) those .class files. This whole problem can be easily demonstrated by creating a simple Java project with a single .java source file (just like comment #21) and then zooming off to Resources Navigator and deleting the .class file. Observe that the .class file is *not* regenerated, thus pumping JDT into an inconsistent state. JDT *was* notified of the change in resource(s) (the .class file) but did not incrementally build it when it was perfectly capable of doing so. Kent, way back in comment #5 you requested: >>> Can you please provide us a reproduceable testcase? Here it is! Ed- This is probably the root of your build problems too- ANT regenerates .class files and then when you zoom back to Eclipse and do a Project Refresh, JDT fails to re-re-generate some .class files using its compiler (as you would want it to) thus resulting in a half-baked JDT state with the described behaviour.
Did you miss the part about slowing down incremental builds looking for extremely rare cases, that result in a full build anyway? If you consider deleting individual .class files for fun as something we should waste time looking for, then there is not much point discussing how to improve our integration. Guess how many incremental builds are run by Eclipse users every day, that NEVER need to look for a random deleted .class file. Then assume your Ant script has rebuilt your project & changed every or most of your .class files. What is the point in doing an incremental build that would need to traverse the project looking for each source file to match up to each changed .class file... only to recompile them again? Its slower than a full build!
>>> [Kent] Did you miss the part about slowing down incremental builds looking for extremely rare cases, that result in a full build anyway? I've ran some tests *with* and *without* my change on a Java project (w/5000 .java files, 2 millions lines of source, takes 1m45s to *compile* this proj from scratch in Eclipse, not including scrubbing output folder/copying resources to output folder stages, Linux and Windows, 2.8GHz/1.25GB). Findings: There is no slowdown in *looking* for these extremely rare cases. So there is no impact other to users when these extremely rare cases are not applicable. I've ran the tests and have the numbers to prove it. >>> [Kent] If you consider deleting individual .class files for fun as something we should waste time looking for, then there is not much point discussing how to improve our integration. Ed, Eugene and I are not deleting .class files 'for fun'. We have real world projects that trip up Eclipse JDT because of this problem. And there is not good reason for it. And I don't consider it a waste of time. >>> [Kent] Guess how many incremental builds are run by Eclipse users every day, that NEVER need to look for a random deleted .class file. >>> [Kent] Then assume your Ant script has rebuilt your project & changed every or most of your .class files. What is the point in doing an incremental build that would need to traverse the project looking for each source file to match up to each changed .class file... only to recompile them again? >>> [Kent] Its slower than a full build! What is the point? The point is: CORRECTNESS! Eclipse would automatically default to the *correct* case without user intervention. Currently, Eclipse immediately arrrives at the *incorrect* case, and the user is required to *know* to 'kick' Eclipse by doing Refreshes/Rebuilds until Eclipse stops complaining. What kind of user experience is that? Agreed that this is a corner case where 99% of users will not be here 99% of the time. But when it does happen, it would be nice for Eclipse to still produce the *correct* answer by default, not the *wrong* answer like it currently does! So what if Eclipse just happens to do an incremental build of 5000 .java files in this corner case? At least it is the *correct* and automatic default case! Why not have Eclipse work *correctly* 100% of the time instead of only 99% of the time, failing the rest of the time?? I don't understand, Kent: (a) We have three Eclipse power-users informing of a JDT bug. (b) It has been characterized down to a trivial test case. (c) I have provided the fix. What gives? Why so much resistance?
So the open question is IF the Build Manager added a new type of build (call it Refresh) in addition to Incremental & Full build, how often could JDT do a new Refresh build instead of a Full build? How many different things would we have to look for: 1. Any number of new .class files can appear in an output folder when an external builder compiles source files that are 'unknown' to Eclipse. If the option to not clean the output folder is set, then we want to see these new/changed .class files as if they were in a class folder (ie. recompile any source file that is dependent on that type). How long does it take to realize that these .class files do not come from any known source file? 2. Any number of existing .class files can be recompiled. Can we trust the external builder created these .class files from the same source files & all at once so no additional dependencies need to be propagated between them? Or do we need to recompile them ourselves? We definitely need to consider each change as a structural change & recompile all dependent types, since we did not get to compare each new .class file with the previous one. 3. Any number of existing files (.class files & random resource files) and packages could have been deleted. In this case we need to find all source files that no longer have a .class file and recreate them, as well as propagate these 'new' types as brand new to all dependent types. All the deleted resource files need to copied back to the output folder. And the obvious question to consider is 'when are we saving any time over a full build?' We definitely are not if the entire output folder has been deleted. If every .class file (or a large percentage) has been changed by the external builder, then it depends on whether we trust the output (which we likely will NOT).
Randy, what does comment #48 say? The problem is with refresh invoking an incremental build as if the user had made the changes inside Eclipse. It doesn't work, nor does hacking the incremental builder in simple ways to 'make' it work for toy testcases.
And can you please explain if a 1.5 second Full build is acceptable to you... why are you then complaining that refresh should NOT invoke a Full build instead of an Incremental build?
(In reply to comment #55) > And can you please explain if a 1.5 second Full build is acceptable to you... > why are you then complaining that refresh should NOT invoke a Full build > instead of an Incremental build? Kent, I can't say for everyone, but for me it is ok to have a full build when I explicitly kick refresh in Eclipse, especially on a project root folder. That is what I have to do anyway. If you feel that it is wrong to have that for everybody, why don't make this configurable and let users decide what they prefer...
Just seeing if anybody was reading what I'm writing... Obviously not many Full builds take 1.5 seconds, many take 5-10 minutes. But the question remains: if your primary concern is that the build state is correct, then what is the objection to a full build after refresh? Eugene, can we trust users to know what their external builders are doing? It does not appear that we can. So if build state consistency is the only objective (and I think it should be), then how long it takes to make it happen must be secondary.
Kent- a. I don't think a new build type called 'Refresh' is warranted (never did) That idea originated, I believe, from Eugene in which I never agreed with myself (We ran into the same problem, though disagree on the solution). I maintain that we can solve these problems without confusing/polluting the UI with another obscure build option that most people probably won't intiailly understand anyway. The incremental builder does *not* need to know if changes were discovered via a 'Refresh'. b. "Any number of new .calss files can appear in an output folder..." Absolutely! c. "Any number of existing .class files can be recompiled..." Absolutely! d. "Any number of existing files (.class files & random resource files) could have been deleted..." Absolutely! >>> [Kent] And the obvious question to consider is 'when are we saving any time over a full build?' Kent- this whole problem has absolutely nothing to do with speed, its about *correctness*. Nobody is complaining about speed. Everybody is complaining about *correctness*. >>> [Kent] If every .class file (or a large percentage) has been changed by the external builder, then it depends on whether we trust the output (which we likely will NOT). agreed 100%... Don't trust it. Incrementally rebuild it. And that is what my change does. And it does it correctly. >>> [Kent, from comment #48] The refresh launching an incremental build cannot possibly be guaranteed to produce a consistent result in all cases. Theoretically, Yes it can. Practically, it should strive to do the best job possible, within reason. [well, of course, barring any explicit evil trickery that evil-doers might do just to explitly trip it up, but, all build systems have this possibility... its not an interesting case though. I mean, hey, if somebody *really* wants to come up with some sort of pathalogical trick/hack to intentially trip up the build state, then, yay, congratulations and all more the power to 'em! =)] So, yes it can. And it should, for a *well intentioned* external build system (eg., an external ANT script) that truly wants to rebuild the world in a well-intentioned manner. And even if there are other cases whereby it is impossible, so what? The whole point is that my fix *increases* the number of cases where we *can* result in a *correct* incremental build! It increases correctness! >>> [Kent] The problem is with refresh invoking an incremental build as if the user had made the changes inside Eclipse. Correct. But there should be no difference between making changes within Eclipse or outside of Eclipse. >>> [Kent] It doesn't work, Correct: It does not currently work right now. >>> [Kent] nor does hacking the incremental builder in simple ways to 'make' it work for toy testcases. Apply my two lines of source code to IncrementalImageBuilder.java and it *will work*! It works for toy testcases and it works for my 2million LOC testcase. (a similiar fix would need to be applied to also copy over any deleted (or changed) random resources files as well upon a 'Refresh'). >>> [Kent] And can you please explain if a 1.5 second Full build is acceptable to you... why are you then complaining that refresh should NOT invoke a Full build instead of an Incremental build? I (and others) am complaining that refresh should result in a *correct* incremental build. Performance was never an issue. Correctness is.
(In reply to comment #57) > Just seeing if anybody was reading what I'm writing... > > Obviously not many Full builds take 1.5 seconds, many take 5-10 minutes. > > But the question remains: if your primary concern is that the build state is > correct, then what is the objection to a full build after refresh? The objection is that the user needs to know to do this. They need to know to kick eclipse and do it. Why make the user do something when we don't have to make them do it? The Java builder is perfectly capable of resulting in a *correct* incremental build... Lets do it then! > > Eugene, can we trust users to know what their external builders are doing? > It does not appear that we can. Agreed. Ecipse should only look out for itself. If the build engineer's setup is not correct, then hey, its a bug in the build system that needs to be fixed. But if an external build system is setup to build the same was the Eclipse would build (i.e., high fidelity) then one build system should not trip up the other build system. Both build systems should take care of themselves and not trip up. > So if build state consistency is the only objective (and I think it should > be), then how long it takes to make it happen must be secondary. Yes!
(In reply to comment #57) > Just seeing if anybody was reading what I'm writing... > > Obviously not many Full builds take 1.5 seconds, many take 5-10 minutes. That is why my original suggestion was to run full build only when one or all output directories had been removed. If I reading you correctly your build is recreating those directories, so you know when they had been removed. Don't you? > But the question remains: if your primary concern is that the build state is > correct, then what is the objection to a full build after refresh? > > Eugene, can we trust users to know what their external builders are doing? > It does not appear that we can. I'm not complaining about the cases when external build is modifying classes compiled by Eclipse. I'm looking at the more reasonable case when Eclipse got a cleaned project after running external tool. As I said before, most common case is the "clean" target in ant or maven. > So if build state consistency is the only objective (and I think it should > be), then how long it takes to make it happen must be secondary. Not sure where you are going with this. Anyway, there is an indentified case when you found that target dir had been deleted, so it is a signal that something happens and chances are that you need a full build. Where I am wrong?
Randy, the proposal is that Refresh should be changed to invoke a Refresh (or a normal Full) build AND that this will be faster than any incremental story. No where did I suggest a new UI command. A Refresh build would be invoked by the Build Manager across all builders in the workspace. As a user you could not tell the difference. You would still choose the same Refresh command in the UI. Why argue that speed is not important, that only correctness is, but then say we have to make incremental builds handle abnormal changes? It makes no sense at all. Incremental builds are meant to be very fast. Full builds are not since they start from scratch, but help us recover from any form of change.
Eugene, I think the common case is Ed's scenario and not yours (inherited Ant script generates a couple new source files AND compiles every source file in project, replacing every .class file & not deleting them)... but it doesn't matter. >> If I'm reading you correctly your build is recreating those directories No the incremental builder does NOT recreate missing package folders... but it will ensure that a package folder exists before writing a .class file. That is why 'some' package folders may reappear. The output folder itself is a different story. We will always guarantee that it exists at the start of a build. >> Not sure where you are going with this. My point is that refresh should launch a Full build instead of an Incremental build, and if it did you would never have seen the problems you do now. In your case, because every .class file was deleted, a Full build is the fastest way to rebuild your project. Spending any time figuring out that every source file needs to be compiled is a waste of time.
(In reply to comment #62) > >> If I'm reading you correctly your build is recreating those directories > > No the incremental builder does NOT recreate missing package folders... but it > will ensure that a package folder exists before writing a .class file. That is > why 'some' package folders may reappear. > > The output folder itself is a different story. We will always guarantee that > it exists at the start of a build. Hmm. That is where full build could be forced... But it probably doesn't matter considering later conclusion. > >> Not sure where you are going with this. > My point is that refresh should launch a Full build instead of an Incremental > build, and if it did you would never have seen the problems you do now. Probably you are right. > In your case, because every .class file was deleted, a Full build is the > fastest way to rebuild your project. In my case yes. I think it will be acceptable. > Spending any time figuring out that every > source file needs to be compiled is a waste of time. You can tell this right away when see no output folder. So, do we have complete agreement now? If so can you please reopen this issue?
"You can tell this right away when see no output folder." But in your case, the Maven builder created .class files in the output folder after it deleted everything, did it not? BTW: you can already have the behaviour of Refresh + Clean/Full build by turning off automatic builds, and launching builds when you want (incrementally with Ctrl-B or full builds with Clean...).
Ed, are you still listening? I believe you said the overlap between the Ant script & Eclipse is all of the source files? If so, I would like to track down why a Clean/Full build does not produce a consistent result everytime.
(In reply to comment #64) > "You can tell this right away when see no output folder." > > But in your case, the Maven builder created .class files in the output folder > after it deleted everything, did it not? Not in the project where I had problems. It all started at the one when maven's clean goal was deleting Eclipse's output directory. > BTW: you can already have the behaviour of Refresh + Clean/Full build by > turning off automatic builds, and launching builds when you want > (incrementally with Ctrl-B or full builds with Clean...). Ctrl-B is too much. I have about 50 projects in workspace and I don't really want to rebuild them all when only one of them was refreshed and I sort of expected not to loose authomatic builds when this issue will be resolved...
"Ctrl-B is too much. I have about 50 projects in workspace and I don't really want to rebuild them all when only one of them was refreshed and I sort of expected not to loose authomatic builds when this issue will be resolved..." STOP! ;) Crtl-B is the exact same as an automatic incremental build. It is NOT a Full build. You must select Clean... to fully rebuild every project. When you refresh 1 project with auto build on, all dependent projects are built to see the changes (but since its not a normal incremental build, this currently does not work - but it will if you do a Full build - for now). And when we have a solution, you can turn auto builds back on... but for now, you should turn them off & do Crtl-B for an incremental build when you make changes inside Eclipse, and Clean for a full build after you have run your external builder.
Glad to see this reopened. So, with this new 'refresh build' upon 'refresh' proposal: Are we saying that, if I have 5000 .java files, if I go behind Eclipse's back and use Command Prompt+Notepad.exe to modify one (1) .java file, and then zoom back to Eclipse and do a Project->Refresh, that Eclipse JDT is gonna rebuild the world? If not, then what would happen in this case?!?
We'll likely deal with cases that have no changes in the output folder as a normal incremental build. But you already said that consistency is what matters, not speed so do not expect many cases to be optimized. I won't be surprised if we end up doing full builds most of the time... something you can already do yourself after a refresh by selecting Clean.
Ah, I see... the proposed solution is to consider the output folder(s) only in the decision to to a full rebuild... My question is: Why should we differentiate between changes in source folder(s) vs changes in output folder(s) ?!? It would be arbitrary to do so. So, said another way: with this new 'refresh build' upon 'refresh' proposal: Are we saying that, if I have 5000 .class files, if I go behind Eclipse's back and delete one (1) .class file, and then zoom back to Eclipse and do a Project->Refresh, that Eclipse JDT is gonna rebuild the world? Yes, it would be correct, but would be unneccessarily slow, i.e., we would have done a full rebuild when we didn't have to... Why should JDT arbitrarily default to rebuild the world? It is given the correct delta (one (1) deleted file) from Platform/Resources, and is perfectly capable of doing a fast/correct incremental build! It seems like the proposal is to 'special case' this whole thing... And to make it always slow, even when it does'nt have to be slow. It is not a special case. It is the general case of performing an incremental build based on any changes from anywhere, i.e., from either source folder(s) and/or output folder(s). So, I guess what I'm saying is, why not consider these lines (or some variant)?!? > if (sourceDelta != null) > if (!findAffectedSourceFiles(sourceDelta, nameEnvironment.binaryLocations, javaBuilder.currentProject)) return false; (and do something similar for non-java resource files too) Fast when it can be fast. Always correct.
Randy, your 2 lines do not accomplish anything close to what you think they do. The Incremental builder does not find a source file to recompile when its .class file is deleted or changed. It finds source files in the delta & propagates changes to other .class files IF they are tagged as structurally changed. You really need to make up your mind between correctness vs. speed for builds after a refresh. You've said correctness is the only thing that matters. So... wait until we 'know' that this build follows a refresh, then we'll see what cases actually happen & could be optimized.
Fair enough, Kent- A correct solution first, and optimize later...
1. If we go into a solution watching changes in output folder, we need to be careful of cases where other builders (in Eclipse) are also generating classfiles into it. Currently, this causes no harm as long as they don't overlap; and this is the case where we introduced the flag for not cleaning the output folder to be resilient with other players. 2. If we did consider binary changes, it would require to correlate these to managed source files (i.e. did we build it at some point?) to realize if someone messed with our binaries. 3. Last scenario which worries me a bit: what if somone injected a binary file which is now hiding another one on the classpath. It is not managed by us, but can we ignore it ? (no). So if we care about *any* change, then how do we deal nicely with (1) ?
What about we do look at the binary delta for the output folder, as if it was a class lib folder on the classpath. All changes are considered as structural (since no better info to refine it). If amount of binary structural changes goes beyond some threshold (1/3 source files), then incremental reverts to full build.
In normal situations (normal user not messing with external tools), the output folder binary delta should be empty all the time, so it shouldn't cost to traverse it.
Yes hopefully the 'normal' scenario is we 'own' the output folder & its delta is empty all the time. I'm not convinced a threshold is useful in real scenarios. I expect we will mostly see one of 3 cases after refresh: - no binary changes at all, but a few source changes that we can incrementally build - a few source changes + the entire project was recompiled so every .class file appears changed - or delete all of our generated .class files without recompiling any source files (may add some .class files for 'unmanaged' source files) In either of the last 2 cases, we are wasting time looking past the first changed/deletion of a managed .class file.
*** Bug 107831 has been marked as a duplicate of this bug. ***
Kent, is there are plans to address this issue in 3.1.x or 3.2 streams?
Likely 3.2 Its harder to convince the people in charge that it 'will not' have any side affects. Some developers may have written builders that put .class files into the output folder (or optimized existing .class files), and will not be happy if we're constantly rebuilding their project because of it.
If I may express my oppinion no self-respecting developer would rely on such behavior of another application. If someone is indeed rebuilding or optimizing contents of output directory it should be reasonable to expect such people are first copying that directory to some safe place where no outside forces can mess with it and their scripts can safely work there.
Sorry, I wasn't clear. Its perfectly reasonable that an optimizing builder detects .class files and tweaks them. In this case, if the Java Builder notices the changed .class files & realizes it did not write them -> then a full build is necessary to make the project consistent again. But its a never ending cycle of optimizing & rebuilding which undoes the optimizations.
Released changes to HEAD to detect 'unknown' changes to managed .class files. A full build will be started when such changes are detected. Added BuildpathTests testCorruptBuilder(). If everyone can please try this out in the near future to see if it takes care of your specific problem - let us know if there is a case we missed.
"Verified for 3.2 M5 using build I20060214-0010
So it took less than a week for someone to complain about this change. See bug 129082 Anyone have any ideas why we shouldn't pull this change?
(In reply to comment #84) > So it took less than a week for someone to complain about this change. > > See bug 129082 > > Anyone have any ideas why we shouldn't pull this change? There are several things to look at: -- Why does it happens when automatic build is disabled? -- JDT should provide class post processor extension point to allow tools like TPTP do their own instrumentation. Another option would be to provide an API to notify JDT about such changes. So, such changes would not be considered as foreign/external. -- Generally, TPTP and other tools shouldn't be saving extra resources and backup classes within target folder, but there are not enough flexibility in Eclipse to dynamically change target folders.
Well actually the problem is existing tools that optimize .class files without the builder throwing away their changes by recompiling the source file (as was already mentioned in comment #79 - 81). This change breaks their code. Adding a new hook for them is not backwards compatible.
(In reply to comment #86) > Well actually the problem is existing tools that optimize .class files without > the builder throwing away their changes by recompiling the source file (as was > already mentioned in comment #79 - 81). > > This change breaks their code. > > Adding a new hook for them is not backwards compatible. Ok. If such tools are using Eclipse resource API to modify those files (which they should do, when working from within Eclipse) then it should be possible to detect such changes and notify JDT. It still would make sense to provide a hook, so they could pick it up in new versions, especially those alligned for Calisto. As a last resort you could allow to disable this feature in preferences, so it could be a workaround for those who have to use old tools that can't pickup new hook.
Well I think its going to be a new option which enables this change - not the other way around. Very few users have other tools modifying their output folder at all. They do not need the builder spending time looking for such changes in the first place and the optimizers out there, do not want the builder undoing their changes.
(In reply to comment #88) > Well I think its going to be a new option which enables this change - not the > other way around. That's acceptable > Very few users have other tools modifying their output folder at all. > > They do not need the builder spending time looking for such changes in the > first place and the optimizers out there, do not want the builder undoing their > changes. I am afraid you underestimating the number of such tools. Actually there are lot of tools that could benefit from such changes. The list include but not limited to the following: -- performance monitoring and profilers (including TPTP and other profilers) -- code coverage tools (Cobertura, Agitar, etc) -- AOP tools such as JBoss AOP, AspectWerkz, and few others -- DB and XML mapping tools including Hibernate, JDO, etc -- basically any tools that are doing custom bytecode transformations for some special needs (e.g. testing, problem detection, concurrency, and so on and so forth) Currently most of such tools can't provide incremental postprocessing for the bytecode compiled by JDT and have to do it in a separate pass which slows down the launching...
No they don't. They just have to write a builder that runs after the Java Builder and optimize the .class files that have been incrementally compiled. Without the change from this PR, the Java builder would never replace their optimized .class files.
(In reply to comment #90) > No they don't. > > They just have to write a builder that runs after the Java Builder and optimize > the .class files that have been incrementally compiled. > > Without the change from this PR, the Java builder would never replace their > optimized .class files. Right. I am always forgetting that. So, after all it is TPTP who is doing things wrong way? Though it would save on performance if class file could be transformed in memory before written to the file system...
We've added an option to control when output folders will be searched during incremental builds. Until JDT/UI adds support for it (see bug 133668), you can enable the option by setting: org.eclipse.jdt.core.builder.makeOutputFolderConsistent=enabled in the jdt.core.prefs of your .settings folder. Without the option enabled, the builder's behaviour will be the same as it was before we released the changes for this PR.
Renamed the option to: org.eclipse.jdt.core.builder.recreateModifiedClassFileInOutputFolder=enabled
