Community
Participate
Working Groups
Today if the compiler suspects that the classpath has changed it rebuilds the project. We have a build machine where we do headless, ant based eclipse builds. We then distribute the up-to-date workspace to our developers. Unfortunately even through we are all using the same JRE it is not always installed in the same location and the builder thinks that the classpath has changed (even though it really hasn't). This triggers a rebuild (which lasts hours) on the developers machine. We would like the builder to do a better job in eliminating these unnecessary compiles. Perhaps the CRC of each of the jars that was used in the build could be taken, and stored somewhere, and then later used by the builder to catch some of these superficial classpath changes.
From the Java builder standpoint any JAR/ZIP change is hard to react to. The fact it has changed is hard to figure, and computing a CRC would be quite expensive in general. No plan for improving this for 2.1. One thing you could try is to put the JRE inside the workspace directly ? Given it seems huge anyway, this may not be so bad... How big is your workspace that it takes hours to build ?
I understand it may not be possible for 2.1, please consider it for post 2.1. The CRC could be cached and only calculated when the file timestamp changes, which would be quite rare. The workspace is big, it was 318MB, 46K resources, it is probably bigger now.
We would certainly not close it, since we want to reconsider this situation post 2.1. Did you try using an internal JRE instead ? Have a JRE project as part of the build workspace, and use this one on your classpath instead ? Note that it would be nice if the JRE support did allow to use internal JARs. Darin - can you please comment on why internal JARs are prohibited currently ?
Tagging as RC2 for investigation. No actual commitment we would improve this at this stage.
Deferring
reopen
To start with, there is no CRC already stored in a jar file. It will be very expensive (time + space) to read in an entire jar file to produce a CRC value. There are 2 ways to change a jar file: 1. A jar file is replaced with a different version in place. JavaCore already notices that the timestamp of an external jar file is different & provides a delta for the file, thus causing a full build. Core resources does the same for internal jar files. When the jar file is replaced with a file that has the same timestamp (very unlikely case), the change is not detected and would be very expensive to notice compared to how many times it actually happens. 2. A jar file is moved to another location. Currently causes a full build. We could keep track of timestamps and ignore moved jar files that have the same timestamp, but not all filesystems keep the same timestamp when moving files. We also cannot verify that the file is identical since the old one likely no longer exists. --------------- A CRC is the only full-proof approach to catch all cases, but is prohibitively expensive when compared to the frequency of this problem. The obvious solution to installing a common build is to include the JRE with it (it can easily be included with the drop & located at the root of the drive, it does not need to be nested inside).
I think we could be clever about this, so that we didn't have to calculate the CRC very often. While there is no CRC in the JAR itself, we could keep track of these codes outside of the jar file (just like we do for timestamps) As a reference point, I calculated the CRC on rt.jar. On my 1.2GHz laptop it took 1.6s on a cold machine and 240ms warm (i.e. on the second attempt). Considering the build that we would be saving takes hours, you could calculate a lot of CRC's in that time. I haven't read the license agreement recently but I thought there were a lot of restrictions about how you can and can not distribute a JRE.
Its not only just rt.jar... it is every jar file needed by your workspace. We cannot single out a single jar file (besides in jdk 1.4, rt.jar is replaced with several jar files). We also run on top of several different VMs that name their class library jars with different names. So for a decent size workspace with 250 jar files, we should add several minutes to the build to handle a case that has a much simpler solution? Why can you not include the JRE with your drop so that its installed in the same place on everyone's machine?
Full drive paths are used by several tasks (other than the Java builder) to store information about the workspace. So to avoid re-executing other tasks, you already need to instruct your developers where to install the workspace... otherwise tasks such as indexing all the source & jar files will be redone. The obvious simple solution for avoiding a full rebuild is to also instruct them where to install the JRE. I suggest we close this PR. Agreed?
Closing. There just isn't an efficient way to handle this case without impacting every other build.
