Community
Participate
Working Groups
Build Identifier: I20100608-0911 java version "1.7.0-ea" Java(TM) SE Runtime Environment (build 1.7.0-ea-b76) Java HotSpot(TM) Client VM (build 17.0-b05, mixed mode, sharing) eclipse.buildId=I20100608-0911 java.version=1.7.0-ea java.vendor=Sun Microsystems Inc. BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=nl_NL Framework arguments: -product org.eclipse.epp.package.java.product Command-line arguments: -os win32 -ws win32 -arch x86 -product org.eclipse.epp.package.java.product -clean Error Thu Jun 24 09:56:10 CEST 2010 An internal error occurred during: "Install download1". java.lang.IllegalArgumentException: Comparison method violates its general contract! at java.util.ComparableTimSort.mergeHi(ComparableTimSort.java:835) at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:453) at java.util.ComparableTimSort.mergeForceCollapse(ComparableTimSort.java:392) at java.util.ComparableTimSort.sort(ComparableTimSort.java:191) at java.util.ComparableTimSort.sort(ComparableTimSort.java:146) at java.util.Arrays.sort(Arrays.java:1160) at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.hasValidMirror(MirrorSelector.java:317) at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.downloadArtifact(SimpleArtifactRepository.java:517) at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getArtifact(SimpleArtifactRepository.java:576) at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.getArtifact(MirrorRequest.java:248) at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.transferSingle(MirrorRequest.java:223) at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.transfer(MirrorRequest.java:179) at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.perform(MirrorRequest.java:115) at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getArtifact(SimpleArtifactRepository.java:565) at org.eclipse.equinox.internal.p2.artifact.repository.simple.DownloadJob.run(DownloadJob.java:63) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54) Reproducible: Always Steps to Reproduce: 1. Had a JDK7 early access installed and is default Java 2. Downloaded and unzipped Eclipse IDE for Java Developers (Helios) 3. tried to install Subclipse plugin from marketplace or install new software 4. exception in details section happens
I have the same problem on final Helios release using Linux 64bit with Java 1.7.0-b97
Could you please try with a stable build of the 1.6 JRE? I suspect that this is a VM bug.
(In reply to comment #2) > Could you please try with a stable build of the 1.6 JRE? I suspect that this is > a VM bug. It works with 6u20 for me, the point is that JDK7 has a new implementation of Array.sort() (ComparableTimSort) - so either that is bugged (but I don't expect that) or whatever MirrorSelector tries to sort doesn't implement compareTo correctly. I don't see org.eclipse.equinox.internal.* in the web SVN client so I don't know what it is sorting on to see if that implements compareTo correctly or to make a test case for a JDK bug.
We are sorting MirrorInfo objects here, I've been looking at the compareTo method and it looks ok to me. What I can imagine is a synchronization problem because MirrorInfo objects are returned to the outside world. So when thread 1 has selected a mirror and is using it, and thread 2 starts selecting a mirror, if thread 1 changes the MirrorInfo object between comparisons happening on the thread 2 sort then the sort can become inconsistent leading to the IllegalArgumentException. I think what we need is for the MirrorInfo increment/decrementFailureCount and setBytesPerSecond methods to be synchronized on the MirrorSelector and not just the MirrorInfo object.
Sorry I actually found it, I just needed to navigate to the right bundle first. It could be a threading problem if a resetFailure timer fires during a sort as that could change the order. The sort is synchronized on MirrorSelector blocking updates from MirrorSelector#reportResult() but the timer decrease failure only synchronizes on the MirrorInfo itself. The JDK7 Array.sort() might be more sensitive to that. The first failure timer fires after 30 seconds and I got failures in my error log: !ENTRY org.eclipse.equinox.p2.repository 2 0 2010-06-24 09:49:30.363 !MESSAGE Connection to http://eclipse.ialto.org/releases/helios/201006230900/aggregate/plugins/org.eclipse.net4j.jms.api_3.0.0.v20100614-1655.jar.pack.gz failed on Unable to parse header: </a></div>. Retry attempt 0 started !ENTRY org.eclipse.equinox.p2.repository 2 0 2010-06-24 09:50:14.208 !MESSAGE Connection to http://eclipse.ialto.org/releases/helios/201006230900/aggregate/plugins/org.eclipse.net4j.jms.api_3.0.0.v20100614-1655.jar.pack.gz failed on Unable to parse header: </a></div>. Retry attempt 0 started !ENTRY org.eclipse.core.jobs 4 2 2010-06-24 09:50:42.693 !MESSAGE An internal error occurred during: "Install download0". If you want I can attach my complete error log. I see Andrew just beat me to this possibility ;-).
Created attachment 172622 [details] proposal Here is a possible patch. We need to be careful of deadlocks between the MirrorSelector and MirrorInfo.
As a datapoint: I've been using Eclipse 3.6RC3 and earlier milestones with OpenJDK 7 for quite a while, and only started getting this error now with the Helios release. Is it possible that some change between RC3 and release made this issue more visible?
Nothing changed in this code for quite a while. It may just be that there were no mirror available.
Wow that code has become complicated since I last saw it! Another option would be for the MirrorInfo instances to be immutable and be copied on modification: class MirrorInfo { MirrorInfo incrementFailureCount() { MirrorInfo info = clone(); info.failureCount++; return info; } ... etc ... } That would simplify the code by removing a mass of synchronization and make the sorting stable. Given the complexity of this code and the deadlock risk with all that concurrency and synchronization, I would suggest just fixing in 3.7.
(In reply to comment #2) > Could you please try with a stable build of the 1.6 JRE? I suspect that this is > a VM bug. This solved my problem, installed the jdk 6 on Linux
*** Bug 327165 has been marked as a duplicate of this bug. ***
Let's not this patch rot.
Happened to me also for 1.7.0-ea-b130 (Developer preview Milestone 12)
*** Bug 346838 has been marked as a duplicate of this bug. ***
*** Bug 347606 has been marked as a duplicate of this bug. ***
Still happening with 1.7.0-ea-b144 . I expect this to become more of an issue as Eclipse misteriously fails installing updates under Java 7.
I agree. We have been delaying it hoping that it was a bug in the JRE but we will definitely look into it so we have a fix for 3.7.1.
*** Bug 347391 has been marked as a duplicate of this bug. ***
Still a valid bug. Using C:\Users\xxx>java -version java version "1.7.0-ea" Java(TM) SE Runtime Environment (build 1.7.0-ea-b145) Java HotSpot(TM) 64-Bit Server VM (build 21.0-b15, mixed mode) Plugin sources checked out from CVS. console error log: !ENTRY org.eclipse.core.jobs 4 2 2011-06-16 01:10:38.522 !MESSAGE An internal error occurred during: "Install download0". !STACK 0 java.lang.IllegalArgumentException: Comparison method violates its general contract! at java.util.ComparableTimSort.mergeHi(ComparableTimSort.java:835) at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:453) at java.util.ComparableTimSort.mergeCollapse(ComparableTimSort.java:376) at java.util.ComparableTimSort.sort(ComparableTimSort.java:182) at java.util.ComparableTimSort.sort(ComparableTimSort.java:146) at java.util.Arrays.sort(Arrays.java:472) at org.eclipse.equinox.internal.p2.artifact.repository.MirrorSelector.hasValidMirror(MirrorSelector.java:319) at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.downloadArtifact(SimpleArtifactRepository.java:577) at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getArtifact(SimpleArtifactRepository.java:680) at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.getArtifact(MirrorRequest.java:260) at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.transferSingle(MirrorRequest.java:235) at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.transfer(MirrorRequest.java:191) at org.eclipse.equinox.internal.p2.artifact.repository.MirrorRequest.perform(MirrorRequest.java:122) at org.eclipse.equinox.internal.p2.artifact.repository.simple.SimpleArtifactRepository.getArtifact(SimpleArtifactRepository.java:664) at org.eclipse.equinox.internal.p2.artifact.repository.simple.DownloadJob.run(DownloadJob.java:63) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
I've digged deeper into this bug and finally found the root cause. It's not an multi-threading issue nor JRE7's new TimSort algorithm. It's the Comparator implementation introduced with Bug 297805. The proposal as of 2010-06-24 is 'not good' and will definitely not fix the problem. Additionally the unit tests throw compile time errors. The current implementation violates the Comparator.compareTo contract. Which says transitive and so on ... (see Java docs of JRE1.4.2+). Because the current Comparator implementation is difficult to predict, I've implemented a new one, based on solid mathematics ;-) This is an short outline of the source. I will commit patch soon (next 1-2 days). ---------------------- This Comparator uses a vector space classification algorithm and implements some kind of Rocchio Classification. In theory, when sorting multiple mirrors, we want to know which one is the best in terms of its attributes 'bytesPerSecons', 'failureCount' and 'initialRank'. This Comparator needs three initial query attributes, which mean to search for the fastest mirror, with lowest failure and nearest to the initial rank. By calculating the vector space distance from query to Object 1 and query to Object 2, we implicitly know, which mirror is better. There are two weight factors, which directly influences sorting results. They are computed by using a one real live mirror list of eclipse.org and tweak as long as the results look good as a list ;-) See test case fore more details. Mathematical basics used in MirrorInfoComparator#compare(MirrorInfo, MirrorInfo) Given two vectors Q and T, where Q is query and t is Target and Q_a (or T_a) are attributes of each vector, this is the formula, which is computed to each MirrorInfo object. -> -> q * t (dot product) sim(q,t) = --------------------------- / || -> || || -> || \ | || q || * || t || | (euclidean lengths) \ / N --- \ > Q * T / ai ai --- i = 1 sim(q,t) = ----------------------------------------- / ___________ ___________ \ | | N | N | | | --- | --- | | _ | \ 2 _ | \ 2 | | \ | > Q * \ | > T | | \| / ai \| / ai | | | --- | --- | \ | i = 1 | i = 1 / ---------------------- This will result in a sorted mirror list like the attached similarity_example_results.txt
Created attachment 198111 [details] Example of a sorted list of mirrors.
To get to the heart of this bug, selecting of mirrors will not work under JRE7. Possible workarounds: * use JRE6 OR * when using JRE7, theres a small rarely documented feature set system property java.util.Arrays.useLegacyMergeSort=true This should use old implementation and should not bring up the bug (I haven't tested, but you can search for the property ;-) )
Created attachment 198147 [details] fixed comparator implementation incl unit tests Here's the patch, using the vector space classification as described above. When using the new unit test case against the old Comparable implementation, you will get the IllegalArgumentException, as described in this bug.
Martin, thank you very much for your contribution. However there is a significant problem with your attachment. Your patch includes OpenJDK source code, but by attaching a patch here it means the code is being contributed under the Eclipse Public License, which is not compatible. Can you please do the following: - Attach a new patch containing *only* the fix - If it is possible to write a unit test that doesn't include JDK source code, please attach that separately. Note we will probably be running our JUnit tests over the next year using a Java SE 7 VM, so I would expect you could just invoke the class library sort and the test would correctly fail when run in java 7 without your fix.
The content of attachment 198147 [details] has been deleted by Eclipse Webmaster <webmaster@eclipse.org> who provided the following reason: Requested by John A. Patch contains code that isn't ip clean The token used to delete this attachment was generated at 2011-06-17 10:50:14 EDT.
Created attachment 198271 [details] fixed comparator implementation incl unit tests (without GPL component) Sorry for the inconvenience. I wasn't sure about license incompatibility. That's why, I fully declared the source in my (old and now removed) patch. I've attached the patch again, without the GPL components, now. Because of the fact, that the TimSort algorithm from JRE7 was only used to detect wrong implementations of Comparator interface, the test now only relies on the 'hope', that with 1000 randomly shuffled tries, a wrong implementation of Comparator interface will be discovered. (when transitive contract is not fulfilled, it may result in wrong sort order.) Regards Martin
Created attachment 198308 [details] General comparator test Instead of hoping, you could just test the comparator contract for your test data :). See attached test code (doesn't check if one compare throws an exception then the other does as well). I don't have Eclipse code checked out to add it to your patch and test it.
Thanks for this nice improvement Martin! John I will let you handle the review and commit.
Yes, I will be getting this into 3.7.1 soon. Thanks Martin!
*** Bug 344561 has been marked as a duplicate of this bug. ***
I have released the fix to HEAD. I will back-port to 3.7.1 stream after a successful integration build. The contributed test didn't have the property of demonstrating the problem before applying the patch. I.e., the test relies on the new implementation. However I hacked up your test to run on the MirrorSelector implementation in 3.7, and I found that it did indeed get the IllegalArgumentException periodically when run with Java 7 (in my case a recent IBM Java 7 build). Interestingly it often didn't fail until 50-100 iterations which might explain why not everyone is getting this error. Anyway, this gave me confidence that the test case is suitable for catching future regressions and I have included it.
*** Bug 352049 has been marked as a duplicate of this bug. ***
>I have released the fix to HEAD. I will back-port to 3.7.1 stream after a >successful integration build. John, can you put the fix into the maintenance stream now? I'd like to have this in an M-build before Java 7 goes GA.
Backport into 3.7.x is covered by Bug 352089. Closing.
(In reply to comment #22) > Possible workarounds: > * use JRE6 > OR > * when using JRE7, theres a small rarely documented feature > set system property java.util.Arrays.useLegacyMergeSort=true > This should use old implementation and should not bring up the bug > (I haven't tested, but you can search for the property ;-) ) Does the system property really help? I've seen two cases where only switching back to 1.6 fixed the problem. Using the property only made the IAE go away but the install was still stuck.
(In reply to comment #35) > (In reply to comment #22) > > * when using JRE7, theres a small rarely documented feature > > set system property java.util.Arrays.useLegacyMergeSort=true > > This should use old implementation and should not bring up the bug > > (I haven't tested, but you can search for the property ;-) ) > > Does the system property really help? I've seen two cases where only switching > back to 1.6 fixed the problem. Using the property only made the IAE go away but > the install was still stuck. Since enabling this property in eclipse.ini there were no problems installing additional plugins/features. (Eclipse 3.7 release, jdk 1.7.0-b142+)
*** Bug 353396 has been marked as a duplicate of this bug. ***
*** Bug 354693 has been marked as a duplicate of this bug. ***
*** Bug 355317 has been marked as a duplicate of this bug. ***
*** Bug 356053 has been marked as a duplicate of this bug. ***
*** Bug 360057 has been marked as a duplicate of this bug. ***
I don't want to be mean but this bug is far from being closed. eclipse.ini was just an workaround. Do you expect that normal users should dig the internet to find workaround for such bugs? Probably this is why most people hate eclipse: the user experience.
Hi, this bug fix was already delivered with Eclipse 3.7.1 (a.k.a. SR1). So I expect, that normal user are able to download this release or in case of they already have Eclipse 3.7.0 installed, they can just simply press on 'Check for updates'. That easy. Best regards Martin (In reply to comment #42) > I don't want to be mean but this bug is far from being closed. eclipse.ini was > just an workaround. > > Do you expect that normal users should dig the internet to find workaround for > such bugs? > > Probably this is why most people hate eclipse: the user experience.
(In reply to comment #43) Yes in order to reproduce the bug in 3.7.0 you go to "Check For Updates" and try to get 3.7.1, which subsequently fails because of these errors. It is that easy! That is exactly what I was doing before I found this. > Hi, > > this bug fix was already delivered with Eclipse 3.7.1 (a.k.a. SR1). > So I expect, that normal user are able to download this release > or in case of they already have Eclipse 3.7.0 installed, they can just > simply press on 'Check for updates'. > That easy. >
To clarify comment #22 for those on JRE7 who can't upgrade to 3.7.1 because of this bug: Add the following line to your eclipse.ini: -Djava.util.Arrays.useLegacyMergeSort=true Then run the update to 3.7.1 from inside Eclipse again, it should succeed now. Afterwards, you should be able to drop that line from eclipse.ini again.
(In reply to comment #45) > To clarify comment #22 for those on JRE7 who can't upgrade to 3.7.1 because of > this bug: > > Add the following line to your eclipse.ini: > -Djava.util.Arrays.useLegacyMergeSort=true > Then run the update to 3.7.1 from inside Eclipse again, it should succeed now. > Afterwards, you should be able to drop that line from eclipse.ini again. I still have the issue, using OpenJDK7 eclipse.buildId=I20110613-1736 java.version=1.7.0_147-icedtea java.vendor=Oracle Corporation BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=en_US Command-line arguments: -os linux -ws gtk -arch x86_64
(In reply to comment #46) > I still have the issue, using OpenJDK7 > > eclipse.buildId=I20110613-1736 > java.version=1.7.0_147-icedtea > java.vendor=Oracle Corporation > BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=en_US > Command-line arguments: -os linux -ws gtk -arch x86_64 I20110613 sounds like you're still on 3.7.0. You'll have to upgrade to fix the problem, e.g. using the trick above.
(In reply to comment #47) > I20110613 sounds like you're still on 3.7.0. You'll have to upgrade to fix the > problem, e.g. using the trick above. I did use the trick above and it didn't work for me. Maybe because I'm using OpenJDK instead of Sun? I've gotten around it by switching Java to 1.6, updating, now switching back to 1.7. In fact the only reason I'm using 1.7 is to workaround a different Eclipse/JVM bug where it randomly crashes on 1.6 which may or may not have been fixed with the update.
*** Bug 363477 has been marked as a duplicate of this bug. ***
Bug 364735 looks very closely related - but is still happening on Eclipse 3.7.1, and not related to Equinox.
Thx for the reference Mark
*** Bug 379986 has been marked as a duplicate of this bug. ***
I can confirm that this bug is present in Juno, Build id: 20120614-1722, running JRE 7u5. Pasting the suggested workaround, -Djava.util.Arrays.useLegacyMergeSort=true, into eclipse.ini allows any plugin installation process to reach 45%, but no further. Can I change the fix status back?
(In reply to comment #53) > I can confirm that this bug is present in Juno, Build id: 20120614-1722, > running JRE 7u5. > > Pasting the suggested workaround, > -Djava.util.Arrays.useLegacyMergeSort=true, into eclipse.ini allows any > plugin installation process to reach 45%, but no further. > > Can I change the fix status back? Please file a separate bug. This bug is about the broken comparator which got fixed.
*** Bug 388466 has been marked as a duplicate of this bug. ***
