Community
Participate
Working Groups
Build Identifier: 3.5.2 As a project for a university course, I have written a test suite generator algorithm with an oracle to test Eclipse Compare's three-way diff algorithm. I generated and ran 59136 tests resulting in the following summary: Verdict Number Percentage -------------------------------- Fail 6110 10,33% Inconclusive 2160 3,65% Success 50866 86,02% Total 59136 100,00% I wrote a *very* slow diff3 implementatation which resulted in the following: Verdict Number percentage ------------------------------ Fail 0 0,00% Inconclusive 1458 2,47% Success 57678 97,53% Total 59136 100,00% So, the failures found can be fixed. It is interesting to note that in build version 2010-05-31 the successes improved by 1.1% over 3.5.2: Verdict Number Percentage Improvement over 3.5.2 --------------------------------------------------- Fail 4146 7,01% 33,54% Inconclusive 3484 5,89% 78,71% Success 51506 87,10% 1,10% Total 59136 100,00% See the report (report.pdf) for all the details. See the four attached java-files for the test suite and oracle implementations. Reproducible: Always Steps to Reproduce: 1. Use Eclipse for RCP/Plug-in Developers Galileo-SR2 2. Checkout version tag R3_5_2 of the following packages - org.eclipse.compare/plugins/org.eclipse.compare - org.eclipse.compare/plugins/org.eclipse.compare.core - org.eclipse.compare.tests - org.eclipse.core.tests.harness - org.eclipse.core.tests.resources - org.eclipse.test.performance - org.eclipse.test.performance.(your OS-identifier) 3. Install the test-suite found in the four attached java files into package org.eclipse.compare.test 4. Run the BBTestSuite.testRandomTests or BBTestSuite.testSystematically as a junit test 5. Read of the results in report.txt placed in the project root of org.eclipse.compare.tests
Created attachment 177932 [details] Detailed report on the test suite and oracle
Created attachment 177933 [details] The four java files of the test suite and oracle implementation
What kind of files were you comparing? If those files were large our diffing algorithm probably used capping the get the results faster (bug 292831 is about one of such cases). We're currently working on providing a way to disable the capping in the algorithm (bug 307280).
(In reply to comment #3) > What kind of files were you comparing? If those files were large our diffing > algorithm probably used capping the get the results faster (bug 292831 is about > one of such cases). We're currently working on providing a way to disable the > capping in the algorithm (bug 307280). The files used was very small, less than 20 lines.
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.
