Community
Participate
Working Groups
Perf tests results for N20071015-0010 and I20071016-0800 builds show regression between -0.5% and -8.1% for following tests: - FullSourceWorkspaceSearchTests#testIndexingOneProject(): -8.1%<t<-1.3% - FullSourceWorkspaceSearchTests#testSearchField(): -5.4%<t<-0.4% - FullSourceWorkspaceSearchTests#testSearchType(): -4.9%<t<-0.9% - FullSourceWorkspaceTypeHierarchyTests#testPerfAllTypes(): -4.8%<t<-2.7% I open only one bug as they all happened since the same build and so, I assume that these regressions should have the same origin... The standard errors on these tests are OK, so the regressions seem to be real hence need to be investigated. Duration for these tests are between 1 and 3s approximately, so that makes the regression not really noticeable for users => set minor severity
It sounds not to be specific to Index or Search as some other JDT/Core local perf tests show a small regression (AST, Build and Type Hierarchy) => modify the summary accordingly... Unfortunately, the regression value is not enough significant to clearly understand what could be the origin of this problem. I'll wait a little bit to see if this regression is still observable in following builds...
I have run performance tests using JDT/Core v_819 on top of I20071010-1200 build and I cannot reproduce the regression => so it sounds to be due to a JDT/Core required plug-in. I continue to investigate to have more information on this, but as the concerned tests read lot of resources, I suspect that the regression may come from Platform/Resource plug-in...
Looking at difference between v20071008 and v20071015 versions of project org.eclipse.core.resources, I saw a change which may explain this small regression: in File.getCharset(boolean) method now calls a new method checkSynchronized() (also done in getContentDescription()). I confirm that the impacted JDT/Core tests go through this new method call while reading the resources (parse to build the AST, parse to compile the java file or parse to locate the matches during a search request...). Move to Platform/Resources for further investigation
Make some quick measures on one of our impacted tests confirms the time difference of 4% when the call to the checkSynchronized() method in File.getCharset(boolean) is done or not.
The issue is caused by the fix for the bug 186984.
I will address the issue during M4. Thank you for patience.
Back to JDT/Core land after having opened bug 209617. We'll try to see if some optimization in JDT/Core may mask this Platform/Resources perf regression...
I've opened bug 209617 to track work in Platform/Resources land (see comment 3)
(In reply to comment #8) > I've opened bug 209617 to track work in Platform/Resources land (see comment 3) > Obviously should read bug 209167...
Looking at the performance results on the build page, there have been several builds since the resources change that had no regression for the search tests. Only FullSourceWorkspaceTypeHierarchyTests seems to have a consistent regression since the resources change was released.
(In reply to comment #10) > Looking at the performance results on the build page, there have been several > builds since the resources change that had no regression for the search tests. > Only FullSourceWorkspaceTypeHierarchyTests seems to have a consistent > regression since the resources change was released. > I completely disagree with this. Look at JDT/Core results for last build I20071107-1300 on Win XP Sun 1.4.2_10 (3 GHz 2 GB) and you'll see that the regression is definitely noticeable for several tests of FullSourceWorkspaceSearchTests: testIndexingOneProject(), testSearchField(), testSearchBinaryMethod() and testSearchType(). You can see also a regression for FullSourceWorkspaceASTTests#testDomAstCreationProjectJLS3(). I agree this regression looks more noticeable for Windows box than for Linux ones, but it definitely exists and is not due to a change in JDT/Core (see comment 4 explaining why).
Close as WONTFIX as there won't be more performance tests on 3.4...
