Bug 305116 - [index] Improve performance of indexes results tables
Summary: [index] Improve performance of indexes results tables
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 3.6   Edit
Hardware: All All
: P3 normal (vote)
Target Milestone: 3.6 M7   Edit
Assignee: Satyam Kandula CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 289057
Blocks:
  Show dependency tree
 
Reported: 2010-03-09 03:55 EST by Frederic Fusier CLA
Modified: 2010-04-29 03:22 EDT (History)
1 user (show)

See Also:
frederic_fusier: review+


Attachments
Proposed patch (3.77 KB, patch)
2010-04-15 02:38 EDT, Satyam Kandula CLA
no flags Details | Diff
Revised Patch (4.00 KB, patch)
2010-04-19 02:14 EDT, Satyam Kandula CLA
no flags Details | Diff
Revised Patch (3.76 KB, patch)
2010-04-20 02:17 EDT, Satyam Kandula CLA
frederic_fusier: iplog+
frederic_fusier: review+
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Frederic Fusier CLA 2010-03-09 03:55:48 EST
The fix for bug 289057 was intentionally curbed to be as safe as possible for M6. However, there are some other identified places where some more improvements could be done. This bug intends to track the complementary work to do in this area...
Comment 1 Satyam Kandula CLA 2010-04-15 02:38:33 EDT
Created attachment 164936 [details]
Proposed patch

When the results are being added while looking up the category table, they are searched to see if it is already present. However, it is not necessary in some cases. I have tried to take advantage of this conditions in this attached patch

I am working on the performance numbers - I will update once I have them.
Comment 2 Satyam Kandula CLA 2010-04-15 08:54:26 EDT
Here are the numbers running on the windows machine with the eclipse full source and the project including the offending jar.

FullSourceWorkspaceSearchTests#testSearchAllTypeNames takes 29.36s(Patch)vs  109.2s(Head)
FullSourceWorkspaceSearchTests#testNewSearchAllTypeNames takes 3.03s(Patch) vs 10.84s(Head)
FullSourceWorkspaceSearchTests#testSearchAllTypeNameMatches takes 3.31s(Patch) vs 
11.52s(Head)

All the other search tests look more or the same. 

I am still running the tests on the eclipse full source without the offending jar.
Comment 3 Satyam Kandula CLA 2010-04-16 06:35:54 EDT
For FullSourceWorkspaceSearchTests on the eclipse full source, there is a small improvement ~1-2% of performance compared to the Head with this  patch.
Comment 4 Satyam Kandula CLA 2010-04-16 06:36:28 EDT
Frederic, Can you review this patch?
Comment 5 Frederic Fusier CLA 2010-04-16 11:03:24 EDT
(In reply to comment #4)
> Frederic, Can you review this patch?

I do not understand the logic of checkBeforeAdd boolean? In some places it's outside the for loop and on others it's inside. It seems to me to have a big impact on the behavior of the called method addQueryResult(...).

Could you explain your change a little bit more by adding comments on the new lines?

Another cosmetic remark:
	EntryResult result = null;
	if (checkBeforeAdd)		
		result = (EntryResult) results.get(word);

Is usually written as:
	EntryResult result = checkBeforeAdd ? (EntryResult) results.get(word) : null;

I know that some code guidance disapprove the usage of the ternary conditional operator, but we prefer to use it in JDT/Core code when the expressions are not too long to save 2 extra lines of code...

TIA
Comment 6 Satyam Kandula CLA 2010-04-16 13:29:32 EDT
Frederic, 
Thanks for the comments. I will put a better comment and use the ternary conditional. 
The results hashtable will not contain the document name for the first category lookup. Hence, the get() is un-necessary. Subsequent category lookups may have the document names and thus the boolean checkBeforeAdd should be true for the subsequent category lookups. So, this flag is getting set at the end of the category lookup for loop. This is not set if the first category doesn't have any results. Hope this explains.
Comment 7 Satyam Kandula CLA 2010-04-19 02:14:48 EDT
Created attachment 165230 [details]
Revised Patch

Added a better comment and used the terenary expression as per Frederic's comment. Also changed the condition for the initialization of checkBeforeAdd. That could be initialized to false instead of depending on the presence of the memoryIndex.
Comment 8 Frederic Fusier CLA 2010-04-19 05:41:59 EDT
(In reply to comment #7)
> Created an attachment (id=165230) [details]
> Revised Patch
> 
> Added a better comment and used the terenary expression as per Frederic's
> comment. Also changed the condition for the initialization of checkBeforeAdd.
> That could be initialized to false instead of depending on the presence of the
> memoryIndex.

OK, I get it now... Hence, I do not think the local variable checkBeforeAdd is necessary... You can replace it with <code>i > 0</code> and that will do the trick with, I think, a more obvious meaning. Then, I would also change the parameter name as multipleCategory or something like that, which would also means more the reason of the check than the fact you'll do it (which is redundant while reading the code...)
Comment 9 Satyam Kandula CLA 2010-04-20 02:17:30 EDT
Created attachment 165368 [details]
Revised Patch

Changed the variable name to prevResults -- I couldn't think of a better name. Also changed the way the prevResults is being set. Now it is set to (results != null).
Comment 10 Frederic Fusier CLA 2010-04-20 03:37:02 EDT
Comment on attachment 165368 [details]
Revised Patch

Patch looks good to me. The name is not ideal but let's take it until we find another better one :-)
Comment 11 Frederic Fusier CLA 2010-04-20 03:40:33 EDT
(In reply to comment #10)
> (From update of attachment 165368 [details])
> Patch looks good to me. The name is not ideal but let's take it until we find
> another better one :-)

Released for 3.6M7 in HEAD stream.
Comment 12 Frederic Fusier CLA 2010-04-28 09:36:34 EDT
Reopened as performance results of I20100425-2000 build do not show any improvement for this test... :-(
Comment 13 Olivier Thomann CLA 2010-04-28 12:57:42 EDT
Satyam,

Please try to fix the problem with the released patch. If we get a fix for tomorrow, we might release it for M7.
Comment 14 Olivier Thomann CLA 2010-04-28 14:27:54 EDT
Satyam, please make sure get Frédéric to check the patch on his local performance machine before it is released.
Thanks.
Comment 15 Satyam Kandula CLA 2010-04-29 00:19:07 EDT
(In reply to comment #12)
> Reopened as performance results of I20100425-2000 build do not show any
> improvement for this test... :-(
Frederic, 
This bug was actually meant to primarily improve the testcase provided in bug 289057. Further to that, I could see performance improvement of testNewSearchAllTypeNames compared to M6.   
Yes, it doesn't improve the performance of  FullSourceWorkspaceTypeHierarchyTests#testPerfAllTypes(), which is anyways being tracked by bug 306170. 
Am I missing anything?
Comment 16 Frederic Fusier CLA 2010-04-29 03:21:17 EDT
(In reply to comment #15)
> (In reply to comment #12)
> > Reopened as performance results of I20100425-2000 build do not show any
> > improvement for this test... :-(
> Frederic, 
> This bug was actually meant to primarily improve the testcase provided in bug
> 289057. Further to that, I could see performance improvement of
> testNewSearchAllTypeNames compared to M6.   
> Yes, it doesn't improve the performance of 
> FullSourceWorkspaceTypeHierarchyTests#testPerfAllTypes(), which is anyways
> being tracked by bug 306170. 
> Am I missing anything?

No, you're right... I'm definitely sorry, it seems I mixed these two bugs :-S
Comment 17 Frederic Fusier CLA 2010-04-29 03:22:15 EDT
Hence, verified for 3.6M7 looking at I20100425-2000 performance results