Bug 578359 - JDT Indexing Became Much Slower In 4.21 (2021-09)
Summary: JDT Indexing Became Much Slower In 4.21 (2021-09)
Status: NEW
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 4.23   Edit
Hardware: PC Windows 10
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: JDT-Core-Inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords:
Depends on:
Blocks:
 
Reported: 2022-01-24 12:27 EST by Andy Lee CLA
Modified: 2024-03-25 12:15 EDT (History)
2 users (show)

See Also:


Attachments
Sampler snapshots from VisualVM to illustrate performance difference (1.15 MB, application/x-zip-compressed)
2022-01-24 12:27 EST, Andy Lee CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Andy Lee CLA 2022-01-24 12:27:49 EST
Created attachment 287886 [details]
Sampler snapshots from VisualVM to illustrate performance difference

JDT indexing performance seems to have gotten significantly slower in 4.21 for my workspace configuration:

Time to fully reindex in 4.20: 9 minutes
Time to fully reindex in 4.21: 42 minutes

These times were measured using these steps:
 - Close Eclipse
 - Delete the .metadata/.plugins/org.eclipse.jdt.core folder from the workspace
 - Launch Eclipse
 - Open the "Open Type" dialog and measure how long it takes for indexing to finish and results to appear

Based on profiling the Eclipse process, the primary culprit seems to be a dramatic increase in File.length calls being performed by the indexing thread via ClasspathJar.findPackageSet (File.length sadly being a notoriously slow call in Windows / NTFS environments).

I've attached sampling snapshots from VisualVM 2.0.3 that illustrate the details of the indexing performance in 2021-06 (4.20) versus 2021-12 (4.22).
Comment 1 Jeff Johnston CLA 2022-03-18 15:51:51 EDT
Hi Stephan,  could this be related to your change for Bug 574603?

namely ClassPathJar.decorateWithExternalAnnotations() which calls scanContent()

If yes, would it be possible to add a boolean parameter to findPackageSet() to not check the file length and just verify the time stamp of cached entries so decorateWithExternalAnnotations() could call scanContent() with said boolean value and pass it along to alleviate the extra work?
Comment 2 Stephan Herrmann CLA 2022-03-18 17:21:23 EDT
Hi Jeff, did you see the linked bug 579205, in particular bug 579205 comment 11 ff.?

Still your proposed fine-tuning could be interesting for those who enable that new option. Thanks. 

Do you agree to make this a duplicate of bug 579205?
Comment 3 Eclipse Genie CLA 2024-03-25 12:15:05 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.