Bug 573479 - DefaultProjectClasspathEntry.expandProject revisits already visited libraries
Summary: DefaultProjectClasspathEntry.expandProject revisits already visited libraries
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Debug (show other bugs)
Version: 4.19   Edit
Hardware: PC All
: P3 normal (vote)
Target Milestone: 4.21 M2   Edit
Assignee: Andreas Huber CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
Depends on:
Blocks:
 
Reported: 2021-05-11 07:24 EDT by Andreas Huber CLA
Modified: 2021-08-03 10:35 EDT (History)
2 users (show)

See Also:


Attachments
example workspace to reproduce the performance issue (239.87 KB, application/x-zip-compressed)
2021-05-14 08:31 EDT, Andreas Huber CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Andreas Huber CLA 2021-05-11 07:24:33 EDT
DefaultProjectClasspathEntry.expandProject() performs a depth first iteration over the projects classpath. It tries to prune already visited sub trees by checking if the entry is already in the expandedPath list. One code path however transforms the entry before adding it to the list. In that case the pruning does not work.

I have a workspace with 150 deeply nested projects and lots of dependencies that are defined in each of the projects. Startup of a simple unit test can take up to 70 seconds. Most of that time is spend in DefaultProjectClasspathEntry.expandProject().
Comment 1 Eclipse Genie CLA 2021-05-11 07:26:13 EDT
New Gerrit change created: https://git.eclipse.org/r/c/jdt/eclipse.jdt.debug/+/180466
Comment 2 Andreas Huber CLA 2021-05-14 08:31:15 EDT
Created attachment 286382 [details]
example workspace to reproduce the performance issue

The attached exampleWorkspace.zip can be used to reproduce the issue. 

1. Create a new Eclipse workspace in an empty directory.
2. Download the zip and extract it into a sub-folder of the workspace.
3. Import the projects into Eclipse. 
    a) Execute `gradlew cleanEclipse eclipse`. This will generate the necessary project files.
    b) Import projects: File / Import / General / Existing Projects into Workspace
       Don't forget to check *search for nested projects*.
4. You should have 102 projects (root, main and sub-project-1 to sub-project-100). All projects have the same dependencies.
5. Run the Junit test *ExampleTest* in project *main*.
6. The start-/end-time logged in the header of the console view gives you the full test duration including the launch time.

On my machine the fix improves the launch time from 45s to 5s.
Comment 3 Andreas Huber CLA 2021-05-14 08:40:07 EDT
(In reply to Andreas Huber from comment #2)
> On my machine the fix improves the launch time from 45s to 5s.

The measurements were done on Windows. On Linux we see much better performance, because JavaProject.canonicalizedPath() is a noop on case-sensitive file systems, which is likely on Linux systems.
Comment 4 Andrey Loskutov CLA 2021-07-25 11:27:33 EDT
(In reply to Andreas Huber from comment #2)
> Created attachment 286382 [details]
> example workspace to reproduce the performance issue

Thanks, that helps.

> On my machine the fix improves the launch time from 45s to 5s.

I have not the fastest Windows notebook but I don't see 45 seconds, just ~5 seconds by default on 4.21 master without patch. With both patches from this bug and bug 567595 applied the launch time improves to 1 second, so still good improvement. 

Thanks Andreas, will merge as soon as gerrit tests finish.
Comment 6 Andreas Huber CLA 2021-08-03 10:35:53 EDT
Verified together with 567595.
The performance improved as expected.
I tested TestNG and Junit tests.
I did not come across any class path related issues.