Bug 570896 - Improve AliasManager performance
Summary: Improve AliasManager performance
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Resources (show other bugs)
Version: 4.19   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: 4.20 RC1   Edit
Assignee: Alex Blewitt CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
Depends on:
Blocks: 571430
  Show dependency tree
 
Reported: 2021-02-04 01:53 EST by Jörg Kubitz CLA
Modified: 2021-04-12 04:07 EDT (History)
2 users (show)

See Also:

Attachments
Java call stack of most memory allocations during compile (2.35 KB, text/plain)
2021-02-04 01:53 EST, Jörg Kubitz CLA 		no flags Details
Screenshot of the flightrecording result (140.02 KB, image/png)
2021-02-12 03:49 EST, Jörg Kubitz CLA 		no flags Details
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Jörg Kubitz CLA 2021-02-04 01:53:51 EST 
Created attachment 285440 [details]
Java call stack of most memory allocations during compile

Compiling a large eclipse workspace takes a lot of memory allocations (and thus GC and time). By far most of memory allocation in my case is caused by
AliasManager.getComparator() which creates new instances of org.eclipse.core.runtime.Path during comparison. 
Since the comparison inside the Treemap is called at least log(n) times for all n resources the problem grows more then linear with the project size.

I would like to contribute an improvement which would avoid memory allocation during compare at all.

However it seems yet unclear (and wrong) to me why the AliasManager comparator does not use URI.compare() for comparing.
There are subtile difference between URI.compare() and AliasManager:
AliasManager/Path compares fully case sensitive while URI.compare treats
scheme and host part case insensitive.
Comment 1 Eclipse Genie CLA 2021-02-10 03:51:53 EST 
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.resources/+/176070
Comment 2 Eclipse Genie CLA 2021-02-10 08:39:49 EST 
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.resources/+/176081
Comment 4 Jörg Kubitz CLA 2021-02-12 03:49:34 EST 
Created attachment 285531 [details]
Screenshot of the flightrecording result

Added Screenshot of the flightrecording showing many GB of Memory allocated by compare.
Comment 5 Eclipse Genie CLA 2021-03-23 05:39:52 EDT 
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.resources/+/178253
Comment 8 Lars Vogel CLA 2021-03-24 06:15:41 EDT 
Thanks a lot Jörg for the fix and Alex for spending the time to help Jörg bringing this in.

Jörg, do you see an improvement with the next successful I-build for your use case in real life?
Comment 9 Jörg Kubitz CLA 2021-03-24 06:19:47 EDT 
I cannot test ibuilds on our real product. But i testet a cherry pick on our real product before hand. The problem was gone.
Comment 10 Eclipse Genie CLA 2021-03-25 12:52:03 EDT 
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.resources/+/178384
Comment 12 Eclipse Genie CLA 2021-03-28 13:11:49 EDT 
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.resources/+/178468
Comment 14 Eclipse Genie CLA 2021-03-28 15:00:30 EDT 
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.resources/+/178475
Comment 15 Eclipse Genie CLA 2021-03-28 18:11:45 EDT 
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.resources/+/178478
Comment 16 Eclipse Genie CLA 2021-03-29 00:06:02 EDT 
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.resources/+/178479