Bug 550729 - Improve performance of ModuleContainer and ModulClassLoader
Summary: Improve performance of ModuleContainer and ModulClassLoader
Status: RESOLVED INVALID
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: Framework (show other bugs)
Version: unspecified   Edit
Hardware: PC All
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: equinox.framework-inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
Depends on:
Blocks: 566485
  Show dependency tree
 
Reported: 2019-09-04 03:15 EDT by Lars Vogel CLA
Modified: 2020-11-30 05:49 EST (History)
5 users (show)

See Also:


Attachments
Yourkit trace (176.67 KB, image/png)
2019-09-04 17:04 EDT, Lars Vogel CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Lars Vogel CLA 2019-09-04 03:15:16 EDT
From https://bugs.eclipse.org/bugs/show_bug.cgi?id=550579#c20

----
If ModuleContainer resolve kicks in, it takes 12% of the time and here are its hotspots.   Many thousands of filters are parsed but a great many are duplicates, i.e., 90% are duplicates.  That seems amenable to using cached instances.  Also, Character.isWhitespace is tested a lot (by the parser's skipWhitespace); a specialized method that is fast for ASCII whitespace would seem simple, safe, and helpful.

Most of the cost of Filter.compare is creating versions.  It seems to me that org.osgi.framework.Version.valueOf(String) might use cached instances as well. After all, how many different versions are there likely to be?
----

See also https://bugs.eclipse.org/bugs/attachment.cgi?id=279765
Comment 1 Lars Vogel CLA 2019-09-04 03:16:33 EDT
BJ and Tom, please have a look.
Comment 2 Thomas Watson CLA 2019-09-04 08:44:35 EDT
Lars, much of the recent performance issues being investigated lead me to believe the profiling for performance is focusing on cleaned configuration start only.  While performance for that scenario shouldn't be abismal it is much more important to optimize the cached configuration.  Resolution should not happen in general from a cached restart of the framework.
Comment 3 Lars Vogel CLA 2019-09-04 08:49:44 EDT
(In reply to Thomas Watson from comment #2)
> Lars, much of the recent performance issues being investigated lead me to
> believe the profiling for performance is focusing on cleaned configuration
> start only.  While performance for that scenario shouldn't be abismal it is
> much more important to optimize the cached configuration.  Resolution should
> not happen in general from a cached restart of the framework.

Definitely, I see you also have registered for a Yourkit licence, would be great if you could trace the startup and report issues for the cached configuration.
Comment 4 Lars Vogel CLA 2019-09-04 09:13:11 EDT
(In reply to Lars Vogel from comment #3)

> you could trace the startup and report issues for the cached configuration.

You may want to wait a bit with that. We already have a bunch of patching pending, and it might be more efficient to wait until some of them have been merge. To need to waste energy in finding the same issues.

Would be great if you could investigate this issue in ModuleContainer in the meantime, even though it might be more important for an non-cache runtime.
Comment 5 Ed Merks CLA 2019-09-04 11:04:07 EDT
Yes, I'd mentioned to Lars that from past experience, measuring a self-hosted startup has overhead (resolution) that doesn't normally happen after a first start has cached the module database, i.e., until after the next update...
Comment 6 Lars Vogel CLA 2019-09-04 11:28:49 EDT
(In reply to Ed Merks from comment #5)
> Yes, I'd mentioned to Lars that from past experience, measuring a
> self-hosted startup has overhead (resolution) that doesn't normally happen
> after a first start has cached the module database, i.e., until after the
> next update...

I think removing "Delete Configuration..." In the launch config will do the trick.
Comment 7 Lars Vogel CLA 2019-09-04 17:04:46 EDT
Created attachment 279776 [details]
Yourkit trace
Comment 8 Lars Vogel CLA 2019-09-04 17:12:31 EDT
(In reply to Lars Vogel from comment #7)
> Created attachment 279776 [details]
> Yourkit trace


I a cached configuration ModulClassLoader#defineClass becomes the bottleneck. Maybe this can be also improved?
Comment 9 Ed Merks CLA 2019-09-05 04:12:25 EDT
(In reply to Lars Vogel from comment #8)
> (In reply to Lars Vogel from comment #7)
> > Created attachment 279776 [details]
> > Yourkit trace
> 
> 
> I a cached configuration ModulClassLoader#defineClass becomes the
> bottleneck. Maybe this can be also improved?

It think this mostly comes down to the cost of calling java.lang.ClassLoader.defineClass(String, byte[], int, int, ProtectionDomain) which one would have to assume has been optimized to death.
Comment 10 Lars Vogel CLA 2019-09-05 08:17:12 EDT
Adding Paul, who did some very impression performance work recently.

Paul, do you see options to improve the performance of ModulClassLoader (and ModuleContainer)?
Comment 11 Thomas Watson CLA 2019-09-05 09:20:57 EDT
(In reply to Lars Vogel from comment #8)
> (In reply to Lars Vogel from comment #7)
> > Created attachment 279776 [details]
> > Yourkit trace
> 
> 
> I a cached configuration ModulClassLoader#defineClass becomes the
> bottleneck. Maybe this can be also improved?

Use https://adoptopenjdk.net/ choose J9 and enable shared classes support for eclipse:

https://blog.openj9.org/2018/10/30/openj9-shared-classes-with-eclipse-equinox-4-9/
Comment 12 Thomas Watson CLA 2019-09-05 09:46:06 EDT
(In reply to Lars Vogel from comment #3)
> (In reply to Thomas Watson from comment #2)
> > Lars, much of the recent performance issues being investigated lead me to
> > believe the profiling for performance is focusing on cleaned configuration
> > start only.  While performance for that scenario shouldn't be abismal it is
> > much more important to optimize the cached configuration.  Resolution should
> > not happen in general from a cached restart of the framework.
> 
> Definitely, I see you also have registered for a Yourkit licence, would be
> great if you could trace the startup and report issues for the cached
> configuration.

I've been doing that as I have time and have found some small improvements, but nothing that gives drastic speed up, recent examples:

Bug 549406
Bug 546606
Comment 13 Lars Vogel CLA 2019-09-05 09:48:57 EDT
(In reply to Thomas Watson from comment #12)on.
> 
> I've been doing that as I have time and have found some small improvements,
> but nothing that gives drastic speed up, recent examples:
> 
> Bug 549406
> Bug 546606

Thanks, Tom
Comment 14 Lars Vogel CLA 2019-09-11 04:24:53 EDT
Cache startup time:
Time to load bundles: 5
Starting application: 1393
Application started in : 6666ms


Startup time after installing EGit

Time to load bundles: 7
Starting application: 3675
Application started in : 9692ms
Comment 15 Lars Vogel CLA 2020-11-30 05:49:03 EST
Please reopen if you plan to work on this issue.