Community
Participate
Working Groups
I have a couple of large files in my code base, e.g. this one with over 20k lines (mostly Javadoc): https://raw.githubusercontent.com/jOOQ/jOOQ/master/jOOQ/src/main/java/org/jooq/impl/DSL.java Whenever I save the file, I get a delay of roughly 3s - 5s until the UI is ready again. In a profiling session, I haven't noticed any specific CPU hot spot, but clearly, memory consumption of two things is excessive (for details, see screenshot 01-allocation-pressure.png): Class Average Object Size(bytes) Total Object Size(bytes) TLABs Average TLAB Size(bytes) Total TLAB Size(bytes) Pressure(%) char[] 776.18 7'150'944 9'213 185'530.806 1'709'295'312 27.64 org.eclipse.jdt.internal.compiler.lookup.TypeBound[] 2'853.35 6'865'160 2'406 507'134.318 1'220'165'168 19.73 That looks like a lot of memory for only a few save actions. Some analysis on char[]: ------------------------ Most of this happens somewhere downstream of Binding.computeUniqueKey(), and inside of that, most is in org.eclipse.jdt.internal.compiler.lookup.MethodBinding.genericSignature() Note that jOOQ has types like Record1<T1>, Record2<T1, T2>, ... Record22<T1, T2, ..., T22>, so computing the entire generic signature as a string does seem to cause overhead. Details in screenshot 02-stack-trace.png. Some remarks: 1) A lot of these methods use StringBuffer instead of StringBuilder. While it is possible that the two perform equally well in benchmarks (http://alblue.bandlem.com/2016/04/jmh-stringbuffer-stringbuilder.html), there are a lot of effects (including method size) where perhaps escape analysis won't work reliably anymore and synchronisation cannot be omitted. Since these buffers are used method-locally only, it may be a really low hanging fruit to just replace them by StringBuilder. 2) A lot of character data is copied from StringBuffer to char[] to String back to StringBuffer in order to calculate these Binding.computeUniqueKey() strings. I suspect that the algorithm could be drastically accelerated, if a shared StringBuffer (or rather StringBuilder) would be passed around these methods instead of manually allocating new char[] all the time. 3) Are string-based keys really necessary? Perhaps, an entirely different key format that costs much less to compute would be optimal here. Some analysis on TypeBound[]: ----------------------------- The memory consumption here is also excessive, and it all happens inside of org.eclipse.jdt.internal.compiler.lookup.BoundSet() This class is created quite a few times through the copy() method, and each time it allocates an array unincorporatedBounds of size 1024. Is this really necessary?
Created attachment 268712 [details] Allocation pressure
Created attachment 268713 [details] Stack trace for char[] allocations
Created attachment 268714 [details] Stack trace for TypeBound[] allocations
While compilation is still slow, I cannot reproduce the allocations that I've reported at the time when using 4.7.1a This means that this particular problem has either been fixed (and wasn't so significant), or it was only a symptom of a bigger problem, which exposes itself elsewhere now. I'm closing this to avoid the noise of having it open, reporting a new issues instead.
I was wrong, sorry. I can still reproduce the TypeBound[] allocations originating from the BoundSet constructor. It's just that there are now other memory-intensive allocations happening, which seem to further worsen compilation speed.
Just linking the dots regarding one observation: (In reply to Lukas Eder from comment #0) > Some analysis on TypeBound[]: > ----------------------------- > > The memory consumption here is also excessive, and it all happens inside of > org.eclipse.jdt.internal.compiler.lookup.BoundSet() > > This class is created quite a few times through the copy() method, and each > time it allocates an array unincorporatedBounds of size 1024. Is this really > necessary? This was introduced via Bug 442245. Could mean we are trading cpu vs. memory. I'm not aware of any measurement justifying the 1024.
@Lukas, can you give simple instructions for me to setup an environment in which i can compile your DSL.java from comment 0 (ideally without maven or such)? Alternatively, could you repeat your measurements from an I-build after we reduced the size of the suspicious array?
Sure, here's a copy of the Github repository with only the relevant code inside, and without Maven: https://github.com/lukaseder/jOOQ-Eclipse-Bug-517703 I'm using JDK 1.8.0_141 to build this. I can reproduce similar signatures in JMC's memory allocation views, even if the UI feedback is a bit faster with the reduced amount of code (and without Maven). Let me know if you need anything else.
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant. -- The automated Eclipse Genie.
