Community
Participate
Working Groups
When a class is unloaded from the JVM the debugger receives a class unloaded event and gets the signature of the unloaded class. It then calls VirtualMachineImpl.removeKnownRefType() to try and remove the unloaded class from the cache of known reference types. To do this it tries to get a reference type ID using the JDWP classesBySignature command, but since the class has been unloaded it will never successfully get the ref type ID for the unloaded class, so will either do nothing to the cache, or potentially remove the wrong class. This causes a problem on Oracle JDK & OpenJDK 8 with heavy use of invokeDynamic because the LambdaForms used to implement MethodHandles and invokeDynamic can generate a large number of classes, and these may get garbage collected in large numbers. Issuing large numbers of classesBySignature calls (1 per GCed class) is very expensive since the JVM has to iterate over every loaded class to compare the signatures, and can cause applications to freeze for up to 5 minutes.
This bug can cause a huge problem when debugging an app using invokeDynamic.
(In reply to Duncan MacGregor from comment #0) Can you attach a sample project where you are facing this problem ?
Created attachment 248986 [details] Example project which shows the problem. I've attached an example project that should demonstrate the problem. Requires the ASM 5 library to link against. Generates a large number of classes and then garbage collects to make them unload. Under normal operation each load of 1000 classes will take approximately the same time, with occasional slight slowdown for garbage collection. Under the debugger long pauses (of up to 2 minutes on my machine) can be observed. To observe significant pauses it is critical that there are both a large number of classes still in memory & that a large number have been unloaded.
Currently, we cache the value by ID, and to get the Id for signature we are using the JDWP classesBySignature command. Should we use one more cache to store boolean for a signature. If we have stored ids for the signature we can flag that. If all the ids for a signature is unloaded, we need to flag that again. So before invoking classesBySignature we can check in this new cache and then if required invoke classesBySignature.
I think you need a cache mapping signatures to (sets of) IDs. when a you discover a signature has been unloaded you then check how many IDs have that signature. If you have IDs against a signature then you need to check the status of each and remove ones which are no longer valid. I would be very cautious about optimising the case where you only have one ID against a signature as that could be an artefact of having just removed a whole bunch of unloaded classes. I think classesBySignature should never be used in this area as it will not return any IDs for the classes that have been removed, so you'll never remove them from your cache.
I've written a quick proof of concept fix for this, but it has the potential to build up a useless map of signatures to expired reference type IDs, so it's not exactly production quality. I'm going to try turning it into something more robust by introducing a ReferenceTypeCache class which can manage all the internal cleanup neatly, and return all cached reference types for a signature. I don't think any API changes are needed on VirtualMachineImpl as fCachedReftypes is private and isn't returned by any methods. There is a potential performance hit as signatures will have to be fetched for types when they are cached (if they haven't been set already).
Created attachment 249237 [details] Proposed patch. Okay, this seems to work for me, in my test case there is still a short pause when classes are unloaded but it's much shorter than before. I couldn't find any unit tests that were really focused on this area, and I'm not sure how you guys implement tests for internals
We face same issue in our application. Sarika, do you plan to follow up on this? If not, we (me & Simeon) should take this.
(In reply to Andrey Loskutov from comment #8) > We face same issue in our application. > > Sarika, do you plan to follow up on this? > If not, we (me & Simeon) should take this. I don't plan to work on this right now.
New Gerrit change created: https://git.eclipse.org/r/149760
I've taken the patch suggested by Duncan MacGregor: https://git.eclipse.org/r/#/c/149760/ I've not examined what the TypeCache does, which is most of the change. But with the case in which we observe classes which can't get unloaded by the JVM due to JDT, the problem is gone with the patch.
@Simeon Do you plan to complete this for 4.14? Else you can move it to 4.15.
If you still plan to fix it in 4.14 RC1, please change the target milestone.
Removed the target, please put the target when planned.
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant. -- The automated Eclipse Genie.
