Description Jon Seymour 2010-10-24 20:15:23 EDT

The following describes a scenario which causes AspectJ to hold on arbitrarily large amounts of memory that is not recoverable until the application is restarted.

The circumstances arise due to the interplay of soft and weak references, and the fact that the AspectJ Dump class is typically only referenced during application startup.

Assumptions:

* EAR containing Spring EJBs
* Spring EJBs use AspectJ point cuts
* Periodic cycling on Spring EJBs such that all references to the Spring EJBs are removed
* Spring contexts pinned by soft references.
* a period of low-load, followed by a period of high load

Suppose that the container has a large amount of memory and periodically executes an EJB that typically does not do much. This will cause an EJB to be created, and then a Spring container to be created. The type analysis performed by AspectJ on Spring's behalf will cause a ReflectionWorld to be added to the WeakHashMap maintained by the AspectJ weaver.Dump class. If the EJB then becomes idle, it is then subject to removal. This causes the Spring container to be destroyed.

However, if the Spring container is reachable via a Soft reference, then the weak references may not be immediately eligible for reclamation. 

Several cycles of the EJB can cause an arbitrary number of ReflectionWorld's to be added to Dump's WeakHashMap. If the pressure on the JVM caused by these cycles is low enough, the Soft reference pinning the objects will not get cleared.

Eventually though, the JVM will be forced to reclaim the soft references. This will make the weak references in the WeakHashMap weakly reachable, resulting in the reference queue associated receiving all the ReflectionWorld's that were previously softly reachable and are now only weakly reachable.

The thing is, if this reference queue only gets cleared if the WeakHashMap is used again. But the nature of the WeakHashMap used by the Dump object is that it may only ever get touched during application initialisation. If the server is suddenly subjected to high load, the EJBs will never be released so there is never another re-initialization of the EJBs which means that the Spring container does not get re-initialized which means that the WeakHashMap is not subject to the access that would cause it to purge its reference queue.

The net result is that long idle periods that do not put soft references under pressure, followed by a bursty period that does, can dump large numbers of effectively unreclaimable objects onto the reference queue used by the WeakHashMap used by the Dump class. The objects are effectively unreclaimable because the WeakHashMap used by the Dump class may never be referenced again, and as such its reference queue will not be polled.

This suggests that the WeakHashMap used by the Dump class needs either an active monitoring thread to clear it or regular polling as the consequence of some other use of AspectJ that does get performed regularly through the lifetime of an AspectJ container.