Community
Participate
Working Groups
The nesting and subsequent use of a third party Apache Velocity library (org.eclipse.m2e.archetype.common/velocity-1.5.jar) can cause Velocity initialization to fail, resulting in the loss of functionality in m2e and other eclipse.org features (depending on the order of use), specifically the Dali Java Persistence Tools project. When this initialization error occurs, one of the primary Dali features (Entity generation from tables) will not work. A workspace restart is then required to work around the error. A short summary of the problem is that whichever Velocity library is used first (in the IDE) will become the only Velocity library that will work due to the fact that Velocity eagerly checks for classloader discrepancies during initialization. More details can be found in bug 394616 comment #8. This issue can also break the New Maven Project from archetype functionality in m2e. If Dali entity generation is performed before Maven project creation in a given workspace session, velocity initialization will fail and as a result the Maven project creation will fail. Stack trace will be attached. The Simultaneous Release requirements specifically state, "Any third-party plug-ins that are common between projects must be consumed via Orbit". And, "The Simultaneous Release must not have duplicate third-party libraries" [1] Part of this rule is to prevent these types of classloader problems. [1] http://wiki.eclipse.org/SimRel/Simultaneous_Release_Requirements#Re-use_and_share_common_third_party_code_.28partially_tested.29
Created attachment 224696 [details] m2e project creation failure - stack trace
There are no immediate plans to work on this issue unless a quality patch is provided.
The importance of this issue has become more evident in the past week. Multiple eclipse adopters are now hitting this issue in testing as Maven tooling and other velocity dependent functionality are used in the IDE. This is a big issue for the Eclipse Java EE Package as m2e is being included into this package for the first time in Kepler. This issue could greatly affect the overall user experience for Java EE package users who use the Maven tooling, as it or other Velocity dependent feature won't work when used in the same session.
After looking at the source it is not obvious to me as to how I would go about contributing a patch for this issue. Given that this is a cor sim-rel requirement issue, I think it is reasonable to expect that the m2e project would be willing/able to fix this problem.
As I explained in bug 394616, we do not know the root cause of the problem, i.e. multiple versions of velocity are expected to coexist in separate bundles. Thread context classloader "leak" is the only explanation I currently have, but I could not find the source of the leak. Changing m2e velocity packaging may mask the root cause of the problem, but I am very uncomfortable making any changes until and unless I understand what exactly going on.
(In reply to comment #5) > As I explained in bug 394616, we do not know the root cause of the problem, > i.e. multiple versions of velocity are expected to coexist in separate > bundles. Thread context classloader "leak" is the only explanation I > currently have, but I could not find the source of the leak. I agree that there does appear to be some sort of classloader leak here, but I don't think it is related to m2e mismanaging the TCCL. I don't see any problems in the MavenModelManager, as the context classloader is appropriately managed by preserving the old classloader and setting it back in a finally clause. After some additional research, it would appear that we are running into an inherent limitation in the ContextFinder in Equinox (and perhaps more accurately stated as a limitation of the Java VM/TCCL and OSGi). The classloader that initially loads the Velocity class is being cached (since Velocity uses Class.forName(...) to load the class), and subsequently reused outside of the scope of the m2e bundle that loaded it. As a result, when Dali or any other Velocity user comes along and tries to initialize Velocity, it will fail, as the ContextFinder will use the initially cached classloader to load the class, resulting in the failure of Velocity to initialize. See the following links for more information: http://blog.bjhargrave.com/2007/07/contextfinder-in-eclipse-is-broken.html https://bugs.eclipse.org/bugs/show_bug.cgi?id=127963 https://bugs.eclipse.org/bugs/show_bug.cgi?id=127963#c22 (this comment is key) >Changing m2e > velocity packaging may mask the root cause of the problem, but I am very > uncomfortable making any changes until and unless I understand what exactly > going on. It would appear that the root of the problem is that TCCL + Equinox/ContextFinder + Class.forName(...) = leaks. Given the environmental limitations, it would seem that the least intrusive solution for this problem would be for all Velocity consumers to use the Orbit provided Velocity bundle to prevent the leak at the source. If everyone doesn't play by this rule, there will be issues.
I am still not fully decided, but I will tentatively consider making this change for Kepler SR1. Something does not add up. First, m2e sets TCCL before calling into Velocity, so Equinox ContextFinder should not be involved. Second, even if Equinox ContextFinder gets used somehow, I believe this should result in Exception during RuntimeInstance.initializeResourceManager(). Any thoughts on why the code throws NPE? Also, current Orbit version of velocity is not a singleton as far as I can tell... but I guess we can pretend 1.5 is the only version of Velocity available and noone will ever try use a different bundle version.
(In reply to comment #7) > I am still not fully decided, but I will tentatively consider making this > change for Kepler SR1. > > Something does not add up. First, m2e sets TCCL before calling into > Velocity, so Equinox ContextFinder should not be involved. After looking at the stack attached to this bug, I did some more digging, and I think perhaps the leak is that the injection mechanism used to inject the VelocityComponent (which must build and initialize the VelocityEngine, which constructs the RuntimeInstance) is not using the Maven TCCL that is used to "merge" the templates. As a result, the ContextFinder may end up being used to test the classpath integrity. >Second, even if > Equinox ContextFinder gets used somehow, I believe this should result in > Exception during RuntimeInstance.initializeResourceManager(). Any thoughts > on why the code throws NPE? It's a bit mystifying. Perhaps you are hitting the same initialization problems noted in the Dali bug, but that this 'Initializable' exception is caught and managed and not reported via the Eclipse Error Log? The only thing I can think of is that the RuntimeInstance tried to build the ResourceManager and failed, yet the VelocityComponent is still injected. Subsequent attempts to use the velocity engine fail with an NPE because the ResourceManager was never initialized. Although it seems there is a check to ensure that the ResourceManager was initialized, and that flag must be set to true in this case. > > Also, current Orbit version of velocity is not a singleton as far as I can > tell... but I guess we can pretend 1.5 is the only version of Velocity > available and noone will ever try use a different bundle version. It would actually work fine for Dali if you use a different bundle version (assuming the velocity bundle name is the same), as we do not specify an upper boundary on our requires bundle. As a result, if 1.7.1 was available and used by others, we would work just fine. This does assume loose version range specifications by velocity users. In summary, I think I may have discovered a potential classloader "leak" in the Maven code, although I cannot confirm this as I didn't find the injector source that builds the VelocityComponent, but am guessing this part of the puzzle. However, I am very confident that regardless of this issue, m2e should be using the velocity Orbit bundle which would solve this issue for Kepler, and you can deal with any lingering issues in the Maven/m2e code base afterwards. Otherwise, this m2e bug is going to cause everyone else (including m2e) to fail or find a way to work around this issue with some other classloader hackery. The easy, safer, and effective fix would be to simply move to using the Orbit bundle in m2e.
To be clear, I do not plan to make this change for kepler (sr0) but will consider it for kepler sr1. The proposed change is not easy or safe. First, it will split maven archetyper runtime classpath between two bundle classloaders. This should work, but at very least we will have to give users time to verify their archetypes still work. Second, it will make m2e development environment setup more cumbersome, at least initially.
Btw, I could not convince PDE to notice orbit velocity bundle, neither in plugins/ nor in dropins/ folders... will try some other time.
Just to update those who may not be following the blocked bug...was able to work around this issue by insulating our calls to the VelocityEngine by setting the TCCL for these calls with a known-to-be-good classloader (avoiding the use of the ContextFinder).
I've changed m2e to use Orbit velocity bundle but this completely broke Archetype integration, I get tons of exceptions like below when I open m2e import project wizard. Given that Dali implemented a workaround for the problem with Velocity already, I do not plan any more work on this issue unless somebody provides a quality patch. DEBUG: Sisu - Initialize: org.codehaus.plexus.velocity.DefaultVelocityComponent@61631478 [INFO] Setting property: classpath.resource.loader.class => 'org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader'. [INFO] Setting property: velocimacro.messages.on => 'false'. [INFO] Setting property: resource.loader => 'classpath'. [INFO] Setting property: resource.manager.logwhenfound => 'false'. [ERROR] Problem instantiating the template loader. Look at your properties file and make sure the name of the template loader is correct. Here is the error: [ERROR] java.lang.ClassNotFoundException: org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader cannot be found by org.apache.velocity_1.5.0.v200905192330 at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:501) at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:421) at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:412) at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:107) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:188) at org.apache.velocity.util.ClassUtils.getClass(ClassUtils.java:76) at org.apache.velocity.util.ClassUtils.getNewInstance(ClassUtils.java:95) at org.apache.velocity.runtime.resource.loader.ResourceLoaderFactory.getLoader(ResourceLoaderFactory.java:48) at org.apache.velocity.runtime.resource.ResourceManagerImpl.initialize(ResourceManagerImpl.java:134) at org.apache.velocity.runtime.RuntimeInstance.initializeResourceManager(RuntimeInstance.java:594) at org.apache.velocity.runtime.RuntimeInstance.init(RuntimeInstance.java:241) at org.apache.velocity.app.VelocityEngine.init(VelocityEngine.java:116) at org.codehaus.plexus.velocity.DefaultVelocityComponent.initialize(DefaultVelocityComponent.java:79) at org.sonatype.guice.plexus.lifecycles.PlexusLifecycleManager.initialize(PlexusLifecycleManager.java:338) at org.sonatype.guice.plexus.lifecycles.PlexusLifecycleManager.manageLifecycle(PlexusLifecycleManager.java:296) at org.sonatype.guice.plexus.lifecycles.PlexusLifecycleManager.onProvision(PlexusLifecycleManager.java:148) at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision(ProvisionListenerStackCallback.java:108) at com.google.inject.internal.ProvisionListenerStackCallback.provision(ProvisionListenerStackCallback.java:55) at com.google.inject.internal.ProviderInternalFactory.circularGet(ProviderInternalFactory.java:68) at com.google.inject.internal.InternalFactoryToInitializableAdapter.get(InternalFactoryToInitializableAdapter.java:45) at com.google.inject.internal.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:46) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1018) at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40) at com.google.inject.Scopes$1$1.get(Scopes.java:59) at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:41) at com.google.inject.internal.InjectorImpl$3$1.call(InjectorImpl.java:965) at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1011) at com.google.inject.internal.InjectorImpl$3.get(InjectorImpl.java:961) at org.sonatype.guice.bean.locators.LazyBeanEntry.getValue(LazyBeanEntry.java:83) at org.sonatype.guice.plexus.locators.LazyPlexusBean.getValue(LazyPlexusBean.java:49) at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:253) at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:245) at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:239) at org.eclipse.m2e.core.internal.MavenPluginActivator.lookup(MavenPluginActivator.java:357) at org.eclipse.m2e.core.internal.MavenPluginActivator.getArchetype(MavenPluginActivator.java:393) at org.eclipse.m2e.core.internal.archetype.ArchetypeCatalogFactory.getArchetyper(ArchetypeCatalogFactory.java:71) at org.eclipse.m2e.core.internal.archetype.ArchetypeCatalogFactory$InternalCatalogFactory.getArchetypeCatalog(ArchetypeCatalogFactory.java:108) at org.eclipse.m2e.core.ui.internal.wizards.MavenProjectWizardArchetypePage.getAllArchetypes(MavenProjectWizardArchetypePage.java:541) at org.eclipse.m2e.core.ui.internal.wizards.MavenProjectWizardArchetypePage.getArchetypesForCatalog(MavenProjectWizardArchetypePage.java:516) at org.eclipse.m2e.core.ui.internal.wizards.MavenProjectWizardArchetypePage$15.run(MavenProjectWizardArchetypePage.java:557) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
Forgot to mention, I pushed m2e packaging changes to https://git.eclipse.org/c/m2e/m2e-core.git/log/?h=396554_orbitVelocityBundle
I'm getting the same initialization error in Luna 4.4 with the current version of all plugins as of October 2014. It still appears that whichever plugin loads velocity first, from anywhere, wins, even when m2e consumes from Orbit. I use the velocity editor plugins as well as Dali, m2e, Guice, and JBoss Tools, all of which do something with velocity. So if for example a velocity editor window is open when Eclipse loads, that velocity wins and m2e loading fails. The issue is with the way velocity uses Class.forName() to load classes. See http://velocity.apache.org/engine/releases/velocity-1.5/apidocs/org/apache/velocity/util/ClassUtils.html. A good first step might be to update m2e from velocity 1.5 to velocity 1.7. v1.7 was released November 2010, v1.5 was released March 2007. I believe the update should be seamless from a code perspective. That may help avoid the dreaded error 'The specified class for ResourceManager (org.apache.velocity.runtime.resource.ResourceManagerImpl) does not implement org.apache.velocity.runtime.resource.ResourceManager; Velocity is not initialized correctly.'. Not sure why bug #394616 was closed since it suggested changes in both velocity and the Orbit bundle for velocity, which have not been implemented? Additional info at http://stackoverflow.com/questions/8603546/velocity-in-osgi-how-to-load-templates-from-classpath
https://issues.apache.org/jira/browse/VELOCITY-694 Make Velocity OSGI ready, marked fixed in velocity 1.7. Corrected velocity manifest.mf. Perhaps I can try hacking my local Eclipse to change m2e archetype to use v1.7, to see if it blows up. However I have to create a SHA for the new jar, change the manifest, and it appears that plexus-velocity depends on velocity 1.5 (latest v1.1.8 not updated since 2009, but it's only 4 classes, it appears to no longer be part of plexus-components project). If Dali implemented a workaround, I hope M2E would implement the same workaround rather than relying on one other project not to conflict. There are many many plugins which use velocity, but (for heavy maven users) m2e really is the only one which *has* to work. Also it would be helpful if m2e fails to load velocity (used just for archetype?) that it doesn't kill the entire plugin.
As I mentioned in comment #12, I've attempted to update m2e to use orbit velocity bundle but it didn't work. My failed attempt is available from 396554_orbitVelocityBundle branch [1], you probably want to use it as starting point. Wiki [2] explains m2e development environment setup and configuration. Also, there is a good chance bug #410997 will have to be addressed before we have reliable long-term solution to this problem. [1] http://git.eclipse.org/c/m2e/m2e-core.git/log/?h=396554_orbitVelocityBundle [2] https://wiki.eclipse.org/M2E_Development_Environment
Actually I was thinking m2e could be updated to velocity 1.7 without using Orbit. Right now v1.5 and plexus-velocity are bundled internally with that plugin. Get the update working first, update published, see if the classloader problem still exists, then try to convert to Orbit. Hopefully #410997 will update an Orbit for velocity v1.7, not 1.5, doesn't look like it's been touched in over a year. Sorry, I haven't ever done any Eclipse plugin development. Thanks for the links though, a good opportunity to try, when I have some time. I am very familiar with maven, the plugin appears to require that. There's a number of annoyances with m2e that I'd love to patch. (src/test ahead of src/main in Eclipse classpath, don't exclude *.* when testing from Eclipse, add version update checking from maven-versions-plugin).
I doubt updating embedded velocity version will help. There will be still two (identical) copies of velicity classes, loaded by different classloaders. JVM will still consider them distinct classes and will not allow casting between the two. To fix the problem, either velocity should become OSGi friendly or there must be one copy of velocity only.
https://issues.apache.org/jira/browse/VELOCITY-694 Do you think that v1.7 update made velocity more OSGI friendly, or is more required? Sorry I'm not an OSGI user and can't tell, they apparently updated the bundle dependency declarations. That was a manifest-only change, not a code change to use a better classloader, but #410997 says the manifest should 'import what they export' and act as a passthrough. The current velocity-1.7-dep.jar manifest.mf does not appear to do that (no velocity packages in Import-Package, no "uses" of the packages it exports). I wouldn't rely on changes from velocity itself, given the few changes made over the last few years. Singleton bundle is probably a better solution, and from what I've seen the templates still work like in the older version of velocity, though I don't know how you could prevent other plugins from bundling their own velocity version, or users from manually adding additional velocity bundles (as I have done). Thanks so much for your feedback on this.
"import exported packages" is not enough. The best solution is to fix classloading in velocity, include fixed version in Orbit and change m2e and others to use the Orbit version. Users of the fixed velocity version will work regardless of any other velicity versions available. There is no technical reasons why this cannot be done in Orbit, but this may go against Orbit bylaws (I don't really know, just guessing) Alternatively, all eclipse project will have to agree to use the same velocity bundle. This will work as long as there is only one version of the bundle installed. There is no way to force single version of velicity, however. Even if the bundle will be marked as 'singleton', nothing stops other bundles to embed velocity classes.
(In reply to Igor Fedorenko from comment #20) > "import exported packages" is not enough. Why is this? Is it because many people use "require bundle" instead of "import package"? > The best solution is to fix classloading in velocity, include fixed version > in Orbit and change m2e and others to use the Orbit version. Users of the > fixed velocity version will work regardless of any other velicity versions > available. There is no technical reasons why this cannot be done in Orbit, > but this may go against Orbit bylaws (I don't really know, just guessing) I can confirm, we in Orbit do not "fix" third party code. But do encourage bugs to be open on the third party, and see what they say. > Alternatively, all eclipse project will have to agree to use the same > velocity bundle. This will work as long as there is only one version of the > bundle installed. There is no way to force single version of velicity, > however. Even if the bundle will be marked as 'singleton', nothing stops > other bundles to embed velocity classes. As I've said else where, I doubt having a singleton will really help (except for some cases, and may cause more problems that it fixes). What I don't understand, exactly, is how these velocity classes are "touching" each other in Eclipse use cases. Does someone "export" it, as part of their API? Are "velocity objects" arguments, parameters, or return types from some API? I've not looked at "1.5" (which is the one in Orbit) but have looked at the 1.7 code and it claims to always consult the Thread Context classloader first which, I know in the case for "Xerces" was part of the cure for for "conflicting classes". That's in methods such as ClassUtils.getClass(clazz), ClassUtils.newInstance(clazz). So, clients would have to use those methods, and not simply try to create their own "new Instance()" ... to all use same class with same classloader. Does any of that information help? Or, maybe it'd help to explain to me what the problem is?
(In reply to David Williams from comment #21) > (In reply to Igor Fedorenko from comment #20) > > "import exported packages" is not enough. > > Why is this? Is it because many people use "require bundle" instead of > "import package"? > "import exported packages" solves the problem with inconsistent wiring of public packages, what we have here is entirely different problem. Here is a quote from comment #6 (In reply to Neil Hauge from comment #6) > > See the following links for more information: > http://blog.bjhargrave.com/2007/07/contextfinder-in-eclipse-is-broken.html > https://bugs.eclipse.org/bugs/show_bug.cgi?id=127963 > https://bugs.eclipse.org/bugs/show_bug.cgi?id=127963#c22 (this comment is > key) > ... > > It would appear that the root of the problem is that TCCL + > Equinox/ContextFinder + Class.forName(...) = leaks. Given the environmental > limitations, it would seem that the least intrusive solution for this > problem would be for all Velocity consumers to use the Orbit provided > Velocity bundle to prevent the leak at the source. If everyone doesn't play > by this rule, there will be issues. So, in other words, we'll see this problem as long as there are two or more copies of velocity classes anywhere in eclipse runtime and at least one of the copies continues to use Class.forName. As far as I can tell, velocity 1.7 still uses Class.forName and is still affected by the problem, see ClassUtils.java around line 73 http://svn.apache.org/viewvc/velocity/engine/tags/1.7/src/java/org/apache/velocity/util/ClassUtils.java?view=markup
(In reply to Igor Fedorenko from comment #22) > (In reply to David Williams from comment #21) > > (In reply to Igor Fedorenko from comment #20) > > > "import exported packages" is not enough. > > > > Why is this? Is it because many people use "require bundle" instead of > > "import package"? > > > > "import exported packages" solves the problem with inconsistent wiring of > public packages, what we have here is entirely different problem. Here is a > quote from comment #6 > > (In reply to Neil Hauge from comment #6) > > > > See the following links for more information: > > http://blog.bjhargrave.com/2007/07/contextfinder-in-eclipse-is-broken.html > > https://bugs.eclipse.org/bugs/show_bug.cgi?id=127963 > > https://bugs.eclipse.org/bugs/show_bug.cgi?id=127963#c22 (this comment is > > key) > > > ... > > > > It would appear that the root of the problem is that TCCL + > > Equinox/ContextFinder + Class.forName(...) = leaks. Given the environmental > > limitations, it would seem that the least intrusive solution for this > > problem would be for all Velocity consumers to use the Orbit provided > > Velocity bundle to prevent the leak at the source. If everyone doesn't play > > by this rule, there will be issues. > > So, in other words, we'll see this problem as long as there are two or more > copies of velocity classes anywhere in eclipse runtime and at least one of > the copies continues to use Class.forName. > > As far as I can tell, velocity 1.7 still uses Class.forName and is still > affected by the problem, see ClassUtils.java around line 73 > > http://svn.apache.org/viewvc/velocity/engine/tags/1.7/src/java/org/apache/ > velocity/util/ClassUtils.java?view=markup Well, that is the form that uses three arguments, though, with 'loader' being Thread.currentThread().getContextClassLoader() which, I've heard, is "the right" way to call "Class.forName"? Oh, just re-read Tom's comment in 127963 comment 22 that says there is a difference between tccl.loadClass and Class.forName(... tccl). Something definitely sounds "off" (or, complicated) with that ... but, I'm beginning to understand slightly better. Thanks,
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. As such, we're closing this bug. If you have further information on the current state of the bug, please add it and reopen this bug. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant. -- The automated Eclipse Genie.
Moved to https://github.com/eclipse-m2e/m2e-core/issues/
