Community
Participate
Working Groups
Created attachment 248314 [details] Thread dump With the Debian package of Hudson 3.2.1 on Ubuntu 14.04 we observed the following deadlock. The thread dump taken with jstack is attached. The Hudson instance does not start up anymore. Please let me know if you need more information.
Thanks for the report. We will take a look at it. Is this reproducible always?
If you are in this state, it's reproducible always, e.g. Hudson doesn't start up anymore. It runs into this deadlock. We suspect it has something to do with plugin updates we made: Here is the lsit of hpi files of that instance: -rw-r--r-- 1 hudson nogroup 4784082 Aug 6 11:30 plugins/analysis-core.hpi -rw-r--r-- 1 hudson nogroup 12960339 Aug 6 11:30 plugins/birt-charts.hpi -rw-r--r-- 1 hudson nogroup 2551295 Aug 6 11:30 plugins/checkstyle.hpi -rw-r--r-- 1 hudson nogroup 1218662 Aug 6 11:30 plugins/cvs.hpi -rw-r--r-- 1 hudson nogroup 11591121 Aug 6 11:30 plugins/findbugs.hpi -rw-r--r-- 1 hudson nogroup 3612295 Okt 28 16:19 plugins/git.hpi -rw-r--r-- 1 hudson adm 13924 Aug 6 09:15 plugins/github.hpi -rw-r--r-- 1 hudson nogroup 5686188 Aug 6 11:31 plugins/groovy-support-plugin.hpi -rw-r--r-- 1 hudson nogroup 2943036 Aug 6 11:30 plugins/jfreechart-plugin.hpi -rw-r--r-- 1 hudson nogroup 2021210 Aug 6 11:30 plugins/jna-native-support-plugin.hpi -rw-r--r-- 1 hudson nogroup 14095165 Okt 28 16:19 plugins/maven3-plugin.hpi -rw-r--r-- 1 hudson nogroup 2431754 Okt 28 15:04 plugins/maven3-snapshots.hpi -rw-r--r-- 1 hudson nogroup 8754101 Aug 6 11:30 plugins/maven-plugin.hpi -rw-r--r-- 1 hudson nogroup 8431374 Aug 6 11:30 plugins/pmd.hpi -rw-r--r-- 1 hudson nogroup 2529187 Okt 28 16:18 plugins/rest-plugin.hpi -rwxr-x--- 1 hudson adm 10691 Aug 6 09:15 plugins/scis-ad.hpi* -rw-r--r-- 1 hudson nogroup 515900 Aug 6 11:31 plugins/ssh-slaves.hpi -rw-r--r-- 1 hudson nogroup 8108325 Okt 28 16:18 plugins/subversion.hpi -rw-r--r-- 1 hudson nogroup 2509228 Aug 6 11:31 plugins/tasks.hpi -rw-r--r-- 1 hudson adm 12493 Aug 6 09:15 plugins/text-finder.hpi -rw-r--r-- 1 hudson nogroup 2783303 Aug 7 10:56 plugins/warnings.hpi -rw-r--r-- 1 hudson nogroup 2837078 Aug 6 11:30 plugins/xpath-provider.hpi For example Git, Maven3 und Subversion plugins were installed or updated on Okt 28. After that, the problem did occur, but it may be a coincidence...
What versions of the plugins do you have installed?
How can I determine the plugin versions without starting Hudson?
You can check the plugin manifest (under HUDSON_HOME). For git for example: .hudson\plugins\git\META-INF\MANIFEST.MF
Created attachment 248368 [details] installed plugins
We managed to start Hudson again by removing the maven3 stuff from the plugin folder (see below) und removing those jobs from the jobs folder which we believe used the maven 3 plugin. I attached the remaining installed plugins. Removed plugins: Hudson Maven3 Plugin 3.0.4 Hudson Maven3 SNAPSHOT Monitor 3.0.1
But the deadlock is not completely gone in this state. We were able to start Hudson a few times, but then the deadlock happened again ;-(
From the thread lock, the issue seems to be scis_ad plugin. - hudson.plugins.scis_ad.ScisSupportOffer.<init>() @bci=81, line=36 (Interpreted frame) This plugin seems to do Hudson.getItems() when its extension was initialized hudson.model.LazyTopLevelItem.item() @bci=0, line=150 (Interpreted frame) - hudson.model.LazyTopLevelItem.hasPermission(hudson.security.Permission) @bci=1, line=285 (Interpreted frame) - hudson.model.Hudson.getItems() @bci=46, line=1311 (Interpreted frame) This locks with Hudson startup which do Hudson.getItems() which in turn ensure all extension points are loaded. Can you remove the scis_ad plugin and start Hudson. We need to look at scis_ad plugin.
I can reproduce the bug with the scis_ad plugin installed. It works without issues without the plugin.
I'm glad you could reproduce the issue. We'll try again without the scis_ad plugin and let you know how it goes...
Without the scis_ad plugin our Hudson instance runs stable again. Thanks for your help.
