Bug 446251 - broken or incomplete maven 3 installations on windows7tests slave
Summary: broken or incomplete maven 3 installations on windows7tests slave
Status: CLOSED FIXED
Alias: None
Product: Community
Classification: Eclipse Foundation
Component: CI-Jenkins (show other bugs)
Version: unspecified   Edit
Hardware: PC Windows 7
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: CI Admin Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 460979 (view as bug list)
Depends on: 412933
Blocks: 427529
  Show dependency tree
 
Reported: 2014-10-08 03:02 EDT by Matthias Sohn CLA
Modified: 2015-09-01 09:01 EDT (History)
9 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matthias Sohn CLA 2014-10-08 03:02:40 EDT
moving this problem from bug 427529

I am trying to setup a jgit build on the windows slave "windows7tests" in order to gain
test coverage on windows.

We need at least Maven 3.0 or newer to run the jgit build.

I tried all existing Maven installations on that slave and all seem to suffer from missing or wrong configuration. See the failed builds of the job
https://hudson.eclipse.org/hudson/job/jgit-windows-test/

Than commented on bug 427529:

(In reply to Matthias Sohn from comment #4)
> I created a jgit job on the shared Hudson and tied it to the "windows7tests"
> node.
> https://hudson.eclipse.org/hudson/job/jgit-windows-test/
> 
> The job fails with the following error
> 
> [INFO] Using Maven 3 installation: Maven 3.1.0
> [INFO] Checking Maven 3 installation environment
> [ERROR] Failure: hudson.AbortException: Missing required file:
> /shared/common/apache-maven-3.1.0/bin/mvn.bat
> ERROR: Missing required file: /shared/common/apache-maven-3.1.0/bin/mvn.bat
> 
> It seems the Maven installation isn't working. I also tried Maven 3.0.5
> which hits
> the same error.

That's because /shared/common is not a valid path on Windows. If I recall additional configuration needs to be done on Hudson master under the slave setup for windows7tests to configure equivalent paths for all the Maven tools defined to point to the Windows style location of the same tool.
Comment 1 Eclipse Webmaster CLA 2014-10-15 11:57:30 EDT
I've installed Maven 3.1.1 and have updated the windows slave config to point at the correct path.

-M.
Comment 2 Matthias Sohn CLA 2014-10-15 12:18:00 EDT
thanks :-)

Unfortunately this slave has only 1 executor and there are some jobs which seem to take 5 hours to run, one of them is now in my way to try the shiny new Maven installation.
Any chance to get another executor ?
Comment 3 Matthias Sohn CLA 2014-10-15 12:19:52 EDT
why is this Maven 3.1.1 installation called "Maven 3.1.0" ?
Comment 4 Eclipse Webmaster CLA 2014-10-15 14:59:42 EDT
Because the slave doesn't support custom tools labels.  The only tools you can define are those defined on the master.  So I took a quick look and since someone has installed Maven 3.1.1 in /shared I've added it to the master and updated the label for the windows slave.

-M.
Comment 5 Matthias Sohn CLA 2014-10-15 17:00:38 EDT
this doesn't yet work, Hudson complains:
"Missing required file: /shared/common/apache-maven-3.1.0/bin/mvn.bat"

https://hudson.eclipse.org/hudson/job/jgit-windows-test/10/console
Comment 6 Eclipse Webmaster CLA 2014-10-17 14:04:15 EDT
Ah I added the 3.1.1 install into the Legacy Maven installs by mistake.  I've moved it into the Maven 3 group, So I updated your job to use it(but might have missed a maven parameter), and it's no longer complaining about maven not being found.

-M.
Comment 7 Matthias Sohn CLA 2014-10-17 15:39:58 EDT
now the job fails with [1]

FATAL: java.lang.ArrayIndexOutOfBoundsException: 0
org.hudsonci.utils.tasks.OperationFailure: java.lang.ArrayIndexOutOfBoundsException: 0
	at org.hudsonci.utils.tasks.PerformOperation.execute(PerformOperation.java:75)
	at org.hudsonci.maven.plugin.builder.MavenBuilder.perform(MavenBuilder.java:169)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:630)
	at hudson.model.Build$RunnerImpl.build(Build.java:175)
	at hudson.model.Build$RunnerImpl.doRun(Build.java:137)
	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:429)
	at hudson.model.Run.run(Run.java:1366)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:145)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 0
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.choose(RemoteInvocationHandler.java:295)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:273)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:258)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:218)
	at hudson.remoting.UserRequest.perform(UserRequest.java:114)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:283)
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at hudson.remoting.Engine$1$1.run(Engine.java:60)
	at java.lang.Thread.run(Unknown Source)

this looks like a bug in Hudson or some plugin

I then tried to wipe the workspace to see if restarting from scratch helps but
this failed with an IOException saying that a git pack file can't be deleted.
This looks like Hudson or one of its plugins has still an open file handle on
the repository's pack files. Again this looks like a bug :-(
After that partial deletion the repository is corrupt and the job fails since jgit chokes on this corruptness [2].

I guess to recover from this mess you need to restart the slave to get rid of the open file handle.

[1] https://hudson.eclipse.org/hudson/job/jgit-windows-test/17/console
[2] https://hudson.eclipse.org/hudson/job/jgit-windows-test/19/console
Comment 8 Eclipse Webmaster CLA 2014-10-17 16:12:20 EDT
I restarted the slave and removed the jgit workspace data but still see the array out of bounds(https://hudson.eclipse.org/hudson/job/jgit-windows-test/20/console)

-M.
Comment 9 Matthias Sohn CLA 2014-11-04 10:39:43 EST
Looks like I am stuck here, I tried to reduce the configuration of this job but it always fails with the infamous ArrayIndexOutOfBoundsException, finally I tried wipe workspace but this corrupted the git repository since the workspace deletion failed due to some locked files.

Can you fix this by erasing the workspace on OS level ? We should then forward this problem to the Hudson team, hopefully they can find out what's causing the ArrayIndexOutOfBoundsException.
Comment 10 Eclipse Webmaster CLA 2014-11-04 13:35:48 EST
I've force removed the workspace and restarted the slave jar.

-M.
Comment 11 Matthias Sohn CLA 2014-11-05 02:11:28 EST
I found bug 385566 which looks like the problem we are facing here. Could you check if this helps to fix the problem ?
Comment 12 Eclipse Webmaster CLA 2014-11-06 09:52:11 EST
Ok I think I've managed to get the windows slave agent running with Java 1,6.

-M.
Comment 13 Matthias Sohn CLA 2014-11-06 19:20:14 EST
the job still fails with the same exception

moving this bug to Hudson team to get their opinion on this problem
Comment 14 Matthias Sohn CLA 2014-11-06 19:22:05 EST
latest log of the jgit test job on windows slave which is still failing with the exception 

[ERROR] Failure: java.lang.ArrayIndexOutOfBoundsException: 0
FATAL: java.lang.ArrayIndexOutOfBoundsException: 0
org.hudsonci.utils.tasks.OperationFailure: java.lang.ArrayIndexOutOfBoundsException: 0
	at org.hudsonci.utils.tasks.PerformOperation.execute(PerformOperation.java:75)
	at org.hudsonci.maven.plugin.builder.MavenBuilder.perform(MavenBuilder.java:169)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
	at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:630)
	at hudson.model.Build$RunnerImpl.build(Build.java:175)
	at hudson.model.Build$RunnerImpl.doRun(Build.java:137)
	at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:429)
	at hudson.model.Run.run(Run.java:1366)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
	at hudson.model.ResourceController.execute(ResourceController.java:88)
	at hudson.model.Executor.run(Executor.java:145)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 0
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.choose(RemoteInvocationHandler.java:295)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.perform(RemoteInvocationHandler.java:273)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:258)
	at hudson.remoting.RemoteInvocationHandler$RPCRequest.call(RemoteInvocationHandler.java:218)
	at hudson.remoting.UserRequest.perform(UserRequest.java:114)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:283)
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at hudson.remoting.Engine$1$1.run(Engine.java:60)
	at java.lang.Thread.run(Unknown Source)

is here https://hudson.eclipse.org/hudson/job/jgit-windows-test/36/console
Comment 15 Matthias Sohn CLA 2014-11-06 19:22:41 EST
@hudson team: please advise how to get rid of this exception
Comment 16 Winston Prakash CLA 2014-11-07 12:46:55 EST
(In reply to Matthias Sohn from comment #15)

Unfortunately https://hudson.eclipse.org/hudson is a very old version of Hudson (2.x).

Hudson 2.x is almost 5 years old and released outside of Eclipse. Its maintenance is discontinued long time ago in favor of Hudson 3.x which is released from Eclipse and Hudson team is only concentrating on Hudson 3.x.

The maven plugin in Hudson 3.x is modified to support Maven 3.x which is fully tested by Hudson QA. Is it possible to move to Hudson 3.x, may be by getting a HIPP for JGit project.
Comment 17 Matthias Sohn CLA 2014-11-07 18:15:54 EST
this is the windows slave of the shared Hudson instance which we were advised to use since HIPP's can't have slaves yet.

Webmaster could you update this Windows slave to the current Hudson release ?
Comment 18 Winston Prakash CLA 2014-11-07 19:15:31 EST
(In reply to Matthias Sohn from comment #17)

If Webmaster is planning to add slaves to HIPP, following are my recommendations

- Use latest Hudson 3.2.1

- Enable Team Management 

- Add Sysadmin(s) to the Hudson instance

- Create teams and add users to that team (Team Admin can provide necessary permissions to other team members)

- Add public slaves (only sysadmin can create/delete/modify these), but teams can build their jobs in them.

- Team admin can add Team private nodes if necessary

- Team admin can selectively make jobs visible to public.


More details about 
https://wiki.eclipse.org/Hudson-ci/features/Team_Concept_User_View
Comment 19 Eclipse Webmaster CLA 2014-11-11 11:32:24 EST
(In reply to Matthias Sohn from comment #17)
> 
> Webmaster could you update this Windows slave to the current Hudson release ?

This is the only windows slave we have, and my understanding is that in order to update the slave we'd have to update the shared Hudson instance (bug 412933) .

-M.
Comment 20 Matthias Sohn CLA 2014-11-11 11:54:08 EST
(In reply to Eclipse Webmaster from comment #19)
> (In reply to Matthias Sohn from comment #17)
> > 
> > Webmaster could you update this Windows slave to the current Hudson release ?
> 
> This is the only windows slave we have, and my understanding is that in
> order to update the slave we'd have to update the shared Hudson instance
> (bug 412933) .
> 
> -M.

yes, this is probably a prerequisite
Comment 21 Mikaël Barbero CLA 2015-02-27 04:45:59 EST
*** Bug 460979 has been marked as a duplicate of this bug. ***
Comment 22 Mikaël Barbero CLA 2015-09-01 08:54:10 EDT
The jgit-windows-test job is now running on the new shared instance https://hudson.eclipse.org/shared/job/jgit-windows-test/.
Comment 23 Matthias Sohn CLA 2015-09-01 09:01:39 EDT
thanks :-)