Bug 476362 - Migrate or remove Hudson jobs on the shared instance
Summary: Migrate or remove Hudson jobs on the shared instance
Status: CLOSED FIXED
Alias: None
Product: z_Archived
Classification: Eclipse Foundation
Component: Buckminster (show other bugs)
Version: unspecified   Edit
Hardware: PC Mac OS X
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Mikaël Barbero CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 412933
  Show dependency tree
 
Reported: 2015-09-01 11:42 EDT by Mikaël Barbero CLA
Modified: 2019-02-25 14:40 EST (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mikaël Barbero CLA 2015-09-01 11:42:51 EDT
We are in the process to deprecate the current Hudson shared instance at Eclipse (see https://bugs.eclipse.org/bugs/show_bug.cgi?id=412933 for details). You have the following jobs on this instance for the Eclipse Buckminster project:

* https://hudson.eclipse.org/hudson/job/buckminster-head/
* https://hudson.eclipse.org/hudson/job/buckminster-maintenance/
* https://hudson.eclipse.org/hudson/job/buckminster-voicetools-legacy-nightly/

We offer two migration paths for these jobs:
* Use the new up to date Hudson shared instance at https://hudson.eclipse.org/shared/. It is a recent Hudson installation configured the same way as the old instance was to minimize migration work.
* Use a dedicated Hudson instance for this project (what we call a HIPP, Hudson Instance Per Project), e.g., https://hudson.eclipse.org/buckminster/. This way, you can create as many jobs as you want, install all the Hudson plugins you want, etc. This is the preferred path of migration as the shared instance (old or new) will be deprecated in the mid term. See https://wiki.eclipse.org/Hudson#HIPP for more details.

Whatever the path you choose, the plan is: 
* first to copy the job to the new instance (shared or HIPP). 
* make it work
* deactivate the job on the old shared instance
* after a while without issue and after validation from your side, remove the job from the old shared instance.

What is your preferred way? Is there any preferred time when I can do the migration? Is there any useless job that could be nuked?
Comment 1 Thomas Hallgren CLA 2015-09-01 11:49:42 EDT
I think the best path forward for Buckminster is to use HIPP. That way it will be easier to switch the Buckminster instance used for building Buckminster itself once a new release has been made.

I think the only job we care about and is worth preserving is the one named buckminster-head. The rest can be nuked.

Any time is a good time to do this. Just let us know when we can test.

Thanks.
Comment 2 Mikaël Barbero CLA 2015-09-01 12:02:38 EDT
Thank you for your quick reply. I have scheduled the creation of your HIPP and will keep you posted when it will be available and I have migrated the job.
Comment 3 Mikaël Barbero CLA 2015-09-16 11:42:22 EDT
Buckminster HIPP is now setup at https://hudson.eclipse.org/buckminster/. Sorry for the delay

Committers should now be able to login and create jobs. Please note that unlike the shared instance you will need to login with your _email_ address as your username.

If you require additional plugins feel free to reopen this bug and list them.

Finally we can also optionally add the Buckminster HIPP user to the buckminster project group which would allow the HIPP user to have write access anywhere the project group has access to including downloads area. We can also give it write permissions to your git repositories. This could be useful if you'd like to use Hudson for automated build promotion but keep in mind there is some risk in giving Hudson more permissions than it needs. If you'd like to to enable this please reopen and let us know.

I deactivated the main job on the shared instance (https://hudson.eclipse.org/hudson/job/buckminster-head/) and deleted the maintenance one. Please let me know when everything is ok on your side and I can delete the job on the old shared instance. Thanks
Comment 4 Thomas Hallgren CLA 2015-09-16 17:28:30 EDT
I tried running the build. Something is not right.

https://hudson.eclipse.org/buckminster/job/buckminster-head/272/console

Is it possible to ssh to the machine and configure things outside of Hudson (such as a Buckminster install to use for the build)?
Comment 5 Mikaël Barbero CLA 2015-09-17 02:24:42 EDT
(In reply to Thomas Hallgren from comment #4)
> I tried running the build. Something is not right.
> 
> https://hudson.eclipse.org/buckminster/job/buckminster-head/272/console

The jdk configuration were misconfigured (due to the migration). My bad. I restored it to jdk1.7 as it is on the old shared instance. It seems to run well now.

> Is it possible to ssh to the machine and configure things outside of Hudson
> (such as a Buckminster install to use for the build)?

No, we don't provide any ssh access to HIPP machines. it provides us flexibility to move HIPPs from machine to machine depending on workload, disk usages... If needed, we can set you an administrator of your HIPP and you will be able to change the Hudson conf.
Comment 6 Thomas Hallgren CLA 2015-09-17 03:30:31 EDT
(In reply to Mikael Barbero from comment #5)
> If needed, we can set you an administrator of your HIPP and
> you will be able to change the Hudson conf.

Yes. Please do. I thought that was one of the major advantages when using HIPP (aside from also being able to do stuff on the actual machine, but obviously that was a misconception).
Comment 7 Thomas Hallgren CLA 2015-09-17 03:36:08 EDT
(In reply to Mikael Barbero from comment #5)
> (In reply to Thomas Hallgren from comment #4)
> ... It seems to run well now.
> 
Not quite. It errors out because it's unable to copy the build result into the download area. The directory is set up to accept writes from the Hudson build account "hudsonBuild". Obviously that has changed somehow. What account is used by the HIPP instance?

Failed to copy /home/hudson/genie.buckminster/.hudson/jobs/buckminster-head/workspace/output/org.eclipse.buckminster.site.eclipse_1.7.0-eclipse.feature/site.p2/artifacts.jar to /home/data/httpd/download.eclipse.org/tools/buckminster/updates-4.5/artifacts.jar due to can't write to read-only destination file /home/data/httpd/download.eclipse.org/tools/buckminster/updates-4.5/artifacts.jar
Comment 8 Mikaël Barbero CLA 2015-09-17 03:50:12 EDT
(In reply to Thomas Hallgren from comment #6)
> Yes. Please do. I thought that was one of the major advantages when using
> HIPP (aside from also being able to do stuff on the actual machine, but
> obviously that was a misconception).

Done. Of course, with great powers come great responsibilities, so be cautious with what you do ;)

(In reply to Thomas Hallgren from comment #7)
> Not quite. It errors out because it's unable to copy the build result into
> the download area. The directory is set up to accept writes from the Hudson
> build account "hudsonBuild". Obviously that has changed somehow. What
> account is used by the HIPP instance?
> 
> Failed to copy
> /home/hudson/genie.buckminster/.hudson/jobs/buckminster-head/workspace/
> output/org.eclipse.buckminster.site.eclipse_1.7.0-eclipse.feature/site.p2/
> artifacts.jar to
> /home/data/httpd/download.eclipse.org/tools/buckminster/updates-4.5/
> artifacts.jar due to can't write to read-only destination file
> /home/data/httpd/download.eclipse.org/tools/buckminster/updates-4.5/
> artifacts.jar

I just added your HIPP user to the project's group. It will grant write access to the HIPP wherever project committers have. It is much more secure than using ACLs for the shared instance user ;). It will take a hour or so to sync the ldap groups to the HIPP machine and will require a reboot of Hudson after that. Will keep you posted when it will be effective.
Comment 9 Mikaël Barbero CLA 2015-10-08 07:12:27 EDT
Sorry for the delay in catchup. The permissions are good now. Could you check that your builds are alright?
Comment 10 Mikaël Barbero CLA 2015-10-13 11:12:58 EDT
May I delete the jobs on the old shared instance now?
Comment 11 Mikaël Barbero CLA 2015-11-25 05:36:37 EST
(In reply to Mikael Barbero from comment #10)
> May I delete the jobs on the old shared instance now?

ping ;)
Comment 12 Thomas Hallgren CLA 2015-11-25 11:33:23 EST
Yes, by all means. Go ahead and delete them. Sorry for late reply.
Comment 13 Mikaël Barbero CLA 2015-11-26 06:59:03 EST
Done. Thanks.