Community
Participate
Working Groups
As it seems the Hudson installation formerly available at http://build.eclipse.org/hudson is no longer available, could we get it installed for use in running the Galileo build? I put a copy of the hudson.war on the machine at /shared/galileo/build Installation instructions are here: http://hudson.gotdns.com/wiki/display/HUDSON/Installation+and+Execution
Going to look at this tomorrow as I had originally put the Hudson installation on the eclipse infrastructure.
After some investigation, I think the problem with the installation of hudson on the eclipse machines is how we've configured apache to pass through the requests to the winstone container running the application. I've found a proposed solution on the web and forwarded on the information to the eclipse webmaster. Hoperfully, somebody there will have a chance to take a quick look at it and see if we can make the changes.
Great, thanks for your help on this Adrian.
As discussed in email, Hudson is now running but cannot yet be used for Galileo build due to a lack of write permissions to /shared/galileo/build. It was concluded that Hudson should be configured as a service and with access restrictions to keep just anyone from configuring/running builds on build.eclipse.org.
I'm taking a look at this right now. So expect the hudson instance to be up and down today. We'll need to add the 'restricted' user to the callisto-dev group, Bjorn can you approve that? Beyond that I'm going to look into moving us from http to https and adding a set of ldap access rules to the frontend to limit access to people in the callisto-dev group, or is there another group that would be a better choice? -M.
I'd like to have the connection to hudson run over HTTPS, but I can't find a way to make it happen. If I leave hudson 'alone' and add the proxy rules to the https vhost config I get the following error in the apache logs: [Tue Dec 23 14:38:23 2008] [error] ajp_read_header: ajp_ilink_receive failed [Tue Dec 23 14:38:23 2008] [error] (120006)APR does not understand this error code: proxy: read response failed from (null) (localhost) However if I try and turn on Hudsons 'internal' https handling I get the following error on the console: [Winstone 2008/12/23 15:13:35] - Error during HTTPS listener init or shutdown java.security.NoSuchAlgorithmException: SunX509 KeyManagerFactory not available Given the potential for abuse by un-authenticated users, this has to be authenticated and over an encrypted channel, and adding yet another login is not going to happen. Any ideas? -M
From a quick Google search, it looks like we need to specify the IbmX509 algorithm, as we're using an IBM JRE and not Sun's.
(replied to bugzilla mail instead of directly posting here, this is how you can specify IBM's key manager). Matt, I got this from the command line help section of Hudson (and the winstone servlet container): --httpsPort = set the https listening port. -1 to disable, Default is disabled --httpsListenAddress = set the https listening address. Default is all interfaces --httpsDoHostnameLookups = enable host name lookups on https connections. Default is false --httpsKeyStore = the location of the SSL KeyStore file. Default is ./winstone.ks --httpsKeyStorePassword = the password for the SSL KeyStore file. Default is null --httpsKeyManagerType = the SSL KeyManagerFactory type (eg SunX509, IbmX509). Default is SunX509 You could change the SSL Key Manager type to something that would make sense, maybe the IbmX509 ? Hope this is some help. Adrian.
*** Bug 260021 has been marked as a duplicate of this bug. ***
Ok, I've managed to get hudson running over https in our test environment. I just need to iron out the authentication and make the changes on build to get things running. Hopefully this will go live later today. -M,
Moving this to servers (from website).
(In reply to comment #10) > Ok, I've managed to get hudson running over https in our test environment. I > just need to iron out the authentication and make the changes on build to get > things running. Hopefully this will go live later today. Awesome! Is it safe to assume that this will be available for committers (or perhaps just those in the 'signers' or 'users' (ie., modelingBuild@build, dashBuild@build) group?) to use for any builds run on build.eclipse.org, not just for Galileo?
Hudson is now available via https://build.eclipse.org/hudson . In order to access hudson you will need: 1) Your committer ID and password 2) To be a member of the callisto-dev group Right now apache is handling the authentication, but if security or management becomes an issue then I'll turn on hudsons built in tools. The privileges for the hudson process are really restrictive so it should only be able to access callisto-dev data(sorry Nick). -M.
(In reply to comment #13) > Hudson is now available via https://build.eclipse.org/hudson . In order to > access hudson you will need: > 1) Your committer ID and password > 2) To be a member of the callisto-dev group I used 1) and thought I was part of 2) but no luck.
(In reply to comment #14) > (In reply to comment #13) > > Hudson is now available via https://build.eclipse.org/hudson . In order to > > access hudson you will need: > > 1) Your committer ID and password > > 2) To be a member of the callisto-dev group > I used 1) and thought I was part of 2) but no luck. I tried nickb and nickboldt@gmail.com and nickboldt+bugzilla@gmail.com with accompanying passwords and nothing worked. I'm in signers group for sure.
(In reply to comment #13) > The privileges for the hudson process are really restrictive so it should only > be able to access callisto-dev data(sorry Nick). You make me sad. So be it. Come, Patsy. > None shall pass. What? > None shall pass, be they born of woman, a committer, or in group 'signers'. I have no quarrel with you, good Sir Webmaster, but I must use this app. > Then you shall die. I command you as one of the clones to stand aside! > I move for no man (except Denis or Mike). (apologies to http://www.sacred-texts.com/neu/mphg/mphg.htm)
Sigh, $#%%%#$% typo in the group name. Nick, Rich you should both be able to login now, as well as anyone else in the callisto-del group. I don't know what the callitso-dev group is, but I'm pretty sure that nobody is a member. -M.
And again. Not my day. -M.
Login works now, thanks. I set up a galileo.gen project that will get a project from CVS to a /shared/galileo/build/config directory. I expect permissions are not correct by this error: started FATAL: Failed to mkdirs: /shared/galileo/build/config java.io.IOException: Failed to mkdirs: /shared/galileo/build/config at hudson.FilePath.mkdirs(FilePath.java:452) at hudson.model.AbstractProject.checkout(AbstractProject.java:664) at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:261) at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:235) at hudson.model.Run.run(Run.java:817) at hudson.model.Build.run(Build.java:88) at hudson.model.ResourceController.execute(ResourceController.java:70) at hudson.model.Executor.run(Executor.java:90)
Ok, I've updated the permissions on the root directory from 744 to 774 so please try it again. -M.
Regarding comment #14, are there any plans to have hudson access more than the callisto-del data? I've been using hudson on my internal build server for the past few days and it seems to be working well for starting and tracking builds etc. It would be nice for the eclipse.org hudson to access project specific directories. The way that the Apache community does this is that there is a generic hudson id and each project assigns this user rights in their home directories for build purposes. See http://wiki.apache.org/general/Hudson
(In reply to comment #20) > Ok, I've updated the permissions on the root directory from 744 to 774 so > please try it again. > -M. It works now, thanks.
(In reply to comment #21) > It would be nice for the eclipse.org hudson to access project specific > directories. > See http://wiki.apache.org/general/Hudson +1. Maybe we should reopen bug 260021 for this? Dash's Common Builder needs a web UI, and Hudson's the best option IMHO, compared to CruiseControl and my older DIY solution. If needs be, I'd be open to the idea of separate Hudsons for separate projects, each on their own port - Galileo on :80, Dash on :81, etc. But of course Hudson can handle many projects and build configs thru a single UI, so that'd only be necessary if we don't trust each other w/ a shared UI.
If you look at what the Apache project has done, they have multiple projects on the same ui. From the web, everyone can see what builds are running but you need to authenticate to administer builds. I'm not sure if they have a further level of granularity once you authenticate, can't tell. http://hudson.zones.apache.org/hudson/ One thing I noticed on the build.eclipse.org/hudson that once I authenticated as a committer, I had rights to change the configuration of the hudson server etc. which should probably be limited to the webmasters. I was looking at Hudson in the context of using it within the CBI for our project. I trust other committers, however, it would be makes sense to limit the ability the change the parameters of the build, kick off builds to those committers within that project. I've reopened bug 260021 and added it as a dependancy on the Dash bug 251945 (replace custom web UI with Hudson).
(In reply to comment #24) > One thing I noticed on the build.eclipse.org/hudson that once I authenticated > as a committer, I had rights to change the configuration of the hudson server > etc. which should probably be limited to the webmasters. Speaking of which, can we get email configured properly? https://build.eclipse.org/hudson/configure
Mail should now be working correctly. -M.
(In reply to comment #26) > Mail should now be working correctly. > > -M. It is, thanks. Two things remain to be done: 1. Links sent in email do not work, as they begin with http://build.eclipse.org:8443/ and should begin with https://build.eclipse.org/ I have access to the configuration area, so can change it myself if that's OK. 2. Matt created a directory for publishing "staging" builds, which can later be manually be copied to the main /download/releases/galileo directory (unfortunately, we can't allow the hudsonbuild user to copy there using Hudson itself). This "staging" directory needs to be accessible in order to test builds before promoting. If it is already, I don't know what URL to use. Oh, and I can't access this ~/hudsonbuild/downloads directory with my userid.
> I have access to the configuration area, so can change it myself if that's OK. Sure. > Oh, and I can't access this ~/hudsonbuild/downloads directory with my userid. D'oh, missing x permission on ~hudsonbuild . I've fixed that and verified that I can use your account to access the ~hudsonbuild/downloads area and create files. -M.
Regarding comment #24 here's a link to an article that describes how to configure hudson so users are limited to starting/configuring etc builds on a per project basis. http://weblogs.java.net/blog/johnsmart/archive/2008/09/hudson_projectb.html
(In reply to comment #29) > Regarding comment #24 here's a link to an article that describes how to > configure hudson so users are limited to starting/configuring etc builds on a > per project basis. If we can configure it this way and allow each user to use their own identity to run builds, this would fix the current problem that we cannot promote to the download server using Hudson. Also, I expect it will resolve the current permission issues we have in our build directories.
Changing the bug summary to reflect the reason this is still open. I'd like to get this done soon-ish.
Update: I've been working with a local copy of Hudson to get the kinks worked out in the required setup. Unfortunately I've run into a null pointer exception when trying to use ldap in the 'Project based matrix' setup. It looks like Hudson is trying to bind as the user/group instead of searching when trying to add users/groups to specific projects. I've grabbed a copy of the code, and I'm going to look into what is required fix this. -M.
Thanks for looking into this, Matt. It'll really help to have people able to view test reports, etc.
Ok, so after determining that my local setup was the first problem, and Hudson needing an update was the second, I have a test server running. Here's the plan: We will create an 'admin' group for Hudson, just like we have for PlanetEclipse. This group of committers will be responsible for managing Hudson projects. With Hudsons security setup global permissions override per project settings. So an admin group lets us limit who has global access, and still allows per-project control. When a project wants a new Hudson project, they'll file a bug against a component that Webmaster will set up, and the admin group will create it and set the per project security settings to allow the project group total control of their build. All committers will have read only access via the common group committers are part of. We'll also enable the anonymous user in read only mode. Any volunteers? -M.
(In reply to comment #34) > Any volunteers? Me.
(In reply to comment #34) > > Any volunteers? > I'll help. As long as I'm not the only one :) And someone "teaches" me what's needed. (I'm assuming it's mostly adding a list of ids to some conf file and restarting server?) So, if I can help, fine ... but won't be insulted if you find "real" admins that have experience administering production servers.
I'll volunteer too. I have experience administering an internal hudson server.
Ok, when the current build is finished I'll update Hudson, setup the security and send you all a note on what you'll need to do(really just for David ;). I suspect that there will be some 'pain' over the next couple of days as we get all of the permissions right. However I'll start by adding the callisto-dev group to all of the current projects, so that should minimize disruption. -M.
Ok, we should be all set here. I've updated Hudson, turned on security and added Kim,Andrew and David to the admin group. I'll add an ACL to grant the hudsonbuild user access to the build.eclipse.org staging areas. -M.
(In reply to comment #39) > Ok, we should be all set here. I've updated Hudson, turned on security and > added Kim,Andrew and David to the admin group. Can I be added too?
Created attachment 131855 [details] screenshot - how to add/edit users of a given Hudson job > And someone "teaches" me what's needed. 0. Load up Hudson. https://build.eclipse.org/hudson/ 1. Log in. https://build.eclipse.org/hudson/login?from=/hudson/ 2. Choose the job you want to administer, eg., cbi-linuxtools-Ganymede. https://build.eclipse.org/hudson/job/cbi-linuxtools-Ganymede/ 3. Click 'configure'. https://build.eclipse.org/hudson/job/cbi-linuxtools-Ganymede/configure 4. Note the new section called "Enable project-based security" 5. Add user(s) and grant them permissions. If you hover over the column titles you'll get details re: what each permission means. Build, Workspace, and Update seem like the most useful permissions for committers. Configure & the two Deletes are more for releng people.
Great to see this feature coming in - many thanks. I'm happy to help too, I'll be setting up the STP builds and can pitch in with the general stuff as required.
I think we're done here. Please feel free to reopen if I've missed something. -M.
