Re: [cross-project-issues-dev] Hudson shutdown wait from hell

[Denis Roy <denis.roy@xxxxxxxxxxx>]

On 02/08/2012 02:49 PM, David M Williams wrote:
> In general, I'll say there's been very little communication about the state
> of Hudson 

>From the webmaster's headquarters here in freezing cold Ottawa, here's
what I know:

-- Matt is investigating running Hudson on Jetty instead of Winstone. 
That effort is progressing slowly (ie, it doesn't appear to be that
simple).  Many thanks to Jesse and Winston for their support here.

-- We've reduced the number of executors on slave1 to 3.  It definitely
does not like having 8 executors, although the box can definitely
support the load.

-- We've set up slave6 with 1 executor.  Slave5 has been set up with 2
executors, but since it's an IA64

-- Once we have enough slaves, we'll take slave1 down and split it into
smaller slaves with one executor each

-- IBM has *once again* provided us with free hardware for builds, with
two generously-equipped POWER7 servers.  The catch is that these servers
are in the Oregon State University's Open Source Lab (OSUOSL).  We'll be
setting these up as slaves as soon as we have the cycles to do so.  Many
thanks to IBM for providing us with hardware.

-- We're currently running Hudson 2.1.2... We should perhaps upgrade to
2.2.0, or perhaps even use the 3.0.0 milestone that is available at
Eclipse...  Any thoughts?

-- We don't have all the disk bottlenecks we had last year.  That's good
news  :)

Denis


> .... seems several comments made or questions asked on this list
> with no response ... Hudson restarted, configuration changes all without
> comment. Is there some resistance to that? Everyone too busy to
> communicate? Is their a better channel? Hudson bug list? Do we need a
> "hudson-and-infrastructure" mailing list? Just asking.
>
> The queue is down to 26 now ... in the 10 minutes I took to write this
> note ... if that rate holds, it will be clear in 90 minutes or so? :/
>
> Thanks,
>
>
>
>
>
>
> From:	Miles Parker <milesparker@xxxxxxxxx>
> To:	Cross project issues <cross-project-issues-dev@xxxxxxxxxxx>,
> Date:	02/08/2012 02:22 PM
> Subject:	[cross-project-issues-dev] Hudson shutdown wait from hell
> Sent by:	cross-project-issues-dev-bounces@xxxxxxxxxxx
>
>
>
> Hi all,
>
> I wanted to bring up a Hudson annoyance and see if people had ideas for
> improving this. What's happening now is that any time Hudson gets sent a
> shutdown, everyone is locked out until the last job in queue finishes.
> Which is a) Good news for the people with running builds, b) Bad news for
> everyone else. Observing that most of the time most of us are  in category
> b) I vote for making things work better for group b). Nothing against
> PTP :D, but they happen to be in group a) this time around and the last run
> took 22 hours. :O But there are lot's of long builds out there. This means
> that snapshots are delayed for everyone.
>
> So I'm wondering if it might be possible to have some kind of policy where
> builds are terminated with prejudice under shutdown. I'm not sure if a)
> this is even supportable OOTB in Hudson, and b) whether that would have the
> possibility of FUBAR'ing anyone project builds. As project builds should
> not be relying on previous state, I would say that the answer to b is
> probably no. I also imagine allowance should be made for key builds such as
> aggregator. IIRC in the past when in release panic mode we were triggering
> hard shutdowns from time to time.
>
> thoughts?
>
> Miles