Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [wtp-dev] Stability of I-builds during release shutdown


I think the issue here is that adopters of WTP need a two things:

1) they need stability so they can develop new function instead of catching up with interface changes
2) they need fast turnaround time for critical fixes so they can proceed with testing

These are valid requirements and we need to start satisfying them at this point in the cycle. We need to stop making interface changes and focus exclusively on quality. If we do that then our I-builds will have high quality and adopters will be able to move to them quickly.

I suggest we continue with weekly I-builds. I believe no one is planning any interface changes at this time. Let's monitor the situation and take a checkpoint a week after M5 is released. I'd like to ask Tim and Thomas to immediately flag any regressions during this time so we can understand the cause.

Arthur Ryman,
Rational Desktop Tools Development

phone: +1-905-413-3077, TL 969-3077
assistant: +1-905-413-2411, TL 969-2411
fax: +1-905-413-4920, TL 969-4920
mobile: +1-416-939-5063, text: 4169395063@xxxxxxx
intranet: http://labweb.torolab.ibm.com/DRY6/



Timothy Deboer/Toronto/IBM@IBMCA
Sent by: wtp-dev-bounces@xxxxxxxxxxx

06/29/2005 08:50 AM

Please respond to
"General discussion of project-wide or architectural issues."

To
naci.dai@xxxxxxxxxxxxx, "General discussion of project-wide or architectural issues." <wtp-dev@xxxxxxxxxxx>
cc
Subject
Re: [wtp-dev] Stability of I-builds during release shutdown






Hi Naci,


I agree about N-builds - they are only useful for getting the very latest changes, with absolutely no stability guarantee. I do not think we should change our N-builds at all.


What I think both Thomas and I are getting at is that for external teams, weekly I-builds are not enough. If somebody is waiting for a change or a bug fix and it is done incorrectly or misses the next I-build, you can easily wait two weeks to get a build that you can use. During early milestones we might be able to slow down the I-builds; during intermediate milestones once a week is ok; but during release shutdown this lag in the cycle is too much - our turnaround time for builds will keep people from verifying bugs or getting through a complete scenario with WTP.


The Eclipse platform does an RC-build, stops producing I-builds for a couple days so that people can test, and then does an I-build every few days (spinning until clean) until the next RC. All I'm suggesting is that we do something similar to ensure that we have a stable RC- or I-build at least once a week, and preferrably more like twice. Since we'll be in shutdown, it should be easier (and safer) for component leads to release their map files more often.


Thanks,

Tim deBoer
WebSphere Tools - IBM Canada Ltd.
(905) 413-3503  (tieline 969)
deboer@xxxxxxxxxx


Naci Dai <naci.dai@xxxxxxxxxxxxx>
Sent by: wtp-dev-bounces@xxxxxxxxxxx

06/29/2005 08:02 AM

Please respond to
naci.dai and "General discussion of project-wide or architectural issues."

To
"General discussion of project-wide or architectural issues." <wtp-dev@xxxxxxxxxxx>
cc
Subject
Re: [wtp-dev] Stability of I-builds during release shutdown







I will have to add to the discussion  as some of the suggestions are not workable.

N-Build, or a nightly build, is a build constructed from the cvs head directly.  This build at the best of the times indicate the status of the latest changes.  It is not meant for adoption and/or testing, but it will give an indication of the current status of the code.  The fundamental property is that this builds configuration is *not* managed.  An n-build is not reproducible.  Current N-Build frequency is once every 12 hours if there are any changes.  However, N-Builds have not been used very successfully in the last 10 months.  N-Builds do not offer good feedback to produce I-Builds.

I-Build, the integration build is configuration managed build.  The component owners release their cvs tagged code into this build.  These are maintained in a set, called the build map.  Each integration build has a unique cvs tagged mapped.  It is reproducible.  An explicit cross-component effort and testing takes place to declare  an I-Build.  Only way to get feedback on the quality and status of an I-Build is to try to produce that I-Build repeatedly until satisfied.

Back to the main point,  we should and do have only one Integration build per week.  The fact that there are many trial I-Builds produced in the open confuses the issue, we will start restricting access to these warm up/trial builds, as they are only meant as a feedback to the committers to fix the problems before the I-Build is declared.

One I-Build per week maybe too frequent in itself, it gives little breathing room.  However, this was necessary because of the loose coordination between the teams, and number of changes accumulated in two weeks made the integration harder and often impossible.  We can switch to a longer frequency after we are convinced that the code base has stabilized to afford a longer integration period.  Currently the number of changes each week are massive to keep the time short.  Remember the M1 & M2 horror, they were the integration builds due to the lack of integration builds.

Post R0.7, wtp adopters can use M-builds, or communicate the I-Build they are based on so that we continue maintaining it to assist less painful adoption.

Finally,  the quality of an I-Build can only be good (all tests pass, smoke testing approved, component owners approve, etc),  and, it should not regress wrt to a prior build.  We had hard time achieving this goal due to test quality issues and api changes, and relaxed our policy during the M5 cycle.  Relaxed means we declared some builds with test failures.  Post M5, we should activate the strict policy again.

My experience trying to improve I-Build quality by suggesting good practices over email has been pretty much useless, we will have to enforce the practice locally within each team.  Peer-pressure and good quality feedback  is the mechanisms for achieving the level of quality we seek.  WTP is the second largest eclipse project (platform being the first), and probably the largest geographically in terms of number of teams on how they are distributed so we will  have to find the different ways to manage it ourselves.

In terms of build frequency, can I have people declare opinions on the following survey:

a) Produce an I-Build  once a week (Thursday - as usual)

b) Produce an I-Build  once every two weeks and more only when needed.


I also support frequent *trial*  I-Builds (4 hours etc. more if needed), prior to declaring each build.  Doing this the day before is too short as the team distribution crosses the day-night barrier so we will have to extend it to at least 48 hours.

Yes. We observed similar pattern.

 
Adopt an I-Build is a very costly process for us. It is measured in man-days.
 
Every time, we adopt I-Build that doesn’t go as far as the previous one, we take big hit on productive and we would not able to proceed with some implementation. Without proceed with implementation, we won’t discover critical bug that need to discover before 0.7.
 
Because we only have 2 weeks, I would think that we want to go one level further. We should respin nightly build every 4 hours, and make it the first priority to have all compile error and JUnit test fixed once it is discovered in nightly build. And, all JUnit test should be deemed critical.

 
This suggestion might appear to be costly. But, I believe the saving from the unstable build’s cascade effect is beyond the cost. After all, compile error and JUnit tests need to be fixed anyway. Fixing it early enable people/team in the downstream to start working on that earlier.

 
In short, I counter propose:

 
1/ We move the expectation of a nighlty build up, as what we would had expected from a I-Build

 
2/ We respin the nightly build every 4 hours.

 
3/ We fix all compile error and all JUnit error within a I-Build.

 
4/ We produce a warm-up I-Build on Thursday.

 
 
Thomas

 
 





From:
wtp-dev-bounces@xxxxxxxxxxx [mailto:wtp-dev-bounces@xxxxxxxxxxx] On Behalf Of Timothy Deboer
Sent:
Tuesday, June 28, 2005 6:19 PM
To:
wtp-dev@xxxxxxxxxxx
Subject:
[wtp-dev] Stability of I-builds during release shutdown

 

Hi everyone,


The 0.7 release candidate builds are proposed for the 13th and 22nd, with the release at the end of the month. What I haven't seen is what happens between these times - presumably we are going to run nightly I-builds or something similar. My concern is that with constant I-builds typically comes constant churn - every team ends up taking a turn at breaking the build and we have brown-outs that are nobody's fault - most of the build is good, but there are some less stable parts which change from build to build. Our past I-builds are a good example - although they are fairly stable on Tuesday, we typically have a series of build and JUnit failures until sometime Thursday when everyone starts focusing on it.


To ensure that we have frequent, high-quality builds that can be used for testing and external teams building on top of WTP, I propose one of the following:


1. We do less frequent I-builds (e.g. every 3-4 days), but use warmups like we have done with previous I-builds. For example, we could start spinning I-builds on every Monday morning and spin every 4 hours until we get it clean, then do it again on Thursday.


2. We stay vigilant - every nightly I-build should be as good as the previous milestone/release candidate. Any compile errors or JUnit failures are deemed critical and must be fixed asap by a respin.


Thought? Opinions? Other suggestions?


Thanks,

Tim deBoer
WebSphere Tools - IBM Canada Ltd.
(905) 413-3503  (tieline 969)

deboer@xxxxxxxxxx




_______________________________________________
wtp-dev mailing list

wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev
 


--
Naci Dai,
eteration a.s.
Inonu cad. Sumer sok. Zitas D1-15
Kozyatagi, Istanbul 34742
+90 (533) 580 2393 (cell)
+90 (216) 361 5434 (phone)
+90 (216) 361 2034 (fax)

http://www.eteration.com/
mailto:nacidai@xxxxxxx
mailto:naci@xxxxxxxxxxxxx_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev

_______________________________________________
wtp-dev mailing list
wtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/wtp-dev


Back to the top