>Date: Wed, 17 May 2006 16:05:58 -0700
>From: "Tim Buss" <TBuss@xxxxxxxxxx>
>Subject: [alf-dev] Build SCM Use Cases
>To: "ALF Developer Mailing List" <alf-dev@xxxxxxxxxxx>
>Content-Type: text/plain; charset="us-ascii"
>As discussed today in our meeting rather than too course grained it
>seems perhaps we are focusing on detail a bit too much and should
>probably consider (at least briefly) more scenario level use cases to
>get at what is important for an ALF vocabulary. The use cases currently
>in play are at a function level and its easy to get bogged down in
>detail while missing the big picture.
>With that in mind we came up with a few "scenarios" and discussed some
>1. Scheduled Build
>The build is schedule to happen at a particular time. It is general
>configured to build a particular configuration (build this version of
>these components) or follow a particular predefined pattern (eg. Lable
>latest, get it and build it). The usual purpose of this kind of build
>is to provide a known version for the next day's work
As a detail, scheduled builds can be either recurring, or one-time.
>2. On Demand Build
>This build is unscheduled and occurs at the request of a user or tool
>for ad hoc reasons. It may follow a predefined configuration or be
>parameterized to allow the build configuration to vary with the build.
>The usual purpose of this kind of build is to handle unusual
>circumstances (eg the nightly scheduled build failed - tgb. not really
>"unusual" I suppose :) 0
I would not limit describing on-demand builds to just "unusual"
circumstances. Reaching a certain milestone may be reason to do
initiate a build for example. And as you point out, an automated
build failing is not unusual.
>3. Triggered (Continuous) Build
>This build is unscheduled and occurs, typically, as the result of an
>event being raised. A typical event may be caused be changes to the
>code stored in the SCM. It will typically be configured to build the
>latest of a particular set of code.
There are two underlying methods to achieve this from a functional
perspective- polling method and triggers/raised events. They each
have their pros and cons, but generally I think the trigger method is
more efficient and has more benefits. Typically polling is done by
the build tool, whereas a trigger would be initiated by the SCM tool.
>4. Patch Build
>This build may be scheduled or unscheduled and is generally a subset of
>a larger build. The main difference seems to be how it is specified (ie
>the query used to get the appropriate source code) and the likleyhood
>that it will be associated with a difference report.
So in 1-3 are outlined ways to initiate a build at a functional level,
and with #4 now this is more of a category or type of build. I agree
that typically a patch build would not be initiated via a continuous
build event. However there are other categories of builds you may
want to consider if you are introducing such a notion:
I don't know how deep you want to go into this, but you can see
introducing categories of builds opens up the discussion quite a bit :-)
At the end of the day all these types of builds are initiated in 1
of 3 ways outlined earlier, but they all have different use cases,
some varying more than others.
>5. Checkin of Derived Objects
>Really a variation rather than a scenario. Some build need to checking
>the objects that they produce for various reasons
There are some build categories where this makes sense, if you want
to preserve the derived objects created from the build.
Other categories of builds, like an incremental build, you do not
want to necessarily check in all the derived objects if they existed
prior to the build. A simple timestamp comparison of the derived
object relative to the beginning build time may resolve this however
in the build logic itself. Debug builds are another example- these
derived objects may have extra compiled data in the executables that
could behave differently, such as writing to temporary files, produce
extra system or warning messages, expose internal data within the
application, etc. just for the purpose of a particular troubleshooting
exercise. The common use case is you provide a user or customer a
debugged version of an application your software team develops to help
uncover an issue. It may be applied as part of a packaged install,
but it could be just a standalone artifact like a DLL or EXE that the
user copies to the designated location manually. These types of
debug-built objects are transitory in nature, and you may not want to
save them, especially since they are typically much larger than their
corresponding non-debug versions, and may have no use beyond that
troubleshooting exercise. A debug build would be almost exclusively
>It seems from this that builds differ in three main ways
>1. What triggers them: scheduled, triggered, on demand
>2. How they are specified. What "query" is used to define the source to
This can be described as the scope of the build possibly as well.
This also encourages best practices to have all needed artifacts for
a build under source control. It may be just me on the use of the
term "query" , but that is not a standard SCM way to think about
retrieving files to populate a workspace, even if the tool in fact
uses an underlying RDBMS to store them internally :-) But I get the
idea of what you are saying, just trying to think if there is a better
way to describe it.
>3. What they do with the result. Check it in, prepare difference report
I would say builds also differ by category, which corresponds at a
higher level to the underlying reason for the build and how will it be
used or consumed. This may even be aligned with the business
objectives of the organization in addition to internal consumers of
the build output. A very common use case for internal build output
consumption is to initiate a suite of test cases, smoke tests, upon a
successful build. Touching on the testing domain is out of the scope
of discussing use cases for how SCM and build interact, but there is
no doubt that build and test are tightly coupled within the
application software lifecycle because test consumes the build results.
I would add that builds also need to provide a status result that in
the ALF world would be an event raised - typically success or failure.
Success could be broken down further into two sub-categories:
warnings (non-fatal errors, conditions, etc.) or no warnings. However
I don't think the SCM system would need to monitor or act on such
build status result - the SCM system is passive when dealing with the
build system except for the case of continuous builds where it directs
the build system to do something (is my assumption there correct?)
I want to note here that an important task builds must perform also
includes environment setup among other things, but that happens
outside the scope of the SCM repository. So making sure a particular
library version is installed at the right location could be one such
example. This library can be under version control, which is
extracted first into the workspace as part of the build process, then
from there the build logic takes care of any necessary setup. This
also implies that the build logic must also keep track of the results,
should that involve the SCM system again later in the process such as
checking in files mentioned in (3) above.
>>From 1 it seems that we will need an SCM to raise an ALF event when
>things stored in the SCM are changed. It seems to me that this is
>related to the object we have defined and we could define ALF events for
>changes to a subset of those objects. We need to explore what this
This is the right way to think about it- you then need to map a
define object representing a set of files in the SCM system to a
corresponding build to initiate.
>Looking at the list I might guess that Element (NewVersion), Change-Set
>(creation, update), Branch/Stream (creation, update), Configuration
>(creation, update), Baseline (creation, update?), Component (creation,
>update) as possible events that would trigger a build. This seems
>fairly straight forward with the right intensity of hallucination.
Basically anything that changes the source namespace, or the
contents of the namespace, should trigger a build. Some of these
metadata items associated with SCM may or may not result in a build
being needed. And actually it gets a bit more complicated than that
depending on how you define "source". For example, it is a best
practice to version documentation with source code on a project and
logically group versions of them together somehow with a label or
baseline, but you don't want to initiate an unnecessary build when
your doc set changes. So something to think about is a way to exclude
or ignore certain files from triggering a build.
>The "query" aspect seems to imply that we may need to define an ALF SCM
>"query language" that could be used for Checkout/Lock, Get, Report etc.
>and carry over to CheckIn, Lable etc. This is probably not a shock but
>may be a bit more onorous than the first blush "let's have a vocabulary"
>statement. A structured query will probabably be a better approach to
>use with Web services than a object system that represents the query.
Is defining a query language more complex than just defining a small
set of common SCM operations that need to be supported?
I guess until we flesh out the other domain interactions with SCM such
as tracking, test, deploy, etc. we don't really know the full set of
use cases of an outside service requesting something from the SCM
system, as well as what types of events the SCM system should be
raised for another application/service to act on that event. The good
news is I think most of the SCM tools can support common basic file /
version type of operations. Introducing task-based SCM into the
picture is where are the lowest common denominator starts to go away.
>"101 things I might do with this build" also seems fairly straight
>forward to define. Things I want to do seem to be limited to the
>complete set of non-admin SCM functions :) - Get, Check out, Check in,
>New Check in, Lable, Report etc.
>>From this I conclude that perhaps our to date current approach address
>item 3 and we should continue that but we also need to examine the
>"query" and "event" definitions to come close to a full picture. If
>this makes sense then someone(s) with some SCM domain knowledge should
>probably have a go at these sooner rather than later.
To examine all the possible event definitions relating to SCM, you
need to analyze all the other constituents, stakeholders, domains,
etc. in the envisioned ALF ecosystem that would need to interact with
SCM before you have your full picture.
Between build and SCM specifically, identifying the use case
categories of builds might be helpful to get the big picture, but I
think the underlying SCM actions they boil down to for initiating a
service request, or having the SCM system raise an event, will be very
common, again not factoring in task-based SCM scenarios. You can only
initiate a build a certain number of ways, and the set of files the
build needs to act upon or depends upon are in the SCM system that you
need to extract somehow. When you are done building, you may or may
not need to manipulate the resulting derived objects.
Director of Product Management
Buildforge / IBM Rational Software
512 225 0436
alf-dev mailing list