>Date: Wed, 17 May 2006 16:05:58 -0700
From: "Tim Buss" <TBuss@xxxxxxxxxx>
Subject: [alf-dev] Build SCM Use Cases
To: "ALF Developer Mailing List" <alf-dev@xxxxxxxxxxx>
Message-ID:
<55646D9DC2748E4CACDD632FF2E53982019FEAED@xxxxxxxxxxxxxxxxx>
Content-Type: text/plain; charset="us-ascii"
As discussed today in our meeting rather than too course grained it
seems perhaps we are focusing on detail a bit too much and should
probably consider (at least briefly) more scenario level use cases to
get at what is important for an ALF vocabulary. The use cases currently
in play are at a function level and its easy to get bogged down in
detail while missing the big picture.
With that in mind we came up with a few "scenarios" and discussed some
variations.
1. Scheduled Build
The build is schedule to happen at a particular time. It is general
configured to build a particular configuration (build this version of
these components) or follow a particular predefined pattern (eg. Lable
latest, get it and build it). The usual purpose of this kind of build
is to provide a known version for the next day's work
As a detail, scheduled builds can be either recurring, or one-time.
2. On Demand Build
This build is unscheduled and occurs at the request of a user or tool
for ad hoc reasons. It may follow a predefined configuration or be
parameterized to allow the build configuration to vary with the build.
The usual purpose of this kind of build is to handle unusual
circumstances (eg the nightly scheduled build failed - tgb. not really
"unusual" I suppose :) 0
I would not limit describing on-demand builds to just "unusual"
circumstances. Reaching a certain milestone may be reason to do
initiate a build for example. And as you point out, an automated
build failing is not unusual.
3. Triggered (Continuous) Build
This build is unscheduled and occurs, typically, as the result of an
event being raised. A typical event may be caused be changes to the
code stored in the SCM. It will typically be configured to build the
latest of a particular set of code.
There are two underlying methods to achieve this from a functional
perspective- polling method and triggers/raised events. They each
have their pros and cons, but generally I think the trigger method
is more efficient and has more benefits. Typically polling is done
by the build tool, whereas a trigger would be initiated by the SCM
tool.
4. Patch Build
This build may be scheduled or unscheduled and is generally a subset of
a larger build. The main difference seems to be how it is specified (ie
the query used to get the appropriate source code) and the likleyhood
that it will be associated with a difference report.
So in 1-3 are outlined ways to initiate a build at a functional
level, and with #4 now this is more of a category or type of build.
I agree that typically a patch build would not be initiated via a
continuous build event. However there are other categories of
builds you may want to consider if you are introducing such a
notion:
unit builds
integration builds
incremental builds
system builds
clean builds
test builds
release builds
production builds
milestone builds
nightly builds
special builds
debug builds
etc.......
I don't know how deep you want to go into this, but you can see
introducing categories of builds opens up the discussion quite a
bit :-)
At the end of the day all these types of builds are initiated in
1 of 3 ways outlined earlier, but they all have different use
cases, some varying more than others.
5. Checkin of Derived Objects
Really a variation rather than a scenario. Some build need to checking
the objects that they produce for various reasons
There are some build categories where this makes sense, if you
want to preserve the derived objects created from the build.
Other categories of builds, like an incremental build, you do not
want to necessarily check in all the derived objects if they
existed prior to the build. A simple timestamp comparison of the
derived object relative to the beginning build time may resolve
this however in the build logic itself. Debug builds are another
example- these derived objects may have extra compiled data in the
executables that could behave differently, such as writing to
temporary files, produce extra system or warning messages, expose
internal data within the application, etc. just for the purpose of
a particular troubleshooting exercise. The common use case is you
provide a user or customer a debugged version of an application
your software team develops to help uncover an issue. It may be
applied as part of a packaged install, but it could be just a
standalone artifact like a DLL or EXE that the user copies to the
designated location manually. These types of debug-built objects
are transitory in nature, and you may not want to save them,
especially since they are typically much larger than their
corresponding non-debug versions, and may have no use beyond that
troubleshooting exercise. A debug build would be almost
exclusively initiated on-demand.
It seems from this that builds differ in three main ways
1. What triggers them: scheduled, triggered, on demand
2. How they are specified. What "query" is used to define the source to
get.
This can be described as the scope of the build possibly as well.
This also encourages best practices to have all needed artifacts
for a build under source control. It may be just me on the use of
the term "query" , but that is not a standard SCM way to think
about retrieving files to populate a workspace, even if the tool in
fact uses an underlying RDBMS to store them internally :-) But I
get the idea of what you are saying, just trying to think if there
is a better way to describe it.
3. What they do with the result. Check it in, prepare difference report
etc.
I would say builds also differ by category, which corresponds at
a higher level to the underlying reason for the build and how will
it be used or consumed. This may even be aligned with the business
objectives of the organization in addition to internal consumers of
the build output. A very common use case for internal build output
consumption is to initiate a suite of test cases, smoke tests, upon
a successful build. Touching on the testing domain is out of the
scope of discussing use cases for how SCM and build interact, but
there is no doubt that build and test are tightly coupled within
the application software lifecycle because test consumes the build
results.
I would add that builds also need to provide a status result that
in the ALF world would be an event raised - typically success or
failure. Success could be broken down further into two
sub-categories: warnings (non-fatal errors, conditions, etc.) or no
warnings. However I don't think the SCM system would need to
monitor or act on such build status result - the SCM system is
passive when dealing with the build system except for the case of
continuous builds where it directs the build system to do something
(is my assumption there correct?)
I want to note here that an important task builds must perform
also includes environment setup among other things, but that
happens outside the scope of the SCM repository. So making sure a
particular library version is installed at the right location could
be one such example. This library can be under version control,
which is extracted first into the workspace as part of the build
process, then from there the build logic takes care of any
necessary setup. This also implies that the build logic must also
keep track of the results, should that involve the SCM system again
later in the process such as checking in files mentioned in (3)
above.
From 1 it seems that we will need an SCM to raise an ALF event when
things stored in the SCM are changed. It seems to me that this is
related to the object we have defined and we could define ALF events for
changes to a subset of those objects. We need to explore what this
means.
This is the right way to think about it- you then need to map a
define object representing a set of files in the SCM system to a
corresponding build to initiate.
Looking at the list I might guess that Element (NewVersion), Change-Set
(creation, update), Branch/Stream (creation, update), Configuration
(creation, update), Baseline (creation, update?), Component (creation,
update) as possible events that would trigger a build. This seems
fairly straight forward with the right intensity of hallucination.
Basically anything that changes the source namespace, or the
contents of the namespace, should trigger a build. Some of these
metadata items associated with SCM may or may not result in a build
being needed. And actually it gets a bit more complicated than
that depending on how you define "source". For example, it is a
best practice to version documentation with source code on a
project and logically group versions of them together somehow with
a label or baseline, but you don't want to initiate an unnecessary
build when your doc set changes. So something to think about is a
way to exclude or ignore certain files from triggering a build.
The "query" aspect seems to imply that we may need to define an ALF SCM
"query language" that could be used for Checkout/Lock, Get, Report etc.
and carry over to CheckIn, Lable etc. This is probably not a shock but
may be a bit more onorous than the first blush "let's have a vocabulary"
statement. A structured query will probabably be a better approach to
use with Web services than a object system that represents the query.
Is defining a query language more complex than just defining a
small set of common SCM operations that need to be supported?
I guess until we flesh out the other domain interactions with SCM
such as tracking, test, deploy, etc. we don't really know the full
set of use cases of an outside service requesting something from
the SCM system, as well as what types of events the SCM system
should be raised for another application/service to act on that
event. The good news is I think most of the SCM tools can support
common basic file / version type of operations. Introducing
task-based SCM into the picture is where are the lowest common
denominator starts to go away.
"101 things I might do with this build" also seems fairly straight
forward to define. Things I want to do seem to be limited to the
complete set of non-admin SCM functions :) - Get, Check out, Check in,
New Check in, Lable, Report etc. >
From this I conclude that perhaps our to date current approach address
item 3 and we should continue that but we also need to examine the
"query" and "event" definitions to come close to a full picture. If
this makes sense then someone(s) with some SCM domain knowledge should
probably have a go at these sooner rather than later.
To examine all the possible event definitions relating to SCM,
you need to analyze all the other constituents, stakeholders,
domains, etc. in the envisioned ALF ecosystem that would need to
interact with SCM before you have your full picture.
Between build and SCM specifically, identifying the use case
categories of builds might be helpful to get the big picture, but I
think the underlying SCM actions they boil down to for initiating a
service request, or having the SCM system raise an event, will be
very common, again not factoring in task-based SCM scenarios. You
can only initiate a build a certain number of ways, and the set of
files the build needs to act upon or depends upon are in the SCM
system that you need to extract somehow. When you are done
building, you may or may not need to manipulate the resulting
derived objects.
Thoughts, Volunteers?
Thanks,
Tim Buss
Serena
Doug Fierro
Director of Product Management
Buildforge / IBM Rational Software
dfierro@xxxxxxxxxx
512 225 0436
http://www.buildforge.com
------------------------------------------------------------------------
_______________________________________________
alf-dev mailing list
alf-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/alf-dev
