[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
RE: [alf-dev] The role of source control in ALF
|
Clearly there are some challenges coming up with a reasonable common
semantic definition. My guess is that much of this has to do with how
you address elements, either individually or as a group, stored SCM. In
a general way this can be considered common to most operations you want
to perform in SCM. From this view, get and put are just some operations
to which the common addressing model applies. The details of how the
file transfers are done are just a detail
>From another view, fundamentally, an SCM system is a specialized file
store for which the primary use cases are "get" and "put" in some guise
or other. These are expensive operations and have some difficulties in
a distributed environment particularly when they apply to groups of
file. We have to address that since it would do us no good to define a
common semantic model for SCM and then find the performance issues make
its primary use impractical. I think this was the essence of Mark's
original point, said from a slightly different perspective.
Much of ALM is going to involve moving files so I definitely agree that
"get and "put" need to be available to ALF. We will need to figure out
practical ways to use them, how this effects the SCM vocabulary and
whether it implies the existence of some common service.
Tim
-----Original Message-----
From: alf-dev-bounces@xxxxxxxxxxx [mailto:alf-dev-bounces@xxxxxxxxxxx]
On Behalf Of Richard Title
Sent: Wednesday, March 08, 2006 6:43 AM
To: 'ALF Developer Mailing List'
Subject: RE: [alf-dev] The role of source control in ALF
Thanks, Tim. You make some good points.
However, here's the rub. You are talking about SCM systems as if they
are just file-servers. "Get me the source files" (so I can build, or run
tests, or whatever) is the kind of request you make of a file server.
SCM systems are more complex than that. Each files is versioned, and
possibly there are branches in the version graph. So the question you'd
need to ask an SCM system is more like:
"Get me the latest set of sources on branch BOB" [CVS, base ClearCase]
"Get me the set of sources tagged with label LARRY" [CVS, base
ClearCase] "Get me the set of sources corresponding to baseline BARRY"
[ClearCase UCM] "Get me the content of stream STAN [ClearCase UCM,
AccuRev] "Get me the set of sources corresponding to snapshot SNIDELY"
[AccuRev] ...(I'm sure there are more examples, once you look at more
SCM systems)
So now you have the problem of defining: what's a branch? What's a
label?
What's a baseline? What's a stream? What's a snapshot? It's not just a
matter of different SCM systems using different terms for the same
thing.
The actual *semantics* vary from one SCM system to another. Even when
the same term appears, the semantics might not be the same (e.g. a
ClearCase "stream" does not correspond exactly to an AccuRev "stream",
though they're conceptually similar).
Quoting from Brian Carroll's vocabulary draft: "The problem with getting
n different tools to communicate with each other is that, in the general
case, it requires n(n-1)/2 pair-wise data mappings. In short, the number
of integrations grows roughly as the square of the number of tools. But
the promise of ALF is that the cost be much lower - that the cost be
linear.
[...] That linear growth in cost can only be realized if the
participants in ALF agree on vocabularies that each will use."
In other words, it's fundamentally about standardization.
So for SCM systems, we need standardized definitions of things like
"branch"
and "label". And since these will be defined in terms of "version"s of
"source-controlled files" ("element"s in some SCM systems), these terms
need definitions as well.
Part of the reason I pointed to JSR 147 is that it attempts to define
terms such as these. They are defined operationally: For example, a
"version" is created by doing a "checkin" operation on some file. The
content of the file becomes the (immutable) version content. (That's
just to give the flavor...
the complete object model would take a lot more words to describe). I
think this is the only rigorous way to define terms, i.e. in terms of
the overall object model supported by the tool.
So, looping back to the original question on the table: I think you need
to include the individual file-level operations in ALF ("checkout",
"checkin"
on individual files) *even if no particular ALF tool will use these
low-level operations*. You need them as building blocks. The
higher-level operations ("get me [some specified set of source files]")
are ill-defined without the lower-level building blocks.
I hope this makes sense.
Richard Title
AccuRev
-----Original Message-----
From: alf-dev-bounces@xxxxxxxxxxx [mailto:alf-dev-bounces@xxxxxxxxxxx]
On Behalf Of Tim Buss
Sent: Tuesday, March 07, 2006 7:29 PM
To: ALF Developer Mailing List
Subject: RE: [alf-dev] The role of source control in ALF
Thanks for the discussion. Here are my thoughts.
The role of source control in ALF should be the same role that source
control plays in the the Application Lifecycle. Thus ALF is probably
not so likely to be concerned with creating a service flow to allow an
engineer to check out a single file to work on themselves but is very
likely to be concerened with checking a project out to create a build
for QA to test. That said it not so easy to say what will or will not
be important to any particular customer of ALF. It may be that in some
environments each file checkout should get logged somewhere central and
ALF could provide this by handing the event raised.
The issue of "get" target locations, really file transfer, is one of
performace and scalability. While I don't see any particular issue with
a web service being use to transfer a file of typical source code size,
it is not likely to be effective to have a BPEL engine copy large
amounts of data through the BPEL process. A better approach will likely
be to abstract the location so that it can be configured, perhaps as
part of a common service or perhaps as part of an assumed service define
by a service vocabulary and supported by the relavent tools. Maybe
both. In this way the BPEL engine can just request the data be "got" to
some given named location and in doing so, can assume that the data is
"got" to a place that the requestor can access.
A common service that abstracts such named locations seems like it might
be a good approach since I'm sure there are use cases beyond source
control where moving files around is necessary. The purpose of the
service would be to abstract the tools from knowing the exact nature of
the shared location (ie File share or ftp site or box of soft peat) and
being abot to transfer them localay or access them through a means that
is convenient to them (eg a file share) but maybe not to the source
service which knows only how to do ftp.
In the POC we made the assumption that all the tools could be configured
to operate on a know shared location. This is not an unreasonable
assumption in the real world but clearly we are taking advantage of a
feature that all the tools in the POC happened to have available to
them. It was not the goal of the POC to address this problem but to
prove that we could use SOA technology to do something that is
recognizable as part of ALM. The fact that the POC illustrates some
things that we will need to build to realize the vision is encouraging.
On the point about Ant and scripting in general, clearly there will be
some confusion. The POC use case could have been scripted using Ant or
windows batch files or other tools so, what, in fact, does it
illustrate? This is a good question that is partly answered by the goal
and assumptions of the POC. That said, for an individual developer
working alone, ALF is not going to compete with a script. However,
individual developers are not generally concerned with application
lifecycles. ALF can span machines, departments, locations and
potentially companies. Ant and other such scripting tools may remain
the most convenient way to perform certain point tasks. Allowing these
to be invoked via a web service brings these point task into an ALM
process providing a broader level of control and automation.
Tim
-----Original Message-----
From: alf-dev-bounces@xxxxxxxxxxx [mailto:alf-dev-bounces@xxxxxxxxxxx]
On Behalf Of Richard Title
Sent: Tuesday, March 07, 2006 9:02 AM
To: 'ALF Developer Mailing List'
Subject: RE: [alf-dev] The role of source control in ALF
Absolutely, you need both sorts of operations:
1. Get a version of an individual file (file content is streamed over as
part of the reply), and 2. Populate an entire workspace. This would be
more efficiently implemented by pointing to a target directory and
telling the SCM system "update"
(assuming it has access to the directory), as opposed to streaming all
the individual files through the SOAP interface.
It's not either/or. You can support both kinds of operations in the SCM
vocabulary.
Richard Title
AccuRev
-----Original Message-----
From: alf-dev-bounces@xxxxxxxxxxx [mailto:alf-dev-bounces@xxxxxxxxxxx]
On Behalf Of Eric Minick
Sent: Tuesday, March 07, 2006 11:19 AM
To: ALF Developer Mailing List
Subject: Re: [alf-dev] The role of source control in ALF
It's certainly possible to do web service file transfers and for a
single file it makes sense. But if tools from ALF partners like
Urbancode and ViewTier are going to really take advantage of ALF and if
we ever wanted to bring CruiseControl on board, we need to be able to
tell ALF "get me all the source for this project" and get back hundreds
of megs of source, libraries, documentation, etc, etc. A web service
might be able to do this, but it's a less obvious fit. The Borland guys
do some of this in the StarTeam API, but I suspect some ALF
implementations would have to resort to using the SCM's command line
tool under the covers. To me that says that all the services have to be
able to access some place on some disk, but I'm not sure they need to
all access the same one. Couldn't data be pushed out to wherever a given
service wants it?
Let's pretend CruiseControl is ALF enabled. How would that work?
Presumably, instead of polling the SCM for changes like it does now, it
waits for check-in notification ALF. More specifically, code committed
on the SCM raises an event and ALF determines that a build should
happen. It tells CruiseControl to build project x. Lovely. Now Cruise
needs the source out of CVS. It seems to me, that it should be able to
ask a local ALF interface to deliver a module, using some branch, to
some local directory. Whether that happens via web-service or running
some commands could be an implementation choice.
Ah, but there's the rub. How is an ALF enabled metric tool going to get
to the code and build artifacts that Cruise is sitting on? Either the
the build information must go out to the tool or the tool must have
access to that same spot on the physical disk. The performance centered
approach would be to have the tool on the same box with access to the
same files, but either would work.
-- Eric
Richard Title wrote:
>Again, why do you think it is a fundamental problem to return
>file-contents in a SOA? It's just a stream of bytes.
>
>By the way, another SCM-related standard worth looking at, in addition
>to JSR 147, is WebDAV DeltaV (see http://www.webdav.org/deltav/ ).
>Again, it's not SOAP, so it's not directly usable by ALF. But some of
>the ideas might
be
>useful. It's an extension of HTTP to support versioning. It does have a
>"GET" and "PUT" operations to obtain/update file/version contents.
>
>Richard Title
>AccuRev
>
>
>
>-----Original Message-----
>From: alf-dev-bounces@xxxxxxxxxxx [mailto:alf-dev-bounces@xxxxxxxxxxx]
>On Behalf Of Mark Phippard
>Sent: Tuesday, March 07, 2006 9:51 AM
>To: ALF Developer Mailing List
>Subject: RE: [alf-dev] The role of source control in ALF
>
>Scott,
>
>Thanks for replying. I will take a closer look at the requirements, I
>had only scanned them in the past.
>
>I didn't mean to imply that source control plays no role in ALF. For
>example, I can see the source control system sending events to ALF,
>such as when files are committed, so that ALF can perhaps initiate a
>service flow of some kind.
>
>What I was getting at is an issue that the POC demo actually raises. A
>lot of the tools and areas that we are talking about for ALF operate on
>source code. Would we ever envision a "GET" request delivering the
>results via an SOA? I do not even know how that could be done.
>WS-Attachments maybe? This seems impractical, so it seems like either
>all of these services are going to need access to the same physical
>location, or these tools are going to still need their own point to
>point interface to the source control system in order to get the source
code.
>
>So perhaps my points were two-fold:
>
>1) Source control-related vocabularly probably needs to focus more on
>what types of events a source control system should raise. Since this
>scenario obviously makes sense and can provide a lot of value.
>
>2) When trying to explain ALF to someone, maybe it would make sense to
>look for other areas to focus on other than source control?
>
>Related question:
>
>Would we envision a source control system being able to ask
>questiosn/seek approval via ALF? For example someone wants to checkout
>or check-in some source code. Could the source control system send a
>request to ALF to see if it was OK? For example, maybe the related
>issue has not been approved yet or something like that.
>
>Mark
>
>alf-dev-bounces@xxxxxxxxxxx wrote on 03/07/2006 09:38:42 AM:
>
>
>
>>I think you have to separate out a bit the use cases that are modeled
>>in
>>
>>
>the
>
>
>>POC and the long term possibilities integrating SCM and other tools.
>>
>>
>Issue
>
>
>>tracking is probably the most obvious: SCM integrates with ITS tools
>>to version and manage change packages/change palettes/change sets
>>(insert
>>
>>
>your
>
>
>>tool's nomenclature here) that address specific defects across
>>multiple release branches. But for many of the other tools in the ALM
>>suite, we
>>
>>
>think
>
>
>>there are plenty of cases where objects need to be properly versioned
>>
>>
>and
>
>
>>managed at the infrastructure level, and see SCM as fundamental to the
>>success of ALF-enabled workbench.
>>
>>The POC though has really only modeled the most superficial SCM use
>>
>>
>cases,
>
>
>>which I think reflects more urgent short-term priorities for the
>>project
>>
>>
>and
>
>
>>does not properly model the role of SCM and the SCM repository in the
>>overall process flow. Richard Title here at AccuRev has submitted some
>>
>>
>cases
>
>
>>to build out the story a bit (and attempted to map out some common
>>vocabulary among several leading SCM tools)--these are still pending
>>
>>
>review
>
>
>>from the requirements group--but I think these will be a start to
>>better representing the role of SCM in the suite.
>>
>>I believe Kelly has posted these cases on the ALF requirements
>>
>>
>site...take a
>
>
>>look and see what you think. We'd welcome some additional feedback and
>>
>>
>agree
>
>
>>wholeheartedly that the SCM needs some further attention in the ALF
POC.
>>
>>Best, Scott
>>
>>
>>
>>
>>Scott McGrath | Senior Product Manager AccuRev, Inc. | www.accurev.com
>>781.325.0652w | 617.834.2339m
>>
>>
>>-----Original Message-----
>>From: alf-dev-bounces@xxxxxxxxxxx [mailto:alf-dev-bounces@xxxxxxxxxxx]
>>
>>
>On
>
>
>>Behalf Of Mark Phippard
>>Sent: Tuesday, March 07, 2006 9:15 AM
>>To: ALF Developer Mailing List
>>Subject: [alf-dev] The role of source control in ALF
>>
>>Watching the POC demo and writing my web service for Subversion made
me
>>really start to question the role of source control in ALF. I am sure
>>that there will be cases where having it supported will be useful, and
>>having a service interface to a source control tool cannot hurt, but I
>>wonder how well it is really going to work in the real world. I think
>>
>>
>the
>
>
>>whole concept of an SOA kind of falls apart if ALL information that
>>
>>
>needs
>
>
>>to be exchanged cannot be done via the service interface. In the case
>>
>>
>of
>
>
>>source control tools, the information that needs to be exchanged is
>>ultimately the source itself. Do you eventually envision defining a
>>vocabulary or whatever, where the source control service would
actually
>>deliver the artifacts via the web service? That seems like it would
be
>>hard to do and probably be overly resource and network intensive.
>>
>>If all you can do is send an instruction to a source control tool
>>
>>
>telling
>
>
>>it to get source to some folder accessible via the web service that is
>>going to be very limited in how you can deploy and build your solution
>>
>>
>and
>
>
>>service flows. Take the POC demo as an example. You are doing a GET
of
>>
>>
>
>
>
>>the source code so that you can build it, test it and scan the source
>>
>>
>for
>
>
>>security bugs. This only works in the POC demo because all of the
>>services have access to the same location. How is ALF going to move
>>beyond that in the real world? Are users going to have to have all of
>>their source-related services deployed on the same server? It seems
>>
>>
>like
>
>
>>in a real world deployment you are going to wind up just using the
>>features that are built into OpenMake, or Ant or whatever build tool
is
>>used to let it go get the source from the source repository and then
>>
>>
>build
>
>
>>it.
>>
>>Is there a trick up your sleeve that I am not aware of, or is this
>>
>>
>always
>
>
>>going to be a real issue? If it is, then I think we need to start
>>defining alternate reasons why ALF is going to provide value. The
>>
>>
>problem
>
>
>>with the POC demo is that there are probably hundreds of tools already
>>
>>
>out
>
>
>>there that do the same thing much easier. The only parts of that demo
>>that you could not do with a simple Ant script are the parts related
to
>>the issue tracker and even that could be done relatively easily if the
>>issue tracker had Ant integration.
>>
>>Thanks
>>
>>Mark
>>
>>
>>
>>
>>
>>
>>_______________________________________________
>>alf-dev mailing list
>>alf-dev@xxxxxxxxxxx
>>https://dev.eclipse.org/mailman/listinfo/alf-dev
>>
>>
>>
>>
>_______________________________________________________________________
____
_
>_
>
>
>>Scanned for SoftLanding Systems, Inc. and SoftLanding Europe Plc by
IBM
>>
>>
>Email
>
>
>>Security Management Services powered by MessageLabs.
>>
>>
>>
>_______________________________________________________________________
____
_
>_
>
>
>_______________________________________________________________________
____
_
>_
>Scanned for SoftLanding Systems, Inc. and SoftLanding Europe Plc by IBM
>Email Security Management Services powered by MessageLabs.
>_______________________________________________________________________
____
_
>_
>_______________________________________________
>alf-dev mailing list
>alf-dev@xxxxxxxxxxx
>https://dev.eclipse.org/mailman/listinfo/alf-dev
>
>_______________________________________________
>alf-dev mailing list
>alf-dev@xxxxxxxxxxx
>https://dev.eclipse.org/mailman/listinfo/alf-dev
>
>
_______________________________________________
alf-dev mailing list
alf-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/alf-dev
_______________________________________________
alf-dev mailing list
alf-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/alf-dev
**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they are
addressed. Any unauthorized review, use, disclosure or distribution is
prohibited. If you are not the intended recipient, please contact the
sender by reply e-mail and destroy all copies of the original message.
_______________________________________________
alf-dev mailing list
alf-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/alf-dev
_______________________________________________
alf-dev mailing list
alf-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/alf-dev
