[news.eclipse.technology.alf] Re: Upcoming validation release review

["Tim Buss" <t b u s s @ S e r e n a . c o m>]

Jochen,

I don't disagree with any of your points. The essential problem, I believe
is one of terminology and lack of definition. Looking a the eclipse project
lifecycle it appears have two main phases. This from the the "Eclipse
Development Process v033 Jan 26 2006"
http://www.eclipse.org/projects/dev_process/


" ....


Validation
<http://www.eclipse.org/projects/dev_process/validation-phase.php> - After
the project has been created, the purpose of the validation phase is to
establish a fully-functioning open-source project. In this context,
validation (also known as incubation) is about developing the process, the
community, and the technology.

* · While it might seem easy, history has shown that it takes experience to
run an open, transparent, inviting, permeable, and predictable software
development process. And history has shown that it takes a significant
investment of time and energy to nurture a community of tool users and
framework users and committers around a new project.

* Similarly, the validation phase is about developing the technology to
Eclipse Quality
<http://www.eclipse.org/projects/dev_process/eclipse-quality.php> . The
project will most likely produce a number of 0.N releases during this phase
in order to garner feedback from the community on their APIs and tools.

* Validation/incubation is a phase rather than a place - new projects may be
incubated under any existing top-level project. This is different from the
way Apache <http://incubator.apache.org/> uses a place to encapsulate the
phase. Incubating under an existing top-level project is a significant
investment in time and energy by the top-level PMC, so most projects
incubate under the Technology top-level project due to Technology's
experience in this area.

* Validation ends with an Checkpoint Review
<http://www.eclipse.org/projects/dev_process/validation-phase.php#Checkpoint
Review> .

· Implementation
<http://www.eclipse.org/projects/dev_process/implementation-phase.php> - The
project team has demonstrated that they are an open-source project with an
open and transparent process; an actively involved and growing community;
and Eclipse Quality
<http://www.eclipse.org/projects/dev_process/eclipse-quality.php>
technology. The project is now a mature member of the Eclipse Community.
Major releases continue to go through Release Reviews
<http://www.eclipse.org/projects/dev_process/release-review.php> .


...."


At first glance the terms "Validation" and "Implementation", would seemed to
indicate to use that one should do planning, proof of concept and
prototyping during the "Validation" stage and implementation during the
"Implementation" stage. Given this presumption then nothing I read in this
definition or in other material really disavows me of that understanding.
Nor did any guidance we received. I think this is the essence of our
confusion. This accounts for, to take an example, our contention that
Bugzilla is not the tool of choice for providing feedback during the
"Validation" stage but only toward the end of the "Implementation" stage. I
would contend that the purpose of the Validation stage and its exit criteria
are unclear and, evidently, the "Implementation" stage is misnamed.

Based our immediate experience and on these discussion I think the project
lifecycle might be more correctly described as follows:

***

"Incubation"

After Creation review and project is considered in "Incubation". It should
create a number of milestones, grouping sets of useful functionality that
might be of interest to potential users of the project that it reasonably
expect to deliver. As each milestone is reached the project should schedule
a "release review" to publicize the completed milestone to the community and
solicit input from the broader community. At some milestone, when the
project feels it has completed some significant portion of its initial goal
and has built some active community it should schedule a "validation
review". This is essentially a a "release review" but with an additional
examination of the state of the process followed by the project and the
strength of the community that surrounds it. If a project passes the
Validation review then it moves from "Incubation" to "Active".

"Active"

The project proceeds to deliver on its milestone holding release reviews as
appropriate with the goal of creating a major release.

***

Obviously I'm missing a few important details - this is just off the top of
my head - but replacing "Validation phase" and "Implementation phase" with
"Incubation phase" and "Active phase" and making milestones and release
reviews the main thrust, I think, clears up the ambiguity of the current
description and makes expectations much clearer. It also provides a
framework to make them concrete which would be helpful.

Given this interpretation, much of the current discussion while valid, seems
moot to the issue at hand, and clearly we were mistaken to have applied for
a Validation review at this point. While, it is now clear, we should have
made more of an effort to understand the process, ALF as well as other
incubator projects could have certainly used guidance from the eclipse
organization. Given that we are apparently expected to have had milestones
and release reviews before the validation review then this could have been
pointed out when the subject of validation review was first brought up
several months ago.

Best Regards,
Tim Buss




"Jochen Krause" <jkrause@xxxxxxxxxxxxxx> wrote in message
news:ec09ct$2ev$1@xxxxxxxxxxxxxxxxxxxx
> Ali,
>
> Sorry for focusing below on the items where I see problems. There are
> some areas where ALF is very advanced (planning, meeting minutes,
> conducting explicit architecture work). See below:
>
> Ali Kheirolomoom wrote:
> > Thank you for your feedback.
> >
> > As we understood the goals of the Incubation/Validation phase as
> > documented on Eclipse--which is the focus of this review--we do believe
> > that we have adequately fulfilled the requirements as expected of a
> > project in this stage.   However, as evident by the concerns raised by
> > the Eclipse Review Board, there appears to be a general misunderstanding
> > of the nature of ALF.  As such, the ALF team withdrew from the review to
> > address these concerns.  Listed below please find a quick summary
> > stating our position.
> >
> > To date we have completed two milestones namely, .1 and .2, the
> > deliverables of which were announced in the project plan published to
> > the ALF site.  The bulk of this work was to establish the base
> > architecture for ALF, to provide a base implementation of that
> > architecture, and to execute a POC with other interested parties to test
> > the Architecture and to create a working sample that would allow someone
> > to run, debug and otherwise test the base implementation using eclipse
> > as the development/test environment.  In addition we have published a
> > detailed architecture and development plan for ALF Security and have
> > successfully launched two ALF vocabulary groups, namely SCM and BUILD,
> > with regularly attended and documented conference calls.  We did not
> > explicitly announce the availability of milestone .1 and the recent
> > milestone .2.  Instead we tend to announce the availability of
> > milestones as they occurred.  We do agree that a more formal
> > announcement of availability is an advantage and will do so in the
future.
> >
>
> Quoting the validation phase part of the development process document
> (http://www.eclipse.org/projects/dev_process/validation-phase.php):
>
> "Heartbeat Builds
>
> We have learned that projects cannot successfully build their three
> communities without regular milestone releases (we recommend six to
> eight week periods), nor can it do so without regular, successful,
> heartbeat builds (we recommend at least nightly). Projects need regular
> milestones and builds so that their three communities can continuously
> pick up, integrate, test, use, and provide feedback on the latest
> features and improvements. Projects that do not provide frequent builds
> are effectively working in the closed rather than in the open."
>
> This might be one reason for the different perception of ALF between the
> community and the project team themselves. It is obvious that the
> project team knows about all the progress that you made, but it is very
> difficult for an outsider to experience this progress.
>
>
> > Clearly as part of the implementation and the ultimate 1.0 release we do
> > expect to create unit tests and solicit the submittal of any bugs found.
> >  However we don't see these activities as necessarily a part of a
> > validation phase where the primary intent is to establish an active
> > community, and publish and validate the architecture through community
> > discussions and POC sample development.
>
> There are two items in the "exiting validation phase criteria" that
> address the above topic:
>
>   - Open and transparent Bugzilla with a described and documented bug
> process.
>   - The project has working policies and procedures for developing,
> specifying, testing, and getting feedback on APIs.
>
> Again, this is not about the quality of your work, but about the process
> that you use to implement the code: Using or not using bugzilla (which
> is defacto hardly being used at ALF). And providing test cases for the
> API that you are providing.
>
>
> So, why is this important at all?
>
> Because it enables the community to evaluate and pick up ALF. I am sure
> that every company that is contributing to ALF has a set of rules how to
> manage projects (SCM, bug tracking, ...). If you pursue a project and
> you use a different SCM, no bug tracking, different communication
> channels you make it difficult to your environment to find out what is
> going on. Relying on some standards (even if they are mediocre in some
> cases) makes this much easier.
>
> > In the mean time the ALF WIKI,
> > alf-dev mailing list, and the ALF newsgroup have been found adequate for
> > community collaboration.  We accept that if a project starts as a large
> > initial contribution the character of the validation phase might be more
> > oriented to transforming that code to some acceptable form and that in
> > that case bugs and unit tests might be created.
>
> I strongly disagree. It does not matter if there is a code contribution
> or not. During validation you should prove that you develop the code in
> an open and transparent manner. And that you care about documenting
> quality. Bugzilla is not only intended for your use. By enabling
> developers to use your code (builds, new and noteworthy) they will start
> to enter bugs for you.
>
> > However, ALF is being
> > created from scratch and all the specifications and code has been
> > created specifically for the ALF project.  The only reason there was any
> > "initial contribution" at all was due to the complexity and our lack of
> > understanding of the IP process.  As indicated in our published project
> > plan, we have allocated resources to perform functional and load testing
> > as the project proceeds through the implementation phase.  This work is
> > slated to start as part of milestone .3 and increase through and beyond
> > the creation of the Release candidate.  ALF is totally committed to
> > quality, to building the community, to providing leadership to users,
> > developers, committers and commercial organizations.
>
> It seems that you want to resolve a lot of the critics above with your
> next milestone, but that you want to pursue those things in the
> implementation phase. I would suggest to do all those things in the
> validation phase. It might be a good idea to have a couple of milestones
> and a release (< 1.0) before planning to exit the validation phase.
>
>
> >
> > It should be noted that the ALF project is not a plug-in project for
> > Eclipse IDE but rather an infrastructure extending beyond the developer
> > IDE and focused on distributed interoperability and integration, by way
> > of establishing a common platform and vocabulary. As you know, there are
> > a number of Eclipse projects underway that stretch the traditional
> > domain of Eclipse as a desktop environment. Projects such as the SOA
> > tools project, the Web Tools Project, and others either incorporate or
> > provide server-side components.  In keeping with the Eclipse approach of
> > extensible client-side frameworks, we have set forth an architecture for
> > an extensible server-side framework.  In addition, we are investigating
> > a real-time interactivity between the distributed tools orchestrated by
> > the ALF framework and the plug-in tools inside the Eclipse IDE through
> > bridging ALF events and OSGi events.  We have identified and documented
> > related synergies with the Corona project to address this use scenario.
> >
> > The principle API in ALF is the Event Manger WSDL.  This underwent
> > considerable discussion as part of its original design and was tested
> > from a practical standpoint as part of the POC effort and has been
> > available for public review for quite some time.  The ALF Team is about
> > to initiate another review of this WSDL based on what was learned in the
> > POC so we can move forward to finalize this API.  The other important
> > API work in ALF is represented by the SCM and BUILD vocabulary groups.
> > There are not currently any APIs specifically intended for use within
> > eclipse since they are not currently deemed a necessity to establish
> > ALF.  ALF does use Eclipse Plug-ins to design, configure and deploy
> > ALF-based applications including configuration of parameters used by the
> > framework.  The ALF Configuration Plug-in is the primary ALF-specific
> > tool and exists today as a working illustration of our intent.  We also
> > hope to leverage WTP and the BPEL Designer.  The recent ALF sample went
> > to some length to provide a configuration that works with WTP.
> >
>
> It is great that you are creating an extensible server-side framework,
> and I think that the Eclipse mission statement covers this ground. But
> the mission statement is talking about "extensible frameworks and
> exemplary tools", not only frameworks. If "tool" in the classic sense is
> to narrow for your project, we may want to interpret this as "useful
> implementation built on the framework". The main idea behind the eclipse
> mission is that we need a working implementation of a "tool" to prove
> that the framework we are building is useful. At the same time the
> exemplary implementation helps adopters by providing an example on how
> to build something on top of the framework.
>
>
> > There are a number of users outside of ALF immediate sponsors who are
> > either actively investigating or working with ALF.  All these can be
> > considered active tool users.  However there cannot be an "active tool
> > user" community in the traditional sense until ALF implementations exist
> > for a number of products. The ALF team has and continues to actively
> > recruit vendors to ALF-enable their products. In addition,  the ALF team
> > will ALF-enable various Open source products as indicated in the ALF
> > project plan.
>
> The community that you are talking about is the "plug-in developer
> community", even if plug-in does not fit well in the ALF case. The
> "active tool user" community means developers that are simply using the
> exemplary implementations that showcase your frameworks. An exemplary
> implementation could address one of the open source products as
> mentioned above. This implementation can then be used by the "tool
> users". It is great a great achievement of the ALF project to have a
> strong community of commercial adopters (plug-in developers).
>
>
> >
> > The current feedback just leaves us scratching our heads. Are these
> > criteria applicable to a server-side framework such as AFL?  Given the
> > stated desire by Eclipse to expand beyond the developer IDE and host the
> > distributed server-side frameworks such as ALF, it would be crucial to
> > properly refine the exit criteria corresponding to the various stages in
> > the Eclipse Lifecycle based on the type of the project.
> >
>
> I hope that you can now better understand our feedback. For the items
> stated above I don't see much difference between a client and a
> server-side framework with regard to the development process. But this
> includes some interpretations on my part, and others will have different
> interpretations. I would welcome to make the development process easily
> applicable to both client and server-side projects.
>
> Best Regards,
>
> Jochen Krause
>
> P.S: As I am on vacation right now it may take some days before I reply
> on this thread.
>
>
> > We look forward to contributing to the ALF project and the broader
> > Eclipse Open Source Community.
> >
> > Regards,
> > ALF Team
> >
> >
> >
> >
> >
> >
> > "Jochen Krause" <jkrause@xxxxxxxxxxxxxx> wrote in message
> > news:eblcjh$4pr$1@xxxxxxxxxxxxxxxxx:
> >
> >> To comply with the Eclipse "open source rules of engagement" I am
> >> resending a letter that was sent yesterday to the ALF project leads by
> >> email to this newsgroup.
> >>
> >>
> >> Dear ALF project leads,
> >>
> >> As electeted Eclipse board members we would like to clarify the status
> >> of the ALF project with respect to the criteria of the validation
review
> >> with you.
> >>
> >> Our motivation for this discussion is not to pick on you or to
> >> discourage you in any way, but to honour the eclipse development
> >> process. We think that sticking to the values defined in the
development
> >> process (http://www.eclipse.org/projects/dev_process/)  is essential
for
> >> the long term viability of Eclipse as an open source community and for
> >> maintaining the Eclipse quality brand. On first sight, it seems that
ALF
> >> is not meeting all of the criteria set forth by the process
> >>
(http://www.eclipse.org/projects/dev_process/validation-phase.php#Exitin
> >> g%20Validation), namely:
> >>
> >> - no active framework user community as the only public download
> >> available does not implement a framework
> >> - no active tool user community
> >> - bugzilla is not used (6 bugs, no closed bug, some of these are there
> >> for testing purposes, no commented bug)
> >> - there are no established policies for testing (there is almost no
test
> >> code) or API feedback
> >>
> >> There is also a requirement to meet the "Eclipse quality" standards,
> >> that we may need to discuss as well.
> >>
> >> As there is very little time left before the release review we would
> >> like to ask you to postpone the review for a couple of weeks to give us
> >> and you the chance to discuss the above topics collaboratively and
> >> before the review.
> >>
> >> Best Regards,
> >>
> >> Jochen Krause  Jeff McAffer  Mike Taylor  Tim Wagner  Todd Williams
> >