Bug 220871 - Revise Eclipse Development Process
Summary: Revise Eclipse Development Process
Status: RESOLVED FIXED
Alias: None
Product: Community
Classification: Eclipse Foundation
Component: Process (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P1 enhancement (vote)
Target Milestone: ---   Edit
Assignee: Bjorn Freeman-Benson CLA
QA Contact:
URL: http://www.eclipse.org/projects/dev_p...
Whiteboard:
Keywords:
: 108303 (view as bug list)
Depends on:
Blocks: 203296 213788 214177 218090
  Show dependency tree
 
Reported: 2008-02-28 21:28 EST by Bjorn Freeman-Benson CLA
Modified: 2013-09-13 16:18 EDT (History)
13 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Bjorn Freeman-Benson CLA 2008-02-28 21:28:54 EST
Annual revision to accommodate the continued evolution of the Eclipse community and projects.
Comment 1 Bjorn Freeman-Benson CLA 2008-02-29 03:00:17 EST
Revised section 2 "Principles" and section 7 "Revisions".
Comment 2 Bjorn Freeman-Benson CLA 2008-03-07 01:21:18 EST
I have finished the first draft and posted the first blog article discussing the revision.
Comment 3 Martin Oberhuber CLA 2008-03-07 07:07:12 EST
What's the rationale behind kicking out principles
   2.2 Quality Culture
   2.3 Collective Reputation
?

I think that these principles are an important part of what Eclipse is now, and they do guide some daily operations at Eclipse. The important point is that these are perhaps less stringent in Incubation projects, but should be guidance for mature projects. Or am I getting anything wrong?
Comment 4 Nick Boldt CLA 2008-03-07 11:58:13 EST
Infinite project nesting? Oh, gods, the horror! The horror! This sounds like a Very Bad Idea, esp. considering we still haven't sorted out bug 198541 (Standardized groups on dev.eclipse.org). 

In Modeling, we've made great strides while limited to TLP > Project > Component-cum-Subproject. I'm curious -- what's the usecase for actually needing TLP > Project > Subproject > Subsubproject > Subsub[...]project? Who really needs more than Grandpa Burger > Papa Burger > Teen Burger [1] ?

[1] http://en.wikipedia.org/wiki/A%26W_%28Canada%29

(Then again, mybe I'm just paranoid that someone in Modeling will decide it's time to reorg again.) ;-)

If I read the proposed changes correctly and the definition of component and project have now merged, does this mean I will be able to close bug 203296 (The download page urls need to have ?component= rather than ?project=) as WONTFIX?

Comment 5 Bjorn Freeman-Benson CLA 2008-03-07 12:23:43 EST
(In reply to comment #3)
> What's the rationale behind kicking out principles 2.2 and 2.3

Just because the words are not in this document does not mean they are not important. I struck them out because they have been a source of confusion by readers and I was unable to come up with a good way to rewrite these sections to be more clear without making them too wordy.

(In reply to comment #4)
> considering we still haven't sorted out bug 198541

We haven't? That's news to us at the EMO, because we're busy implementing it.

> In Modeling, we've made great strides while limited to TLP > Project >
> Component-cum-Subproject. 

You are welcome to continue that structure. Nothing in the new process requires that have more nesting depth.

> I'm curious -- what's the usecase for actually needing TLP ...

Modeling is one of them. Modeling currently uses components as sub-projects. That's not kosher under the current process, but will be under the new process. Once I changed the wording to allow that, it made no sense to restrict the number of levels. Practically speaking, projects will choose their own practical level of hierarchy and I doubt it will be much different from the way things are done today.

> does this mean I will be able to close bug 203296 ... as WONTFIX?

Yes. (See, something for everyone! :-)
Comment 6 Nick Boldt CLA 2008-03-07 13:34:09 EST
(In reply to comment #5)
> > considering we still haven't sorted out bug 198541
> We haven't? That's news to us at the EMO, because we're busy implementing it.

Sorry, I should have said "completed bug 198541." I know it's in progress, I'm just worried that one day you'll have to handle eclipse-equinox-p2-subthing-subthingelse-dev as a group name. Or modeling-mdt-uml-uml2tools-extensionthingy-releng. Blecch.
 
> You are welcome to continue that structure. Nothing in the new process requires
> that have more nesting depth.

Right, but if you allow it someone will ask for it. *shudder*
 
> > does this mean I will be able to close bug 203296 ... as WONTFIX?
> Yes. (See, something for everyone! :-)

W00t! 

Comment 7 Doug Gaff CLA 2008-03-07 13:41:43 EST
The quality culture section was definitely wordy. But perhaps a single sentence
talking about quality and collective reputation could be added to the Ecosystem
section. It's not the end of the world if they're not there, IMO.

"Eclipse as a brand is the sum of its parts (all of the projects), and projects
should strive for the highest possible quality in extensible frameworks,
exemplary tools, transparent processes, and project openness."

I suggest renaming "Clear and Concise" to "Clear, Concise, and Evolving" or
something like that.
Comment 8 Kenn Hussey CLA 2008-03-07 13:59:31 EST
Thanks for the updates, Bjorn. A couple of comments...

"The Eclipse Projects are organized hierarchically. The top of the hierarchy
are the set of Top Level Projects. Each Top Level Project contains one or more
Sub-Projects. Each Sub-Project contains zero or more Sub-Projects. The term
Project refers to either a Top-Level Project or a Sub-Project. Projects may be
referred to as Sub-Projects or Components, but the choice of common name does
not change the characteristics of the Project."

Will existing "components" (in Modeling for example) be grandfathered as
sub-projects under the new definition?

"Container Projects do not have file infrastructure: no Unix group and no
repository."

Container projects do, however, have Web file infrastructure in CVS, and it's
common for them to also have (shared) release engineering infrastructure (e.g.
in a "releng" CVS module).

"By design, committers exist only at the leaves of the tree, thus even with a
tall hierarchy of projects, the committers in each (leaf) project are in full
control of that project: there are no committers in a parent project who could
override the collective decisions of the sub-project."

This was in your blog entry, not in the process. I think it is important to
capture the relationship between (a) container project lead(s) and its
operating sub-projects; does a container project lead not have any authority
over the contained operating projects?
Comment 9 Bjorn Freeman-Benson CLA 2008-03-07 14:11:23 EST
(In reply to comment #8)
> Will existing "components" (in Modeling for example) be grandfathered as
> sub-projects under the new definition?

It will be your choice. If you want them to have separate committer lists, then they will need to be sub-projects/components.

> Container projects do, however, have Web file infrastructure in CVS, and it's
> common for them to also have (shared) release engineering infrastructure (e.g.
> in a "releng" CVS module).

Stand by for my blog post on Monday about that very topic.  As a hint: container projects with web sites, etc. where those websites have a different committer list will need to have a sub-project for that web site and committer list.

Do you think I should another example to the dev process document explaining this?

> This was in your blog entry, not in the process. I think it is important to
> capture the relationship between (a) container project lead(s) and its
> operating sub-projects; does a container project lead not have any authority
> over the contained operating projects?

Project leads have some authority (about as much authority as they do today), but they cannot impose committers on sub-projects (just as project leads cannot impose committers on projects today). In reality, the position of project lead today (and in the proposed future) is more honorary and extra work than powerful. Not to scare anyone away or anything :-)

For example, when there are IP questions with a CQ, Janet's team often contacts the Project Lead instead of the whole dev mailing list. Thus if a CQ were submitted for project A.B by someone in project A.B.C.D, then the project lead for A.B would get involved. (The reason you might ask for approval at the A.B level is that then it would cover all the projects A.B.* rather than just the single A.B.C.D.)

(In reply to comment #6)
> just worried that one day you'll have to handle
> eclipse-equinox-p2-subthing-subthingelse-dev as a group name. Or
> modeling-mdt-uml-uml2tools-extensionthingy-releng. 

That doesn't bother us at all because it's all going to automated and thus we won't have to type those long group names and, subject to the limits of Linux, we just don't care :-) It does have the benefit of being very descriptive!
Comment 10 John Arthorne CLA 2008-03-07 15:00:57 EST
> An Operating Project has a self-managing set of Committers. The Committers of an > Operating Project have the exclusive right to elect new Committers to their 
> Project – no other group, including a parent Project, can force a Project to 
> accept a new Committer.

Perhaps it is covered elsewhere, but I think there is a need to reach outside the self-managing group in cases where there are insufficient active committers to hold a new committer election. I..e, if there are 2, 1, or 0 committers on the project, they cannot obtain the three required +1 votes to add a new committer. This possibility becomes quite real in the face of arbitrarily nested sub-projects. There are a number of components in the Eclipse TLP that fall into this category.
Comment 11 Bjorn Freeman-Benson CLA 2008-03-07 23:52:03 EST
(In reply to comment #7)
> The quality culture section was definitely wordy. But perhaps a single sentence
> talking about quality and collective reputation could be added to the Ecosystem
> section. It's not the end of the world if they're not there, IMO.

Done.

> I suggest renaming "Clear and Concise" to "Clear, Concise, and Evolving" or
> something like that.

Done.

(In reply to comment #10)
> Perhaps it is covered elsewhere, but I think there is a need to reach outside
> the self-managing group in cases where there are insufficient active committers

Good point - added.
Comment 12 John Arthorne CLA 2008-03-10 00:35:33 EDT
From Section 6.3:

"All Projects are required to participate in at least one Review per year."

If a TLP has a release review, are all the nested projects implicitly part of that review, or do they require separate reviews?  Can multiple projects that don't share a common top-level project share a single review? For example if Equinox moved to the RT top-level project, could it still share a single review with the Platform project, or does it require an separate review?

From Section 6.4:

"During the process of developing software and preparing a Release, various nightly and integration builds are are made available..."

-> Typo (are are)

"Do not include any links on the project website, blogs, wikis, etc. that might encourage non-developers to download and use nightly builds, release candidates, or any other similar package"

-> This contradicts the next paragraph which states:

"Milestones and release candidates are "almost releases" intended for adoption and testing by early adopters. Projects are allowed to have links on the project website, blogs, wikis, etc. to encourage these outside-the-committer-circle early adopters to download and test the milestones and release candidates"
Comment 13 Bjorn Freeman-Benson CLA 2008-03-10 03:49:12 EDT
(In reply to comment #12)
> If a TLP has a release review, are all the nested projects implicitly part of
> that review, or do they require separate reviews?  

I used the word "participate" for exactly that reason: so that nested projects could participate without having to have separate Reviews. I believe that it will be necessary for the TLP Review to state all the sub-projects that are included in the Review because not all sub-projects of a project will necessarily be included. But for those that are, yes, one single review will suffice.

> Can multiple projects that
> don't share a common top-level project share a single review? 

Hadn't considered that case, but I don't see why not - the only requirement is that there be a public Review so that the community-at-large is aware of state changes and IP clearance.

> -> Typo (are are)

Fixed.

> -> This contradicts the next paragraph which states:

Hmm, I thought the wording was clear, but how about if I change it to "non-early-adopters". (I meant "developers" to include "people who are helping the team develop", i.e., the contributors and early-adopters - just not the general public who doesn't understand the distinction between "build", "milestone", and "release".)
Comment 14 John Arthorne CLA 2008-03-10 07:58:41 EDT
The contradiction I saw was with respect to release candidates.  The first paragraph essentially says "don't blog about release candidates", and the second one says "you can blog about release candidates". Which is it?
Comment 15 Martin Oberhuber CLA 2008-03-10 15:51:11 EDT
(In reply to comment #13)
> the team develop", i.e., the contributors and early-adopters - just not the
> general public who doesn't understand the distinction between "build",
> "milestone", and "release".)

Why do you think the general public wouldn't understand the distinction between "build", "milestone", "release candidate" and "release"? Have there been any negative experiences in the past?

I'd be confident in the public understanding these distinctions, but perhaps I'm too much a computer guy to understand the general public that's consuming Eclipse downloads...
Comment 16 Nick Boldt CLA 2008-03-10 22:21:29 EDT
(In reply to comment #15)
> (In reply to comment #13)
> > the team develop", i.e., the contributors and early-adopters - just not the
> > general public who doesn't understand the distinction between "build",
> > "milestone", and "release".)
> 
> Why do you think the general public wouldn't understand the distinction between
> "build", "milestone", "release candidate" and "release"? Have there been any
> negative experiences in the past?

We just provide a link on our downloads pages (the contents of which I found invaluable when *I* first started working with Eclipse projects) [1]. I've never heard any negative feedback in 3 years.

[1] http://www.eclipse.org/modeling/downloads/build-types.php

This could this be generalized and standardized somewhere, eg., as linked from http://wiki.eclipse.org/index.php/Development_Resources or http://www.eclipse.org/projects/dev_process/development_process.php

Incidentally, what's a "build" build? Is that the same as Nightly, Maintenance or Integration?
Comment 17 Bjorn Freeman-Benson CLA 2008-03-11 00:01:15 EDT
(In reply to comment #14)
> The contradiction I saw was with respect to release candidates.  The first
> paragraph essentially says "don't blog about release candidates", and the
> second one says "you can blog about release candidates". Which is it?

The point I was trying to make is that projects are, of course, welcome and encouraged to have a community of experts assist them in evaluating and testing their builds, but that they should refrain from advertising builds as being complete/done/a release.  So, blogging, posting on the download page, etc. about a new nightly build or release candidate being available for the early adopters to help test - that's great; blogging about a release candidate being an official release - not great.

We have at least four audiences for our project communications: committers (actively writing code), contributors (actively helping write code), early-adopters (actively helping with testing), and users (passive consumers). The first three understand the limitations and differences between a build, a milestone, and a release. I am arguing (as does Apache) that the fourth group does not understand that distinction and thus we (the project teams) need to be careful in our communications to them.

I obviously don't have the wording right here, so if you have any suggestions?

(In reply to comment #15)
> Why do you think the general public wouldn't understand 
> the distinction between...

Because, in fact, not even all the project teams understand the distinction based on the number of teams who trying to get IP approval for interim builds of other open source projects while claiming they are "releases".

> I'd be confident in the public understanding these distinctions, but perhaps
> I'm too much a computer guy to understand the general public that's consuming
> Eclipse downloads...

I'm afraid I'm going to disappoint you: hang out in the eclipse.newcomer newsgroup for a while...

(In reply to comment #16)
> We just provide a link on our downloads pages (the contents of which I found
> invaluable when *I* first started working with Eclipse projects) [1]. I've
> never heard any negative feedback in 3 years.

That's fine. In fact, it should be required.

> Incidentally, what's a "build" build? Is that the same as Nightly, Maintenance
> or Integration?

Where did I write that?

Comment 18 Nick Boldt CLA 2008-03-11 00:15:22 EDT
(In reply to comment #17)
> (In reply to comment #16)
> > Incidentally, what's a "build" build? Is that the same as Nightly, Maintenance
> > or Integration?
> Where did I write that?

Martin's comment 15.

Comment 19 Doug Gaff CLA 2008-03-11 00:17:05 EDT
This link off the main dev process page is broken:

http://www.eclipse.org/projects/dev_process/top-level-phase.php
Comment 20 Doug Gaff CLA 2008-03-11 01:02:34 EDT
http://www.eclipse.org/projects/dev_process/release-review.php

There are two Section 2.7's.
Comment 21 John Arthorne CLA 2008-03-11 12:46:48 EDT
It sort of worries me that the "just enough process" principle has been removed. It looks like it has been replaced by:

"This document imposes requirements and constraints on the operation of the Projects, and it does so on behalf of the larger Eclipse community. It is an explicit goal of the Development Process to provide as much freedom and autonomy to the Projects as possible while ensuring the collective qualities benefit the entire Eclipse community."

Philosophically, this goes above conferring negative rights to the community (committers must not do things that harm the community), and confers positive rights on the wider community (commiters must act to the benefit if the wider community).  This implies a greater burden on committers, and given the very large size of the community, this burden can be significant.  Of course we must strike a balance here, but my fear is that we are slowing drifting towards a process burden that causes people to question why they should bother putting their code under the Eclipse umbrella. I know of cases where this has already happened, and ironically I think the community may lose out in the end if this increases.

Sorry to raise such a murky objection, because I think overall the Eclipse development process today is good and strikes a fair balance. I just wanted to observe the general trend and the corresponding change to the basic principles of the development process.
Comment 22 Nick Boldt CLA 2008-03-11 23:00:45 EDT
http://eclipse-projects.blogspot.com/2008/03/edp-2008-2-components.html sounds great. This works for a project like Tools.GEF, which has shared-committer-access components GEF, Draw2D, and Zest [Case A] and for a project like Modeling.EMFT, which has firewalled independent components with distinct committer lists [Case B]. "Just enough process" to make both scenarios work, and the flexibility (I'm assuming) to split/merge if needs arise in future (eg. if GEF decides to get more strict or EMFT decides to combine components into ubercomponents). 

I'm guessing this would also allow the scenario where there's one committer for multiple components, where to minimize overhead those components could be merged into a single grouping (eg., Christian Damus' EMF Query, EMF Transaction, and EMF Validation components). Or does that break the, ahem, model? :)
Comment 23 Bjorn Freeman-Benson CLA 2008-03-11 23:58:02 EDT
(In reply to comment #21)
> It looks like it has been replaced by

No, there is no replacement - that wording that you quote has been part of the Eclipse Development Process for over a year now. (Use the diffs link http://www.eclipse.org/projects/dev_process/development_process.php?version=diffs to see that.)

> Philosophically, this goes above conferring negative rights to the community
> (committers must not do things that harm the community), and confers positive
> rights on the wider community (commiters must act to the benefit if the wider
> community).  

I believe that this has been part of the Eclipse Development Process from the beginning, but who knows, I could be wrong about that. The Eclipse Foundation has always been about a commercial open source, in other words, open source projects whose explicit goal is to build extensible frameworks on which the ecosystem can build commercial products. In order to do that, the wider community does have positive rights (has always had positive rights). The key, as you point out, is to balance things.

> my fear is that we are slowing drifting towards a
> process burden that causes people to question why they should bother putting
> their code under the Eclipse umbrella. 

I'm actually hoping that each revision of our processes ends up with less burden for the projects and greater benefit for the community. For example, this revision speeds up Reviews while still retaining the benefits for the ecosystem. 

(In reply to comment #22)
> "Just enough process" to make both scenarios
> work, and the flexibility ...

Correct.

> I'm guessing this would also allow the scenario where 
> there's one committer for
> multiple components, where to minimize overhead those components could be
> merged into a single grouping 

Correct. The EDP only specifies that *if* there are separate committer lists, then there are separate projects/components. If there are multiple bits of code (sub-projects/components/whatever) that have the same committer list, then it's perfectly ok to have one "official" project.
Comment 24 Kenn Hussey CLA 2008-04-11 14:07:16 EDT
I assume, under the revised process, that it would be less of an issue for components (nested projects) to have their own newsgroups than it has been in the past?
Comment 25 Bjorn Freeman-Benson CLA 2008-04-12 10:42:25 EDT
(In reply to comment #24)
Correct.
Comment 26 Bjorn Freeman-Benson CLA 2008-04-28 21:29:32 EDT
*** Bug 108303 has been marked as a duplicate of this bug. ***
Comment 27 Nick Boldt CLA 2008-07-04 12:55:57 EDT
The target milestone here is June 08. When will this be ratified?
Comment 28 Bjorn Freeman-Benson CLA 2008-07-05 17:24:32 EDT
(In reply to comment #27)
> When will this be ratified?

I no longer have any idea. I've changed the target to Q3, but who knows.
Comment 29 Bjorn Freeman-Benson CLA 2008-08-07 15:02:24 EDT
Sent PDF to Mike for the Board.