On March 22, I will be presenting my revised edition of the Eclipse Development Process to the Eclipse Board of Directors for their approval. I’ve been discussing the upcoming changes in this blog, and in a handful of bug reports, so I’m pretty comfortable with the ideas. But like many things in life, the delivery date has snuck up on me and I find myself scrambling a little to get all the words down right.

As I said, the ideas are all out there and I’m in a process of wordsmithing it all at the moment. The original document was built in the days before XHTML and stylesheets were popular, so I’ve been trying to fix up as much of the markup as possible while I’m at it while trying not to disturb the content any more than necessary (so that the viewcvs diff remains at least a little useful). I’ve tried to add annotations in places where significant changes have occurred. You can turn annotations on by clicking the appropriate link on the right side of the page. It occurred to me to let the document use more of the screen. Right now, it’s crammed into a relatively narrow band down the middle of the window. But then I thought of Ward’s TenWordLine and decided to leave it alone.

There are three major changes in the document.

First, I’ve removed the notions of Container- and Operating-Projects. Now we just have projects. All projects can opt to either have code or not. Projects can have downloads, builds, websites, etc. or not. Projects can opt to do roll up builds of sub-projects. I have also added some words that attempt to actually define the term “project” and have taken steps to deformalize the use of the term “sub-project” (a “sub-project” is simply a way of talking about a project that has a parent). I’ve talked about this particular change at length already.

Second, the notion of Incubators is made formal. Incubators do not do releases. They do not graduate. They do not require continuation reviews (or reviews of any form). Incubators remain perpetually in the Incubation phase. I have also already blogged about this.

Third, I’ve made an existing project creation loophole formal. New projects can be created directly from a restructuring review (I’ve also consolidated Move and Restructuring Reviews) without going through the proposal and creation phases under certain conditions (the scope must be preserved). This one, which came up as a side effect of Bug 241041 and the recently-announced Mylyn Restructuring, is potentially one of the more controversial changes.

There’s other less significant changes, like the elimination of the notion of a review call in favour of a “review period”. There’s nothing to prevent a call from happening, but there is no mention of in the process. I’ve also acted on some excellent suggestions for some wording changes in the section on project leadership.

The original document uses capitalization similar to what you might find in a legal document. I am inclined to change it to instead follow more common language rules. This will mess up the diff, so I may just put that particular exercise off for a while; I don’t believe that changing the capitalization of the text will require further scrutiny from the Eclipse Board of Directors.

And yes, there’s hyperlinks to the old version. Good idea.

There is some awkward wording in the current version of the Eclipse Development Process with regard to project leadership. I’d like to try and clean that up.

Here’s what I’m thinking:

---

4.6 Leaders

There are two different types of project leadership at Eclipse: The Project Management Committee (PMC) and Project Leads. Both forms of leadership are required to:

* ensure that their project is operating effectively by guiding the overall direction and by removing obstacles, solving problems, and resolving conflicts;
* operate using open source rules of engagement: meritocracy, transparency, and open participation; and
* ensure that the projects and its sub-projects (if any) conform to the Eclipse Foundation IP Policy and Procedures.

4.6.1 Project Management Committee (PMC)

Top-level projects are managed by a Project Management Committee (PMC). A PMC has one or more PMC Leads and zero or more PMC Members. Together the PMC provides oversight and overall leadership for the projects that fall under their top-level project. The PMC as a whole, and the PMC Leads in particular, are ultimately responsible for ensuring that the Eclipse Development Process is understood and followed by their projects. The PMC is additionally responsible for maintaining the top-level project’s charter.

PMC Leads are approved by the Board; PMC members are elected by the existing PMC Leads and Members, and approved by the EMO(ED).

4.6.2 Project Lead

Eclipse Projects are managed by one or more Project Leads. Project Leads are responsible for ensuring that their project’s committers are following the Eclipse Development Process, and that the project is engaging in the right sorts of activities to develop vibrant communities of users, adopters, and contributors. The initial project leadership is appointed and approved in the creation review. Subsequently, additional Project Leads must be elected by the project’s committers and approved by the EMO(ED).

In the unlikely event that a member of the project leadership becomes disruptive to the process or ceases to contribute for an extended period, the member may be removed by the unanimous vote of the remaining Project Leads (if there are at least two other Project Leads), or unanimous vote of the project leadership of the parent project.

In exceptional situations, such as projects with zero active committers or projects with disruptive committers and no effective Project Leads, the Project Leadership Chain has the authority to make changes (add, remove) to the set of committers and/or Project Leads of that project.

4.6.3 Project Leadership Chain

The leadership for a project is composed of the project’s Project Lead(s), the leadership of the parent project (if any) and the PMC Leads and PMC Members for the top-level project.

---

This is being discussed on Bug 300002. Your comments here or there are most welcome.

I worry that the language we use is so overloaded that it’s easy to get confused. I’ve been working with one group who is “graduating” some code and committers out of their Incubator Project. Naturally, they feel that they need to do a Graduation Review. Except that Graduation Reviews are what we use to move a project from the Incubation Phase into the Mature Phase. What they really want to do is a Move Review; they’re moving some existing code and committers from one project into another. The Incubator Project is going to remain in the Incubation Phase. No graduation involved. Does your head hurt yet?

The current version of the Eclipse Development Process (EDP) has relatively little to say about Move Reviews. Lately, I’ve been thinking that Move Reviews really are the same thing as Restructuring Reviews. With both, you really just need to specify what is changing, and why (and get your IP Log approved). Isn’t a move just a restructuring of code?

Frankly, the whole thing is more than just a little bit confusing. I find myself wondering if collapsing moves and restructures into a single thing might help. Or will it make things more complicated?

Here’s what I’m thinking:

---

Restructuring Review
The purpose of a Restructuring Review is to notify the community of your intent to make significant changes to one or more projects.

“Significant changes” includes:

• Movement of significant chunks of functionality from one project to another;
• Modification of the project structure, e.g. combining multiple projects into a single project, or decomposing a single project into multiple projects; and/or
• Change of project scope

A Restructuring Review may include the movement of significant chunks of code. A move is considered significant if it has an impact on the community (i.e. if segments of the community will notice that the code has moved). This may include entire projects, bundles, and features, but likely excludes small (<250 LOC) fragments, code snippets and individual files. The IP Log of all moved code must be reviewed prior to the start of the review period (this, typically, is a subset of the project’s IP Log). If all of the code is moved out of a project, a Termination Review for that project can be combined with the Restructuring Review.

A Restructuring Review may necessitate the construction of one or more new projects. This tends to occur when an existing project is decomposed into two or more projects. In this case, a Restructuring Review is similar to a Creation Review. Any new projects that are created as part of a Restructuring Review must have their scope explicitly specified as part of the review. The scope of any new project must be a subset of the scope of the original project. Likewise, the set of committers assigned to a new project must be a subset of the committers of the original project (additional committers can be elected to the new project after it is created). Any new projects that fall outside of the scope of the original project, or wish to establish a different set of committers, must undergo the full project creation process.

Committers can be moved along with code into a new project as part of the project provisioning process. Committers cannot be moved along with code into an existing project. In this case, the existing project must elect the new committers into the project.

Ultimately, it is left to the discretion of the project leadership, PMC, and EMO to determine how the process can be leveraged to best serve the interests of the community.

A project should socialize pending changes using established communication channels prior to initiating the review. A Restructuring Review must provide the community with at least one week to review and respond to the changes. Prior to the start of that review period, the community must be provided with (via the EMO) completed review documentation that describes in specific terms what will be changed as part of the restructuring.

This may include:

• Name, description, scope, and committer lists of new projects that need to be created;
• Source and target locations for moves of source code directories;
• Reorganization of builds and downloads;
• Contribution questionnaires (CQs) that need to be moved or piggy-back CQs that must be created;
• Location of the approved IP Log; and
• Other information that helps the community understand the change.

---

When it comes down to it, the entire point of any review is to tell the community what you’re doing and give them a chance to voice their opinion.

As always, I want to hear your thoughts on the topic. This is your process. We have several bugs open; you can add your comments there. Or as comments on this blog. Or by private email (I’m the one wayne that works at eclipse.org). Whatever suits you best.

It started as a bit of a weekend distraction, but turned into something a little more.

I’ve always thought that the Eclipse Projects Gateway page isn’t quite living up to its potential. It seems to me that our user and adopter community should be able to use this page to find out more about the various projects at Eclipse. The page is also looking a little long-in-the-tooth and I thought it could use a little bit of an upgrade. My work-in-progress is here. Keep in mind that it’s very early. Be kind.

The first thing that you should notice is that the page design is clearly influenced by the Xtext-look that seems to be sweeping the landscape. With time, the buttons on the right will likely change. But the main focus of my attention has been the bottom of the page.

I’ve used a little JavaScript magic (with a lot of help from the YUI library) to make a mini project browser. As you click on projects, a short-summary of the project is generated. The hope is that I can make this into something that’s pretty usable and quick. Right now, most of the rendering is done using some PHP on the server, but I may consider moving the data to the browser and letting it do all the work. The hope is that this mini-summary is a little lighter weight than the project summary pages and therefore a little easier for users and adopters to browse. At least that’s the direction I think it’s heading.

One concern I have is that the tree structure probably only makes sense to people who already understand the Eclipse project structure. The initial list of top-level projects does very little to help new users and adopters find their way. I (we?) need to think of a better way to present this to the user. I’m thinking of a flat list populated initially with those projects that are either (a) the most active, or (b) have the most complete information available. Your comments are welcome.

One of the things that I’ve tried to do is to keep the implementation modular. I will likely turn this project browser (and the “Recent Activity” list) into Open Social gadgets that can be used inside the Eclipse Helios workbench. I think that’d be pretty cool. Right?

The IDE for Education (ide4edu) project has made some progress over these last few months with the help of some undergraduate students. Most of the effort has been around making Eclipse just a little easier to use during those first few months of learning.

For the first time user, the flexibility of the JDT is more than is required. Initially, we forked bits of the JDT code and tweaked it to suit our needs. In a relatively short period of time, it became clear that we needed to be a little more radical with our efforts. Forking just isn’t the right way to go. For starters, we ended up with a lot of code that accesses non-public APIs. More important, however, it was just really hard to make significant changes to things like wizards. So, we completely reimplemented the New Java Project and New Class wizards.

Here’s the New Class wizard:

Given our audience, we decided that we could make some assumptions on the user’s behalf. We ask only for the information that we actually need. At this point, the appearance of the wizard is dynamic. The very first time the user asks for it, they are only asked for the name of the class (current implementation also asks for a package, but we’ll probably hide that as well). We assume that, if the workspace is empty (i.e. they’ve never actually created a project of their own), than they don’t need to be bothered to understand things like projects. When they create their first class, we build the first project automatically. As the student’s understanding increases (as determined by the state of the workspace), additional fields are exposed.

When the “Project Name” field is available, the user is free to type whatever name they want there. If the name they type is for a project that doesn’t exist, the project is created. Automatically created projects are built using what we believe are reasonable defaults for students in their first few weeks and months of education.

We’re trying really hard to keep these wizards simple. The dynamic nature of the wizards has been the source of some debate. Is it correct to change the appearance of the wizard? We’ve discussed providing some mechanism to let the user know that the change has occurred. We’ve also talked about making a more general mechanism for determining the competency level of the student so that we can better tune more aspects of the delivery.

One thing that fell out of this implementation is a clear separation between the user interface and the underlying behaviour. This is implemented in two layers: there is a NewJavaClassConstructor type that knows how to actually build a class (and project if necessary), and there is a corresponding wizard that knows how to use the NewJavaClassConstructor to do the the heavy lifting. This separation should allow us to leverage the constructor behaviour in other contexts (I dream of coming up with an entirely different create user interface paradigm).

We’re still pretty far away from a proper release. We have a lot of testing to do, still have a lot more code using non-public APIs than I’m comfortable with, have some work going on to introduce a Scheme environment for students, and are hopeful for some Prolog support. In the meantime, I’m turning my attention to fixing the build (basing it on Athena) so that we can do a better job of getting what we have into the hands of more students to get more feedback.

Hey students and mentors, the Google Summer of Code 2010 has been announced. Keep ide4edu in mind when you’re proposing your summer project!

In my previous post discussing pending change in the Eclipse Development Process (EDP), I tried to present my view of what projects should look like at Eclipse. Scott suggested that a few pictures might help. It’s important to me that these concepts be understood. Therefore, I present: The. Best. Graphics. Ever.

First, there’s the relatively simple version.

A project has a parent. In this case, the parent is a top-level project. The project also has resources. I’ve shown a web site, VCS, access to the build server, and space on the downloads server in this image. I’ve also shown a group of committers. This group of committers has write access to all project resources. If you have write access to one of these resources, you have write access to them all. A committer is a committer is a committer. Projects that need finer grained access control to their resources will have to manage that access via social convention. Social convention seems to be working very well in several projects already, including CDT, Equinox, RAP, and more.

Over the past few years, we’ve tried (for reasons I don’t quite understand) to avoid using the word subproject. This next image, however, shows just that.

Here, we see that a project, in addition to the various resources and committers is has, can also have zero or more subprojects. Each of these subprojects has its own distinct set of resources. Each subproject also has its own distinct group of committers. Sorry to get technical with the language, but the group itself is distinct. Two distinct groups can actually share committers. As I mentioned in my previous post, each group of committers translates into a single UNIX group that has uniform access to the project’s resources.

A project has only the resources it chooses to have. That mid-level project could, for example, decide that it doesn’t need a VCS to maintain source code. That hypothetical project would, in effect, be just like what we call a “Container Project” in the EDP today.

This opens a question: what is the point of those container projects anyway? Well, I think they provide all sorts of different kinds of value. By making them a little more flexible, we leave it to the projects themselves to decide if and how they want to leverage the structure. Projects can still choose to do (or not do) roll up builds, roll up IP Logs, roll up downloads, roll up reviews, etc.

Ultimately, it should result in less work for projects that want to introduce subprojects (for whatever reason). With the current EDP, an operating project must become a container project in order to have even a single subproject. This means that the existing project has to move it’s code into another separate subproject. You can still do that with the change proposed here. You just don’t have to.

I think it’s the ability to choose what’s right for your project that’s key.

I’ve been wrestling with this one for a while… Just what is the role of a container project? I asked this question more formally in Bug 301065. I invite you to make your comments. The Eclipse Development Process is, after-all, your process.

Frankly, I think it’s time to do away with the notion of container projects. I think that we can move forward with a single notion of a project. Here’s what I’m thinking.

• A project has exactly one group of committers that have write access to the various resources owned by that project. From the Webmaster’s point-of-view, this means that each project has exactly one UNIX group.
• Projects manage access to their various parts using social convention (this is already happening today on most Eclipse projects)
• A project has zero or one of each of the following resources: a source code repository, a website, some space on the downloads server, some space on the archive server, access to a build server, etc.
• A project must have at least one project leader.
• A project may have one or more subprojects. Subprojects have their own distinct group of committers, and resources. We may refer to the organization of projects using whatever language makes sense to the target audience and broader community (e.g. “parent project”, “child project”, “subproject”, …).
• A project’s source repository should be separate and distinct from the source repository of its “parent” project.

There is no roll up of committers; the set of committers on a project is exactly that set of people who have been explicitly elected into that role for the project (i.e. being a committer on a “child” project does not give you any automatic rights on the “parent” project).

A project may choose to do roll up builds and provide roll up downloads of its subprojects

The entire leadership chain of a project must be consulted for approval of those things that require approval. A project must, for example, obtain approval from the PMC of its top-level project and the project leadership of any parent project to do a release.

Nesting of projects is limited to a depth of “three”. i.e. top-level.parent.child

I think that the most controversial thing here is the notion that a project can have its own code repository and subprojects. For what it’s worth, this is already happening in some Eclipse projects. I’ve always found the limitation against this very artificial. The one part of this that bothers me is overlapping repositories. I don’t think that subprojects should exist within the source code repository of the parent project: this just leads to confusion with our Dash tooling and adopters (who have to sort out which parts of the repository belong to which project). Confusion leads to anger. Anger leads to pain. Pain leads to suffering. Of course, some projects are doing this today, so maybe I’m overstating the problem (we should be able to sort out any possible issues with Dash).

Probably the second most controversial thing suggested here is the notion of a single group of committers for all of a project’s assets. Experience has shown us that managing fine-grain control to the various resources just doesn’t work well. Nobody seems to understand who should (and does) have access to what. For me, the responsibility that comes with being a committer includes the responsibility to know where you’re supposed to be working. On the Examples project, for example, committers working on the Toast example, just don’t commit on the Slideshow example. They just don’t. This, despite the fact that they have full access. Frankly, if you’re not sure that you can trust a developer to follow social conventions, then maybe you need to rethink nominating them as a committer.

At least that’s what I think. What do you think?

The Eclipse Development Process (EDP) defines “incubation” as a phase that projects go through on their way to maturity. New projects start in incubation to allow the project’s committers time to familiarize themselves with the state of being an Eclipse project, give them time to start developing a following in the community, and allow them some leeway to make and correct early mistakes. Projects in incubation can make releases, but they must be pre-1.0 releases, and must be labeled as “Incubating”. Frankly, I think that incubation is a fantastic idea and intend to keep it very much a part of the EDP.

An interesting use of the incubation phase as emerged and is the topic of discussion on Bug 300000 (which sounds a little like the title of a really bad sci-fi movie): the perpetual incubator. Many mature Eclipse projects now have projects that are intended to remain in perpetual incubation. These never-grow-up “Peter Pan” projects are an excellent place to innovate, test new ideas, and grow functionality that may one day be moved into the mature project. It is a curious, however, that the very notion of an incubator project is essentially (though not explicitly) forbidden by the EDP in its current form.

With the next revision of the EDP, I’d like to acknowledge the notion of an “Incubator” project. In Bug 300000, Holger Voorman and others propose that we adopt the name “Labs” for this notion to conform with precedents set by other open source organizations. Frankly, though, I believe that we’ve already established our own precedent and intend to run with it (I’m sure you’ll let me know if you disagree).

As I see them, Incubators are operating projects that never have releases; they do not require yearly continuation reviews; and they are not part of the release train. Incubators have builds, and downloads. They conform to the standard incubation branding are subject to the IP due diligence rules outlined for incubating projects. Incubators do not graduate.

I’d like to avoid creating any new process around the notion of Incubators. I’m hopeful that it will be enough to change some of the language in the EDP (and the Incubation HOW TO document) to allow projects to simply declare themselves as an incubator and leave it at that.

Unfortunately, I think that we may have a language issue. An “Incubator” project is in the incubation phase. However, a project in the incubation phase is not necessarily an incubator. A project like Mint, that one day intends to graduate, is in incubation. The Examples and the Equinox Incubator projects never intend to graduate and so are incubator projects in the incubation phase.

Maybe it’ll help if I stop thinking about it so much.

These are heady times in Eclipse project-land. I feel busier than a… well, I’m really, really busy.

We have creation reviews pending for the Modeling Team Framework and Object Teams proposals. Object Teams is an already-established, mature project that’s moving to Eclipse. The logistics of the move are interesting enough (the project itself is darned interesting to boot). This project will enter eclipse.org under incubation, but intends to graduate with their first release; this makes a lot of sense for a project with established code base as it allows their developers time to get used to the Eclipse Development Process, but then quickly establish themselves as mature technology. There is some overlap with this project and some existing Eclipse project, but discussions are underway to see how these technologies can work together, or carve out their own unique corners.

New project proposals are flowing it at an incredible pace.

The Enterprise Modules Project, code-named Gemini, was proposed in late 2009 and is due for creation soon. This project will “provide a home for subprojects that integrate existing Java enterprise technologies into module-based platforms, and/or that implement enterprise specifications on module-based platforms.” As a start, the reference implementations for eight OSGi Enterprise Expert Group specifications will be created as subprojects. Gemini is complemented by Dynamic Enterprise Application Platform Project, Virgo, which intends “to provide a runtime platform for the development of server-side enterprise applications built on top of Equinox, and optionally using modules from the Gemini project.”

Also queued up are the Graphiti, JavaScript Development Tools, Mangrove - SOA Modeling Framework, Mylyn Reviews, and ScalaModules proposals. There are about a dozen other proposals pending that I’d love to tell you about, but will have to wait.

There seems to be no end of interest in bringing projects to Eclipse. This is very gratifying to me and I do love working with the hard-working folks involved. Parts of our process can be frustrating though, and I’m working hard to address them while balancing the important principles that define what it means to be an Eclipse project. I’ll be presenting the Eclipse Board with an overview of the changes to the Eclipse Development Process that I’m working on in mid-February. I intend to have a draft ready for presentation to the Board in time for EclipseCon. In the meantime, I’ll be discussing the changes here and in Eclipse Bugzilla. If there are specific issues that you’d like addressed as part of this revision, either open a bug or email me directly (I’m the only “wayne” at the Eclipse Foundation).

I made a mention to Mike a couple of weeks back that I wanted to make a few changes to the Eclipse Development Process (EDP). He seemed to think that evolving our process was a grand idea and reminded me that changes to the development process require approval from the Eclipse Board. I knew that already, but it was still nice of him to remind me (he’s one of those give-you-gentle-reminders-and-let-you-get-it-done sorts of bosses). But I digress.

I was originally thinking of only a few small changes. But I’ve gathered up a bit of steam and have decided that maybe it’s worth trying to make some more fundamental changes. I’ve been in this role for about nine months; after nine months of immersion in our process, I have some ideas where I think we can do a better job. But before I start making wholesale changes, I though it might be prudent to try and bring some of my ideas before the community to gather feedback. Keep in mind that–ultimately–any changes we make to the development process do have to be approved by the board, are bounded by our bylaws, etc.

There are several key principles that I believe underly the EDP (in no particular order, because they’re all important):

• Transparency
• Openness
• IP cleanliness
• Quality

I believe that most of what we find in the EDP is intended to support one of these key principles. Correct me if I’m wrong.

There’s one more thing that I’ve added to this list:

• “Do the right thing” ahead of process.

Recently, I’ve been thinking a heck of a lot about reviews. Release reviews in particular. I’ve had many conversations with committers about release reviews and why we do them. Years ago, we required that project members “meet” the community on a scheduled call to present their case for release. Attendance on those calls was never particularly strong, and so the call melted away and we’ve effectively changed over to a “review period” of a week that infrequently involves some kind of call. The project still has to provide review “docuware” and the working theory is that interested parties from the community read this carefully prepared documentation and then theoretically pose questions that are theoretically answered. I don’t mean to sound overly theoretical, but my experience is that most of these documents are read by Anne and me, and few others outside of the project. Correct me if I’m wrong.

I’ve written in the past about how valuable I think these documents are. This hasn’t changed. I still think that it’s valuable to a project to capture what they’ve done leading up to the review. But I wonder if the energy that goes into this document could be better directed at something with a little more shelf-life. Like, perhaps, a “new and noteworthy” document for the release.

So here’s what I’m thinking. Keep in mind that we’re still very early in this process and that this is little more than an idea at this point.

At the beginning of a development cycle, projects are required to provide a project plan in the standard format. The project plan document has a handy spot for listing release dates. Through the plan, the release dates are easily accessible by interested parties (users, contributors, adopters, and the broader community). We can even parse out these dates and include them on the “What’s New” page and RSS feed.

So each project can use their plan (in the standard format as required by Board mandate) as the mechanism to notify the community of the release. This gives the community the project’s entire development cycle to get involved, ask questions, challenge assumptions, and what-not.

When a project is ready to complete their release (as documented in their project plan), they get IP Log approval, put the finishing touches on their “new and noteworthy” documentation, and open a bug that states that they’re ready to release. Either the project’s PMC or the EMO (or both) adds their +1 to the bug and Bob’s-your-uncle. The approval is required to ensure that the aforementioned principles (and documentation requirements) have been observed. By this point, however, approval should be little more than a rubber stamp since the PMC and EMO will have been aware of the pending release for some time.

Frankly, if the community hasn’t gotten involved by this point, I’m not sure how adding an additional week of review time will improve matters…

I like this approach because the two documents that are needed (the project plan and “new and noteworthy”) (a) are things a project should be doing anyway, (b) are, at least in the case of the project plan, actual requirements set down by the Eclipse Board, (c) convey the really important information that is contained in the review documentation, and (d) are immediately useful. Correct me if I’m wrong.

Again, this is just an idea right now. It’s something I’d like to explore. In my mind, this should be easier for projects and more valuable for the community. I believe that it is a natural evolution of the process.

Thoughts?