Community
Participate
Working Groups
In order to simplify the maintenance of unix groups so as to allow the eclipse.org infrastructure to continue to scale, I propose that we standardize the groups <-> project mappings. Specifically, I propose: * each project X has a group X-dev containing all the committers on that project * each project X with component Y has a group X-Y containing all the committers on that component of that project These groups will be automatically maintained by scripts that run off the official Foundation database of Committers. You can see the committers and components of a project through the [members] link in the project-specific box on the portal http://portal.eclipse.org/ or the committers of a project (but not components) via the http://www.eclipse.org/projects/lists.php?list=allcommitters link. Using this new standardized, webmasters will no longer add and remove people from groups. To add someone to a group, they must be elected as a committer on that project or component (via the portal). To remove someone from a group, they must be removed as a committer from that project or component (currently via email to emo@).
Slight modification: * each project X has: ** a group X-dev containing all the committers on that project ** a group X-admin of committers with access to the download server ** for each component Y, a group X-Y containing all the committers on that component
Slight modification: * each project X has: ** a group X-dev containing all the committers on that project ** a group X-web containing all the committers on that project who have access to the project's website ** a group X-admin of committers with access to the download server ** for each component Y, a group X-Y containing all the committers on that component
(In reply to comment #2) +1 This would be so nice.
Am I ever going to get this correct? * each project X has: ** a group X-dev containing all the committers on that project ** a group X-web of committers who have access to the project's website ** a group X-download of committers with access to the download server ** for each component Y, a group X-Y containing all the committers on that component
+1 . This would totally rule.
Created attachment 75139 [details] Description of existing groups -> new groups Attached is an audit/evaluation/description of what would happen to the existing unix groups under this new policy.
For more information, see my blog post on this topic http://eclipse-projects.blogspot.com/2007/08/standardized-groups-scaling-our.html
I like it. Being one of the oldest projects, our groups are a mess. We tried to clean them up a couple of years ago but this would help finish the job. +1
(In reply to comment #5) > +1 . This would totally rule. +1 from me (also on behalf of Ed). For Modeling, we currently have this mapping: Project: X, Component: Y [group X-Y] /cvsroot/modeling/org.eclipse.X/org.eclipse.Y [group X-Y-releng] /cvsroot/modeling/org.eclipse.X/org.eclipse.Y.releng - or - [group X-Y] /cvsroot/modeling/org.eclipse.X/org.eclipse.X.Y [group X-Y-releng] /cvsroot/modeling/org.eclipse.X/org.eclipse.X.Y.releng There was some back and forth last year when Modeling.MDT was born about whether to put everything in X-Y or X-Y-releng, and the consensus at the time was for the latter; I preferred the simpler X-Y but was outvoiced so if the EMO and webmasters prefer to make things simpler, I'm all for it. Note that this means I'll end up being in fewer linux groups but will need to be voted in as a committer on more projects than I am now in order to continue my work as .releng guy for the projects I maintain. I'm cool with that. ;-)
+1. Great idea.
+1, this should make things interesting in regards to that bug I have open about a Foundation XMPP server :P (https://bugs.eclipse.org/bugs/show_bug.cgi?id=126089)
+1, that would help.
+1
Platform UI doesn't work this way as we are effectively 4 components databindings jface workbench ide We have at least 2 committers who are databindings only and one who is JFace only (Tom Shindl). I don't actually object to this idea as all of the people in question are trusted people but it will change the dynamic of how commit rights work on the platform. Right now only 2 of us are active in the ide (Boris and I) so were we starting from scratch it might look even more restricted.
There are even more subcomponents in Platform UI - Common Navigator and Tabbed Properties. Unless there are other teams that have subcomponents with special access rights, it seems that we need to accommodate the 1:1 mapping between groups on dev.eclipse.org and components, either by being less fine-grained, or creating new components, or a combination of both. I'm not sure if this bug is the best place to discuss this though.
I need a bit of clarification on the proposed change. What is the meant by the term component? I assume it's the same as a bugzilla component? For example, eclipse->platform and component swt? Our Core, UI and SWT teams have committers that have rights to specific projects but not the entire scope of projects within the component. For example, SWT has committers that contribute libraries to a specific fragment such as org.eclipse.swt.gtk.linux.s390. The Core teams have committers on filesystem fragments with the same characteristics. The UI team have committers with rights to a subset of the entire UI component. Is there any way that this level of granularity can be preserved under the the proposed changes? Also, I'm not really clear how the permissions for a project where all our committers require access would be implemented. For instance, all committers need to access org.eclipse.releng for the builds. If the foundation is proposing generating rights from a database it would be nice to see a list of the proposed changes in advance to avoid any disruptive changes :-)
The access to releng components is also a concern for the modeling projects since someone like Nick acts as an releng admin for many components and for most of those components he's not really a committer for the component in general and folks may want to restrict his rights to access only the releng/web infrastructure. (Of course he won't be upset if folks have to do this themselves!)
Kim, the attachment lists all existing groups. It seems that Bjorn's current suggestion is to delete the sub-component groups (for example plat-jfaces aka JFace, which is part of the Platform UI component but has active committers who don't have commit rights for all of Platform UI).
There seems to be some confusion over what this is. And I can see why. Correct me if I'm wrong Bjorn, but this is purely related to the Unix group permissions on the files at eclipse.org. A component == a group. It's purely mechanical. When we vote for committers, we're voting for them to be added to the Unix group to give them write permission to the files owned by that group, either through CVS/SVN or directly via ssh (and actually, you can change the CVS files directly with your power, but I didn't tell you that ;). This effort seems to be simply on standardizing group names to allow for automation of adding a new committer. And while we're at it, it's also an opportunity to clean up. The groups don't have to match bugzilla components, and in a lot of cases can't since the fixes to some bugs span groups. One issue I see, though, is that committer votes that were for specific components would need to be revisited. If I gave someone committer rights to a certain set of files owned by a group, and this reorg gives them access to more files, then I believe we would need to have another vote. And there is an issue if it's less files. This is what happened when the CDT reorg'ed their groups a couple of years ago where we essentially moved everyone from the component level to the project level, i.e. ended up with one component.
I'd forgotten about my committer rights in org.eclipse.build.tools (but not in its parent, org.eclipse.releng.basebuilder). That adds another argument for subcomponent access... I'd need something akin to group "platform-releng-basebuilderbuildtools", which if I'm not mistaken is 4th-level nesting: Bugzilla: Classification > Product > Component > ??? Other: PROJECT > Project > Component > subcomponent eg. Eclipse > Platform > Releng > basebuilderbuildtools The only way around this in Bugzilla would be to create releng.basebuilderbuildtools as a new component [1] or to create a new Product for "Platform Releng", like TPTP has for its subprojects [2], and shuffle everything around. [1]https://bugs.eclipse.org/bugs/describecomponents.cgi?product=Platform [2]https://bugs.eclipse.org/bugs/enter_bug.cgi?classification=TPTP So... are PMCs cool with effectively promoting people from subcomponent access to component access in order to comply with the maximum 3 level nesting? Or, are they cool with having to reorg their project/product structure in order to shoehorn their existing setup into this new structure? I see only these two paths, if we're going to make this work (for the platform, anyway). One could also make the claim that emf.query.releng is a subcomponent of emf.query which is a component of emf which is under the Modeling Project, giving rise to the 4th level nesting again: Modeling > EMF > emf.query > releng I'd happily be a "subcomponent" committer for the .releng modules I maintain... but I'd really prefer to not have to go through another Bugzilla reorg to make it happen. We just finished one, and I'm fairly certain Matt and Denis have better things to do that shuffle the bits one more time. ;-) That said, we already have two Releng components in bugzilla which serve my needs: Modeling > EMF > Releng (for EMF product/component releng) Modeling > Modeling > Releng (for cross-product/component releng)
Sorry, I missed the attachment which of course is very useful :-) It seems that for the swt fragments the current rights are not restricted to the fragment for individuals who just contribute the binary. The group is plat-swt which is the same group that is assigned to the rest of the swt components. This is unexpected since I think the intent of the committer vote was only for the fragment. The existing core filsystem the fragments seem to be limited by different groups, but you are proposing deleting the specific groups and giving them rights to all groups covered by plat-core. I agree with Doug that this reorganization will mean that committers will have rights to more projects than was the original intent of their vote. Some teams may not be in a position to extend committer rights to an entire component to an individual who only contributes a binary fragment a few times a year.
Now is probably the opportunity to iterate a few pain points vis-a-vis the Eclipse project specifically, and their groups and project layout -- from the Webmaster's side of things. Most often, the CVS-committer-group-related requests from the Eclipse project do not explicitly state the group names when asking for committer rights or for the creation of new directories. We often need to ask, or we simply take a stab at our best educated guess. Contrast this request: Bug 170992. Clear. To requests like these: bug 172308 or bug 171302 "the swt team needs access" requires us to know what constitutes the SWT team. plat-swt makes sense by looking at who's in it with my own human eyes, but to someone who doesn't know, the lack of a standardized component<->group mapping convention requires us to investigate. Worse yet, bug 157711. We can't give someone access to one directory, as requested, so we end up tossing them in the group that owns the directory. Most new committer requests come in this format anyway (grant new committer X access to these 7 directories ... ) so in the end, we likely end up granting *more* access than what's required. The only team on the Eclipse project that actually seems to have a clear grasp on the unix groups is Equinox. This is not meant to be a stab, complaint or criticism. It does outline the need to standardized group<->component<->project mappings.
Can we not keep the groups the way they are today and just change their names to fit the standard, i.e., every group you have now is a component. Then if you want to clean things up you can have the appropriate votes to do that over time. And I believe Denis is right and a lot of that has to do with the new committer request form. It should have asked explicitly what Unix groups to add the committer too, not the "component" or whatever it was.
> This is not meant to be a stab, complaint or criticism. It does outline the > need to standardized group<->component<->project mappings. > No. Actually, it outlines the fact that a simple story doesn't capture the complexity of dealing with commit rights for Eclipse. As several people have identified above, we need finer granularity than the component level. Instead of attempting to shoehorn our requirements into the simple strategy, we should be looking at ways to resolve the communication/visibility issues that Denis has identified.
(In reply to comment #14, comment #15, comment #16, and comment #24) As a sort of meta comment, the Eclipse Development Process (the EDP) describes Top-Level Projects, Projects, and Components. It does not mention Sub-Components. (http://www.eclipse.org/projects/dev_process/development_process.php#4_Structure_and_Organization) The Bylaws (http://www.eclipse.org/org/documents/Eclipse%20BYLAWS%202003_11_10%20Final.pdf section 7.1) use the term "Subsystem" instead of "Component" but have the same description. Even the Eclipse Charter (http://www.eclipse.org/eclipse/eclipse-charter.php) only describes Components and not Sub-Components. The Eclipse Charter does mention "Ports", but they appear to be the same as Components with a slightly different initial membership rule. We can always improve the EDP to be the way we (collectively) want - I even added a clause to last year's revision effectively forcing us to do that (http://www.eclipse.org/projects/dev_process/development_process.php#7_Revisions). (In reply to everyone) Projects can have as many Components as they want. Components can even overlap. (In reply to comment #19) Doug, yes, you are correct: "A component == a group. It's purely mechanical. When we vote for committers, we're voting for them to be added to the Unix group to give them write permission to the files owned by that group..." and that "This effort seems to be simply on standardizing group names to allow for automation of adding a new committer." (In reply to comment #23) It's not just about what groups a new committer should be a member of because (according to the Charters) the Component membership controls who gets to vote on issues affecting that code. We could just say "if you're a member of unix group X then you can vote on issues affecting the code owned by unix group X". (In reply to comment #24) This proposal is not really about communication - it's really about improving sysadmin efficiency. (I talked a bit about this in my blog post.) In order to continue providing more services for the Eclipse eco-system (e.g., recently blogs.eclipse.org and live.eclipse.org) we (the EMO) needs to provide existing services more efficiently (fewer webmaster hours per request). If we can automate the group management from a database and then provide access to that database to the project teams, then it will allow the webmasters to help you/us in other ways. (P.S. "access to the database" mean "in a way that follow the legal, charter, and process rules", i.e., new election -> add to groups; etc.) We (the EMO) are open to alternate proposals about how to structure this change as long as it meets the requirements of: (1) less work per request and (2) follows the legal, charter, and process rules. Like I said in my blog post: it would be great if each of us could have a dedicated 24x7 IT person; then reality impinges and we have to find a lower-cost solution. We're listening...
In a follow-up to Bjorn's comment #25: what we're looking for is something standardized so that we can easily write tools to handle it. Tools are the thing that let us (the webmasters) be more efficient helping you, and thus give us more time to do the things you really want to have improved or implemented. Less work per request could be done while keeping human hands completely involved in the process but in order for us to really be efficient we need a solution that lets us automate large chunks of the process. Keeping the existing group structure won't work in a sustainable way so we need a solution that does that, and which standardizes things enough that the tools to manage it are supportably simple. I personally like Bjorn's suggestion and I think it makes a lot of sense for 9 out of 10 projects. For those who don't see it making sense for them, please help us come up with suggestions to make the world better for everyone. We really do want to help, but we need to shake things up a bit to do it in such a way that it beneftis the whole community.
I think we've established that the administrators want to define component == group and that component != bugzilla component. Thus committers will be voted to be in a group/component and generally only existing group members will vote on adding new members to that group. This says nothing directly about how group access permissions are assigned to the hierarchical structure of files and directories in the file system or CVS. I assume that a particular directory subtree could belong to multiple groups so in that sense, there is a degree of flexibility in what each group is given access to. So one group could be given access to a big subtree (the whole tree) while another "subgroup" could be given access to smaller portions of that same subtree. That pretty much seems to give the flexibility that some projects seem to be looking for. Is this a consistent and sensible interpretation of what's being proposed? I think the feature it misses is that a group that controls all of the big tree would not get to vote for adding members to a group that has access to only a smaller part of the tree. Is that a bad thing? Do we need to define a component hierarchy so that higher level groups have voting rights to control entry into the lower level groups; note that this becomes problematic in that lower level groups would appear not to have voting writes for the higher level groups yet entry into those groups impacts them directly. This is what seems to be forcing the group structure to be only one level deep. Does anyone see a way to define a sensible voting pattern for a hierarchical component structure?
(In reply to comment #27) > This is what seems > to be forcing the group structure to be only one level deep. Well, it's more the larger Eclipse community, through the Eclipse Development Process and the Project Charters, that are forcing the one level deep structure. Fortunately, those are changeable through public discussion and votes. We (EMO and webmasters) will support with tools and portals and all whatever hierarchical structure is chosen. This (Top - Sub - Component) happens to be the current structure.
What is the time frame for doing this work? Anything definite? We in WTP are planning a bit of re-structuring in a few weeks (organizationally) which might effect some of the webtools groups, so easier to do after that. In the mean time, 1. who is in wsvt-home? (How can I query?) This was for the old "WSI" project which merged info WST (code wise) and suspect all the necessary committers have already been carried over ... but, seems it should be checked. 2. wst-jsdt-dev should not be deleted ... at least until it's determined how/when to merge it with WST, or some other (new) wtp project. There's people there actively doing work, and as far as I know that's their only access group. 3. Should the projects under the top level project of "webtools" be changed to have webtools in their name? For example, should "jsf-dev" be "webtools-jsf-dev"? Seems that would help "tools and automation" substantially. In general, let me say, I don't think "we" ever use these groups directly, in any scripts or anything ... so, what ever is easiest for administration is fine with us.
(In reply to comment #29) > What is the time frame for doing this work? Anything definite? This fall. Not imminent because we want to find the right solution for everyone (hence this bug for discussion). > 1. who is in wsvt-home? (How can I query?) getent groups | grep wsvt-home > 2. wst-jsdt-dev should not be deleted ... If, after your restructuring, there is a jsdt component of the WST project, then there will be a wst-jsdt group, so you should be fine. > 3. Should the projects under the top level project of "webtools" be changed to > have webtools in their name? For example, should "jsf-dev" be > "webtools-jsf-dev"? > Seems that would help "tools and automation" substantially. I went back and forth on this. Then I decided this way to be consistent with the past. But as I type in my explanation here, it doesn't make any sense, so, yes, it should be toplevel-dev, toplevel-sub-dev, and toplevel-sub-component-dev.
(In reply to comment #30) > > 3. Should the projects under the top level project of "webtools" be changed to > > have webtools in their name? For example, should "jsf-dev" be > > "webtools-jsf-dev"? > > Seems that would help "tools and automation" substantially. > > I went back and forth on this. Then I decided this way to be consistent with > the past. But as I type in my explanation here, it doesn't make any sense, so, > yes, it should be toplevel-dev, toplevel-sub-dev, and > toplevel-sub-component-dev. I agree that the longer names are easier to understand. If we add anything more than three levels I think it would start to get unwieldy to type and though the SuSE command line tools all seem to support long names, there are likely others which will chop the names off to (unfortunately) the least specific information. Karl
(In reply to comment #27) > I assume that a particular > directory subtree could belong to multiple groups so in that sense, there is a > degree of flexibility in what each group is given access to. So one group > could be given access to a big subtree (the whole tree) while another > "subgroup" could be given access to smaller portions of that same subtree. > That pretty much seems to give the flexibility that some projects seem to be > looking for. Is this a consistent and sensible interpretation of what's being > proposed? Actually, due to the way unix groups work a single group can have ownership of a file/directory. So that means that anyone who is a member of the higher level group who should have write/read access to the lower level group's files will have to be a member of both. Ideally the group naming will match project structure, be not very deep, and still provide enough flexibility to fit the needs of the projects. Essentially we are trying to remove entropy from our system(s). ;)
In comment #25, Bjorn Freeman-Benson wrote: > (In reply to comment #24) > This proposal is not really about communication - it's really about improving > sysadmin efficiency. > I understand, but at least some of the problems Denis outlines in comment #22 *are* communication problems. Here is an initial stab at an alternative proposal: First, make the groups be first class in the committer tools; so that for example, when a PMC member is asked what things a new committer should have rights to, they should be presented with a list of *groups* rather than be asked for a specific list of CVS and website directories. Since we don't know the groups, we can't provide them, and attempting to guess the directory mappings is obviously error prone. Second, give the PMC for each project overall ownership of the groups that are related to that project. That is, let the PMC define the set of groups. Any restrictions on the nesting of the groups should be removed. As is true now, the members of each group would be responsible for voting new committers to that group. Then, given how infrequently the group<->directory mappings would change in practice, actual association of groups to directories could still be done by the foundation infrastructure team. I am aware that implementing this would require changes to the charter/development process.
(In reply to comment #33) > First, make the groups be first class in the committer tools; ... Sounds like a good idea. > Second, give the PMC for each project overall ownership of the groups that are > related to that project. That is, let the PMC define the set of groups. Ok. As long as the group membership abide by the committer legal agreements and elected project memberships, I can see that working. Denis - do you see operational issues (assuming that group creation/removal/membership is managed by the PMC through the portal)? Mike - what process do you suggest for the following: 1. adding a new person to a group: an election, correct? 2. removing a person from a group: PMC or project lead, correct? 3. creating a new group: PMC? how is this different than adding a new person to a group? one could imagine that if I didn't want to go through the bother of an election to group X, I could create a new group X' with all the members of X plus the new person and then switch ownership of all files from X to X'.. how to avoid that? 4. changing file group ownership? moving a file F from an existing group P to a new group Q is similar to adding the new people Q-P to group P. Should there be an vote to change file ownership? That both seems reasonable and, at the same time, somewhat heavy. Also note that moving F from P to Q has the effect of dropping people the Q-P people from group P (more of less). As per the Development Process, I'm planning a 2007 "improvement" to the Process, so am happy to incorporate these changes and take the simultaneous Charter changes forward to the Board (in other words, I'll do the legwork) if you all will just help me a bit defining the best policies.
(In reply to comment #34) > Mike - what process do you suggest for the following: > 1. adding a new person to a group: an election, correct? Yes. > 2. removing a person from a group: PMC or project lead, correct? Yes. > 3. creating a new group: PMC? how is this different than adding a new person to > a group? one could imagine that if I didn't want to go through the bother of an > election to group X, I could create a new group X' with all the members of X > plus the new person and then switch ownership of all files from X to X'.. how > to avoid that? My model was that new group creation would be uncommon, and heavyweight, akin to setting up a new "subcomponent". I think in terms of assigning rights to subtrees of the servers, so creating a new group that is intended to differ from an existing one based only on the member list seems bogus to me. If the foundation manages the mapping between groups and the directories on the servers, then there is a control point that could potentially be used to prevent such cases. Similarly, I think of the groups as effectively partitioning the members of the project into subareas of responsibility. Adding someone to a group gives them rights at a given level (e.g. the whole Eclipse Project, or just the Platform, or just SWT, or just the Mac port). Note that my argument for assigning "create a new group" privileges to the PMC was effectively just to accept responsibility for the work, given that I want the increased flexibility. If you can suggest a better locus of control, I'm fine with that. [Also, I realize that none of the above is a prescription for the solution. I'm just trying to provide background on where I was coming from.] > 4. changing file group ownership? moving a file F from an existing group P to a > new group Q is similar to adding the new people Q-P to group P. Should there be > an vote to change file ownership? That both seems reasonable and, at the same > time, somewhat heavy. > I'm trying to think of a case when you would want the ownership to *change*. It would be uncommon -- something like creating a new subarea of responsibility, or perhaps doing some large scale clean up of the group<->directory mappings for a crufty, old project [names withheld <g>]. If that's true, a vote would be fine. > Also note that moving F from P to Q has the effect of > dropping people the Q-P people from group P (more of less). > Sorry, I didn't get this. [There's a reason why I'm going on vacation on Friday.] > As per the Development Process, I'm planning a 2007 "improvement" to the > Process, so am happy to incorporate these changes and take the simultaneous > Charter changes forward to the Board (in other words, I'll do the legwork) if > you all will just help me a bit defining the best policies. > What we need is to find a definition that encompasses the informal notion of "subareas" in some kind of manageable fashion. The direct association with groups is the obvious one, but it has problems (as you identified). The most important issue for me is making the process of telling the foundation what rights a new commiter should have as easy as possible, without constraining the ability to partition the work areas in ways that make sense to individual projects.
I'm still trying to figure out how voting would work for nested components. Since files can be owned by at most one group (I didn't know that), it's clear that membership in a parent group would necessarily imply membership in all child groups recursively. So adding someone to a group would involve votes from all the members of the recursive child groups as well as all the members of all the parent groups but not recursively any of those parent's child groups, right? I also assume that a subproject would not itself correspond to a group, so a project might have a single root component, or might have multiple root components, right? It's still not entirely clear that we couldn't accomplish pretty much the same goal with only just one level of component by simply having people be elected to multiple groups if they are to have access to all the code covered by those groups. One might imagine a group splitting and that all the members will be common and after that, the membership might diverge.
(In reply to comment #35) > (In reply to comment #34) > My model was that new group creation would be uncommon, and heavyweight, akin > to setting up a new "subcomponent". I think in terms of assigning rights to > subtrees of the servers, so creating a new group that is intended to differ > from an existing one based only on the member list seems bogus to me. If the > foundation manages the mapping between groups and the directories on the > servers, then there is a control point that could potentially be used to > prevent such cases. Ah, but we don't control the group to directory mappings, and that's where it's sticky. Anyone who is in more than one group can change the ownership of any file/directory to any group they are a member of, if they own the file/directory. So there is no control point there. > Note that my argument for assigning "create a new group" privileges to the PMC > was effectively just to accept responsibility for the work, given that I want > the increased flexibility. If you can suggest a better locus of control, I'm > fine with that. Part of the whole inefficiency of the current situation is the dissimilarity between the projects. Denis is on vacation this week and his vote certainly counts more than mine from the Foundation's standpoint, but in my opinion any solution that essentially givs projects the ability add or remove any groups they like puts us more or less exactly where we are now. > [Also, I realize that none of the above is a prescription for the solution. I'm > just trying to provide background on where I was coming from.] Me too. :) > I'm trying to think of a case when you would want the ownership to *change*. It > would be uncommon -- something like creating a new subarea of responsibility, > or perhaps doing some large scale clean up of the group<->directory mappings > for a crufty, old project [names withheld <g>]. If that's true, a vote would be > fine. This happens all the time, actually. At the component level and lower, people can and do move files around between groups all the time (Note that many projects don't elect committers to just one component, so in most cases people can re-arrange code ownership quite a lot without affecting any legal paperwork). It's only when you need to move whole trees of code around that we are involved, and that's usually when it gets very sticky. Some of the hardest things are when code gets promoted from one top level projects to another, for example. When the projects have completely dissimilar group structures this can become quite a morass. > What we need is to find a definition that encompasses the informal notion of > "subareas" in some kind of manageable fashion. The direct association with > groups is the obvious one, but it has problems (as you identified). Personally I think even the official election of committers only to a specific component is already overkill and introduces a big layer of bureaucracy to all of our processes. Officially sanctioning a further level of administration (if I understand you correctly!) sounds like a step in the wrong direction to me. > The most important issue for me is making the process of telling the foundation > what rights a new commiter should have as easy as possible, without > constraining the ability to partition the work areas in ways that make sense to > individual projects. I agree that should be one of our most important goals. I know that is frustrating for everyone, heck it doesn't help us out any, either. But for us what is also important is making sure that people have access to what they're supposed to, and not to what they're not. With in some cases tens of groups to provision for people this is just not possible. There is a certain point where just the shear number of groups and complication of the bureaucracy makes it likely to fail. Note that on most of the newer projects we have no more than 3-4 groups for the whole project. I realize there are differences in the complexities and size of some of the projects, but isn't there a simple model that doesn't call for infinite permutability but which still meets all the requirements? Here's a metaphor: In system administration you can either build all of your machines the same way and tailor just the very specific details of each one to the task at hand. Or you can build each machine from the ground up to suit a particular purpose. Some people argue #2 is better. Sure performance might be a little better. You might get a few things you don't get with #1. But in the end #1 is far more supportable and is the method used by nearly every large shop in the world. Imagine automated upgrades on an infrastructure built like #2. It's just not going to work in that environment. This is exactly what we're talking about here.
That last comment is me, not Denis. For some reason this browser was logged in as webmaster. Karl
(In reply to comment #37) > (In reply to comment #35) > > (In reply to comment #34) > > > > What we need is to find a definition that encompasses the informal notion of > > "subareas" in some kind of manageable fashion. The direct association with > > groups is the obvious one, but it has problems (as you identified). > > Personally I think even the official election of committers only to a specific > component is already overkill and introduces a big layer of bureaucracy to all > of our processes. I (mostly) agree with Karl. While some projects might need to manage things at the component level, I'm not sure that translates into CVS access. Instead, to me, access and legal documentation should be strictly controlled at the project level. Lower than that, I think, projects could still have components, still have their own processes around that, definitions, committer elections, etc. and if the Committer tools helped manage those elections, that'd be great, but, once someone was elected to a component, I'm not sure that should have any other effect on Eclipse Foundation processes or staff. Instead, I think the component level significance is mostly "social" ... that is, the component level groups should have their own design decisions and document any votes there if needed to decide a direction, if something should be API or not, etc. but, it would just be "peer pressure," so to speak, that would prevent someone from committing code to the wrong component, one which they were not a member of. There is after all, a _record_ of what people do in CVS, so not too much could happen without others in the component from knowing what was done and who did it, so corrective action could be fairly easily taken, especially since this would presumably be rare. But, for all this to be make sense, we'll have to back up and work on our definitions of "top level project", "project", and "component". I know some "components" that I think should be classified as "projects". So ... lots to do.
(In reply to comment #37) > Ah, but we don't control the group to directory mappings, and that's where it's > sticky. Anyone who is in more than one group can change the ownership of any > file/directory to any group they are a member of, if they own the > file/directory. So there is no control point there. > Actually, I was making the argument that you *could* do this, as I indicated in my original comment ("...actual association of groups to directories could still be done by the foundation infrastructure team.") > > Note that my argument for assigning "create a new group" privileges to the PMC > > was effectively just to accept responsibility for the work, given that I want > > the increased flexibility. If you can suggest a better locus of control, I'm > > fine with that. > > Part of the whole inefficiency of the current situation is the dissimilarity > between the projects. Denis is on vacation this week and his vote certainly > counts more than mine from the Foundation's standpoint, but in my opinion any > solution that essentially givs projects the ability add or remove any groups > they like puts us more or less exactly where we are now. > It's not the existence of the groups that is the problem, it's the inefficiency of the process we use to deal with them. How often are new groups created? Is this really a major time sink for *anyone*? The real problem is that currently the foundation infrastructure team has to _know_what_the_mappings_are_. I honestly believe that, if we could remove that part of the puzzle, the rest of this would seem easy. That's why I was trying to give control of the groups to the individual projects, and require the communication to be at the group level. I can't imagine it being harder to be given a list of groups to add someone to, than it is to be told "add them to to project X" and know that that means (X-dev (and maybe X-web and X-admin and X-download and...)). > > What we need is to find a definition that encompasses the informal notion of > > "subareas" in some kind of manageable fashion. The direct association with > > groups is the obvious one, but it has problems (as you identified). > > Personally I think even the official election of committers only to a specific > component is already overkill and introduces a big layer of bureaucracy to all > of our processes. > I'm sorry, but I don't see it that way. To me, earning commit rights to both JDT UI and the JDT Core is as significant as (and probably as difficult as) earning commit rights to both Mozilla and the Linux kernel. > Here's a metaphor: In system administration you can either build all of your > machines the same way and tailor just the very specific details of each one to > the task at hand. Or you can build each machine from the ground up to suit a > particular purpose. Some people argue #2 is better. Sure performance might be > a little better. You might get a few things you don't get with #1. But in the > end #1 is far more supportable and is the method used by nearly every large > shop in the world. Imagine automated upgrades on an infrastructure built like > #2. It's just not going to work in that environment. This is exactly what > we're talking about here. > No, we aren't. Keeping track of the groups that are associated with a project is not of the same order of complexity as keeping a custom pc configuration going, nor is it as amenable to automation and tooling. In any case, here's the bottom line: Earning rights to the Eclipse Project code base takes real commitment, and a *lot* of work. For someone to earn commit rights on all of SWT, for example, is much harder than for them to do so on a particular platform that they are already an experienced developer for. We will not grant anyone commit rights until they have earned them -- it is absolutely not just a "social" constraint for us -- so the simplified rules discussed in this bug would be a significant barrier for us to grow our community. As such, I believe we must find a better solution.
(In reply to comment #40) > That's why I was trying > to give control of the groups to the individual projects, I, too, am a big fan of moving control to the edges/projects and away from the center/EMO. I believe that it matches the philosophy of open source. > No, we aren't. Keeping track of the groups that are associated with a project > is not of the same order of complexity as keeping a custom pc configuration > going, nor is it as amenable to automation and tooling. McQ, I was agreeing with your discussion until this point and then we diverge - or maybe I misunderstood your point. I believe that it is important for us to automate (or transfer to the edges/projects) as much as possible in order to be able to scale the Eclipse Foundation to more projects and more committers. Keeping track of groups is not complex for someone who understands all the groups in a given project, but hundreds of groups for hundreds of projects x components, each with a different set of membership rules (who can join, who can leave, etc) is just not scalable.
(In reply to comment #41) > > No, we aren't. Keeping track of the groups that are associated with a project > > is not of the same order of complexity as keeping a custom pc configuration > > going, nor is it as amenable to automation and tooling. > > McQ, I was agreeing with your discussion until this point and then we diverge - > or maybe I misunderstood your point. I believe that it is important for us to > automate... > Could be just a misunderstanding. I also believe we should be able to automate the process (i.e. what I was trying to say when I wrote "it is ... amenable to automation"), I just don't think that homogenizing the group mapping is a requirement for this. In any case, I'm going to have to drop out of this discussion until I get back from my vacation (i.e. in Sept); I've got less than 1.5 days left and too many people trying to get ahold of me before then.
Based on the feedback so far, here's a revised proposal. This proposal requires that the PMCs and the Board all approve changes to certain defining documents (see below). This proposal is similar to above proposal: 1. Group membership corresponds to project components 2. Component/group membership is managed by the project leads via the portal 3. Group names are consistent across all projects 4. While flexible and useful, this is a change for some projects. This proposal differs: A. Components can be hierarchically nested B. Hopefully this proposal accommodates both the webmasters' need to be more efficient and the projects' need to fragment their committers. Each project X will have three fixed groups: * X-dev containing all the committers on that project * X-web containing the committers with access to the project's website * X-download containing the committers with access to the project's download area Note that project X refers to both top-level projects T and sub-level projects T.W. In the case of sub-level projects T.W, the groups will be named T-W-dev, T-W-web, and T-W-download in order to provide unique group names. Also note that membership in sub-project T.W does not imply membership in top-level project T, nor does membership in top-level project T imply membership in sub-project T.W. These are possible, but require separate membership qualification (elections). Each project X can define a hierarchical set of components a.k.a. subsystems that provide further partitioning of the commit rights of a project. The basic rules are: * The unix group corresponding to a component C.D.E of project T.W will be named T-W-C-D-E-dev (i.e., long descriptive names) * Membership in component C implies membership in all sub-components C.D. * Membership in component C.D does not imply membership in the super-component C. * Only committers on project T.W can be members of components or sub-components of T.W. * Adding a committer to component C of project T.W requires a committer election in which all the members of component C are eligible to vote * Any committer in project T.W may nominate a candidate for an election to any component C of T.W * Removing a committer from component C requires the project lead of project T.W to request removal * If a committer on component C loses their commit rights (due to inactivity or changing employers or resigning or whatever), they lose their membership in component C (and all nested components). * Committer legal paperwork will be maintained at the project level - the EMO will not accept ICAs, MCAs, and ECEFs that restrict the committer to working on only a subset of a sub-project (i.e., one or more, but not all, components). * Creating a new component is considered a significant event that must be announced to the Eclipse community at large. (see http://www.eclipse.org/projects/dev_process/notifying-membership.php) * Removing a component is a rare event, but does not require at-large announcement. It should be announced on the project's development mailing list. * Assigning directories to groups is done at component creation time. * Changing a directory from one group (component) to another group (component) within the same project is rare and must be announced on the project's development mailing list and approved by the EMO. The reason for the required approval is that changing a directories' ownership has the effect of changing the set of committers for those files. The EMO will want to make sure that any existing committers who will be losing commit rights on those files are aware of this change and are ok with it - in other words, we can't just go and remove someone's commit rights arbitrarily. The portal will provide a mechanism for PMCs and project leads to manage the components of their project(s). The webmasters will not be involved in most component/group actions _except_ for adding or removing a group (because they will need to chgrp some directory(s)) and changing the group a directory belongs to (again, because of the chgrp). Committers will be allowed to nominate new committers via the portal as per the rules above. Note 1: Should the Board and PMCs not being willing to change the defining documents, we'll have to revert to a solution similar to that described above (groups for top, sub, and component) as the status quo is not sustainable. Note 2: In the transition from the existing project/group structure to the new project/group structure, it is possible (although highly unlikely) that an existing committer's (X) employer will not provide permission that committer to join the new group structure. In that case, the committer's account will be disabled until the new legal paperwork is completed (just as we do today when a committer changes employers, etc.) Defining Document Changes: * The Eclipse Development Process (http://www.eclipse.org/projects/dev_process/development_process.php#4_Structure_and_Organization) will change to include the description of components as above. * The Bylaws will not need to change as long as the Development Process changes because the Bylaws reference the Development Process document. * The Charters (e.g., http://www.eclipse.org/eclipse/eclipse-charter.php and http://www.eclipse.org/projects/dev_process/Eclipse_Standard_TopLevel_Charter_v1.1.php) will change to: ** remove the section about "Ports" - this is subsumed by nested components ** change the section about "Project Components" to something like "The PMC may decide to divide a Project further into components as defined by the current Eclipse Development Process. Components are established and discontinued by the PMC. When the PMC creates a component it appoints a component lead to act as the technical leader and names the initial set of Committers for the component. The component lead is designated as a committer for the Project and represents the component in discussions and votes pertaining to the Project as a whole." * The Charters that reference the standard Charter will change to reference the new version (version 1.2) (e.g., http://www.eclipse.org/stp/project-charter.html).
Bjorn, I think your proposal answers my question about voting implications for hierarchical groups, but I'm not sure I'm happy with the answer. It seems to me the way this has been proposed is that a super component A with a sub component B can have a vote that will add a committer to A and hence will also indirectly add a committer to B without all the committers in B participating in the vote. That doesn't seem quite fair somehow does it? There's just something about hierarchical components that seems to beg the question of why they are necessary. It's clear that any space can be partitioned into non-overlapping parts and that committers can be elected to any or all of those parts to have access whatever subset of the space they should have access to. Hence what it is about hierarchical groups that provides a greater degree of flexibility than a flat set of mutually exclusive groups? Given that a file or directory can be owned by at most one group, there's really no such thing as a hierarchy of groups in the space where the term actually has an explicit meaning. Any hierarchy in actual permission groups is implicit in their overlapping membership and hence can be managed by deciding who is in what group. Of course that's managed by elections to corresponding components, but the proposed hierarchical election rules don't strike me as fair whereas the rules for a flat set of components seems inherently fair. So let me ask the simple question again with the hope of getting a clear and convincing argument: Why do we need hierarchical groups?
I can give some examples in the platform project where hierarchical components are a good fit. The most obvious example is platform ports. Some components (SWT, Equinox Framework, Platform Resources), have ports to a significant number of platforms. There are cases where we would like to give commit rights to a specific platform port to someone with expertise on that platform, without giving away the keys to the entire component. Those with commit rights on the entire component need to have commit rights to all ports - for example if they change a native method signature they need to be able to fix all ports. It's nicely bounded to give commit rights to, say, the HPUX port of the Equinox launcher, without giving commit rights to the rest of the Equinox Framework component (a particularly sensitive component to which commit rights are carefully controlled). Of course these could be represented as a flat set of components/groups, but presumably this would require up to a dozen different elections when one individual is granted commit rights to the kernel piece of SWT or Equinox. Another example where hierarchies would be useful is in large components such as Platform UI. It takes a very high level of commitment to gain the level of proficiency and trust required for commit rights to the entire UI. Generally, you need to be working on the component as a full time job to attain and maintain proficiency required to work on a component this big. However, there are many more casual contributors who rightfully earn proficiency and trust on smaller pieces - such as JFace, or the IDE plugin, or a particular presentation plugin. It is fitting to reward these contributors with commit rights on the piece of code they are familiar with, again without giving complete access to a fairly sensitive piece of the platform. In this case, a hierarchy of components nicely represents the different scopes of commitment and trust. It's also a great way to encourages contributors on these large components - they don't have the insurmountable hurdle of learning the entire component at once, but instead they can start on a small piece, and work their way up the hierarchy of proficiency and trust to gain access to larger component sub-trees over time. When commit rights on a large component was an all or nothing proposition, we've had very little success in bringing in new committers on the platform - our success stories over the past couple of years of new committers have come from more fine-grained partitioning of commit rights.
(In reply to comment #44) > add a committer to B without all the committers > in B participating in the vote. Hmm, you're right - I'm embarrassed to have missed this implication. That doesn't seem right. (In reply to comment #45) > Of course these could be represented as a flat set of > components/groups, but presumably this would require up to a dozen different > elections when one individual is granted commit rights to the kernel piece of > SWT or Equinox. So how do you answer Ed's (and my) concern that a hierarchical component structure allows one group of people (the higher component) to impose committers on another group of people (the lower component) without the election process considered standard in open source projects? I could imagine a technical fix to the elections issue as follows: * Committers elected to component C do not automatically become committers on C.D * When an election is started for a committer on component C, automatically and simultaneously (via the portal) an election is started on all sub-components C.D. * Each voter of C or C.D is asked only once for a vote (to prevent a flurry of emails). * A vote by a C committer will automatically be propagated to all sub-elections C.D. * A vote by a C.D committer (person X) will apply only to the subcomponents of C that person X is a committer on. This should have the effect of a single election in terms of effort and emails while at the same providing sub-components their proper open source voting rights. Thoughts?
(In reply to comment #46) > Thoughts? Sounds good to me. This would allow managing the commit rights to Platform UI and its subcomponents. I particularly like the idea of making it look like a simple election.
Re comment #46: I personally don't see this as unfair - if you accept that there is a hierarchy of power/control, to me it naturally follows that a higher level can impose things on the lower level. On the other hand, the technical solution you propose sounds fine if it satisfies development process rules. I just don't know how it would work in practice if, for example, someone granted SWT commit rights loses an election on a particular native port. They would have the rights to make breaking changes that would affect that port, but not have the ability to fix them.
Please excuse me if I've missed some of the nuances in the long thread of comments. I'm trying to come up to speed here... Looking at some of the recent comments I have the perception that we are going too far and getting too complex. In essence the proposal is reifying something that was not meant to be (sub-components). In the Eclipse project we have a relatively simple set of requirements. The component (eg.., UI, Resources, ...) teams need to be able to grant commit rights on a finer granularity than the whole component. As has been pointed out, this is for various technical and community reasons. We do not need another entity that can run its own votes or have more restricted access than hosting component. Looking at it this way, the issue (re)stated in comment 46: "...hierarchical component structure allows one group of people (the higher component) to impose committers on another group..." is not actually an issue at all. There is no hierarchy and no sub-components. There are parts of the real components that have a few more committers. In this context I find "* Committers elected to component C do not automatically become committers on C.D" quite problematic. Simpler (IMHO) would be something that uses unix groups. Mike I think had a similar idea. Assuming there is a set of unix groups and a mechanism for project/component leads adding/removing said groups, votes would enumerate the groups to which the candidate was being given access. We would ensure that the groups listed were all part of the same Eclipse project or component as appropriate. Done. There is little extra work (someone already has to spec this info at some point in the election process anyway) and there is more transparency (voters get to see exactly what they are giving access to).
(In reply to comment #48) > Re comment #46: I personally don't see this as unfair - if you accept that > there is a hierarchy of power/control, to me it naturally follows that a higher > level can impose things on the lower level. But there isn't a hierarchy of power/control. The project Charters state "All Projects operate using open source rules of engagement: meritocracy, transparency, and open participation." The hierarchy of power granted by the Charter is the ability for the PMC to remove a Committer who is "disruptive, does not participate actively, or has been inactive for an extended period." The Charter does not give the PMC (or any other structure) the right to impose committers on projects, components, or other sub-groups. In fact, the Charter is fairly specific about the only way to become a Committer is to be elected (or be part of a new project).
(In reply to comment #49) > Assuming there is a set of unix groups and a mechanism for project/component > leads adding/removing said groups, votes would enumerate the groups to which > the candidate was being given access. We would ensure that the groups listed > were all part of the same Eclipse project or component as appropriate. The main question: who gets to vote on adding person P to unix group X? Cases to consider: 1. Person P is being elected to project Q, component W = unix group X. 2. Person P is being elected to project Q, full rights to everything. 3. Person P is already a committer on project Q (some components) and is being added to component W = unix group X.
So I think we're running into a conflict between two things: process definition and strategy, and technical 'hard' requirements. See below for elaboration. (In reply to comment #49) > Simpler (IMHO) would be something that uses unix groups. Mike I think had a > similar idea. > Assuming there is a set of unix groups and a mechanism for project/component > leads adding/removing said groups, votes would enumerate the groups to which > the candidate was being given access. We would ensure that the groups listed > were all part of the same Eclipse project or component as appropriate. > > Done. If this is what Mike was proposing (something which is still not clear to me at least) then I agree that this is almost a workable technical solution. There is still a problem we face in the current scheme which is not resolved in this proposed solution, however. This is the fact that the sheer number of groups _is_ a problem. We have a number of people, particularly releng people, who need to be a member of too many groups to do their job. And when I say "too many" I'm not using a subjective term based on my own judgment. Rather I am speaking of the absolute fact that NFS does not permit us to have people in more than 16 groups. In order to allow these people to do their jobs we currently have had to create separate POSIX access lists at the filesystem level to override group privilege for those people to whole trees of the projects. Thus any time anyone wants to verify the permissions of a particular user there are two places which have to be checked, and trying to map POSIX access lists which are applied for a whole subtree to UNIX groups at the file/directory level to determine what someone really has access to is a problem. Any solution which does not reduce the number of groups is not workable in my opinion. > There is little extra work (someone already has to spec this info at some point > in the election process anyway) and there is more transparency (voters get to > see exactly what they are giving access to). I agree this is a benefit of that proposal. And I like this proposed solution. But what I think we still need is to define how we will limit the number of groups that can be created.
re comment 50: this is only a hierarchy if you make something "sub" something else. In the model I see, you have a Component. That's it. nothing sub. no one imposing anyone on anyone. Everyone elected, ... re comment 51: > 1. Person P is being elected to project Q, component W = unix group X. All the committers in W get to vote because clearly P is managing committers at the component level. > 2. Person P is being elected to project Q, full rights to everything. All the committers in P vote as it seems P is NOT managing committers at the component level. (for example in the Eclipse TLP we never vote in committers across a whole project. > 3. Person P is already a committer on project Q (some components) and > is being added to component W = unix group X. All the committers in W get to vote because clearly P is managing committers at the component level. re comment 52: Understood (group count issues). We clearly have this issue today. For example, many of my project team have this issue since they have been around for a long time. When I talk about "unix groups" it is more in the notional sense then the actual implementation detail sense. Perhaps the implementation strategy, since we have the group count problem anyway, is to solve that by implementing good ACL management facilities. With such facilities in place we could then, for example, toss "unix groups" completely and do everything with ACLs. As long as the ACL infrastructure allows us to identify a set of dirs as being accessible to a symbolically named set of people, we have a group equivalent and can implement the solution. Does that make any sense?
(In reply to comment #53) Jeff, it seems that your proposal is to separate the concept of component and the concept of unix group. The problem is that the founding documents do not allow restricting Committer access this way (e.g., committer is elected to a project or component, but is not given access to all the code of that project or component). I'm not clear on how your proposal to have unix groups that are managed outside the project/component committer relations works with the project Charters?
Not so much separating them as calling them out explicitly. Committers are still voted in on components. Today we have a vote and then someone has to say what access rights the newly elected committer is to get. Why not say that up front? I have to say that I'm not sure about the restricting aspects wrt project charters etc. The charters I looked at (including the standard template charter) does not mandate that committers get write access to ALL of the component's repository. "Committer and given write access to the source code and/or web repository" If we were following this to the letter then why, in today's process, is the project lead asked what CVS commit rights the newly elected committer should have? That is already defined in the charter!
(In reply to comment #55) > ...why, in today's process, is the > project lead asked what CVS commit rights the newly elected committer should > have? I always thought it was because the webmaster do not know which of the hundreds of CVS modules belong to which project and component. So really that question in the election form is about the mapping from project/component to /cvsroot/xxx/yyy directory rather than about allowing committers access to a subset of a project.
Bjorn reported that a XMPP server cannot be maintained by the webmasters until this bug is dealt with.
is this a technical dependency or a workload issue? Seems somehow unrelated technically.
(In reply to comment #58) > is this a technical dependency or a workload issue? Seems somehow unrelated > technically. > At the risk of speaking for Bjorn, I think this is a workload issue. Right now we're adding new features to our infrastructure but we have zero resources to maintain them. The blogs tool is an example of something that desperately needs maintenance but is getting no love. It's a security hazard to run in its current state, with a swiss-cheese set of holes that no one has time to fix.
(In reply to comment #59) > At the risk of speaking for Bjorn, I think this is a workload issue. Right now > we're adding new features to our infrastructure but we have zero resources to > maintain them. The blogs tool is an example of something that desperately > needs maintenance but is getting no love. It's a security hazard to run in its > current state, with a swiss-cheese set of holes that no one has time to fix. At the risk of giving feedback after the decision was taken ;), if the resources are constrained, why did you take on another service like a blog tool when there are alternatives easily available? Sorry, I just had to say something about that and feel free to ignore... But, the focus on this whole thing should be on our IT infrastructure. And, unfortunately, that means pushing some of the responsibility upstream to the projects. I.e., as a project lead, you need to think about the files and directories and how the ownership of those is set up on the Eclipse servers. Every committer has ssh access to the servers and the 'ls -l' command to see what's there. And all requests to the Foundation staff has to be in those terms. And that will allow for automated tools to be set up to keep their heads above water. Having been a SourceForge project admin for a few years, there's no mistaking the value of automation of these services, both to the scalability of the site and to the freedom of the projects to set things up the way they want.
(In reply to comment #60) > At the risk of giving feedback after the decision was taken ;), if the > resources are constrained, why did you take on another service like a blog tool > when there are alternatives easily available? A legitimate question but your conclusion is wrong - there were no easy alternatives (to the blog tool). Specifically, if a blog X is hosted at eclipse.org, then it has a Terms of Use that allow any IP invented on the blog to be incorporated in Eclipse projects without going through the IP review process. If blog X is hosted elsewhere, that is not true and thus all IP has to be carefully reviewed before incorporation in an Eclipse project -- and that IP review process and delay is the number one source of complaints by the project leads. So, yes there were alternatives (other blog hosts), but none that were "easy".
(In reply to comment #58) Mostly workload, but there is a slight technical issue: the XMPP server needs some information about groups in order to show groupings. This bug is about how to keep track of and maintain group information beyond the level of just "project". So we need this information in order to make the XMPP server useful. (Or else any project using the XMPP server would need to define their groups _yet again_ in _yet another_ group descriptor file.)
(In reply to comment #62) > (Or else any project using the XMPP server would need to define their groups > _yet again_ in _yet another_ group descriptor file.) Understood and agreed. Do we need finer grained/different groups than "component committers" in the XMPP usecase? Seems like for chat etc that would be fine.
I thought I would try to summarize the current discussion on this bug. Here's what I think has been said/discussed/concluded: (1) The Foundation, in order to scale our support for the community, needs to have the webmasters spend less time administering the dev.eclipse.org Unix groups. We need to make this change during or before 2007Q4. (2) Three proposals for implementing (1) have been put forth: (2a) Unix group names and membership is based on the top-level/sub-project/component hierarchy. Administration of the groups is done through elections and defections of committers to and from projects and components via the portal. There are no nested sub-components. (2b) Similar to (2a), Unix group names and membership is based on the top-level/sub-project/component/sub-component hierarchy. There are nested sub-components. (2c) Unix group names and membership is separated from the project/sub-project/component hierarchy and is separately administered (via the portal) by the PMCs. (3) Proposals (2b) and (2c) are contrary to the existing founding documents (Charters and Development Process). (2b) is problematic because the documents contain no description of a hierarchy of sub-components. (2c) is problematic because the documents do not provide for a separate fine-grained control of commit rights. These documents can, of course, be revised through the mechanisms outlined in the Bylaws (basically, by a vote of the Board). (4) Proposal (2b) has a problem with the philosophy of open source in that it allows one group of committers to impose something (in this case, new committers) upon another without the other group being allowed to vote. (5) Most of the people who have commented on this bug would be happy with (2a). Members of the Eclipse Top-Level Project have expressed unhappiness with (2a) arguing for hierarchical components (2b) or something else (2c). Are there any facts that I've missed?
(In reply to comment #39) I think David's comment #39 is particularly significant: > Instead, I think the component level significance is mostly "social" ... that > is, the component level groups should have their own design decisions and > document any votes there if needed to decide a direction, if something should > be API or not, etc. but, it would just be "peer pressure," so to speak, that > would prevent someone from committing code to the wrong component, one which > they were not a member of. It's not clear why we are trying to solve a social problem (preventing committers from committing outside their safety zone) through a technical tool. Once we have partitioned the commit rights to a component level, on the order of a dozen people, it seems that finer-grained control should be based on social processes, i.e., you have to answer to your peers. Perhaps one of the authors above (Boris, Jim, Jeff, John, Kim, McQ, Nick, or Tod) could enlighten us/me about why the finer-grained groups are necessary here? Note: I understand that "it has always been done this way", but that's not an answer for why it is necessary. Anyway, thanks for helping me understand...
(In reply to comment #65) > ... Perhaps one of the > authors above (Boris, Jim, Jeff, John, Kim, McQ, Nick, or Tod) could enlighten > us/me about why the finer-grained groups are necessary here? Note: I > understand that "it has always been done this way", but that's not an answer > for why it is necessary. Anyway, thanks for helping me understand... John explained it very well in comment 45. In particular, I would like to emphasize the last sentence: "When commit rights on a large component was an all or nothing proposition, we've had very little success in bringing in new committers on the platform - our success stories over the past couple of years of new committers have come from more fine-grained partitioning of commit rights." I believe that if we cannot continue to use finer-grained access control than at the component level, the Platform UI component probably will have to be split into a number of smaller components. Which might be fine - I could actually see this working for us. I am not so sure about the SWT team which has the additional problem of wanting to manage rights for all the different ports. It seems to me that this is one of those areas where there will always be a lump under the rug. You can try to move it around, but you won't get rid of it. A simple story for the access rights will make life more complicated for the committers. Just think about what happens if we tried to make the access rights story even simpler - we would vote on commit rights for a whole project instead of one of its components, the administration becomes trivial, and we would have only a few unix groups to worry about. On the flip side though, we would need to have more complicated social processes for making sure that nothing bad happens, and committers would have to spend a substantial part of their time double-checking CVS commits.
Re comment #65: > It's not clear why we are trying to solve a social problem (preventing > committers from committing outside their safety zone) through a technical tool. By this logic we could just have one unix group for all eclipse.org committers, and use social conventions to ensure people stick to their projects. Clearly there's some granularity where tooling is helpful. Perhaps the Eclipse project is unusual because of its size - there are components in the platform that have more committers than some entire eclipse.org projects. Over time we'd like to continue growing this platform committer base, and as I mentioned in comment #45, being able to define more fine-grained commit rights helps us do this comfortably. As for the argument about CVS being able to undo bad commits, this is small consolation when it results in bugs being shipped or schedules to be slipped. It's not so much an issue of protection from malicious committers - a well-intentioned and seemingly innocent change at the wrong time in the release cycle can create a lot of churn. With 100+ commits a day, it's hard for anyone to closely track what goes into the repository and undo damage in a timely manner.
Thanks for the summarization. Very useful. (1) +1 (2) The social norm argument can clearly go ad obsurdum in either direction. It may be that in many cases social structures can manage things at the level of a component. Actually, there are other elements in the setup that we use social norms to manage. the Releng projects for example. The way we are setup now, all Eclipse project committers should have commit rights in the releng project. That is how we do build contributions. (3) I actually disagree that the current charters prohibit this approach. As mentioned in comment 55, the charter template says that committers get write access to the source. It does not say that they get all of the source. Indeed, in the case of component based projects, there is generally one repo and committers on a repo get access to only a small portion of the repo. More accurately, newly elected committers should get write access to the code/source to which they were just elected as a committer. If someone was voted into component C under the idea that the would work on port X, subpiece y, ... then that is what they should get access to. Using "groups" (unix or ACL based) we can spell that out clearly and explicitly in the voting process rather than having it hidden at the end as it is now. (4) The issue here is mostly around trying to define a formalized notion of "sub". don't think we need or want that and it leads to questions like this. (5) Not sure if this is a quirk of the Eclipse project, history or large diverse mature projects. The need for flexibility in these matters is largely what has driven me to 2c. I would think that the sysadmins would love it as they get out of the middle. PMCs can create/remove groups, the community votes people into groups, ... The admins "just" need create the mechanism for managing the groups (via ACLs or whatever) and associating them with components/projects. For projects that just have one group/ACL set they can just have a simple option (ALL) but we have the flexibility where we need it.
(In reply to comment #67) > Re comment #65: > > > It's not clear why we are trying to solve a social problem (preventing > > committers from committing outside their safety zone) through a technical tool. > > By this logic we could just have one unix group for all eclipse.org committers, > and use social conventions to ensure people stick to their projects. Clearly > there's some granularity where tooling is helpful. So there are two considerations in managing groups: 1.) Controlling access to the code for purposes of managing the project 2.) Legal considerations guaranteeing a committer has permission/legal authority to commit code to a particular tree At a legal level we are obliged to have a solution that resolves #2. As I read the development process and bylaws I see nothing of #1 anywhere unless I am perhaps missing something? Your statement above simply points to the fact that we have all acknowledged that #1 is still important in some form. Bjorn is not suggesting we eliminate #1. We have historically supported #1 above extensively but it has become increasingly draining on our time, and thus your resources, so we're trying to come up with a middle ground between only taking care of #2 and still helping with #1. As I read things there is not currently a requirement that we implement any of #1. Now I am not proposing we discard #1, so don't anyone flame me, but really I think we should be looking at the whole discussion through that lens. If I've clearly missed something in my reading, someone please point me to where I've missed it--for my own education.
(In reply to comment #68) > I would think that the sysadmins would love it as they get out of the middle. > PMCs can create/remove groups, the community votes people into groups, ... The > admins "just" need create the mechanism for managing the groups (via ACLs or > whatever) and associating them with components/projects. For projects that > just have one group/ACL set they can just have a simple option (ALL) but we > have the flexibility where we need it. Jeff, like I said above I like the idea. What I'm less certain of is success implementing it. A lot of thought and discussion would have to go into building a strategy that can successfully implement this idea while still protecting the general security of the system. And using ACLs vs Unix groups opens other cans of worms that need close consideration. Socially it is a very good solution, technically it is more peril fraught.
Yeah, and the vast majority of us have the utter luxury of basking in the ignorance of what we are suggesting! Sorry. The thing that did occur to me however is that you have this group/ACL problem already. I know lots of people who are members of more than 16 groups. That is because they are involved all over Eclipse. How will the sort of component/group reorg proposed here affect that? We are forever having problems where someone does in the ACL for this dir or that. This problem is not going away. So naively I thought that solving that problem would give you a base for implementing the ACL management needed to solve the problem at hand here. Is that wayward thinking?
I'm wondering if being more concrete might help ... at least help some of us understand what the problem is -- at least a problem for one project :) Is the "Eclipse Project" still considered the only project in the Eclipse Project, and JDT, PDE, and Platform are components? (If so, that might be one problem right there :) But, what I'm suggesting is taking the most complicated one, such as the Platform "Component/Project"? and drawing a tree diagram showing the current components of it, and their sub components? (and their sub components?) Please don't spend a lot of time just to educate me ... but if it's easy to I think a tree diagram of the current project structure with all the committers memberships/relationships shown might help understanding. Can't easily draw a tree diagram or write down? Well then, perhaps that is then the true source of the problem. :) [Honest, just a reminder that instance diagrams help clarify class definitions :) ... to improve understanding ... friendly barbs aside].
> (2) The social norm argument can clearly go ad obsurdum in either direction. > It may be that in many cases social structures can manage things at the level > of a component. Actually, there are other elements in the setup that we use > social norms to manage. the Releng projects for example. The way we > are setup now, all Eclipse project committers should have commit rights > in the releng project. That is how we do build contributions. Right (ad obsurdum) and right (social structures can manage things up to a certain size group of people) and right (releng is a social structure). > (3) I actually disagree that the current charters prohibit this approach. As > mentioned in comment 55, the charter template says that committers get write > access to the source. It does not say that they get all of the source. Well, then we're going to disagree. While English is a reasonable language, it's also not entirely precise. I believe the meaning in the documents is that Committers have access to all the code to which they were elected and that the documents do not define sub-components. Thus elected access = groups = components. > (5) PMCs can create/remove groups, the community votes people > into groups, ... As long as we respect the founding documents, the legal needs, and the principles of open source, yes. To me that means: I. There is no hierarchical sub-component structure. An infinitely wide flat structure, and even one in which a project uses a naming structure to imply hierarchy, but no actual hierarchy. II. People only get added to groups by nomination and voting (as per the charters today). III. Files/directories do not change groups without some kind of vote/public something by the group-that-will-lose-the-files. In other words, I don't want the SWT team to be able to 'steal' org.eclipse.jdt.apt.ui (i.e. chgrp -R eclipse-platform-swt-dev org.eclipse.jdt.apt.ui) without the JDT component team agreeing to that change. IV. There will be a technical limit on the number of unix groups to which a single person can belong. This may affect the number of components a project chooses to use. This is a technical limit (currently 16) that could be remedy in the future by a separate bugzilla (eg bug 157931). We, the EMO, will be happy to modify the portal to support elections to multiple groups simultaneously to minimize the overhead of adding new committers to multiple components. Something like "Please vote for James to be added to PDE components foo, bar, whiz, and bang..."
I do see your point re the charters etc. Perhaps we should just ask the board for interpretive guidance? As for your other points (the ones with roman numeraled points) I was nodding my head (up and down) up to point IV. If that were true and groups were the only mechanism then I, and many people I know, would have to lose commit rights. The side effect you mention (limits on the number of components a project could have) could severely impact the social fabric of a project for purely technological implementation reasons (not to say they are trivial but we would be forced to shape the community to conform to an NFS limitation). Putting in place a component == group structure when we know that the be the implication does not seem right. So, it seems we need to solve problems such as the one in bug 157931 before we can pick any of the options here. It would be pointless to put in place an automated election system where component == group (whatever) only to find that people duly elected could not actually commit because they belong to too many groups.
Created attachment 76017 [details] unix logins currently with more than 16 groups (In reply to comment #74) Hmmm. How many people and components would it really affect? I ran through the current groups on unix and discovered these people who already have more than 16 groups (see attachment). What I'm not understanding, is how this bug (standardized groups) is going to make the number-of-groups situation worse? We are already "shap[ing] the community to conform to an NFS limitation"; that's the issue at the heart of bug 157931. Perhaps the reasoning is that using comment #73's I-III will cause there to be more components and thus more groups? I don't understand that (please explain) because here's what I see: 1. What comment #73 is proposing is two simple changes: first, follow the election processes as written and not allow super-components to force committers into sub-components, and second, standardized group names. 2. A project (e.g., The Platform Project) could have the same number of Components as it does now, they would just all be a flat structure rather than hierarchical. 3. Some projects (e.g., The Platform Project) already encounters the NFS limit (e.g., bug 157931). 4. So the only way that I can see the issue in comment #74 being relevant is if, somehow, requiring that projects follow the election process will cause the project to want to create many more components than they currently have. So you see my confusion... I must be missing some key point - could you give me a hint?
(In reply to comment #72) > I'm wondering if being more concrete might help ... at least help some of us > understand what the problem is -- at least a problem for one project :) I can try... > Is the "Eclipse Project" still considered the only project in the Eclipse > Project, and JDT, PDE, and Platform are components? (If so, that might be one > problem right there :) No, the Eclipse top-level project has five sub-projects: Platform, Equinox, JDT, PDE, and Incubator. > But, what I'm suggesting is taking the most complicated one, such as the > Platform "Component/Project"? and drawing a tree diagram showing the current > components of it, and their sub components? (and their sub components?) The Eclipse-Platform subproject has the following components: Ant, Compare, CVS, Debug, Doc, IDE, Releng, Resources, Runtime, Search, SWT, Team, Text, UI, Update, User Assistance, WebDAV As an example of a component with sub-components, here is the breakdown for Eclipse-Platform-UI: - JFace - Data Binding - Common Navigator - Tabbed Properties - Workbench For the first four of these, we have committers with access rights to just the sub-component. If/when we can bring in committers for RCP-specific issues, they would need access to the fifth sub-component.
If I haven't been totally lost in 76 comments so far, it seems to me that most of the technical discussion here has been revolving around CVS access rights, UNIX groups and ACLs. Has anybody explored yet whether there are other possibilities for managing access rights to the repository? Given that project/component == UNIX group would work for most projects except the really large ones: could it be possible for the large components to have all committers in one UNIX group but do the finer-grained permission handling with a different kind of tooling? One that they could perhaps even administer themselves? I did a quick google search and it seems that for CVS there are at least http://cvspermissions.sarovar.org/ and http://cvsacl.sourceforge.net/ Both of these are only active for "cvs remote" kinds of operations, so malicious committers modifying repository files right on dev.eclipse.org under /cvsroot would not be hindered. Could it be sufficient to have such modifications (which would only occur inside one UNIX group == component) avoided by deprecating them through social control? SourceForge seems to be using cvs_acls, for more info search for "Fine-grained write permissions: cvs_acls" on http://sourceforge.net/docs/E04/ In case a project would want to switch to SVN, I'm not sure what kind of access control that would offer...
> This is a technical limit (currently 16) that could be remedy > in the future by a separate bugzilla (eg bug 157931). Exactly. It's in our 2008 roadmap to upgrade our file server to use an authentication mechanism that doesn't impose this ridiculous 1970's restriction of 16 groups.
re comment 75 Thanks for the list. I wonder how many are "on the edge" Say with >13 groups to their name? Under the current proposal, what happens to these people? (the ones over and the ones that will become newly over the limit) Seems we still need an ACL management solution. Ok, onto the other points. In essence yes, the your proposal will cause there to be more groups required. Take a look at Boris' notes in comment 76. Find all the UI team committers and add 4-5 groups to their list. Then how many are over 16? If the only way to have the granularity the *community* wants is to have more components then that is what the community will have to do. NOTE: The community in many of these cases actually does NOT want more components or even *sub* components. They just want a path to letting more people participate without giving away the farm. In any event, does this clarify for you how the group limits will force the community/social to modify its behaviour? This does not seem to be a desirable outcome. As I've said before, the group problem is not going away and it is hard (of me at least) to see a solution to the voting/permissions automation issue without assurance that the solution is not going to cause more problems. Re comment 76: a few clarifications wrt terminology. There is actually no such thing as a "sub-project". There is the Eclipse Top-Level Project and the 5 "projects" that Boris enumerated. Also, Boris can correct me if I'm wrong, but the 5 areas around the Platform-UI are best thought of as "work areas" or some such. They are not intended to be in any way autonomous or somehow separate from the UI component. They are whole owned and operated by the UI component.
> Exactly. It's in our 2008 roadmap to upgrade our file server to use an > authentication mechanism that doesn't impose this ridiculous 1970's restriction > of 16 groups. That is great but shouldn't we have this solution before forcing the creation of more groups?
(In reply to comment #79) > Ok, onto the other points. In essence yes, the your proposal will cause there > to be more groups required. Take a look at Boris' notes in comment 76. I'm just not understanding why *more* groups will be required because there are *already* groups for all those subsets that Boris lists in comment 76. As far as I can tell (because the names are not standardized, hint, hint), those groups are: plat-ui-jface = JFace plat-ui-bindings = Data Binding plat-ui-nav = Common Navigator plat-ui-tabbed = Tabbed Properties ... So how does this bug's proposal result in *more* groups? Note that I'm admitting that the problem of NSF's 16 group limit is real and is a problem (and bug 157931 is about that) (and comment #77 is about that). This bug is about standardized group names. So if the current technology does not support the current number of groups, that is bug 157931; if the proposed solution has the same (or fewer) groups as the current technology, then the problem becomes bug 157931; only if this proposed solution creates more groups is bug 157931 going to cause problems for this bug. And so far you haven't been able to get through my thick skull why there would be more groups under this proposal... > The community in many of these cases actually does NOT want more components or > even *sub* components. They just want a path to letting more people > participate without giving away the farm. Um, huh? So they want a mechanism to restrict write-access but they don't want to use the mechanism (components/sub-systems) listed in the Charter? > Re: There is actually no such thing as a "sub-project". We added the term "sub-project" in last fall's Development Process document because it was in popular use (http://www.eclipse.org/projects/dev_process/development_process.php#4_Structure_and_Organization). "Each Top Level Project contains zero or more Projects (for linguistic clarity, Projects as often referred to as Sub-Projects)." Just an FYI.
I just wanted to comment that aside from the less vs. more groups debate, I think there has been no disagreement that more standardized group names would be a good thing. Looking at the groups I belong to, I realize I have no clue what some of them are. There are many inconsistencies in the current set of group names, and having a more standardized convention for the names of groups would undoubtedly make life easier for whoever ends up managing groups. Minimally, I suggest that groups belonging to project X have the form X-*, and groups belonging to component Y in project X have the form X-Y-*.
Created attachment 76371 [details] plan for implementation of standardized groups Thanks for all the helpful discussion in and around this bug - it has helped clarify the situation and improve the proposed solution for me (and I hope for others). I believe we are to the point of a concrete proposal including actions and dates, and so I've written one in the following attachment. It's a two page PDF and definitely recommended reading for all the interested parties because, as I mentioned, it contains a specific proposal and has action items to assigned to the EMO and the PMCs.
Adding all the PMC Leads to the cc list because now that I've posted a specific proposal with a specific plan, those people will want to read and comment. Especially as the specific plan includes action items and deadlines for the PMCs.
thanks for the concrete plan. Great progress. Some questions/comments... (1) - There is an inconsistency in the terminology. AFAIK (pointed out to me by you or Mike...) there are no "sub" projects. There are TLPs, Projects and Components. It feels like a nit but it actually cues a lot of questions for me (see below for some) - "ui-jface"-style names are confusing and do not match the scheme noted at the top. That is, all group names should be three segments with segments delimited by '-'. In this case, the fact that jface (one component) has some relationship to another component (ui) is largely immaterial. People should be encouraged to pick simple component/group names (e.g., jface). If people want to decorate their group names then they should use a different delimiter (e.g., '.') (2.1) - Can you expand on the phrasing "...will always have a www component of committers who have write-access to web site."? The wording around "component of committers" is strange. - Not sure that having only one www component per project will work. We use this approach in Equinox ok but in other projects it is less clear that committers in component X should have write access to the webspace for all other components. Would be interesting to hear what others think. (2.3) - Do you see a need for a project group even if the project uses components? Are there some things that just need to be associated with the project and not a component? - how are the www and download components from 2.1 and 2.2 handled for a project that does not use components? if they don't have these components then how do they manage commit rights to their web and download resources? Or is it assumed that they don't and all committers get the whole enchilada? (3) - As has been stated before, forcing the group<->component relationship is in fact forcing this limitation on the social structure of the projects. People who are currently over or near the limit (e.g., >= 10 groups now) will be in danger of not being able to work. - since the solution to the technical limitation may well cause us to rethink the way write access is managed (e.g., perhaps everything ends up being done with ACLs), it seems like we are putting the cart before the horse by basing our new voting and committer management approach on groups. Groups may be obsolete once we solve bug 157931. We should be solving the technical problem first and then the voting/management problem (in terms of the new rights management solution) - I understand this technical limitation feel strongly that a thorough migration plan has to be in place before moving on this plan. The set of people currently over 16 and near 16 is not trivial and many of them are very active. Having their activities and further commit rights curtailed would be very much less than optimal. (5) - The deletion of committer status seems a bit eager in this case and may not be in line with the project charters. Typical charters say something like PMCs can revoke committer status for various reasons. (6.7, 6.8) - wording is confusing (e.g., don't think TLPs typically have components) Other: - We also have groups like orbitbuild which seem to be in place to manage access to Foundation managed resources (in this case the build infrastructure). Is this another component? Will be a little strange to have it surface this way... - how are PMC groups managed? - not sure about others but we tend to have a doc component in each project that everyone in the project has access to (we want everyone writing doc). Net result is that for each project that follows the general model set out, becoming a committer in one component involves adding 3 groups. component-dev, project-www and project-doc - currently there seem to be a few groups that are *-all or some such (e.g., eclipse-all, common, ...). What happens to these? - similarly, there are groups for project leads or admin purposes (e.g., equinoxadmin, orbitadmin, ...). What happens to these?
(In reply to comment #85) > - As has been stated before, forcing the group<->component relationship is in > fact forcing this limitation on the social structure of the projects. People > who are currently over or near the limit (e.g., >= 10 groups now) will be in > danger of not being able to work. Let me try to explain why I agree with Jeff that this will be a problem, using the example groups given by Bjorn: (In reply to comment #81) > I'm just not understanding why *more* groups will be required because there are > *already* groups for all those subsets that Boris lists in comment 76. As far > as I can tell (because the names are not standardized, hint, hint), those > groups are: > plat-ui-jface = JFace > plat-ui-bindings = Data Binding > plat-ui-nav = Common Navigator > plat-ui-tabbed = Tabbed Properties We will not need more groups - they largely already exist. Rather, existing committers will have to be in more groups. For example, I am currently *not* a member of the four groups listed above. I am a member of the plat-ui group which can access all the files owned by one of the four groups above. This is possible due to the clever use (read: manual and error-prone) of ACLs by the webmasters.
> - There is an inconsistency in the terminology. AFAIK (pointed out to me by > you or Mike...) there are no "sub" projects. There are TLPs, Projects and > Components. Yes, MDT got burned on this one too when we wanted to do Modeling > MDT > UML2 > Tools and were told "there are no subprojects". We need to be consistent w/ the terminology -- is it "top level project > project" or "project > subproject" ? (My vote's for TLP > Project > Component, FWIW.) > (2.1) "...will always have a www component of committers who have write-access to web site."? I'd guess that should be "...will always have a www group of committers who have write-access to web site. > - Not sure that having only one www component per project will work. We use > this approach in Equinox ok but in other projects it is less clear that > committers in component X should have write access to the webspace for all > other components. Would be interesting to hear what others think. In Modeling we have modeling-home for the whole website, including EMF, EMFT, MDT, M2T, etc. As per bug 198834, PMCs are cool with this limitation. For me it's cool because it allows more code reuse than if we had one website per Modeling project (mdt-home, emf-home, m2t-home, emft-home). These 4 groups do actually exist, but two are functionally useless, and two are deprecated. (Aside: mdt-home and m2t-home are only there to maintain www.eclipse.org/mdt and www.eclipse.org/m2t, which contain little more than an index page which redirects to /modeling/mdt and /modeling/m2t, respectively. emft-home and emf-home still have some unmigrated content, but should be clean by year's end.) > (2.3) > - Do you see a need for a project group even if the project uses components? > Are there some things that just need to be associated with the project and not > a component? Would it be reasonable to set up all projects to have a TLP, a project, and a component, even if it's in fact only got two of the three? For example, the EMF "component" of the EMF "project" within the Modeling "TLP" currently has an emf-dev group, but this could be renamed to modeling-emf-emf. For projects like GMF which don't (yet) have components, they could be modeling-gmf-gmf, which while redundant, just means that this is the "core" GMF and that if some day side projects (sorry, "components") are added, they can be modeling-gmf-foo, in the same way that we added Query to EMF to get group emf-query (which I'd argue ought to be renamed to modeling-emf-query). For other projects, why not adopt the "tools-" or "technology-" prefix? Why shouldn't the group names (or ACL rules, whatever) simply reflect the directory structure for which they're used? Thus, for now, for Modeling > EMF > Query, we have these 5 groups: emf-query: /cvsroot/modeling/org.eclipse.emf/org.eclipse.emf.query emf-query-releng: /cvsroot/modeling/org.eclipse.emf/org.eclipse.emf.query.releng modelingadmin: ~/downloads/modeling emfadmin: ~/downloads/modeling/emf emf-home: /cvsroot/org.eclipse/www/emf (deprecated) modeling-home: /cvsroot/org.eclipse/www/modeling But we could have this instead: modeling-emf-query: /cvsroot/modeling/org.eclipse.emf/org.eclipse.emf.query modeling-emf-query.releng: /cvsroot/modeling/org.eclipse.emf/org.eclipse.emf.query.releng downloads-modeling: ~/downloads/modeling downloads-modeling-emf: ~/downloads/modeling/emf org.eclipse-www-emf: /cvsroot/org.eclipse/www/emf (deprecated) org.eclipse-www-modeling: /cvsroot/org.eclipse/www/modeling This doesn't address the bug 157931 issue, just the idea of confusing/inconsistent names. Maybe it's just a personal preference, but for me, using suffixes "-dev", "-download" and -www" seems inconsistent with the "-$componentName" suffixes, like "jface" or "query"; IMHO, they should be prefixes which map directly to the directory structure, which would make things simpler and easier to read/audit. It would also make the group names potentially generate'able -- is it possible to set up ACL rules based on a database, populated by scraping the directory tree to determine the group names? If so, then all you need is a tool to put people into those groups; the groups themselves would be determined by pre-existing committers/PMCs when they add new components in CVS or folders in ~/downloads/ (and the scraper tool to generate group names, of course). > - how are the www and download components from 2.1 and 2.2 handled for a > project that does not use components? if they don't have these components then > how do they manage commit rights to their web and download resources? Or is it > assumed that they don't and all committers get the whole enchilada? How about something like these: downloads-tools-pdt -> ~/downloads/tools/pdt downloads-eclipse-equinox -> ~/downloads/eclipse/equinox As above, it's assumed that if you're a component owner in the project, we trust you enough to write your stuff using the same group access as everyone in that project. If we split things up so that query had its own downloads-modeling-emf.query group, we wouldn't be able to have cross-component shared files/folders (eg., RSS feeds, update sites, javadoc). Of course this only applies to projects w/ components. If you're only TLP>Project, then you'd have downloads-$TLP-$Project for your ~/downloadss/$TLP/$Project folder. > (3) > - As has been stated before, forcing the group<->component relationship is in > fact forcing this limitation on the social structure of the projects. Only if we limit the definition of a component. In the above example, I suggest that query.releng is a "component", even if by the website/project structure definition it's not. Sure, this means that a component means different things at the filesystem level than at the project structure level, but is that a bad thing? Perhaps. Is it forgiveable? I'll let you decide. ;-) I'm OK with this touch of inconsistency, but only because I've been living with it for years now. The TLP has projects; the projects have components. Our components also have "sister-components" (for releng only) and thus don't appear on the website, but DO appear in Bugzilla. However, there's a break there too in that there's no query.releng component in Bugzilla, just a releng component in the EMF "product". So, we have: web: Modeling > EMF > Query bugzilla: Modeling > EMF > Query, and Modeling > EMF > Releng (shared amongst components, because build problems are often shared) unix group: modeling-emf-query, and modeling-emf-query.releng (one .releng per component because every build is unique, and [people with access to -query] != [people with access to -query.releng]) If we were going to be truly consistent, we'd have to create new components in bugzilla (eg., query.releng). Can this approach be done for other projects? Can we assume that if there's a Bugzilla component then that "component" (be it a component in the TLP>Project>Component sense or in the functional working group sense) would have its own unique unix group? Could the scraper tool I propose above do two things -- scrape the dev.eclipse.org filesystem and also Bugzilla -- and create a set of groups from both inputs (while of course merging the dupes and punctuation-variants)? If so, you'd have a master list of filesystem and Bugzilla components from which to create groups or ACL rules -- and you'd have consistency in the naming too. As to the implementation plan... Bjorn, will you also be exploring bug 199501 so that we can pull from this new data source instead of our own when creating our committer/contributor lists [1] ? [1] http://www.eclipse.org/modeling/project-info/team.php?byProject
(In reply to comment #85) > - There is an inconsistency in the terminology. AFAIK (pointed out to me by > you or Mike...) there are no "sub" projects. See my explanation of the term in comment #81. > - "ui-jface"-style names are confusing ... If people > want to decorate their group names then they should use a different delimiter > (e.g., '.') Ok. Or we could use top.sub.component group names and then allow people to use "-" in their component names. I'm happy either way. > (2.1) > - Can you expand on the phrasing "...will always have a www component of > committers who have write-access to web site."? The wording around "component > of committers" is strange. I guess I should say "will always have a www component (a subset of the committers who have write-access to the web site)". > (2.3) > - Do you see a need for a project group even if the project uses components? No. I thought about this for a while and I couldn't think of anything that wouldn't be associated with components. However, I'm sure someone will tell me that I'm wrong :-) > - how are the www and download components from 2.1 and 2.2 handled for a > project that does not use components? In my (latest) plan, every project has components. If the project does not want to define any components, then it will have three default components for CVS access (everyone), web access (some people), and download server access (a few people). > (3) > - As has been stated before, forcing the group<->component relationship is in > fact forcing this limitation on the social structure of the projects. People > who are currently over or near the limit (e.g., >= 10 groups now) will be in > danger of not being able to work. I'm sorry, Jeff, but you're just not convincing me of this. Currently there are N groups and each group has G(N) members. You haven't convinced me that changing to consistent names and portal managed membership is going to change either N or G(N). I'd like to see a concrete example of where that is going to happen. > The set of > people currently over 16 and near 16 is not trivial and many of them are very > active. Having their activities and further commit rights curtailed would be > very much less than optimal. The webmasters already have a stop-gap measure in place for those people: it is being used right now. Changing the group names is not going to change that stop-gap measure (the stop-gap for those people does not use groups at all). > (5) > - The deletion of committer status seems a bit eager in this case and may not > be in line with the project charters. Hmm. So the committers on a component should have to vote for its deletion? > (6.7, 6.8) > - wording is confusing (e.g., don't think TLPs typically have components) Some do. Yours doesn't but that doesn't mean they all don't. BIRT and STP come to mind. > Other: > - We also have groups like orbitbuild which seem to be in place to manage > access to Foundation managed resources (in this case the build infrastructure). > Is this another component? Will be a little strange to have it surface this > way... There will be some groups that the webmasters will continue to maintain outside of this infrastructure. For the case of orbitbuild, my guess is that it would be a component of the Orbit project. But I leave that decision to Denis et al. > - how are PMC groups managed? What is a PMC group? Either the members of the PMC are committers and thus could belong to a component, or they are not and thus cannot. > - not sure about others but we tend to have a doc component in each project > that everyone in the project has access to (we want everyone writing doc). > ... > becoming a committer in one component involves adding 3 groups. > component-dev, project-www and project-doc Sure. With one election. > - currently there seem to be a few groups that are *-all or some such (e.g., > eclipse-all, common, ...). What happens to these? I don't know what these groups represent, so you'll need to give me more information. Are there files owned by these groups or are they left over from times past but are not being usd? > - similarly, there are groups for project leads or admin purposes (e.g., > equinoxadmin, orbitadmin, ...). What happens to these? These are the groups of people who have access to the download servers. These will be the download component of each project. (In reply to comment #86) > This is > possible due to the clever use (read: manual and error-prone) of ACLs by the > webmasters. Yes - that is the stop-gap measure described above. That will remain until bug 157931 is fixed. My point of disagreement with you is about whether changing the group names (this bug) will cause more people to have to be added to groups and thus cause the stop-gap measure to be needed for more people. I fail to see why that would be true... If we have five things named A, B, C, D, and E and we rename them to M, N, O, P, and Q, it doesn't change those five things. (In reply to comment #87) > > - There is an inconsistency in the terminology. AFAIK (pointed out to me by > > you or Mike...) there are no "sub" projects. There are TLPs, Projects and > > Components. > > We need to be consistent w/ > the terminology -- is it "top level project > project" or "project > > subproject" ? See my answer in comment #81. > Yes, MDT got burned on this one too when we wanted to do Modeling > MDT > UML2 > Tools and were told "there are no subprojects". That was because you wanted to run UML2 Tools as a sub-sub-project and the Board had decided that nested sub-projects was not allowed. (Separate conversation: I'm going to again try to convince them to change their mind this fall/winter process revision.) > Would it be reasonable to set up all projects to have a TLP, a project, and a > component, even if it's in fact only got two of the three? So far this bug and the proposed solution is not about forcing TLPs to have sub-projects if they don't want them. It is forcing all projects to have components, but only for the sake of managing the access rights of the committers (which is already being done through the opaque groups). > > - how are the www and download components from 2.1 and 2.2 handled for a > > project that does not use components? The www and download components exist for all projects and membership in those components is subject to nomination and election just like all other components. > Only if we limit the definition of a component. In the above example, I suggest > that query.releng is a "component", even if by the website/project structure > definition it's not. Sure, this means that a component means different things > at the filesystem level than at the project structure level, but is that a bad > thing? Perhaps. Is it forgiveable? I'll let you decide. ;-) Oh, you lost me. The whole point of standardizing these things (group names and components) is to standardize them, so it would seem to be a mistake to have components mean something different at the filesystem level than the project level. > The TLP has projects; the projects have components. Our > components also have "sister-components" (for releng only) Project teams are free to create as many sister-components as they want. If a project wants to have both foo and foo.releng components, that's fine. The portal election system will allow elections to one at a time or both together. > If we were going to be truly consistent, we'd have to create new components in > bugzilla (eg., query.releng). Yes, I suppose. But let's solve one problem at a time. > Could the scraper tool I propose above do two things > ... ACL rules based on a database, > If so, then all you need is a tool to put > people into those groups; Again I'm a bit confused - what exactly is this scraper tool doing? The tool for putting people into groups is nominations and elections via the portal; that information is then used to generate the ACLs (the ACLs are currently implemented as unix groups). > As to the implementation plan... Yes, this information will be available through the bug 199501 API. But first we need to implement this bug so that we have the information to provide ;-)
(In reply to comment #88) > > (3) > > - As has been stated before, forcing the group<->component relationship is in > > fact forcing this limitation on the social structure of the projects. People > > who are currently over or near the limit (e.g., >= 10 groups now) will be in > > danger of not being able to work. > > I'm sorry, Jeff, but you're just not convincing me of this. Currently there are > N groups and each group has G(N) members. You haven't convinced me that > changing to consistent names and portal managed membership is going to change > either N or G(N). I'd like to see a concrete example of where that is going to > happen. Boris gave a concrete example of this in comment 86. What is happening today is plat-ui-jface is NOT a component but yet there is a group. Through creative use of ACLs, all eclipse.platform.ui committers get write access to the set of things related to plat-ui, plat-ui-jface, ... As discussed, there are some people who only get membership in plat-ui-jface. I am all or regularizing these structures (e.g., jface should probably be a component). If the only mechanism for managing permissions is groups then all plat-ui members will need to permute onto plat-ui-jface, plat-ui-bindings, ... Notice that, Boris for example is not a member of the plat-ui-bindings group though he is one of the driving members. He gets access rights today by virtue of ACLs. Again, I am 100% in favour of regularizing these structures but forcing this now, before solving the group limitation problem, will cause, for example, Boris to gain several groups. We need a plan to ensure it is not disruptive. You seem to say that somehow the current setup will continue. What happens however when we need to elect a new committer on the UI? We run an election and list 5 or so components. What happens when that puts the committer over their limit? Your note suggesting that PMCs and project leads should keep the limitation in mind seems to indicate that the limitation will manifest itself somehow and that creating components should be avoided. Summary: I suspect there is a fundamental miscommunication here. We are assuming something and you are assuming something else. Not sure what that is but perhaps spelling out exactly what happens for people who are at/over the limit would help. How does the election work? How are their actual commit rights tracked and how are the write permissions conferred? What happens to people like Boris who will get a mess of other groups... etc. > The webmasters already have a stop-gap measure in place for those people: it is > being used right now. Changing the group names is not going to change that > stop-gap measure (the stop-gap for those people does not use groups at all). It is not changing the names it is forcing the voting->component->group coorelation. > > (5) > > - The deletion of committer status seems a bit eager in this case and may not > > be in line with the project charters. > Hmm. So the committers on a component should have to vote for its deletion? Dunno but the charter calls for the PMC to do committer deletions. > > (6.7, 6.8) > > - wording is confusing (e.g., don't think TLPs typically have components) > Some do. Yours doesn't but that doesn't mean they all don't. BIRT and STP come > to mind. Interesting. > > - how are PMC groups managed? > What is a PMC group? Either the members of the PMC are committers and thus > could belong to a component, or they are not and thus cannot. Not really sure what the groups are either but I seem to be a member of groups like "eclipse" and "tools" or some such. I assumed that these are a result of me being on those PMCs. > > - currently there seem to be a few groups that are *-all or some such (e.g., > > eclipse-all, common, ...). What happens to these? > I don't know what these groups represent, so you'll need to give me more > information. Are there files owned by these groups or are they left over from > times past but are not being usd? I too have no idea what these are for. All I know is that I am a member of these groups (along with a mess of other people). Perhaps Denis can shed some light. > So far this bug and the proposed solution is not about forcing TLPs to have > sub-projects if they don't want them. It is forcing all projects to have > components, but only for the sake of managing the access rights of the > committers (which is already being done through the opaque groups). We should probably clarify "component" wrt bugzilla etc. There are a set of other things related to components (bugzilla, leads, ...) that have more visibility than commit rights/write access. If the component-group relationship is just an implememtation detail of access management, the list of components is somewhat irrelevant to the community (as the set of groups is today). If adding a component forces the creation of bugzilla buckets etc then it becomes more visible to the community and again, will impact the social structure (not sure we want to have hpux and carbon components in bugzilla) but we do have such groups.
(In reply to comment #77) There has been more comments about ACLs being a stop-gap measure but hard to administer. I suppose this refers to UNIX / file system ACLs. So far, nobody commented on my proposal of using CVS ACLs (comment 77) for fine-grained access control in components, like SourceForge is using. It looks like these would not need to be centrally administered so they could alleviate some problems. Or is this just silly and I'm missing some point completely?
(In reply to comment #89) > Through > creative use of ACLs, all eclipse.platform.ui committers get write access to > the set of things related to plat-ui, plat-ui-jface, ... That is not a true fact, sorry. The plat-ui unix group has 42 members. Only 9 of those members have 'stop-gap' ACLs. Thus clearly the other 33 members of that unix group do not require additional access permissions and thus will not be in additional groups. > Notice that, > Boris for example is not a member of the plat-ui-bindings group though he is > one of the driving members. He gets access rights today by virtue of ACLs. That is correct, Boris is one of those stop-gap people. > ... will cause, for example, > Boris to gain several groups. As Boris is already in the stop-gap ACL, that does not appear to be a problem, does it? Plus, having all the permission information in the database through the use of component = permission group would allow the webmasters to manage the ACL *automatically* and thus be less "error prone", eh? > You seem to say that somehow the current setup will continue. What happens > however when we need to elect a new committer on the UI? We run an election > and list 5 or so components. What happens when that puts the committer over > their limit? Ok, I'll add a note about that to the proposal: I'll state that when a committer is added to the 17th or greater group, an email will be sent to the webmasters so that they can update their ACL scripts (manually or automatically). > [1] How does the election work? > [2a] How are their actual commit > rights tracked and [2b] how are the write permissions conferred? > [3] What happens to > people like Boris who will get a mess of other groups. 1. Same as today but allowing multi-elects. 2a. Better than today because all the information will be in a single relational database. 2b. Same as today: either unix groups (pending bug 157931) or stop-gap ACLs. 3. Same as today. > Dunno but the charter calls for the PMC to do committer deletions. So does my proposal - only PMCs can delete components and thus only PMCs can delete committers. > ...the list of > components is somewhat irrelevant to the community... Unfortunately, that's not what the Eclipse IP Policy says. One of the hallmarks of Eclipse is the careful record keeping about what IP is involved and who is/was allowed to touch it. Committer elections are an integral part of that and thus components are an integral part of that.
(In reply to comment #90) > So far, nobody commented on my proposal of using CVS ACLs (comment 77) for > fine-grained access control in components, like SourceForge is using. Without speaking directly for the webmaster, I think that it unlikely that we are going to install a tool like that on our main production servers. Big red flag to me are: (A) last release is over a year old, (B) there is only one developer on the project, and (C) the average age of the open bugs is >250 days and is growing by approximately 1 day/day.
(In reply to comment #92) Thanks for the stats... actually I was not proposing a specific tool but rather the concept. The tools from my URL might be old indeed, but there might be others better maintained which serve the same purpose. I'm not such a CVS wizard, but perhaps others are?
(In reply to comment #93) > actually I was not proposing a specific tool but rather the concept. Then your comment is probably better address to bug 157931.
(In reply to comment #91) > That is not a true fact, sorry. The plat-ui unix group has 42 members. Only 9 > of those members have 'stop-gap' ACLs. Thus clearly the other 33 members of > that unix group do not require additional access permissions and thus will not > be in additional groups. You seem to assume that only individual users are listed in the ACLs. This is not true - for example, the plug-in org.eclipse.core.databinding is owned by plat-ui-bindings, but all members of plat-ui have access to the plug-in because plat-ui is listed in the ACL. So it remains a very true fact ;-) that: After the change, all members of the plat-ui unix group will also have to be in plat-ui-jface, plat-ui-bindings, plat-ui-tabbed, and plat-ui-nav. This will allow them to keep their access rights to those directories for which plat-ui was listed in the ACL. Then, if some of them end up with more than 16 groups, their user names will have to be added to the ACLs because of the 16 groups limitation.
(In reply to comment #88) >> Yes, MDT got burned on this one too when we wanted to do Modeling > >> MDT > UML2 > Tools and were told "there are no subprojects". > That was because you wanted to run UML2 Tools as a sub-sub-project and the > Board had decided that nested sub-projects was not allowed. (Separate > conversation: I'm going to again try to convince them to change their mind > this fall/winter process revision.) Really? I prefer the 3-level limit, and not just because if you allow a 4-level structure someone in Modeling-land will decide they want me to move stuff around. Again. (The horror!) ;-) > The whole point of standardizing these things (group names and > components) is to standardize them, so it would seem to be a mistake > to have components mean something different at the filesystem level > than the project level. (but...) > Project teams are free to create as many sister-components as they want. > If a project wants to have both foo and foo.releng components, that's > fine. The portal election system will allow elections to one at a time > or both together. So in one case you're saying it's a mistake to have a different project and filesystem structure; then you're saying deviate all you want. Either I have to declare a query.releng component as a child of its parent EMF project, or it's just there at the component level. It's either a red-headed bastard stepsister of the query component (with no parent in Bugzilla or in the project structure), or as a legitimate sibling (in Bugzilla and in the project structure). How can it be both? Or, is this where the 4th level (sub-components) comes in? Babies having babies, components having sub-components? Problem w/ the breaking the 4th wall, erm, level, is that there's no corollary in Bugzilla, so by definition you've created a new unresolvable inconsistency. IMHO, that's worse than having urchin components running around a la Oliver Twist. > Again I'm a bit confused - what exactly is this scraper tool doing? Group Scraper Tool: commandline / crontab script to get filesystem structure (TLP/Project/Component) and Bugzilla structure (Classification/Product/Component) and generate a list of groups/ACLs into which users can be added. Committer-to-Group Assignment Tool: web UI in the Portal for adding, removing, or altering the committer-to-group linkages. List of groups is populated via scraper tool. List of committers comes from database mentioned in bug 199501.
(In reply to comment #96) > So in one case you're saying it's a mistake to have a different project and > filesystem structure; then you're saying deviate all you want. Um, no, I don't think that's what I meant to say. I just meant to say that if a project has two components A and B and wants to have a social convention that A and B are sister components, then the project is welcome to do that. The Charter and the tools do not treat A and B any differently than other components C, D, and E, but the project's social conventions may do so. > Problem w/ the breaking the 4th wall, erm, level, is that there's no corollary > in Bugzilla, so by definition you've created a new unresolvable inconsistency. Ok, so maybe it's a bad idea. Let me solve this one (this bug) first and then we'll work on others of my (bad) ideas.
Created attachment 76505 [details] People that will have more than 16 unix groups after taking into account ACLs (In reply to comment #95) > You seem to assume that only individual users are listed in the ACLs. ... > So it remains a very true fact ;-) that: > After the change, all members of the plat-ui unix > group will [have more groups]. Ah, ok, I was wrong. I did assume that the ACL only contained users. Now that I understand it to contain groups as well, I reran my simulation of the new world and have attached the revised list of people who will have more than 16 unix groups. At Jeff's request, I extended the list down to people with 13-16 unix groups (i.e., people at risk of going over). The number of people with more than 16 unix groups will grow from 24 (today) to 37 (then). I find this small increase (the sum total across all Eclipse projects!) to be a reassuring number: the webmaster's stop-gap measure will easily accomodate 13 additional users. In fact, having the standardized group names and having the group/component membership information in the central database will allow the webmaster to automate the maintenance of the unix groups and stop-gap ACLs which will reduce errors (ref comment #33 and comment #86) and free up staff time for other new committer toys. [Jeff and Boris, thanks for driving through my misunderstanding and thus helping us gather the complete data about the consequences of this change (and discover that it will not be major).]
Created attachment 76506 [details] revised plan for implementation of standardized groups Here is the revised plan, taking into account the excellent feedback received in comment #85, comment #87, comment #89, and the enlightening comment #95.
(In reply to comment #99) This looks good to me, thanks. I guess it is understood that, when a request is made to create new directories, folks will tell webmaster for which component the new directories are for? Some projects' CVS structure is hierarchically identical to their project/component structure, but some projects (such as Eclipse) have a flat structure, where every directory of every component is at the base of their cvsroot. For instance, this is what we need to avoid: BAD: "please create org.eclipse.someplugin. Should have the same permissions as org.eclipse.pluginx plus Bob Dude and Mary Green. Bubba Smith on the component-t should also have commit access." BAD: "please create org.eclipse.someplugin for the jdt-core component. Bob Dude and Mary Green, as well as Bubba Smith on the component-t should also have commit access." GOOD: "please create org.eclipse.someplugin for the jdt-core component."
(In reply to comment #100) As an addendum to the above comment, do note that the following will be technically possible (and easily accomplished) with standardized groups, whereas we cannot do it efficiently now: GOOD: "please create org.eclipse.someplugin for the jdt-core component. The plat-ui component committers, as well as the swt-text-debug component committers should all have commit access" I'll let Bjorn decide if the above is desirable or something we need to avoid.
(In reply to comment #101) > I'll let Bjorn decide if the above is desirable or something we need to avoid. Sorry, no: one component per directory. Under Eclipse rules, committers elect their peers, i.e., the people who have write access to their code. Allowing multiple components write access to a single directory means that committers would then no longer be in control of their peer group (e.g. person X is in component A, but component B can also write to the code but person X has no voting control over component B).
Good we are getting to the bottom of this. Thanks for the revised numbers. Just to clarify, these numbers include all groups that appear in the ACLs? or just updated to include the UI group structure? There are also several situations in the Resources space as well (ports to qnx, hpux, ...)
(In reply to comment #100) > Some projects' CVS structure is > hierarchically identical to their project/component structure, but some > projects (such as Eclipse) have a flat structure, where every directory of > every component is at the base of their cvsroot. I would argue that along with fixing groups, projects should reorg their code so that it too is hierarchichal. In terms of builds, it shouldn't mean much more than updating map files. In terms of developers, it shouldn't mean much more than checking out all the moved folders from CVS anew. (Trust me on this one, we just did this for EMF's feature reorg across all 4 active branches.) This could also mean that TPTP would *finally* get their 190 hyades plugins/features out of /cvsroot/tools/ and put them in a subfolder, like /cvsroot/tptp/hyades/, so that that cvs could be more easily browsed w/ Eclipse. (I know I'm digressing here. I'll put this request in bug 200747.) My point is that CVS's organizational structure should mirror Bugzilla and the filesystem and the groups/ACLs. If it's all consistent, it's more easily maintained and it's easier for auditing tools to be created to ensure adherence to that consistency.
(In reply to comment #103) > Just to clarify, these numbers include all groups that appear in the ACLs? or > just updated to include the UI group structure? Correct. What I did was: 1. For all users, set groups[user] = empty set 2. For each existing unix group G, for all members U of G, groups[U] += G 3. For each single user ACL for user U and directory D, traverse the entire directory tree D and below, collecting the set H of all groups that own files in that directory tree. Then, for each group K in H, groups[U] += K. Note: this adds user U to all the groups that cover directory tree D and thus will allow U write-access to that tree. 4. For each group ACL for group G and directory D, collect H (as in 3). Then, for each member U of G, for each group K in H, groups[U] += K. groups[U] is now the set of all groups that person U needs to be in in order to have the same write-access as person U does today. That's the list I attached.
(In reply to comment #105) > 3. For each single user ACL for user U and directory D, traverse the entire > directory tree D and below, collecting the set H of all groups that own files FWIW, webmaster does not support, endorse or apply different ACLs/permissions any deeper than the component level.
Created attachment 77070 [details] re-revised plan for implementation of standardized groups No comments on this bug for almost a week - that's a record! I'm going to assume that we're very close to finalizing this so I've updated the implementation plan slightly.
Well, I'd still rather have the group names align more closely with bugzilla and dev.eclipse filesystem structure (eg., download_tools_cdt vs. tools_cdt_download), because it would allow simpler auditing of the mappings between groups, directories, and bugzilla components. However, since I'm not the person who'd have to do such auditing/maintenance, I suppose this is something that the Webmasters should weigh in on. Would reordering the "download" and "www" chunks (and possibly dropping the "-dev" suffix entirely) make pattern matching easier for you guys?
(In reply to comment #107) > Created an attachment (id=77070) [details] > re-revised plan for implementation of standardized groups > > No comments on this bug for almost a week - that's a record! > I'm going to assume that we're very close to finalizing this so I've updated > the implementation plan slightly. > Here's a number of comments I hope can improve the document and process ... some are small nits to improve the precise wording, some are larger process questions. 1. In (2) Default Groups, point 1, you say "a www component containing the subset of committers who have write access ....". In WTP, by policy, we actually allow write access to web site by all committers, plus we have some additional people that are committers to the web site, so, technically that's a superset. I'd word it simply as "a www component containing the committers who have write access .... ". 2. In Section (8) Moving Files and Directories ... a) You end by saying "... the same way that new committers are elected". This doesn't see quite accurate, since there's not vote on moving files/directories. I'd just say something like " ... to make sure movement is well documented, component owners shall send a note to the project's PMC list documenting their agreement, and after the PMC reviews the move they will authorize via a note to emo@...". b) Relatedly, in this section you don't mention the scope of the movement. My assumption (and recommentation) would be to specify it applies to any components within the same top-level component. And, that if a component (or project) is to change from one top-level project to another, it also has to go through a Move Review. [At least, that's what we had to do with JEM moving from Tools-VE to WTP]. 3. in (9) Creating new CVS/SVN Modules, should this be 'directories', instead of 'modules'? I'm no expert, but think "modules" in CVS means something slightly different, but I think the UNIX permissions apply (only) to directories. 4. Now, an implementation detail. You mention a spread sheet where the directories and files are listed for each component/project. I assume this means "the highest level directory or directories that contains all the directories and files..." (for example, if we already have a clear "org.eclipse.jsf" directory, seems no need to list all the subdirectories. [Of course, for must of (the old) JST/WST, we have a bit of mess on our hands and will have to decide if easier to re-org CVS or list tons of directories]. 5. One final aspect on process, that made me short of breath as I read it, was the "all at once in December" part of the plan. Might I recommend that this change be "rolled out" by having one or two top-level projects go first, with a few weeks between other projects moving? Would that help ensure any bugs in the process were worked out before it effected everyone? 6. As a side issue, sort of, I'm not sure how our recent "refactoring" in WTP should interplay with this change. Should we wait and get all fixed up with our new projects in December? Or, should we get started now, to get hooked into the voting portal, etc. Can we, in WTP, define our new project groups, and associate them in Portal database, but not associate them with CVS directories yet, but leave existing Unix directory groups in place, with their existing unix permissions? That would seem to accomplish half the task initially, and we could work out the CVS permissions later? Thanks
Created attachment 77092 [details] re(3)-revised plan for implementation of standardized groups (In reply to comment #109) Thanks for the comments David. > In WTP, by policy, we actually allow write access to web site by all > committers, plus we have some additional people that are committers to the web > site, so, technically that's a superset. Everyone who has write access to an eclipse.org CVS repository is a committer. You may have some committers who are only members of website component, that's fine, but they are still committers, hence my use of "subset". > 2. In Section (8) Moving Files and Directories ... > This doesn't see quite accurate, since there's not vote on moving > files/directories. I changed the wording slightly. > b) Relatedly, in this section you don't mention the scope of the movement. I clarified that. > 3. in (9) Creating new CVS/SVN Modules, should this be 'directories', instead > of 'modules'? I've clarified that. > 4. I assume this > means "the highest level directory or directories that contains all the > directories and files..." Yes, I did and now I've clarified that. > 5. One final aspect on process, that made me short of breath as I read it, was > the "all at once in December" part of the plan. Might I recommend that this > change be "rolled out" by having one or two top-level projects go first, I thought about that and am willing to discuss it further, but if we go with a partial roll-out then we (the webmasters) will be supporting a hybrid tool solution for a while. That's more risk and more work. The real question is "is there more or less risk in doing an all-at-once or an incremental deployment"? My feeling is that there is less risk in an all-at-once because: * The database has one schema for all projects * Component membership = unix group memberships and are defined a consistent way Issues that won't be better or worse either way: * Projects may have made errors in specifying component memberships and thus people will suddenly not be able to access files they thought they could. > 6. Should we wait and get all fixed up with our > new projects in December? Or, should we get started now... It's up to you - it won't make any difference in the amount of work we (the EMO) have to do either way. > Can we, in WTP, define our new project groups, and associate them in Portal > database, but not associate them with CVS directories yet, but leave existing > Unix directory groups in place, with their existing unix permissions? That > would seem to accomplish half the task initially, and we could work out the CVS > permissions later? Well, that might work except for two things: (1) directory permissions, project-component memberships, and legal documentation are intertwined and (2) then the webmaster would have to support you in your hybrid mode for a while. (1) leads to more work for the Legal team and (2) leads to more work by the webmasters. So, I'm going to say no - we really need to get this done and thus reduce everyone's workload.
(In reply to comment #108) > Well, I'd still rather have the group names align more closely with bugzilla > and dev.eclipse filesystem structure (eg., download_tools_cdt vs. > tools_cdt_download), because it would allow simpler auditing of the mappings > between groups, directories, and bugzilla components. > However, since I'm not the person who'd have to do such auditing/maintenance, I > suppose this is something that the Webmasters should weigh in on. Would > reordering the "download" and "www" chunks (and possibly dropping the "-dev" > suffix entirely) make pattern matching easier for you guys? Don't use CDT as an example :). I like the proposal, mainly because it pretty much matches what we're already doing in the CDT.
tptp agrees with this and we ask that we have a bulk update window to get all the baselines set up right. We also need a good way on the portal to remove people from the ACL.
(In reply to comment #112) > tptp agrees with this Olivier - there are 111 other comments on this bug, which one are you agreeing with? Perhaps Doug's comment #111 ? > and we ask that we have a bulk update window to get all > the baselines set up right. That's the spreadsheet that you will be providing to the EMO. See item 3 in the implementation plan of the implementation document. > We also need a good way on the portal to remove > people from the ACL. That's explained in section "(7) Removing Committers" of the implementation document.
(In reply to comment #113) > (In reply to comment #112) > > tptp agrees with this > > Olivier - there are 111 other comments on this bug, which one are you agreeing > with? Perhaps Doug's comment #111 ? :) tptp agrees with the renaming effort and will comply with the effort (as long as you do not make me read and fully understand each and every one of the comments ;) tptp is already set up pretty close to this way already. > > > and we ask that we have a bulk update window to get all > > the baselines set up right. > > That's the spreadsheet that you will be providing to the EMO. See item 3 in the > implementation plan of the implementation document. > > > We also need a good way on the portal to remove > > people from the ACL. > > That's explained in section "(7) Removing Committers" of the implementation > document. >
(In reply to comment #110) > Created an attachment (id=77092) [details] > re(3)-revised plan for implementation of standardized groups I'm not thrilled to see the use of the underscore as a convention in the group names. Wouldn't the period make more sense as a separator? I've tested this to be a valid group name (although long as heck to type): eclipse.platform.ui-jface-test The use of a period would also be consistent with our project.component naming convention used elsewhere (IPZilla, our internal database, website, etc.)
+1 for dot and dash. If it was good enough for Morse, it should be good enough for us...
(In reply to comment #116) > +1 for dot and dash. If it was good enough for Morse, it should be good enough > for us... +1. Agreed. As mentioned in comment 87, I'd prefer "modeling-emf-query.releng" or "org.eclipse-www-modeling", where '-' is just a rotated directory separator ('/' or '\') between TLP, Project, and Component, and '.' is used to distinguish components from "subcomponents" and/or for actual directory names, as in /cvsroot/org.eclipse/www/.
FWIW, I liked the suggestion in comment 115. dot separated tlp.project.component based groups. Seems more in keeping with the way we seem to drive other bits of infrastructure.
FYI, I've opened bug 207212 in WTP land, as I think to implement this standardized groupings, we should restructure some of our CVS first. I've created a wiki document where I (try to) list some of the details for WTP. Let me know if I've gotten any of the group names wrong.
(In reply to comment #119) > FYI, I've opened bug 207212 in WTP land, as I think to implement this > standardized groupings, we should restructure some of our CVS first. > > I've created a wiki document where I (try to) list some of the details for WTP. > Let me know if I've gotten any of the group names wrong. > Opps, forgot the link to the wiki document: http://wiki.eclipse.org/WTP_CVS_Restructuring
From what I can tell, the WTP restructuring will require us to create/add/delete groups. Bjorn, in order to avoid duplicated work, are the standards/conventions of attachment 77092 [details] final? Ie, if we apply these rules to the WTP restructure, we're pretty much all set?
Oh, I notice you haven't updated the document to reflect the dot-dash notation proposed in comment 115. Thoughts?
Denis, my reading of the list is that we all like your proposal in #115, toplevelproject.subproject[.component]-whateverelse (So, I think you're free to go with it). Some other requests to simplify things: I think the general rule is turning out to be, for cases such as webtools, which consists of a top-level project, and a number of sub-project, that if the rules were strictly applied, we'd have groups consisting of tlp.sp-dev tlp.sp-www tlp.sp-download This pattern makes sense for tools and technology top level's, but, I doubt few others. We in Webtools, at least, have a policy that anyone in any subproject can write to any web page in webtools. So, we don't need literally need 10 different tlp.sp-www groups ... at most, just one, such as webtools-www Similar for 'downloads'. We want to effectively have one 'downloads' group for all of webtools, which would have just a few members. I don't mind if all the more detailed ones exist, if its just easier to do mechanically, but not sure we'd use them, and would like to have the ability to manage some resources with these higher level groups. I'd even be willing to say that we (WTP) would be willing to manage the unix groups, if given the authority to, since it's pretty easy rule ... "if user added to any webtools.*-dev list, then add them to webtools-www list. And .. our main downloads groups changes so seldom, we'd be happy to manage that as well. Plus, perhaps best considered as a separate group, we also have some directories in cvs we'd like everyone in webtools to have write access to. This will be our "maps" directories and would like anyone be able to release 'map files' into those map directories. So, ideally, we could also have a 'webtools-all' group that was literally just a group of all webtools.*-dev committers. I'd be happy to manage that one as well. I realize this sort of complicates things since doesn't follow the tools and technology patterns ... but I think overall, it would simplify things for all concerned. If we don't do that ... it would be a pretty huge change for us to all have seperate www and downloads directories. Anyone else have any thoughts/needs along these lines? Thanks,
(In reply to comment #123) > I don't mind if all the more detailed ones exist, if its just easier to do > mechanically, but not sure we'd use them, Yes, that would be easiest - the groups will exist and you will have the opportunity to elect people to them, whether they are overall or sub groups. > I'd even be willing to > say that we (WTP) would be willing to manage the unix groups, if given the > authority to, since it's pretty easy rule ... " No, sorry :-( We need to maintain the purity of the IP paperwork around committers and we do that through the election mechanism. So we require that group membership be maintained through elections.
(In reply to comment #124) > (In reply to comment #123) > > > I'd even be willing to > > say that we (WTP) would be willing to manage the unix groups, if given the > > authority to, since it's pretty easy rule ... " > > No, sorry :-( We need to maintain the purity of the IP paperwork around > committers and we do that through the election mechanism. So we require that > group membership be maintained through elections. > I don't think I am making my self clearly heard, here. For the code related (CVS) groups, yes, we would have groups (the -dev ones) and we'd have projects and we'd vote and vote and vote. But, that's not what I'm talking about. I'm talking about the web content groups (for -www groups). These are not the same as code groups, and you've told me before, Bjorn, that the divisions of IP paper work was not so critical here, that is that as long as someone was voted in as a committer on a subproject code group, then we could automatically make them a committer in the larger webtools web content group without further voting. Are you saying that was true a few months ago, but no longer is? What changed? Similar for the map file directories. There is no IP content in there, and it's a simple matter of trying to simplify everyone's life that if someone is a voted-in code committer, we need the ability to say they can also have access to the non-code map directories. Am I being any clearer yet? I'm happy to help in any way. Perhaps a phone call? Pictures?
(In reply to comment #121) > From what I can tell, the WTP restructuring will require us to > create/add/delete groups. > I should clarify, the actual WTP CVS restructuring doesn't require any different groups ... we could use the same ones we are using now, until standardized groups are finalized ... if they are not finalized for us in next few days. We are doing the restructuring in preparation for the new groups, but the new groups don't have to be done first.
projects within projects within projects! Funny how we've gone from no sub-projects but component hierarchies to a project hierarchy with no components ... seems language shouldn't be so hard. :) So ... is one of the requirements that Linux groups match exactly the names of the project, sub-project, or sub-sub-project? And then what about cvs? We have one wild mis-match in WTP ... the Dali sub-project has similar Linux group name, but the cvs location ended up being "org.eclipse.jpt" for a number of historical reasons. Should we enter tooling requirements/suggestions here? If so, 1. The sub-projects should inherit the meta-data of its parent, if it does not explicitly provide it's own meta-data. Our sub-projects in WTP use a lot of the same info, and it seemed silly to have to fill in "website" in all of them, when it was exactly the same for all of them. (multiplied times the 30 other fields of meta data required!). Thanks,
(In reply to comment #127) > So ... is one of the requirements that Linux groups match exactly the names of > the project, sub-project, or sub-sub-project? Correct. It will all be automated, so the scripts will just generate the correct group names. > And then what about cvs? We have > one wild mis-match in WTP Not sure what you question is? If you are asking "can a project have a CVS/SVN location that doesn't match top.sub.sub.sub?" then the answer is "yes, we're not requiring CVS/SVN locations to change. If you are asking "will the group that owns org.eclipse.jpt change?" then the answer is "yes, because the group names will all become consistent and regular". The project -> CVS/SVN mapping is defined in the project meta-data (maintained via the portal). > Should we enter tooling requirements/suggestions here? Please do. > 1. The sub-projects should inherit the meta-data of its parent, if it does not > explicitly provide it's own meta-data. For some meta-data that makes sense (project website), but for others it does not make sense (CVS directories). It's a good idea, though, and we'll add it to the list.
*** Bug 232365 has been marked as a duplicate of this bug. ***
*** Bug 203083 has been marked as a duplicate of this bug. ***
Moving to webmaster ... I think we're pretty much done here, and in the near future I'd like to just reassess if anything is outstanding.
Matt, could I ask you to add this to your ever-growing list of tasks? Please circle back, scribble down what else needs to be done, so we can elaborate a plan to close this as FIXED.
Created attachment 205539 [details] List of groups to modify/delete Ok, here's the list of what needs to be done. We should be able to do this transparently either in one big push, or in batches here and there. -M.
> Ok, here's the list of what needs to be done. > > We should be able to do this transparently either in one big push, or in > batches here and there. Matt, that looks good to me. I would keep this one, though: www-membersonly: DELETE (chown files www-auth)
I'm just waiting to talk to Ian about about something, but beyond that ldap is pretty clean now. -M.
Ok, after talking to Ian I've resolved the last outstanding issue(for me at least). I'm going to close this as 'resolved'. -M
