Community
Participate
Working Groups
+++ This bug was initially created as a clone of Bug #362076 +++ From the cloned bug, seems all our git repos should be tweaked to add receive.denyDeletes=true to their config files, and this should be the default for all new repositories created. I'll open another bug for "relaxing" this constraint with "hooks" if some projects desire it. I suggest a few projects try this out for two weeks before changing all 200 git repos just to make sure there are no obvious unintended effects ... I bet eclipse.ui might volunteer :)
Does the foundation even setup "default" repositories at any point? I thought each project was creating and configuring their own so far. Are you just asking to endorse a recommended set of defaults, or are you asking webmaster to actively reconfigure all the repos?
(In reply to comment #1) > Does the foundation even setup "default" repositories at any point? I thought > each project was creating and configuring their own so far. Are you just asking > to endorse a recommended set of defaults, or are you asking webmaster to > actively reconfigure all the repos? That's funny ... so, should I open a bug for "Eclipse Foundation should manage source code repositories"? :) My ignorance is showing through, but, yes, I am asking for more than a recommendation. Things like "code persistence" can't be left up to each project to handle all on their own, each one having separate rules, procedures, etc. IMHO. I know (i.e. I've read in the first chapter of my git books :) there are way to "change the defaults" of how repos are created ... but, they don't cover the details until later chapters :) So ... after the initial "enforcement" I'm sure the "defaults" could be tweaked, somehow.
David, I can't speak for other teams but in the Eclipse project we managed all our Git migrations and thus created our repositories. Due to the size of our code repository and the associated history, it wouldn't be feasible for them to do the migrations due to the time involved. The webmasters didn't set up our Git repositories for us, we did.
(In reply to comment #0) > I'll open another bug for "relaxing" this constraint with "hooks" if some > projects desire it. Please do not add denyDeletes to our (rt.equinox) repos without some hook to relax this constraint. We definitely want the ability and freedom to manage committer feature branches as we choose.
(In reply to comment #4) > Please do not add denyDeletes to our (rt.equinox) repos without some hook to > relax this constraint. We definitely want the ability and freedom to manage > committer feature branches as we choose. Why wouldn't you want the protection now? The number of developerid/tmpBranch that your developers can create between now and when bug 362363 gets fixed won't number in the thousands ... you can't create an untenable mess in your repo between now and then. A slightly misconfigured tool or a runaway helper script seems to be a real risk, not an imaginary one :-) PW
(In reply to comment #5) > (In reply to comment #4) > > Please do not add denyDeletes to our (rt.equinox) repos without some hook to > > relax this constraint. We definitely want the ability and freedom to manage > > committer feature branches as we choose. > > Why wouldn't you want the protection now? The number of developerid/tmpBranch > that your developers can create between now and when bug 362363 gets fixed > won't number in the thousands ... you can't create an untenable mess in your > repo between now and then. A slightly misconfigured tool or a runaway helper > script seems to be a real risk, not an imaginary one :-) > > PW To put this another way, I won't delete my developerid/tmpBranchs until bug 362363 get resolved and I'm confident I won't accidentally delete master. All this has me a little paranoid right now.
(In reply to comment #5) > (In reply to comment #4) > > Please do not add denyDeletes to our (rt.equinox) repos without some hook to > > relax this constraint. We definitely want the ability and freedom to manage > > committer feature branches as we choose. > > Why wouldn't you want the protection now? The number of developerid/tmpBranch > that your developers can create between now and when bug 362363 gets fixed > won't number in the thousands ... you can't create an untenable mess in your > repo between now and then. A slightly misconfigured tool or a runaway helper > script seems to be a real risk, not an imaginary one :-) > > PW If this is a temporary thing then that is fine. But I want bug 362363 fixed ASAP then.
(In reply to comment #7) > > If this is a temporary thing then that is fine. But I want bug 362363 fixed > ASAP then. I would agree it has to be a temporary scenario, as I still expect to see developer topic branches shared/deleted. For existing repos, I would also agree that it is opt-in. Certainly in the short term I don't think anyone will be making changes to repos without their PMC's or project lead's consent. PW
So ... webmasters, where do we stand on this issue? Can we say we have safe canonical. code/history protecting git repos at Eclipse Foundation yet? Or does this work still need to be done?
Reading through the comments, I get the impression that fixing bug 362363 has to happen first.
(In reply to comment #10) > Reading through the comments, I get the impression that fixing bug 362363 has > to happen first. I don't get that impression. More like bug 362363 should be fixed ASAP after the safety of this bug is enforced. But ... guess I could ask then, what will it take to resolve 363363? Sounds like its close?
I agree with David: first prio should be to protected the repo. This is more important than a committer being able to delete his personal topic branch. We can deal with enabling such deletions afterwards.
If you set receive.denyDeletes = true, then topic branches practically don't exist any more. Now that receive.denyNonFastForwards is set, the only way to update a topic branch is to delete it and then push it again.
It seems to me there is now a practical alternative to those that want to relax the rules about deleting certain types of branches (bug 362363) So, it seems to me there should be no barriers to enforcing an eclipse-wide policy of no deletes (without appropriate bugzillas, authorities, etc). I think this is important for committer safety, but, I also think this is important for "adopter safety". There could (easily) be commercial adopters that expect things certain tags or branches to stay around indefinitely, and if they could easily be removed (either by accident, or on purpose by decision of the project) then I think some of those adopters would want their own "back up" system. I don't think that's the expectation ... is it? Seems the would just be continuing to provide the same level of "foundation service" that we had with CVS. Speaking of backups, there used to be weekly tarballs created of the cvs repositories. I assume there's no "substitute" for the Git repos? When will the weekly CVS tarballs end? (I'm not asking because I have a need for them .. but ... I think these kind of things need to be well communicated to community of committers and adopters. Thanks,
(In reply to comment #14) > Speaking of backups, there used to be weekly tarballs created of the cvs > repositories. I assume there's no "substitute" for the Git repos? The purpose of the tarballs was to get the entire repo, history and all. Git does that naturally. > When will the weekly CVS tarballs end? Likely on December 21, 2012, when CVS is put in read-only mode and updating the tarballs no longer makes sense. I'll add a blurb about that on the index page -- thanks for the reminder.
(In reply to comment #14) > It seems to me there is now a practical alternative to those that want to relax > the rules about deleting certain types of branches (bug 362363) So, it seems to > me there should be no barriers to enforcing an eclipse-wide policy of no > deletes (without appropriate bugzillas, authorities, etc). I'll just point out that the solution in bug 362363 depends on denydeletes and denynonfastforwards both being false, as it depends on the script to deny deletes to main level branches. PW
(In reply to comment #14) > I also think this is important for > "adopter safety". There could (easily) be commercial adopters that expect > things certain tags or branches to stay around indefinitely, and if they could > easily be removed (either by accident, or on purpose by decision of the > project) then I think some of those adopters would want their own "back up" > system. I don't think that's the expectation ... is it? Seems the would just be > continuing to provide the same level of "foundation service" that we had with > CVS. Any downstream consumer who cares about a specific commit in the source code should have their own clone(s) of the repo(s) with their own tags/branches. They should not trust the foundation to not screw up or to continue to exist. With git (and github), this is now brain-dead easy. I think the pre-receive hook/config properties now in place should cover the vast majority of pilot errors and given the vast number of clones of the repos out there, we should be able to recover from any other pilot error that one might sneak through. These errors will not come from the average committer but from the "smart" ones like the project leads who change a config setting temporarily to perform some rare thing and make a big boo-boo instead :-)
> > Any downstream consumer who cares about a specific commit in the source code > should have their own clone(s) of the repo(s) with their own tags/branches. > They should not trust the foundation to not screw up or to continue to exist. > With git (and github), this is now brain-dead easy. > Thanks Bill, perhaps you wouldn't mind educating me ... if a downstream consumer had their own clones of repos, set up automatically to "get the latests", they'd also pick up "deletes" too, right? So ... how does that help? Is that where they'd need "their won tags/branches" as you mentioned, so they could always recover to that point? I'm just asking since this sounds like a "change for adopters" that should be clearly communicated with concrete recommendations. Greatest thanks.
(In reply to comment #18) > Thanks Bill, BJ > perhaps you wouldn't mind educating me ... if a downstream > consumer had their own clones of repos, set up automatically to "get the > latests", they'd also pick up "deletes" too, right? > So ... how does that help? I wasn't suggesting automatic mirroring. Just a clone (or not a clone, but some repo they created) into which the relevant commits exist and are tagged. But an automatic mirror would also be fine (as long as the mirroring process did not touch the tags/branches I made.) Deletes are just the removal of tags/branches under refs/. The commits are still there and eligible for gc at some point in the future unless they are references by some other tag/branch (which I created). > Is that where they'd need "their won tags/branches" as you mentioned, so they > could always recover to that point? Yes. > > I'm just asking since this sounds like a "change for adopters" that should be > clearly communicated with concrete recommendations. I think this is just good business practice. If I built a product based upon open source, I sure would want to make sure I could control access to the relevant source (via a clone). Storage is cheap...
(In reply to comment #19) > (In reply to comment #18) > > Thanks Bill, > > BJ Apologies, I think I've done that before, haven't I. > ... Deletes are just the removal of tags/branches > under refs/. The commits are still there and eligible for gc at some point in > the future unless they are references by some other tag/branch (which I > created). I vaguely understand ... not enough to implement it :) but will assume that those that need it could learn ... once they know they need it :/ > > I'm just asking since this sounds like a "change for adopters" that should be > > clearly communicated with concrete recommendations. > > I think this is just good business practice. If I built a product based upon > open source, I sure would want to make sure I could control access to the > relevant source (via a clone). Storage is cheap... Yes. I'm sure some do and some don't, but in either case, sounds like a change of "putting a copy of the cvs tar balls into the salt mines". To me, it still feels like a loss of service not to have a canonical source code repository to depend on .... automatically, for all projects in the Eclipse Foundation. But, maybe I've just not gotten into the Git mindset yet, and me (and adopters) need to learn that if you want that, you each need to clone each repo and manage its history of tags/branches in what ever way you want.
I agree with David. Of course adopters could keep a copy (and maybe they should as BJ indicates). But this also sends out a wrong signal. With CVS, it was not possible for a "normal" committer to destroy the repo at the foundation. Now, Eclipse changed the underlying technology, which one could say it's an implementation detail, and now there are chances that things get accidentally deleted. This, by default, should simply not be possible except for topic/feature branches.
(In reply to comment #21) > I agree with David. Of course adopters could keep a copy (and maybe they should > as BJ indicates). But this also sends out a wrong signal. I am not sure what signal you are talking about. The foundation should maintain a complete change history in the main branches used to build released products. It is up to the downstream adopters to decide how much trust they want to place in the foundation. Although, if the foundation really cared about a solid change history, I would think they would not allow unannotated tags in the repo. Using unannotated tags in the repos is a mistake. > With CVS, it was not > possible for a "normal" committer to destroy the repo at the foundation. In earlier days, all committers had full shell access and could easily rm -rf a CVS repo. But full shell access was replaced by a limited shell. This was done for other security reasons rather than to prevent committers nuking a repo, but the result included safeguarding the repo from such silly mistakes. So things got "safer" over time. > Now, > Eclipse changed the underlying technology, which one could say it's an > implementation detail, and now there are chances that things get accidentally > deleted. This, by default, should simply not be possible except for > topic/feature branches. With git we are now closing off such possibilities for those with limited shell accounts. We are still at the mercy of full shell account holders... :-)
(In reply to comment #22) > Although, if the foundation really cared about a solid change history, I would > think they would not allow unannotated tags in the repo. Using unannotated tags > in the repos is a mistake. you're not saying, doing one mistake justifies not caring about other mistakes, are you? :) If the issue of unannotated tags is relevant here, I could use some education on how those breach security, am I the only uneducated reader in this respect? I think one of the worries here is: for many committers git is still a bit voodoo and accidents can happen more easily than with, e.g., shell access, where those who have it should know how it works ...
(In reply to comment #23) > If the issue of unannotated tags is relevant here, I could use some education > on how those breach security, am I the only uneducated reader in this respect? It is not a security breach issue. None of this is about security. It is about the repo history. And tags should be part of the repo history. See http://stackoverflow.com/questions/4971746/why-should-i-care-about-lightweight-vs-annotated-tags
(In reply to comment #22) > Although, if the foundation really cared about a solid change history, I would > think they would not allow unannotated tags in the repo. Using unannotated tags > in the repos is a mistake. While I agree that annotated tags should be used to mark releases, using unannotated tags is pretty well a requirement of our PDE build/git integration. It is necessary at this point. PW
(In reply to comment #15) > (In reply to comment #14) > > Speaking of backups, there used to be weekly tarballs created of the cvs > > repositories. I assume there's no "substitute" for the Git repos? > > The purpose of the tarballs was to get the entire repo, history and all. Git > does that naturally. Denis can you clarify what this means. Does this mean you don't do any backups of Git repositories at the foundation, or just that you don't use tarball technology for the git backups?
(In reply to comment #26) > Denis can you clarify what this means. Does this mean you don't do any backups > of Git repositories at the foundation, or just that you don't use tarball > technology for the git backups? We backup all our SCM data to tape. But back in the day, we'd regularly get requests to download "All Of CVS" for research purposes, and that gave birth to the regular production of tarballs for those who wanted all the CVS history. git clone makes that practice obsolete. I'm not sure why David linked those tarballs to the "backups" topic.(In reply to comment #22) > Although, if the foundation really cared about I think "care" is the wrong word to use. The foundation is aware of the many processes and overhead it imposes on its committers and attempts to not restrict committers wherever possible (because we care). We can lock things down further if committers tell us it's what they want.
(In reply to comment #27) > (In reply to comment #26) > > I'm not sure why David linked those tarballs to the "backups" topic.(In reply > to comment #22) > I assumed (I guess mistakenly) they were part of the backup/recovery process. Thanks for explaining.
I don't think denyDeletes is part of of default repo permissions, but we do enforce FF only and we disable forced pushes. For those repos using Gerrit, additional restrictions may be in place. For those projects on GitHub, we cannot impose such restrictions. I think we're implemented the restrictions to satisfy the intent of this bug, so I'll close it as fixed. Please reopen if my assessment is incorrect.
