Bug 345479 - Migrate Eclipse and Equinox projects from CVS to git
Summary: Migrate Eclipse and Equinox projects from CVS to git
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Releng (show other bugs)
Version: 3.7   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.8 M5   Edit
Assignee: Kim Moir CLA
QA Contact:
URL:
Whiteboard:
Keywords: plan
Depends on: 344152 345471 345474 345669 345670 349350 349460 349891 350038 350102 350627 351001 353368 354181 354496 354894 355181 355192 355504 355506 355509 355846 357534 357870 357884 359305 360157 362670
Blocks: 296541
  Show dependency tree
 
Reported: 2011-05-11 13:52 EDT by Kim Moir CLA
Modified: 2012-02-22 03:10 EST (History)
28 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kim Moir CLA 2011-05-11 13:52:09 EDT
This is a plan item bug for the migration of the Eclipse and Equinox project's CVS repositories to git in the Juno timeframe.

As John remarked in the Eclipse Planning call this morning, it would make sense to migrate to git early in the Juno release schedule and continue Indigo SR development in CVS.  

I'm not sure which bucket this bug belongs in since it's an item that belongs to the entire team, but it can start in the releng bucket.
Comment 1 Chris Aniszczyk CLA 2011-05-31 11:01:23 EDT
Sweet, nice way to set the example.
Comment 2 Ian Bull CLA 2011-05-31 11:42:57 EDT
In the past, the Eclipse SDK has included everything needed to build the SDK.  If the Platform is moving to git, should egit move to the platform?
Comment 3 Mike Wilson CLA 2011-05-31 12:49:27 EDT
(In reply to comment #2)
> In the past, the Eclipse SDK has included everything needed to build the SDK. 
> If the Platform is moving to git, should egit move to the platform?
>

That's a good question. In principle, I would be in favour of this, if all parties were interested and we could make the logistics work.
Comment 4 John Arthorne CLA 2011-05-31 12:57:14 EDT
Related to that, we should also consider removing CVS from the Eclipse SDK after we migrate, at least in the 4.x stream. It could be a separate download in our repository for those who need it.
Comment 5 Kim Moir CLA 2011-05-31 13:20:46 EDT
regarding comment #4

It's already a separate download so this would just be a matter of removing the 
CVS feature from the SDK.
Comment 6 Ian Bull CLA 2011-05-31 13:48:09 EDT
I don't want to change the scope of this bug, but it seems that while moving our SCM, it might be an ideal time to revisit the platforms build process too (since it will have to change to some degree).

Yesterday on the equinox/p2 call, we discussed what would be involved in building p2 separately and contributing our bits to the platform (this was just some initial brainstorming, nothing has been decided).  However, if all the platform components (PDE, JDT, etc...) did this, then the platform build would really be a mini-EPP assembly process + integration tests.

This would likely reduce build time and help maintain better separation between our components.  This may also help components build / ship separately (as is the case with the Java 7 work in the JDT).

The reason I bring this up here is because it's not clear (to me) if:
1. This is desirable
2. Assuming it is, would it be easier to do this as part of the migration to git?

As I mentioned, I don't want to change the scope of this bug, but I would like to offer my help with this (especially on the p2 side of things).  If this is something that should be integrated with the migration to git, then we can discuss it on the (separate) relevant bug reports.
Comment 7 Kim Moir CLA 2011-05-31 13:57:08 EDT
Ian, Pascal has opened bug 345443 and bug 345445 for the issues wrt splitting p2 build out of the SDK build.  I think this should be pursued as a separate issue. I would like to first transition to git and then investigate making the build more modular as I think this issue will take more work, esp. wrt running the tests and consolidating the results. I welcome your help :-)
Comment 8 David Carver CLA 2011-05-31 14:28:20 EDT
(In reply to comment #7)
> Ian, Pascal has opened bug 345443 and bug 345445 for the issues wrt splitting
> p2 build out of the SDK build.  I think this should be pursued as a separate
> issue. I would like to first transition to git and then investigate making the
> build more modular as I think this issue will take more work, esp. wrt running
> the tests and consolidating the results. I welcome your help :-)

Part of helping make the build more modular is also looking at breaking up your current CVS repository into more modular and independent git repositories.   There are probably some perfect breaking points already:

org.eclipse.jdt.git
org.eclipse.pde.git
org.eclipse.core.git

etc

These each would then have their own build and could be built independently.  If each created a P2 repository with it's artifacts, these then could be consumed by the downstream builds that depend on them.

Migrating to git is more than just migrating to CVS, it also is an opportunity to optimize the repositories for git itself.
Comment 9 David Carver CLA 2011-05-31 14:29:32 EDT
(In reply to comment #2)
> In the past, the Eclipse SDK has included everything needed to build the SDK. 
> If the Platform is moving to git, should egit move to the platform?

EGit is already moving to Mylyn Builds, can't the platform SDK just consume the EGit p2 artifacts from there.  It doesn't have to be housed in the eclipse platform itself.
Comment 10 Kim Moir CLA 2011-05-31 14:49:51 EDT
Dave, regarding comment #8 the granularity of the Eclipse/Equinox git repos is being discussed in bug 345471.  

This (current) bug is a plan item bug so much of the discussion and work are taking place in the dependent bugs above :-)
Comment 11 Dani Megert CLA 2011-06-10 02:33:56 EDT
(In reply to comment #2 and comment #4)
If the SDK source is in a Git repo then Git should be part of the SDK and CVS has to go away. Personally I would prefer to wait one release with the removal i.e. ship both with 3.8/4.2 and then remove CVS in 3.9/4.3. This will give adopters enough time to react.
Comment 12 Kim Moir CLA 2011-06-10 16:39:04 EDT
We had a meeting regarding the migration yesterday, John wrote some notes here

http://wiki.eclipse.org/Platform-releng/Juno_Git_Migration
Comment 13 James Blackburn CLA 2011-06-10 17:33:11 EDT
For removing binaries the easiest thing to do is to remove the corresponding ,v's before doing the import of your scratch copy of the CVS repo. There are downsides to this though: any tags and branches which included those binaries won't any longer.  This may affect any builds which attempt to build an older tag from git in the future.  If you'll always build old tags from CVS then I guess this is OK.

It's also possible to remove the binary content after the fact using git filter-branch, but I suspect setting up the CVS repo as you want it to look before the import is the easiest way.

> We need to maintain a CVS repository for performing builds on older branches (3.6.2+, 3.5.2+, etc).
>  Until we have a solution for this, we will need to keep our CVS repository writeable after the Git
> migration. We can delete all content from HEAD in CVS to avoid confusion.

I don't see why you'd want to do this.  If it's planned to make releases from the older branches - and you plan to do this long term - surely it makes sense to fix up the map files and the releng plugins on the branches?  Once the git releng has been sorted for HEAD, is it much work to get it going on the branches for the previous 3 major releases?
Comment 14 Kim Moir CLA 2011-06-10 17:54:28 EDT
Thanks for the advice regarding binaries.

Regarding comment #13, James it's really more like seven previous releases.  The issue is the git fetch factory for pde build would have to be backported, build scripts modified and map files update for all those releases so we could fetch via git.  

And for those who may say why don't you switch your build system to one that supports git natively at the same time you're changing SCMs - we don't have the resources to do that right now, and none meet all our requirements (for instance, some don't allow us to compile against the seven BREEs we build with :-)  The goal is to switch to git this summer which is a considerable amount of work in itself.
Comment 15 David Carver CLA 2011-06-10 18:56:46 EDT
Just a note on maven and supporting/compiling with multiple different JDK's.  It can be done, the tycho-osgi-compiler plugin would need to be updated to support the settings that the maven-compiler-plugin already support.

http://unserializableone.blogspot.com/2008/09/compile-and-test-with-different-jdk.html

Anyways, I'd suggest opening an enhancement request with the Tycho folks if it is something that you really need to have.   I know that Tycho will down compile to a specific BREE.
Comment 16 Markus Keller CLA 2011-06-22 12:16:43 EDT
What's the plan for the secondary CVS repositories associated with projects?

E.g. the webpages are currently on /cvsroot/org.eclipse/ , and some of the pages also contain binaries like developer tools that are not officially shipped e.g. http://eclipse.org/eclipse/platform-core/downloads.php or http://eclipse.org/jdt/ui/astview/index.php .

In JDT UI, we reused the old /cvsroot/eclipse/jdt-ui-home/ to host the sources of these tools, so we would also need a new home for these.

Will the web pages stay on CVS for now?
Comment 17 John Arthorne CLA 2011-06-22 13:34:09 EDT
(In reply to comment #16)
> What's the plan for the secondary CVS repositories associated with projects?

This is covered by bug 324116. I thought the Eclipse Foundation was going to take care of this migration, but I have asked for clarification on the bug.

> In JDT UI, we reused the old /cvsroot/eclipse/jdt-ui-home/ to host the sources
> of these tools, so we would also need a new home for these.

I would think you should include those sources in the same repository as your tests and other code, possibly under a different folder if you want to keep them separate.
Comment 18 David Carver CLA 2011-06-22 18:37:45 EDT
(In reply to comment #17)
> (In reply to comment #16)
> > What's the plan for the secondary CVS repositories associated with projects?
> 
> This is covered by bug 324116. I thought the Eclipse Foundation was going to
> take care of this migration, but I have asked for clarification on the bug.
> 
> > In JDT UI, we reused the old /cvsroot/eclipse/jdt-ui-home/ to host the sources
> > of these tools, so we would also need a new home for these.
> 
> I would think you should include those sources in the same repository as your
> tests and other code, possibly under a different folder if you want to keep
> them separate.

Or if they are completely seperate, host them in their own git repository.  It all depends on how often things are needed and accessed.
Comment 19 Alex Blewitt CLA 2011-09-29 15:07:08 EDT
I have raised bug 359466 to request removal of the CVS from the Eclipse SDK and to be made available as a standalone feature
Comment 20 Kim Moir CLA 2011-12-16 17:07:37 EST
I'm going to close this bug.  There are a few remaining issues open but our entire code base has now been migrated to Git.  

The two builder projects are still in CVS but we're going to defer migrating these for now to focus on bug 355430, which is making the 4.2 stream build the primary build.
Comment 21 Denis Roy CLA 2011-12-17 12:13:00 EST
> our entire code base has now been migrated to Git.

Congratulations!  The work done here is nothing short of impressive.
Comment 22 Sergey Prigogin CLA 2012-02-21 20:16:20 EST
Has git.eclipse.org/gitroot/platform/eclipse.platform.releng.git been migrated? If so, please update http://wiki.eclipse.org/Platform-releng/Git_Workflows and http://wiki.eclipse.org/Platform_UI/Git.
Comment 23 Dani Megert CLA 2012-02-22 03:10:53 EST
(In reply to comment #22)
> Has git.eclipse.org/gitroot/platform/eclipse.platform.releng.git been migrated?
Yes: http://git.eclipse.org/c/platform/eclipse.platform.releng.git/

> If so, please update http://wiki.eclipse.org/Platform-releng/Git_Workflows and
> http://wiki.eclipse.org/Platform_UI/Git.
Done.