Re: [jgit-dev] Remote to remote transfer -- without a file system

[Shawn Pearce <spearce@xxxxxxxxxxx>]

On Mon, Aug 2, 2010 at 3:35 AM, Oliver Searle-Barnes <oliver@xxxxxxxxxx> wrote:
> I'm trying to find a way to push changes in a github repository to a heroku
> repository. For the purposes of this discussion they are remote git repos
> over which I have no control regarding setup etc.
>
> Normally I would create a git repository and then add the github and heroku
> remotes. Then I could fetch from github and push to heroku. simple. Here's
> the tricky bit; I want to do this on a 3rd host that doesn't have access to
> a file system (god bless the cloud).

Yea, good luck with that.  Its not something JGit is built to do.  I
don't want to discourage you from trying, but I can tell you right now
that the JGit source code needs a pile of changes to make this happen.

> My current thinking is that I may be able to use jgit to pull from one
> remote and pipe this into a push without ever having creating a repository.
> I realise that I'm going to need to go fairly low level to do this. At this
> point I'm looking to see if
>
> A - if there's any technical reason that makes it impossible?
> B - pointers to get started with the jgit source

The Git network protocol doesn't offer enough information to perform
this transfer efficiently.  In theory you could connect to both sides,
compare the advertised references, then use the destination's
references as "have" lines to the source side, and pipe the pack the
source gives you directly into the destination.  If the destination is
only ever an older copy of the source this would actually work out
well, the destination's current branches would be the latest common
ancestor between the two repositories and the pack created would be
the most efficient pack available.  If however the destination was
more than just a mirror the source may send you more data than you
really needed.  This isn't a problem, it just means the transfer will
take longer than it might otherwise would if you had more information.

To wire the two together you are probably going to need to implement
some of the protocol yourself.  We're basically talking about the
BasePackFetchConnection and BasePackPushConnection classes in the
transport package.  These are the fetch and push clients that you need
to redo some of the logic of.  In particular the
BasePackFetchConnection's negotiate() logic is what sends the "have"
lines, you would need to replace that logic with sending the
destination's advertised refs as "have" lines rather than walking
through history.  You may find it easier to just implement the entire
fetch client from scratch and reuse the SideBandInputStream and
SideBandOutputStream classes to handle most of the protocol
formatting.  Likewise the push code doesn't want to use the writePack
method but instead just wants to pipe the pack from the source through
as-is.

All of the above only applies to the git:// and the smart http://
protocol.  For the dumb http:// protocol, forget it, you can't
implement this without driving yourself insane.

> I haven't been able to find the source code on the eclipse jgit site so I'm
> currently using the version that's kept at http://github.com/spearce/jgit.
> This seems fairly up to date, is this the correct repo?

http://egit.eclipse.org/w/ has our repository listings.  In particular
our master repository is git://egit.eclipse.org/jgit.git.  :-)