Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [jgit-dev] Implementing shallow clones

I've put together a really hacked-up implementation of this, which
seems to basically work.  It doesn't do things quite like you proposed
above, because I ran into some problems.  What I have here is probably
not suitable to check in, but I was wondering if you could take a look
at it and comment on how I could best implement it for real.

The biggest problem I ran into is that I don't think wrapping the
RevCommits using the createCommit() function of RevWalk is going to
work.  The reason is that you seed the RevWalk with start commits
which you get from elsewhere, so those commits haven't yet been
wrapped.  You could assume that all such commits would have depth 0,
and treat them as such if they aren't wrapped, but these commits may
have been parsed already, meaning their parents have been constructed,
and those haven't been wrapped either.  I can't see any good way to be
sure that all commits you'll ever encounter have been wrapped.  I
basically resorted for the time being to just adding a depth field to
RevCommit, and having its setter function automatically propagate the
depth down to any parents which have already been parsed.  This is
hacky and ugly, but at least it illustrates the pattern that will have
to exist in a correct implementation.

The C implementation of git seems to do this by adding a field to the
commit object called "util", which can be used to hold a
temporarily-allocated block of memory, which it uses to store the
depth info.  This would be pretty ugly in Java, but I was wondering if
we could do some kind of tagging thing, where you could decorate a
commit with one or more extra objects, of which a depth tag would be
one case.  That might be sort of overkill for a single int, but
perhaps there are other aspects of the framework which could be
modified to leverage this as well.  Thoughts?

As far as getting the pack built correctly, I went ahead with my
previous proposal to add a SHALLOW flag to RevWalk, which cuts off the
parent search in PendingGenerator, but doesn't trigger blobs/trees to
be marked as uninteresting in ObjectWalk.  This looks like it produces
a correct pack file, although I haven't done very extensive testing
yet.  The whole thing feels kind of ugly, though, and I'm not at all
sure that I've done it correctly in all of the edge cases--is there
any chance you could look over it and see whether this is right, or
whether there might be a better solution?

This whole thing is very preliminary, but I think that any further
improvements to this design will require some pretty extensive
knowledge of how the framework is put together, so I would appreciate
any thoughts you might have.  Hopefully I'll eventually be able to
beat it into shape, and get it to a point where it can actually be
checked in.

Thanks,
Matt

On Thu, Jul 22, 2010 at 9:41 PM, Matt Fischer <mattfischer84@xxxxxxxxx> wrote:
> On Mon, Jul 19, 2010 at 9:55 PM, Shawn O. Pearce <spearce@xxxxxxxxxxx> wrote:
>> Matt Fischer <mattfischer84@xxxxxxxxx> wrote:
>>>
>>> If I can get to the point of producing the set of commits at the
>>> edges, the rest is pretty easy (I can just feed the parents of those
>>> edge commits into the PackWriter as uninteresting commits to get it to
>>> exclude the proper things.)
>>
>> That probably won't work.  If you tag the parents of the edges as
>> uninteresting we'll exclude any blobs that the parent contains.
>> If the commit we are actually sending in response to the deepen
>> request also uses that blob, but the client doesn't yet have that
>> blob, we'll send an incomplete pack.
>>
>> You probably need to do something more sophisticated, like force
>> the edges to be parsed and then whack out their parent array so its
>> an empty list.  Then the traversal won't follow into the parents,
>> and thus stops at the boundary.  You also would need to mark the
>> blobs we know the client already has (from its common have lines)
>> as uninteresting.
>>
> How about adding a new flag to RevWalk called SHALLOW?  Then we could
> add things like RevWalk.markShallow(), in a similar way to how
> uninteresting works now.  The semantics of SHALLOW would be that
> implies a boundary, but unlike UNINTERESTING it does not imply that
> the remote side has the commit.  Therefore, the ObjectWalk would never
> mark the blobs and trees as uninteresting, so they would make it into
> the pack.  I don't know whether that's too special-case a thing to put
> into RevWalk, but it seems like if/when jgit gets support for the
> client side of shallow checkouts (manipulating .git/shallow and all
> that), it might need this flag anyway.
>
>>> Looking through the code, it looks to me
>>> like generating this set would take logic that looks sort of like a
>>> combination of the Topo sort and the Boundary generator, along with
>>> the ability to tag each commit with an integer for its depth as we go
>>> along.
>>
>> Probably true.  Only we really don't want to fatten out the RevCommit
>> structure by default to add the depth tag.
>>
>> But you could create a custom subclass of ObjectWalk that overrides
>> createCommit to be a subclass which does have the depth field.
>> Then use this new subclass only for shallow clone enumeration
>> support, and set the depth as the commits spool out of the next()
>> method.  Since you extended the ObjectWalk you might be able to wedge
>> something into the StartGenerator that sets up a new filter generator
>> to assign the depths as they spool out of the TopoSortGenerator.
>> It should be pretty easy, the depth is the shortest path to a commit
>> so each commit just has to set its depth + 1 into its parents.
>>
> Wrapper class for RevCommit is fine.  I'm wondering whether it would
> be ok to just add the logic to create them directly into RevWalk,
> though.  It could be set up so that if the "we need depth info" flag
> is set, it just creates those wrapper objects.  Then it can also use
> that flag to enforce that it's using Topo sort (because the depth
> tagging won't work correctly if they aren't in topo order.)  If that
> gets put in place, then I can just implement a new filter that picks
> out only commits of a specified depth, and that gives me my shallow
> list.
>
>>> If I were to add this logic, would it be correct to say that I
>>> should be making a new generator which only returns commits of a
>>> certain depth,
>>
>> I think so.  The TopoSortGenerator is incremental, so if you put
>> the numbering generator into the pipeline you can also decide
>> when to abort.  Though aborting here is a bit tricky, if you just
>> want to stop a given depth you need to remove the parents from
>> the underlying PendingGenerator's queue once you realize that the
>> parents are impossibly deep.  You can't stop until the queue is
>> empty, because you might be visting down one branch, need to cut
>> it off, then go visit another branch that is still in the queue.
>>
>>> and somehow add parameters to the RevWalk to tell it
>>> that I want to instantiate this generator?  If so, what is the proper
>>> way to make this generator get the commits in Topo order, and do the
>>> necessary tagging to compute the depth on each node?
>>
>> Maybe just do a "walker instanceof MySpecialWalker" inside of the
>> StartGenerator right after the TopoSortGenerator is created?
>>
>> Like I was saying above, doing this probably requires having both the
>> TopoSortGenerator here and having access to the PendingGenerator's
>> queue so you can fully kill a branch when its gone to the needed
>> depth.
>>
>> Actually, you can kill it by marking it UNINTERESTING.  That'll make
>> it drop out of the queue because PendingGenerator skips over commits
>> that are uninteresting.
>>
> Aborting the scan sounds kind of tricky--is it really necessary to do
> that?  Walking the entire commit history is certainly inefficient
> during a shallow checkout, but it's no more inefficient than it would
> be if you were fetching the whole history, and it seems like that cost
> is probably negligible compared to the time spent sending the data
> across the wire.  If I just implement a filter to pump through the
> whole tree and only return commits of a specified depth, would that be
> that bad?
>
> --Matt
>
From a23fe3eb3c41b281a3adf1032ad4bbfb368247fb Mon Sep 17 00:00:00 2001
From: Matt Fischer <matt.fischer@xxxxxxxxxx>
Date: Mon, 26 Jul 2010 22:39:37 -0500
Subject: [PATCH] Initial implementation of shallow clones

This is pretty hacked-in, and is not suitable for upstream yet, but
it provides a functional implementation that can be used to study
how best to implement a real solution.
---
 .../src/org/eclipse/jgit/lib/PackWriter.java       |   51 +++++++++++---
 .../org/eclipse/jgit/revwalk/PendingGenerator.java |   26 +++++--
 .../src/org/eclipse/jgit/revwalk/RevCommit.java    |   27 +++++++
 .../src/org/eclipse/jgit/revwalk/RevWalk.java      |   30 ++++++++-
 .../eclipse/jgit/revwalk/filter/DepthFilter.java   |   36 ++++++++++
 .../src/org/eclipse/jgit/transport/UploadPack.java |   72 +++++++++++++++++++-
 6 files changed, 219 insertions(+), 23 deletions(-)
 create mode 100644 org.eclipse.jgit/src/org/eclipse/jgit/revwalk/filter/DepthFilter.java

diff --git a/org.eclipse.jgit/src/org/eclipse/jgit/lib/PackWriter.java b/org.eclipse.jgit/src/org/eclipse/jgit/lib/PackWriter.java
index 48f41a5..d7c1854 100644
--- a/org.eclipse.jgit/src/org/eclipse/jgit/lib/PackWriter.java
+++ b/org.eclipse.jgit/src/org/eclipse/jgit/lib/PackWriter.java
@@ -60,6 +60,7 @@
 import org.eclipse.jgit.revwalk.ObjectWalk;
 import org.eclipse.jgit.revwalk.RevFlag;
 import org.eclipse.jgit.revwalk.RevObject;
+import org.eclipse.jgit.revwalk.RevCommit;
 import org.eclipse.jgit.revwalk.RevSort;
 import org.eclipse.jgit.transport.PackedObjectInfo;
 import org.eclipse.jgit.util.NB;
@@ -82,7 +83,7 @@
  * Typical usage consists of creating instance intended for some pack,
  * configuring options, preparing the list of objects by calling
  * {@link #preparePack(Iterator)} or
- * {@link #preparePack(Collection, Collection)}, and finally
+ * {@link #preparePack(Collection, Collection, Collection)}, and finally
  * producing the stream with {@link #writePack(OutputStream)}.
  * </p>
  * <p>
@@ -102,7 +103,7 @@
 	 * Title of {@link ProgressMonitor} task used during counting objects to
 	 * pack.
 	 *
-	 * @see #preparePack(Collection, Collection)
+	 * @see #preparePack(Collection, Collection, Collection)
 	 */
 	public static final String COUNTING_OBJECTS_PROGRESS = JGitText.get().countingObjects;
 
@@ -203,14 +204,14 @@
 	 * Create writer for specified repository.
 	 * <p>
 	 * Objects for packing are specified in {@link #preparePack(Iterator)} or
-	 * {@link #preparePack(Collection, Collection)}.
+	 * {@link #preparePack(Collection, Collection, Collection)}.
 	 *
 	 * @param repo
 	 *            repository where objects are stored.
 	 * @param monitor
 	 *            operations progress monitor, used within
 	 *            {@link #preparePack(Iterator)},
-	 *            {@link #preparePack(Collection, Collection)}
+	 *            {@link #preparePack(Collection, Collection, Collection)}
 	 *            , or {@link #writePack(OutputStream)}.
 	 */
 	public PackWriter(final Repository repo, final ProgressMonitor monitor) {
@@ -221,14 +222,14 @@ public PackWriter(final Repository repo, final ProgressMonitor monitor) {
 	 * Create writer for specified repository.
 	 * <p>
 	 * Objects for packing are specified in {@link #preparePack(Iterator)} or
-	 * {@link #preparePack(Collection, Collection)}.
+	 * {@link #preparePack(Collection, Collection, Collection)}.
 	 *
 	 * @param repo
 	 *            repository where objects are stored.
 	 * @param imonitor
 	 *            operations progress monitor, used within
 	 *            {@link #preparePack(Iterator)},
-	 *            {@link #preparePack(Collection, Collection)}
+	 *            {@link #preparePack(Collection, Collection, Collection)}
 	 * @param wmonitor
 	 *            operations progress monitor, used within
 	 *            {@link #writePack(OutputStream)}.
@@ -261,7 +262,7 @@ public boolean isReuseDeltas() {
 	 * use it if possible. Normally, only deltas with base to another object
 	 * existing in set of objects to pack will be used. Exception is however
 	 * thin-pack (see
-	 * {@link #preparePack(Collection, Collection)} and
+	 * {@link #preparePack(Collection, Collection, Collection)} and
 	 * {@link #preparePack(Iterator)}) where base object must exist on other
 	 * side machine.
 	 * <p>
@@ -389,7 +390,7 @@ public void setThin(final boolean packthin) {
 	/**
 	 * @return true to ignore objects that are uninteresting and also not found
 	 *         on local disk; false to throw a {@link MissingObjectException}
-	 *         out of {@link #preparePack(Collection, Collection)} if an
+	 *         out of {@link #preparePack(Collection, Collection, Collection)} if an
 	 *         uninteresting object is not in the source repository. By default,
 	 *         true, permitting gracefully ignoring of uninteresting objects.
 	 */
@@ -488,19 +489,36 @@ public void preparePack(final Iterator<RevObject> objectsSource)
 	 * @param uninterestingObjects
 	 *            collection of objects to be marked as uninteresting (end
 	 *            points of graph traversal).
+	 * @param shallowObjects
+	 * 			  collection of objects to be marked as shallow (stop
+	 *            traversing graph, but don't assume remote has any of
+	 *            the commits beyond this one)
 	 * @throws IOException
 	 *             when some I/O problem occur during reading objects.
 	 */
 	public void preparePack(
 			final Collection<? extends ObjectId> interestingObjects,
-			final Collection<? extends ObjectId> uninterestingObjects)
+			final Collection<? extends ObjectId> uninterestingObjects,
+			final Collection<? extends ObjectId> shallowObjects)
 			throws IOException {
 		ObjectWalk walker = setUpWalker(interestingObjects,
-				uninterestingObjects);
+				uninterestingObjects, shallowObjects);
 		findObjectsToPack(walker);
 	}
 
 	/**
+	 * @param interestingObjects
+	 * @param uninterestingObjects
+	 * @throws IOException
+	 */
+	public void preparePack(
+			final Collection<? extends ObjectId> interestingObjects,
+			final Collection<? extends ObjectId> uninterestingObjects)
+			throws IOException {
+		preparePack(interestingObjects, uninterestingObjects, null);
+	}
+
+	/**
 	 * Determine if the pack file will contain the requested object.
 	 *
 	 * @param id
@@ -530,7 +548,7 @@ public ObjectId computeName() {
 	 * Create an index file to match the pack file just written.
 	 * <p>
 	 * This method can only be invoked after {@link #preparePack(Iterator)} or
-	 * {@link #preparePack(Collection, Collection)} has been
+	 * {@link #preparePack(Collection, Collection, Collection)} has been
 	 * invoked and completed successfully. Writing a corresponding index is an
 	 * optional feature that not all pack users may require.
 	 *
@@ -814,7 +832,8 @@ private void writeChecksum() throws IOException {
 
 	private ObjectWalk setUpWalker(
 			final Collection<? extends ObjectId> interestingObjects,
-			final Collection<? extends ObjectId> uninterestingObjects)
+			final Collection<? extends ObjectId> uninterestingObjects,
+			final Collection<? extends ObjectId> shallowObjects)
 			throws MissingObjectException, IOException,
 			IncorrectObjectTypeException {
 		final ObjectWalk walker = new ObjectWalk(db);
@@ -841,6 +860,14 @@ private ObjectWalk setUpWalker(
 				walker.markUninteresting(o);
 			}
 		}
+		if (shallowObjects != null) {
+			for (ObjectId id : shallowObjects) {
+				final RevCommit c;
+				c = walker.parseCommit(id);
+				walker.markShallow(c);
+			}
+		}
+
 		return walker;
 	}
 
diff --git a/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/PendingGenerator.java b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/PendingGenerator.java
index e723bce..4f0bb2b 100644
--- a/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/PendingGenerator.java
+++ b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/PendingGenerator.java
@@ -68,6 +68,8 @@
 
 	private static final int UNINTERESTING = RevWalk.UNINTERESTING;
 
+	private static final int SHALLOW = RevWalk.SHALLOW;
+
 	/**
 	 * Number of additional commits to scan after we think we are done.
 	 * <p>
@@ -138,15 +140,23 @@ RevCommit next() throws MissingObjectException,
 				else
 					produce = filter.include(walker, c);
 
-				for (final RevCommit p : c.parents) {
-					if ((p.flags & SEEN) != 0)
-						continue;
-					if ((p.flags & PARSED) == 0)
-						p.parseHeaders(walker);
-					p.flags |= SEEN;
-					pending.add(p);
+				// If commit is shallow, do not traverse parents
+				if ((c.flags & SHALLOW) == 0) {
+					for (final RevCommit p : c.parents) {
+						if ((p.flags & SEEN) != 0)
+							continue;
+						if ((p.flags & PARSED) == 0)
+							p.parseHeaders(walker);
+						p.flags |= SEEN;
+
+						if (p.getDepth() == -1 || p.getDepth() > c.getDepth() + 1) {
+							p.setDepth(c.getDepth() + 1);
+						}
+
+						pending.add(p);
+					}
+					walker.carryFlagsImpl(c);
 				}
-				walker.carryFlagsImpl(c);
 
 				if ((c.flags & UNINTERESTING) != 0) {
 					if (pending.everbodyHasFlag(UNINTERESTING)) {
diff --git a/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/RevCommit.java b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/RevCommit.java
index 2d96bbf..5cade19 100644
--- a/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/RevCommit.java
+++ b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/RevCommit.java
@@ -73,6 +73,8 @@
 
 	private byte[] buffer;
 
+	int depth; // Depth to closest head
+
 	/**
 	 * Create a new commit reference.
 	 *
@@ -81,6 +83,7 @@
 	 */
 	protected RevCommit(final AnyObjectId id) {
 		super(id);
+		depth = -1;
 	}
 
 	@Override
@@ -526,4 +529,28 @@ public String toString() {
 		appendCoreFlags(s);
 		return s.toString();
 	}
+
+	/**
+	 * Set the depth of this commit
+	 * @param depth
+	 */
+	public void setDepth(int depth) {
+		this.depth = depth;
+		if((flags & PARSED) != 0) {
+			// Propagate depth info to parents
+			for(RevCommit p: parents) {
+				if(p.getDepth() > depth + 1) {
+					p.setDepth(depth + 1);
+				}
+			}
+		}
+	}
+
+	/**
+	 * Get the depth of this commit
+	 * @return depth
+	 */
+	public int getDepth() {
+		return depth;
+	}
 }
diff --git a/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/RevWalk.java b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/RevWalk.java
index a8c67c6..747e1cd 100644
--- a/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/RevWalk.java
+++ b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/RevWalk.java
@@ -152,8 +152,10 @@
 	 */
 	static final int TOPO_DELAY = 1 << 5;
 
+	static final int SHALLOW = 1 << 6;
+
 	/** Number of flag bits we keep internal for our own use. See above flags. */
-	static final int RESERVED_FLAGS = 6;
+	static final int RESERVED_FLAGS = 7;
 
 	private static final int APP_FLAGS = -1 & ~((1 << RESERVED_FLAGS) - 1);
 
@@ -250,6 +252,9 @@ public void markStart(final RevCommit c) throws MissingObjectException,
 		if ((c.flags & PARSED) == 0)
 			c.parseHeaders(this);
 		c.flags |= SEEN;
+
+		c.setDepth(0);
+
 		roots.add(c);
 		queue.add(c);
 	}
@@ -322,6 +327,29 @@ public void markUninteresting(final RevCommit c)
 	}
 
 	/**
+	 * Mark a commit as shallow
+	 *
+	 * Shallow commits operate like uninteresting commits, but unlike
+	 * uninteresting commits, they do not imply that the remote already
+	 * has them.  This means that they and their ancestors do not mark
+	 * their trees/blobs as uninteresting, so they will still be included
+	 * in packs sent to a remote.  This allows a client to request only
+	 * a subset of the commits in a repository, but still receive a complete
+	 * set of data.
+	 *
+	 * @param c
+	 * @throws MissingObjectException
+	 * @throws IncorrectObjectTypeException
+	 * @throws IOException
+	 */
+	public void markShallow(final RevCommit c)
+			throws MissingObjectException, IncorrectObjectTypeException,
+			IOException {
+		c.flags |= SHALLOW;
+		markStart(c);
+	}
+
+	/**
 	 * Determine if a commit is reachable from another commit.
 	 * <p>
 	 * A commit <code>base</code> is an ancestor of <code>tip</code> if we
diff --git a/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/filter/DepthFilter.java b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/filter/DepthFilter.java
new file mode 100644
index 0000000..81c4722
--- /dev/null
+++ b/org.eclipse.jgit/src/org/eclipse/jgit/revwalk/filter/DepthFilter.java
@@ -0,0 +1,36 @@
+package org.eclipse.jgit.revwalk.filter;
+
+import java.io.IOException;
+
+import org.eclipse.jgit.errors.IncorrectObjectTypeException;
+import org.eclipse.jgit.errors.MissingObjectException;
+import org.eclipse.jgit.errors.StopWalkException;
+import org.eclipse.jgit.revwalk.RevCommit;
+import org.eclipse.jgit.revwalk.RevWalk;
+
+/**
+ * Includes a commit only if it has a specified depth.
+ */
+public class DepthFilter extends RevFilter {
+	private int depth;
+
+	/**
+	 * @param depth
+	 */
+	public DepthFilter(int depth) {
+		this.depth = depth;
+	}
+
+	@Override
+	public boolean include(RevWalk walker, RevCommit cmit)
+			throws StopWalkException, MissingObjectException,
+			IncorrectObjectTypeException, IOException {
+		return cmit.getDepth() == depth;
+	}
+
+	@Override
+	public RevFilter clone() {
+		return this;
+	}
+
+}
diff --git a/org.eclipse.jgit/src/org/eclipse/jgit/transport/UploadPack.java b/org.eclipse.jgit/src/org/eclipse/jgit/transport/UploadPack.java
index 77cc1a6..d72d529 100644
--- a/org.eclipse.jgit/src/org/eclipse/jgit/transport/UploadPack.java
+++ b/org.eclipse.jgit/src/org/eclipse/jgit/transport/UploadPack.java
@@ -69,6 +69,7 @@
 import org.eclipse.jgit.revwalk.RevObject;
 import org.eclipse.jgit.revwalk.RevTag;
 import org.eclipse.jgit.revwalk.RevWalk;
+import org.eclipse.jgit.revwalk.filter.DepthFilter;
 import org.eclipse.jgit.transport.BasePackFetchConnection.MultiAck;
 import org.eclipse.jgit.transport.RefAdvertiser.PacketLineOutRefAdvertiser;
 import org.eclipse.jgit.util.io.InterruptTimer;
@@ -146,6 +147,12 @@
 	/** Objects on both sides, these don't have to be sent. */
 	private final List<RevObject> commonBase = new ArrayList<RevObject>();
 
+	/** Shallow commits that we will be sending. */
+	private final List<RevCommit> localShallowCommits = new ArrayList<RevCommit>();
+
+	/** Shallow commits the remote already has. */
+	private final List<RevCommit> remoteShallowCommits = new ArrayList<RevCommit>();
+
 	/** null if {@link #commonBase} should be examined again. */
 	private Boolean okToGiveUp;
 
@@ -165,6 +172,8 @@
 
 	private MultiAck multiAck = MultiAck.OFF;
 
+	private int depth = 0;
+
 	/**
 	 * Create a new pack upload for an open repository.
 	 *
@@ -330,8 +339,56 @@ else if (options.contains(OPTION_MULTI_ACK))
 		else
 			multiAck = MultiAck.OFF;
 
-		if (negotiate())
+		if (depth != 0) {
+			processShallow();
+		}
+
+		if (negotiate()) {
 			sendPack();
+		}
+	}
+
+	private void processShallow() throws IOException {
+		RevWalk walk = new RevWalk(db);
+
+		// Find all the commits which will be shallow
+		walk.setRevFilter(new DepthFilter(depth - 1));
+		walk.markStart(wantCommits);
+
+		RevObject o;
+
+		while((o = walk.next()) != null) {
+			localShallowCommits.add((RevCommit)o);
+
+			boolean found = false;
+			for(RevCommit c : remoteShallowCommits) {
+				if(c.name() == o.name()) {
+					found = true;
+					break;
+				}
+			}
+
+			if (!found) {
+				pckOut.writeString("shallow " + o.name());
+			}
+		}
+
+		// Unshallow any commits which we're expanding on
+		for (RevCommit c : remoteShallowCommits) {
+			boolean found = false;
+			for(RevCommit c2 : localShallowCommits) {
+				if(c.name() == c2.name()) {
+					found = true;
+					break;
+				}
+			}
+
+			if (!found) {
+				pckOut.writeString("unshallow " + c.name());
+			}
+		}
+
+		pckOut.end();
 	}
 
 	/**
@@ -354,6 +411,7 @@ public void sendAdvertisedRefs(final RefAdvertiser adv) throws IOException {
 		adv.advertiseCapability(OPTION_NO_PROGRESS);
 		adv.setDerefTags(true);
 		refs = refFilter.filter(db.getAllRefs());
+
 		adv.send(refs);
 		adv.end();
 	}
@@ -372,6 +430,16 @@ private void recvWants() throws IOException {
 
 			if (line == PacketLineIn.END)
 				break;
+
+			if (line.startsWith("deepen ")) {
+				depth = Integer.parseInt(line.substring(7));
+				continue;
+			}
+			if (line.startsWith("shallow ")) {
+				final ObjectId id = ObjectId.fromString(line.substring(8));
+				remoteShallowCommits.add(walk.parseCommit(id));
+				continue;
+			}
 			if (!line.startsWith("want ") || line.length() < 45)
 				throw new PackProtocolException(MessageFormat.format(JGitText.get().expectedGot, "want", line));
 
@@ -568,7 +636,7 @@ private void sendPack() throws IOException {
 		pw = new PackWriter(db, pm, NullProgressMonitor.INSTANCE);
 		pw.setDeltaBaseAsOffset(options.contains(OPTION_OFS_DELTA));
 		pw.setThin(thin);
-		pw.preparePack(wantAll, commonBase);
+		pw.preparePack(wantAll, commonBase, localShallowCommits);
 		if (options.contains(OPTION_INCLUDE_TAG)) {
 			for (final Ref r : refs.values()) {
 				final RevObject o;
-- 
1.6.0.6


Back to the top