Bug 309582 - Implement synchronization feature in EGit
Summary: Implement synchronization feature in EGit
Status: RESOLVED FIXED
Alias: None
Product: EGit
Classification: Technology
Component: UI (show other bugs)
Version: unspecified   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: 0.9.0   Edit
Assignee: Chris Aniszczyk CLA
QA Contact:
URL:
Whiteboard:
Keywords: noteworthy
Depends on:
Blocks:
 
Reported: 2010-04-17 22:57 EDT by Remy Suen CLA
Modified: 2010-06-17 20:15 EDT (History)
17 users (show)

See Also:


Attachments
remote's selection dialog (16.47 KB, image/jpeg)
2010-05-18 17:47 EDT, Dariusz Luksza CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Remy Suen CLA 2010-04-17 22:57:34 EDT
There are currently no Team synchronization participants in EGit. The 'Synchronize' view provides users with an excellent overview of changes in local and remote files and would be a great step forward over the existing tedious workflow of individually clicking files and going 'Compare With > Git Index' ad infinitum (unless I'm missing some other way to do this from within Eclipse).
Comment 1 Remy Suen CLA 2010-04-17 23:49:03 EDT
I've attached a change set to Gerrit.
http://egit.eclipse.org/r/577

What works (or so I think):
You can synchronize via the 'Synchronize' view itself. I haven't yet bothered adding it to the right-click action.
You pick a branch to synchronize against (3-way branch/index/workspace resource), or use HEAD to just go against the index (2-way index/workspace resource).
Right-click and commit.

What's no good:
I wouldn't try to synchronize two different repositories at the same time (even though the UI lets you).
The commit code is ugly. The original CommitAction desperately needs to be refactored.
After you commit, the files don't go away. I haven't figured this one out yet.
I'm not sure those in/out arrows are making a lot of sense. Perhaps they don't because a) decorators don't work and b) pushing/merging isn't there yet.
The 'bin' folders show up, which is a nightmare. How do I check if something's in .gitignore?

So really, at the moment, it can barely be called usable. It's mainly useful for looking at diffs. I was getting really tired of doing 'Compare With > Git Index' ad infinitum.
Comment 2 Remy Suen CLA 2010-04-18 22:26:57 EDT
(In reply to comment #1)
> After you commit, the files don't go away. I haven't figured this one out yet.

We need to call Subscriber's fireTeamResourceChange(ISubscriberChangeEvent[]) to notify the system that those resources have changed. However, we have no way of knowing what resources were actually committed since the user can theoretically select anything in the dialog. Fire a change event on the synchronized projects doesn't seem to work.
Comment 3 Remy Suen CLA 2010-04-20 18:44:21 EDT
(In reply to comment #2)
> However, we have no way
> of knowing what resources were actually committed since the user can
> theoretically select anything in the dialog.

Rethinking about this. Refactoring the commit dialog is not the one-way ticket to resolve this problem because the user is not required to commit from the 'Synchronize' view. Either the commit operation needs to tell subscriber participants about the change or the participants need to listen for resource changes directly from the workspace.
Comment 4 Dariusz Luksza CLA 2010-04-21 02:25:05 EDT
Good job!

Doing 'Compare With > Git Index' for each particular file was also for me very annoying. Because that I've proposed this as a part of my GSoC project. Right now the situation is slightly different, thanks to you, there is basic support for "Synchronization view", but there are also thing that could be improved! Maybe we can work together on that improvements ?

Right now, from a quick overview, I've noticed that there is no support for actions like:

    * Update
    * Overwrite and Update
    * Mark as merged

in right-click menu of "Synchronization view". That actions are very usable in day to day development, specially in Git where you must merge changes locally on your own.

I've noticed that you mentioned in bugzilla bug report that you haven't bothered with right-click menu option to call synchronize action. IMHO this is huge disadvantage because it takes me some time to find out how this awesome new feature can used. If I didn't have source code (especially plugin.xml file) I'll never figure out where it is localized. So this is quite huge disadvantage right now.

One more thing, commit dialog. When I choose "Commit" in "Synchronization view" on a single file, commit action want to commit all changed files. But this rather know bug (304131).
Comment 5 Remy Suen CLA 2010-04-21 06:47:31 EDT
(In reply to comment #3)
> or the participants need to listen for resource
> changes directly from the workspace.

Wrong again. Committing files is not likely to make any changes at the resource layer since you're just writing to the index. We would have to listen for index changes or all Git operations through EGit needs to be done in such a way that ISubscriberChangeEvents are fired off. RepositoryMapping fires events but they are not fine grained enough as we do not actually know what was changed.

(In reply to comment #4)
> Maybe we can work together on that improvements ?

Dariusz, I was actually hoping that you'd take over the code or can it and just use it as a reference during your GSOC endeavour. >:)

My main drive for hacking a quick and dirty implementation of this was really to get feedback from people that know and use Git on a regular basis so that the EGit team can start thinking about what's needed to implement this feature. To be quite honest, I only really started using Git this weekend to push a few random patches to Gerrit. I tried a few months ago but didn't quite understand the process...and I'm not even sure if I do now. ;)

Anyway, there are Eclipse users out there that say they want the 'Synchronize' view to work. But is that what they really want? What does that even mean? Maybe they just want a separate view that shows staged/unstaged changes so they can pick what to commit? And then the 'Synchronize' view is just for push/pull/merge?

I don't know what other people want. I know what I want the 'Synchronize' view to do but again, I do not use Git on a regular basis and am not in a good position to drive this feature forward.

> Right now, from a quick overview, I've noticed that there is no support for
> actions like:
> 
>     * Update
>     * Overwrite and Update
>     * Mark as merged
> 
> in right-click menu of "Synchronization view". That actions are very usable in
> day to day development, specially in Git where you must merge changes locally
> on your own.

I hadn't yet started looking into how to merge in JGit so these aren't available. I am, of course, not saying they're not useful.

> I've noticed that you mentioned in bugzilla bug report that you haven't
> bothered with right-click menu option to call synchronize action. IMHO this is
> huge disadvantage because it takes me some time to find out how this awesome
> new feature can used.

I see I incorrectly assumed that people would know where this was. I see I was in the wrong. For those that are searching, from the 'Synchronize' view, you can use the first tool item in the view's tool bar. Or you can use Ctrl+3 and look search for 'Synchronize...'.
Comment 6 Stefan Lay CLA 2010-04-21 08:32:24 EDT
I'm very happy to see that work starting and I think it's a great start!

(In reply to comment #1)
> 'Synchronize' view provides users with an excellent overview of changes in
> local and remote files and would be a great step forward over the existing
> tedious workflow of individually clicking files and going 'Compare With > Git
> Index' ad infinitum (unless I'm missing some other way to do this from within
> Eclipse).

The place to do this currently is the commit dialog. Here you see the changed files and you can see the diff by double clicking.

(In reply to comment #5)
> Wrong again. Committing files is not likely to make any changes at the resource
> layer since you're just writing to the index. We would have to listen for index
There is a some unclarity in terminology. In Git commiting does not mean writing to the index. The index in fact contains the content which is staged for commit. There are two "layer" of change, the changes already staged and the changes which are not staged. This layering will surely complicate the synchronize view.


> Anyway, there are Eclipse users out there that say they want the 'Synchronize'
> view to work. But is that what they really want? What does that even mean?
> Maybe they just want a separate view that shows staged/unstaged changes so they
> can pick what to commit? And then the 'Synchronize' view is just for
> push/pull/merge?
> I don't know what other people want. I know what I want the 'Synchronize' view
> to do but again, I do not use Git on a regular basis and am not in a good
> position to drive this feature forward.

There may be different flavors of the synchronize view. 
- Compare against the local HEAD. Here no incoming changes are possible, but you can see what you would commit; and you can commit from here.
- Compare against another local branch or tag or commit. Here you see what would happen if you did a merge. A commit makes no sense. Maybe one could trigger a merge from here?
- Compare against a remote branch. Here you would see what happens if you push or pull. Again a commit does not make sense. Maybe one could trigger a push or pull from here?

In any case it should be clearly visible if you compare against HEAD, a local branch, tag or commit or against a remote branch.
Comment 7 Remy Suen CLA 2010-04-21 08:48:16 EDT
(In reply to comment #6)
> The place to do this currently is the commit dialog. Here you see the changed
> files and you can see the diff by double clicking.

Ah yes, I forgot about this for some reasons.

> There is a some unclarity in terminology. In Git commiting does not mean
> writing to the index. The index in fact contains the content which is staged
> for commit.

Yes, you're right, Stefan. It seems clear now that any operation should funnel through a central channel so that participants can listen for changes.

> - Compare against the local HEAD. Here no incoming changes are possible, but
> you can see what you would commit; and you can commit from here.

For this I am curious if people would prefer a separate view instead. So say we have a tree structure like...

JGit
-Staged
--GitIndex.java
-Modified
-Untracked
EGit
-Staged
--CommitAction.java
-Modified
--PushOperation.java
-Untracked
--MergeAction.java

Then one could just use the mouse to do DND to move the files around. Of course, you can still use the context menu if you do not wish to, or cannot, use the mouse. We could either have one view that shows all known repositories (as shown above) or provide allow the user with the ability to switch pages (like you can right now with the 'Console' or 'History' view).

When you're good to go, just commit. What do you think, Stefan?
Comment 8 Robert Munteanu CLA 2010-04-21 09:10:47 EDT
Might be worth taking a look at how HgEclipse implements synchronization for Mercurial : http://javaforge.com/wiki/76412#section-Synchronization+View .
Comment 9 Ismael Juma CLA 2010-04-21 09:36:51 EDT
Great to see that some work is going into this. I agree with Remy that it's not totally clear if the synchronize view is the best way to organise a commit.

The view as suggested by Remy is closer to what git-cola does and I find that approach very nice. It also makes it very easy to stage partial changes for a given file and the like.
Comment 10 Dariusz Luksza CLA 2010-04-21 15:14:58 EDT
(In reply to comment #5)
> Dariusz, I was actually hoping that you'd take over the code or can it and just
> use it as a reference during your GSOC endeavour. >:)

If I'll be accepted as a GSoC student I'll be pleased to take over yours work, If I'll be rejected I'll try to contribute it on regular basis.

> I don't know what other people want. I know what I want the 'Synchronize' view
> to do but again, I do not use Git on a regular basis and am not in a good
> position to drive this feature forward.

No one knows what somebody else want to ;), because that we should discuss it. But it is better to discuss some existing features instead of making assumptions "how it would be if ...". Now where we have base, we can discuss, make changes, check does that fits us, discuss one again and so on ;)

> > Right now, from a quick overview, I've noticed that there is no support for
> > actions like:
> > 
> >     * Update
> >     * Overwrite and Update
> >     * Mark as merged
> > 
> > in right-click menu of "Synchronization view". That actions are very usable in
> > day to day development, specially in Git where you must merge changes locally
> > on your own.
> 
> I hadn't yet started looking into how to merge in JGit so these aren't
> available. I am, of course, not saying they're not useful.


AFAIK JGit right now don't support branch merges, so implementing those features is impossible right now. But this is worth mention so that we could remember about it ;)

> I see I incorrectly assumed that people would know where this was. I see I was
> in the wrong. For those that are searching, from the 'Synchronize' view, you
> can use the first tool item in the view's tool bar. Or you can use Ctrl+3 and
> look search for 'Synchronize...'.

Right now I see that those '...' at the end are very important, and leads directly to wizard in with we can select branch for synchronization

(In reply to comment #6)
> There may be different flavors of the synchronize view. 
> - Compare against the local HEAD. Here no incoming changes are possible, but
> you can see what you would commit; and you can commit from here.
> - Compare against another local branch or tag or commit. Here you see what
> would happen if you did a merge. A commit makes no sense. Maybe one could
> trigger a merge from here?
> - Compare against a remote branch. Here you would see what happens if you push
> or pull. Again a commit does not make sense. Maybe one could trigger a push or
> pull from here?

In my GSoC proposal I've mention only two from above tree scenarios:
- comparing with local HEAD where user could prepare commit (add and removed staged files same as it can be done in git gui)
- comparing with remote/local branch where user can merge files and prepare for push
Comment 11 Stefan Lay CLA 2010-04-22 11:13:32 EDT
(In reply to comment #7)
> For this I am curious if people would prefer a separate view instead. 
...
> When you're good to go, just commit. What do you think, Stefan?
see one paragraph below...

(In reply to comment #9)
> Great to see that some work is going into this. I agree with Remy that it's not
> totally clear if the synchronize view is the best way to organise a commit.
> The view as suggested by Remy is closer to what git-cola does and I find that
> approach very nice. It also makes it very easy to stage partial changes for a
> given file and the like.

Maybe we could have such a view and additionally allow for committing in the
Synchronize view. However, I think that the synchronize view is better suited
for a comparison with a remote repository where you can have incoming and
outgoing changes. In a distributed VCS then you just do a pull or a push
instead of a sync or a commit. And with git you have the complications
introduced by the index.

I created a bug (310128) to discuss a proposal to change the commit dialog such
that the commit can be organized there. But to have a separate view for this
would also be very useful. In a view however there always is the refresh
problem.

(In reply to comment #8)
> Might be worth taking a look at how HgEclipse implements synchronization for
> Mercurial : http://javaforge.com/wiki/76412#section-Synchronization+View .

As far as I understand the synchronization view for Mercurial it is possible to
commit from there. However, if I synchronize not against HEAD but against
something else, what does that mean?

(In reply to comment #10)
> AFAIK JGit right now don't support branch merges, so implementing those
> features is impossible right now. But this is worth mention so that we could
> remember about it ;)

I am confident that this will be available soon so that you can add merge to
the synchronize view in you GSoC project.

> In my GSoC proposal I've mention only two from above tree scenarios:
> - comparing with local HEAD where user could prepare commit (add and removed
> staged files same as it can be done in git gui)
> - comparing with remote/local branch where user can merge files and prepare 
> for push
Basically that are the three scenarios. How do you plan to handle the staged
and unstaged files in the synchronize view? What do you think about keeping
that simple in the view itself and instead allowing for more advanced actions 
in the commit dialog, similar to git-gui (see bug 310128)?
Comment 12 Dariusz Luksza CLA 2010-04-22 15:56:11 EDT
(In reply to comment #11)
> I am confident that this will be available soon so that you can add merge to
> the synchronize view in you GSoC project.

It is great to hear that, but Student Application Period is closed right now so I can't edit my proposal text so that I could add it :(. List of accepted students will be announced April 26 19:00 UTC at https://socghop.appspot.com.

Of course if I'll get slot we can change some things that was declared in application text to get better results!

> Basically that are the three scenarios. How do you plan to handle the staged
> and unstaged files in the synchronize view? What do you think about keeping
> that simple in the view itself and instead allowing for more advanced actions 
> in the commit dialog, similar to git-gui (see bug 310128)?

To be more specific, I was thinking about some extensions of Team Framework to achieve that, because right now it is impossible to achieve it. I'm afraid that dialogs lose theirs readability after we try to pack in to it do many features/controls/data. Dialogs are quite usable when they have to input fields and two or tree buttons. Embedding compare view inside dialog isn't useful IMHO because if we want to use it we must to maximize dialog if we don't do that we must "fight" with scrollbars.

I would prefer have a view where I could stage and unstage files/changes and launch "Compare" by double click on particular file.
Comment 13 Remy Suen CLA 2010-04-22 17:19:03 EDT
(In reply to comment #11)
> However, I think that the synchronize view is better suited
> for a comparison with a remote repository where you can have incoming and
> outgoing changes.

Indeed. It seems to me the 'Synchronize' view should only be for comparing branches in EGit.

> In a view however there always is the refresh problem.

I don't necessarily see this as a major problem. Decorators aren't in "real-time" either. The view only needs to be updated "on the side".

> To be more specific, I was thinking about some extensions of Team Framework to
> achieve that, because right now it is impossible to achieve it.

While making extensions to Team is a noble cause, I am not sure how far we're going to get if we go down that path. This is very likely going to require API additions. And given our backwards compatibility requirements, we won't be able to use 3.7 API for a long while. ;)

> Embedding compare view inside dialog isn't useful IMHO
> because if we want to use it we must to maximize dialog if we don't do that we
> must "fight" with scrollbars.

I think the compare functionality should be off by default (as it is in the current commit dialog). If someone wants it they can turn it on (like how it is in the CVS commit dialog). Alternatively, you could have it so that opening a file in the commit dialog will popup a new compare dialog instead of trying to make room for it in the commit dialog.

> I would prefer have a view where I could stage and unstage files/changes and
> launch "Compare" by double click on particular file.

This would be how I envision this view to work as well.
Comment 14 Dariusz Luksza CLA 2010-04-22 17:35:28 EDT
(In reply to comment #13)
> While making extensions to Team is a noble cause, I am not sure how far we're
> going to get if we go down that path. This is very likely going to require API
> additions. And given our backwards compatibility requirements, we won't be able
> to use 3.7 API for a long while. ;)

In this case we can do it outside of current Team Framework ... maybe some day it will be imported inside Team Framework ;)

> I think the compare functionality should be off by default (as it is in the
> current commit dialog). If someone wants it they can turn it on (like how it is
> in the CVS commit dialog). Alternatively, you could have it so that opening a
> file in the commit dialog will popup a new compare dialog instead of trying to
> make room for it in the commit dialog.

Yes, current CVS commit dialog is quite nice example. To be hones I've personally use it with embedded compare editor not because it is the best solution for this purpose of problem, but because it is quite useful. But maybe we can come up with better solution for this.

In my humble opinion one "pop up dialog" like commit is enough. Launching second "pop up dialog" above commit one (like compare view) could be quite ugly.

Maybe we can add "commit" in button of this "git stage view", and then show user dialog where he can insert message, amend last commit, change author and commiter fields etc. ?
Comment 15 Remy Suen CLA 2010-04-22 17:43:18 EDT
(In reply to comment #14)
> Maybe we can add "commit" in button of this "git stage view", and then show
> user dialog where he can insert message, amend last commit, change author and
> commiter fields etc. ?

I think the expectation is that the view will let users commit via right-click or whatever. Anyway, the discussion about how to commit something or what the dialog should look like can go into Stefan's new bug 310128.

Let's keep this bug open for discussions about the 'Synchronize' view.
Comment 16 Stefan Lay CLA 2010-04-23 04:49:57 EDT
(In reply to comment #12)
> I'm afraid that
> dialogs lose theirs readability after we try to pack in to it do many
> features/controls/data. Dialogs are quite usable when they have to input fields
> and two or tree buttons. Embedding compare view inside dialog isn't useful IMHO
> because if we want to use it we must to maximize dialog if we don't do that we
> must "fight" with scrollbars.
> I would prefer have a view where I could stage and unstage files/changes and
> launch "Compare" by double click on particular file.

Yes, I agree with you. Maybe it would be better to keep the commit dialog simple and offer staging and unstaging in a view. However, I would at the moment prefer not to use the synchronize view for that. I can not imagine how we could integrate it there. But it would still be useful to see the outgoing changes compared to HEAD and to be able to commit from the synchronize view. Outgoing changes may the changes in working tree or changes in the index.

(In reply to comment #10)
> In my GSoC proposal I've mention only two from above tree scenarios:
> - comparing with local HEAD where user could prepare commit (add and removed
> staged files same as it can be done in git gui)
> - comparing with remote/local branch where user can merge files and prepare 
> for push
I rethought about this and I think that you are right: There are in fact only two scenarios. Remote and local branch in fact are similar beacuse you would have to fetch remote branches in the background in order to be able to calculate the differences, right?. However, there is one difference: for local branches you would trigger a merge and for remote branches you would trigger a push.

(In reply to comment #13)
> I think the compare functionality should be off by default (as it is in the
> current commit dialog). 
full ack


(In reply to comment #14)
> In my humble opinion one "pop up dialog" like commit is enough. Launching
> second "pop up dialog" above commit one (like compare view) could be quite
> ugly.
I agree. But if we embed the compare UI in the commit dialog we should make it easy for the user to enlarge the compare are with a single click.
Comment 17 Remy Suen CLA 2010-05-17 20:06:05 EDT
(In reply to comment #7)
> For this I am curious if people would prefer a separate view instead. So say we
> have a tree structure like...
> 
> JGit
> -Staged
> --GitIndex.java
> -Modified
> -Untracked
> EGit
> -Staged
> --CommitAction.java
> -Modified
> --PushOperation.java
> -Untracked
> --MergeAction.java

A suggested implementation has been pushed to Gerrit for review.
http://egit.eclipse.org/r/716

Please see bug 313263.
Comment 18 Dariusz Luksza CLA 2010-05-18 17:47:04 EDT
Created attachment 169029 [details]
remote's selection dialog

I was wondering how I want to use synchronization view with Git in Eclipse and I come up with idea of having two separate synchronization actions:

* Team -> Synchronize with...
* Team -> Synchronize with default

After choosing first option attached dialog will appears. Then user will be able to select (from predefined list of remote's) with branch (or tag) he want to synchronize and with repository (and branch or tag) should be used to synchronize against, then he can set this configuration as default. Finally after clicking on 'OK' synchronize action will fire and compare both selected branches/tags.

In second case ('Synchronize with default'), mentioned above dialog will not appears but EGit will synchronize against repository that is selected as a default one.

What do you think about this approach?

BTW. Where I should save information about last selected default repository?
Comment 19 Chris Aniszczyk CLA 2010-05-19 18:41:53 EDT
(In reply to comment #18)
> Created an attachment (id=169029) [details]
> remote's selection dialog
> 
> I was wondering how I want to use synchronization view with Git in Eclipse and
> I come up with idea of having two separate synchronization actions:
> 
> * Team -> Synchronize with...
> * Team -> Synchronize with default

Can we just have a Synchronize... for now?
Comment 20 Matthias Sohn CLA 2010-05-20 02:51:26 EDT
+1 to start with just Synchronize...

Maybe the approach Mathias found for simple fetch and push 
http://egit.eclipse.org/r/#change,715
could also be used here by selecting a remote in the repository view and using this context to define what to synchronize with each other.
Comment 21 Dariusz Luksza CLA 2010-05-22 08:13:48 EDT
(In reply to comment #20)
> +1 to start with just Synchronize...

Right now patch set 5 contains implemented 'Synchronize with...' action.

> Maybe the approach Mathias found for simple fetch and push 
> http://egit.eclipse.org/r/#change,715
> could also be used here by selecting a remote in the repository view and using
> this context to define what to synchronize with each other.

Also latest patch set (5) adds support for launching synchronization form 'Repositories view' juts fire RMB on branch name and select 'Synchronize'.

Additionally this patch set should also hide all outgoing changes that matches one of the global git ignore statements.
Comment 22 Chris X CLA 2010-05-25 07:20:49 EDT
Hi,

one question: when will the patch sets described here make it into the nightly build of 0.8.0?

Thanks in advance,
Chris
Comment 23 Matthias Sohn CLA 2010-06-17 04:04:32 EDT
(In reply to comment #22)
> one question: when will the patch sets described here make it into the nightly
> build of 0.8.0?

The IP review has been started, as soon as this is approved the change will be merged into master and then included in the central 0.9.0 build (0.8 is over).
Comment 24 Matthias Sohn CLA 2010-06-17 04:08:52 EDT
Dariusz:

for the IP review you need to make the following confirmations regarding this contribution as a comment on this bug (for Remy's part we don't need this since he became committer) :

1.  authored 100% the content they are contributing
2.  have the rights to donate the content to EPL
3.  contribute the content under the EPL
Comment 25 Dariusz Luksza CLA 2010-06-17 04:30:25 EDT
(In reply to comment #24)
> 1.  authored 100% the content they are contributing

I'm a author of this content (apart of Remy's changes).

> 2.  have the rights to donate the content to EPL

Yes, I have full rights to donate the content to EPL.

> 3.  contribute the content under the EPL

Yes, content is contributed under EPL
Comment 26 Chris Aniszczyk CLA 2010-06-17 20:15:12 EDT
Our first cut of the Synchronize view is now merged to master.

If you come across any issues or enhancement ideas, please file separate bugs.

Thanks all for the feedback.