Bug 303925 - Add .gitignore compatibility to JGit
Summary: Add .gitignore compatibility to JGit
Status: RESOLVED FIXED
Alias: None
Product: JGit
Classification: Technology
Component: JGit (show other bugs)
Version: 0.9.0   Edit
Hardware: PC Linux
: P3 enhancement (vote)
Target Milestone: 0.9.0-M1   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: noteworthy
Depends on:
Blocks: 319849
  Show dependency tree
 
Reported: 2010-02-25 11:03 EST by Charley Wang CLA
Modified: 2010-07-14 11:43 EDT (History)
6 users (show)

See Also:


Attachments
Add new class for checking ignored status of file (4.99 KB, text/plain)
2010-03-03 11:33 EST, Charley Wang CLA
no flags Details
Patch adding git ignore logic to jgit (17.15 KB, text/plain)
2010-05-10 20:47 EDT, Charley Wang CLA
no flags Details
Test cases for 303925 (33.90 KB, text/plain)
2010-05-10 21:32 EDT, Charley Wang CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Charley Wang CLA 2010-02-25 11:03:29 EST
Ideally the 'add to .gitignore' button would be toggle-able within EGit, but a logical first step seems to be first checking the file for existing instances of the entry to be added before adding the entry. 

Currently the IgnoreAction will add files to .gitignore multiple times if the user selects the action multiple times.
Comment 1 Charley Wang CLA 2010-03-03 11:33:29 EST
Created attachment 160799 [details]
Add new class for checking ignored status of file

Somewhat more thorough check for whether a file is ignored
Comment 2 Charley Wang CLA 2010-03-03 11:34:37 EST
Create IgnoredResources class for checking whether a resource is ignored. Method from this class is called in IgnoredAction, replacing the check to Team.isIgnoredHint.

Currently:
- Calls Team.isIgnoredHint
- Parses the .gitignore file and checks for exact matches
- Parses the .gitignore file and checks for glob matches
- Skips comments, but does not handle negations, folder-only or any other .gitignore special indicators

I'm not sure if this more detailed check for ignored files is something the EGit 
project wants, so before forging blindly ahead, I thought I'd first attempt a push
and see what you guys think. :)

From what I can see, there are two places in the code that check 
Team.isIgnoredHint, IgnoredAction and DecoratableResourceAdapter. The latter has a TODO that says it should check for .gitignored and .git/info/exclude.
Comment 3 Charley Wang CLA 2010-03-03 12:12:52 EST
Pushed to Gerrit: http://egit.eclipse.org/r/#change,326
Comment 4 Charley Wang CLA 2010-03-05 11:38:07 EST
Compatibility tests for the patch in Gerrit:


***Compatibility***

With .gitignore in same directory:

Successfully ignored
Pattern		File		Notes
/*.stp		/test.stp	
/*.st?		/test.stp	
/*.sta[0-9]	/test.sta1	
/*/*.stp	/src/asdf.stp	.gitignore in parent dir
/foo/		/foo/		where foo is a directory
*.stp		/test.stp
*.stp		/src/test.stp	No slashes in pattern -- match filename


Successfully not ignored
Pattern		File		Notes
/*.stp		/src/asdf.stp	* should not expand to include '/'
/foo/		/foo		Pattern ends with /, only match directories
/*.st?
!/test.stp	/test.stp	Pattern + higher priority negation
#/test.stp	/test.stp	Commented
/src?test	/src/test.stp	? should not expand to '/'
Comment 5 Charley Wang CLA 2010-03-09 11:29:51 EST
Updated compatibility tests

***Compatibility***

Successfully ignored

Pattern		File		Notes
/*.stp		/test.stp	

/*.st?		/test.stp	

/*.sta[0-9]	/test.sta1	

/*/*.stp	/src/asdf.stp	

/foo/		/foo/		where foo is a directory

*.stp		/test.stp	No slashes in pattern -- match filename

*.stp		/src/test.stp	No slashes in pattern -- match filename

!/test.stp
/test.stp 	/test.stp	Negation and higher priority ignore



Successfully not ignored

Pattern		File		Notes
/*.stp		/src/asdf.stp	* should not expand to include '/'

/foo/		/foo		Pattern ends with /, only match directories

/*.st?

!/test.stp	/test.stp	Pattern + higher priority negation

#/test.stp	/test.stp	Commented

/src?test	/src/test.stp	? should not expand to '/'

!/test.stp
/src/test.stp 	/src/test.stp	Negation in highest priority file, look no further

!/test.stp
/test.stp	
!/test.stp 	/test.stp	Highest priority statement in file is negation
Comment 6 Gunnar Wagenknecht CLA 2010-03-11 09:20:06 EST
+1 for a centralized handling of ignored resources in EGit. However, the patch provided in Gerrit should be updated to make better use of Eclipse APIs. I also think there should be a central ingore manager (per project) which is used by other parts of EGit as well.
Comment 7 Gunnar Wagenknecht CLA 2010-03-11 09:21:50 EST
BTW, ideally the "Add to .gitignore" action should not be available on resources which are already ignored.
Comment 8 Charley Wang CLA 2010-03-11 09:34:58 EST
(In reply to comment #6)
> +1 for a centralized handling of ignored resources in EGit. However, the patch
> provided in Gerrit should be updated to make better use of Eclipse APIs. I also
> think there should be a central ingore manager (per project) which is used by
> other parts of EGit as well.

I'd be happy to take a look at this -- could you specify what you mean by Eclipse API's? I wasn't aware of any that may have helped with the .gitignore stuff. 

Re: Not making .gitignore visible, that can also be handled :) I thought I'd focus on the IgnoredResources class first, though.
Comment 9 Gunnar Wagenknecht CLA 2010-03-11 09:52:14 EST
(In reply to comment #8)
> I'd be happy to take a look at this -- could you specify what you mean by
> Eclipse API's? I wasn't aware of any that may have helped with the .gitignore
> stuff. 

Checkout the IResource API. There is API for reading the content using the correct charset, etc. which doesn't require going through java.io.File. You can also hook a change listerner that gets informed whenever the contents of the .gitignore file changes.
Comment 10 Charley Wang CLA 2010-03-11 12:09:19 EST
(In reply to comment #9)
> (In reply to comment #8)
> > I'd be happy to take a look at this -- could you specify what you mean by
> > Eclipse API's? I wasn't aware of any that may have helped with the .gitignore
> > stuff. 
> 
> Checkout the IResource API. There is API for reading the content using the
> correct charset, etc. which doesn't require going through java.io.File. You can
> also hook a change listerner that gets informed whenever the contents of the
> .gitignore file changes.

Hi again,

I'm not sure what you mean by using the IResource API to read. I've changed it to avoid using File:

final BufferedReader br = new BufferedReader(new FileReader(resource.getLocation().toOSString()));

Or would you prefer it be read as bytes as is done in DirCache.readFrom? I believe gitignore files are deliminited by new lines, so readLine() was just a convenient way to parse each item in the file individually.

I'll take a look at using an IResourceVisitor to help create a project ignore manager, and at hooking changelisteners into every .gitignore file :) I agree that having a git ignore manager would be a better way to do it than checking each file individually. Maybe this can work together with DirCache? I will try to figure it out.

BTW, during the day I'm charley on #eclipse, if you ever want to discuss in realtime.
Comment 11 Gunnar Wagenknecht CLA 2010-03-11 14:38:15 EST
(In reply to comment #10)
> I'm not sure what you mean by using the IResource API to read.

Have a look at:
http://help.eclipse.org/help32/topic/org.eclipse.platform.doc.isv/reference/api/org/eclipse/core/resources/IFile.html
and
http://www.eclipse.org/articles/?sort=date&category=Resources
Comment 12 Charley Wang CLA 2010-05-10 20:47:06 EDT
Created attachment 167849 [details]
Patch adding git ignore logic to jgit

I don't seem to be able to do git push review -- I keep getting
rejected by remote. I will try again from work tomorrow.

-Charley

Commit message:

Add compatibility with gitignore specifications

I tried to meet the requirements. Let me know if there's
anything that needs changing :)

I will attach the test cases shortly. Please see my talk page
for some documentation of the covered cases (documentation is
not comprehensive right now)

http://wiki.eclipse.org/User_talk:Charley.wang.gmail.com

Todo:
- Implement more intelligent node parsing.
- Better cache than just a HashMap. Suggestion was to move
  this into RepositoryMapping in EGit.
- The rule for matching e.g. "/src/ne?" to "/src/new/file.c"
  is slow.

Bug-id: 303925
Change-Id: I5c40e237275f2d9031b2b56591ba98c5ce166008
Comment 13 Charley Wang CLA 2010-05-10 21:32:02 EDT
Created attachment 167855 [details]
Test cases for 303925

Test cases for the above.

Added test folder excludeTest for some intialization tests and other multiline/multifolder tests that I couldn't cover by just checking for pattern matches. Jumped through hoops for .git/info/exclude, which I couldn't add to the git repo.

Still having issues with git push review.

-Charley
Comment 14 Chris Aniszczyk CLA 2010-05-10 22:58:35 EDT
This looks interesting.
Comment 15 Matthias Sohn CLA 2010-05-11 03:35:07 EDT
I am happy you are working on this missing feature :)

JGit is strictly EDL license (see [1]) while EGit is developed under EPL license.
Hence all license headers in your jgit patches need to be changed accordingly.

[1] http://dev.eclipse.org/mhonarc/lists/jgit-dev/msg00127.html
Comment 16 Daniele Segato CLA 2010-06-07 10:25:30 EDT
I've experienced the same issue...

On a Linux box, plugin just installed from the updates web site (not nightly).

both .gitignore and .git/info/exclude are completely ignored...
this is serious I think.

any news about this? installing the nightly build could help?
Comment 17 Chris Aniszczyk CLA 2010-06-07 10:32:38 EDT
(In reply to comment #16)
> I've experienced the same issue...
> 
> On a Linux box, plugin just installed from the updates web site (not nightly).
> 
> both .gitignore and .git/info/exclude are completely ignored...
> this is serious I think.
> 
> any news about this? installing the nightly build could help?

We dont' support .gitignore yet and are working on it for inclusion in the 0.9.0 stream... there is a patch that is currently being reviewed in Gerrit.

http://egit.eclipse.org/r/#change,683

We plan to ship 0.9.0 in September 2010 but we are hoping full .gitignore support appears in an upcoming nightly build (please give us a few weeks, we are still in the process of shipping for the Eclipse Helios release)
Comment 18 Daniele Segato CLA 2010-06-08 03:57:53 EDT
@Chris

in the meanwhile can you (or someone else) propose a workaround to "load" the .gitignore manually?

is there something like this?

I always used git from command line, I'm now introducing some fellow-worker to git through egit but I can't give it to them if it doesn't take in care the .gitignore I already committed on the repository.

thanks
Comment 19 Chris Aniszczyk CLA 2010-07-05 14:10:45 EDT
I'm happy with this cut of the patch and will file a CQ.
Comment 20 Chris Aniszczyk CLA 2010-07-05 14:33:19 EDT
CQ created:

https://dev.eclipse.org/ipzilla/show_bug.cgi?id=4302
Comment 21 Matthias Sohn CLA 2010-07-12 18:52:52 EDT
merged http://egit.eclipse.org/r/#change,683 as b878cdcf6b4c2445553dcd1507d5c3008bf56b7b

Charley: do you want to keep this bug open for further steps or do you want to create new bugs for that ? The title of this bug doesn't exactly match the core ignore logic you implemented in change 683.
Comment 22 Charley Wang CLA 2010-07-12 19:06:17 EDT
(In reply to comment #21)
> merged http://egit.eclipse.org/r/#change,683 as
> b878cdcf6b4c2445553dcd1507d5c3008bf56b7b
> 
> Charley: do you want to keep this bug open for further steps or do you want to
> create new bugs for that ? The title of this bug doesn't exactly match the core
> ignore logic you implemented in change 683.

If it is all right with the EGit community, I'd like to change the title and close the bug as the JGit bug 'Implement core .gitignore functionality', then open a new bug for the EGit component. Otherwise, I'd prefer to keep the bug open for further steps.

-Charley
Comment 23 Matthias Sohn CLA 2010-07-14 07:34:10 EDT
(In reply to comment #22)
> If it is all right with the EGit community, I'd like to change the title and
> close the bug as the JGit bug 'Implement core .gitignore functionality', then
> open a new bug for the EGit component. Otherwise, I'd prefer to keep the bug
> open for further steps.
> 

Then go ahead, change the title of this bug and close it and create new bugs for the things to come.
Comment 24 Charley Wang CLA 2010-07-14 09:17:27 EDT
Committed, see http://egit.eclipse.org/r/#change,683
Comment 25 Matthias Sohn CLA 2010-07-14 09:33:40 EDT
merged as b878cdcf6b4c2445553dcd1507d5c3008bf56b7b
Comment 26 Charley Wang CLA 2010-07-14 11:43:29 EDT
(In reply to comment #25)
> merged as b878cdcf6b4c2445553dcd1507d5c3008bf56b7b

Bug 319849 has been created for the EGit side of this equation.