Bug 301775 - Support core.autocrlf
Summary: Support core.autocrlf
Status: RESOLVED FIXED
Alias: None
Product: JGit
Classification: Technology
Component: JGit (show other bugs)
Version: unspecified   Edit
Hardware: Other Windows All
: P3 enhancement with 37 votes (vote)
Target Milestone: 2.0   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: noteworthy
: 320519 364708 (view as bug list)
Depends on: 333216 333269
Blocks: 361503 334248 339397 339670 353867 382067
  Show dependency tree
 
Reported: 2010-02-04 00:58 EST by Robin Rosenberg CLA
Modified: 2016-06-13 08:15 EDT (History)
48 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robin Rosenberg CLA 2010-02-04 00:58:57 EST
Original feature request at http://code.google.com/p/egit/issues/detail?id=54

Synopsis:
EGit/JGit ignores the core.autocrlf flag which may be quite inconvenient for Windows users.

Besides actually doing this we need to figure out where /etc is. So far we have ignored this, but msysgit users typically have the core.autocrlf flag set in /etc/gitconfig. So how do we know it's set?

An intermediate workaround could be to set the flags to false for repositories created by jgit. That way it will be set to the only setting we recognize at the momement, and thus our and msysgit users will see the same setting for this flag.
Comment 1 Robin Rosenberg CLA 2010-02-04 01:20:27 EST
Workaround posted at http://egit.eclipse.org/r/#change,274
Comment 2 Shawn Pearce CLA 2010-08-18 21:25:56 EDT
Marc has started some of this work with http://egit.eclipse.org/r/1179
Comment 3 Marc Strapetz CLA 2010-08-19 06:52:56 EDT
(In reply to comment #2)
> Marc has started some of this work with http://egit.eclipse.org/r/1179

For http://egit.eclipse.org/r/1179 I confirm that:

(a) I've written 100% of the code except of the code snippets which Shawn Pearce has suggested in his reviews

(b) I have a full right for my code and I'm donating it to Eclipse.

(c) Content is contributed under EPL.
Comment 4 Chris Aniszczyk CLA 2010-08-19 09:52:59 EDT
(In reply to comment #3)
> (In reply to comment #2)
> > Marc has started some of this work with http://egit.eclipse.org/r/1179
> 
> For http://egit.eclipse.org/r/1179 I confirm that:
> 
> (a) I've written 100% of the code except of the code snippets which Shawn
> Pearce has suggested in his reviews
> 
> (b) I have a full right for my code and I'm donating it to Eclipse.
> 
> (c) Content is contributed under EPL.

Thanks Marc :)
Comment 5 Shawn Pearce CLA 2010-08-19 12:54:09 EDT
(In reply to comment #3)
> (c) Content is contributed under EPL.

Do you mean the EDL instead?

JGit is only licensed under the EDL.  For JGit contributions we cannot accept EPL code.  I'm pretty sure you mean EDL, but I'd appreciate it if you could clarify here in Bugzilla so the IP folks are happy.
Comment 6 Marc Strapetz CLA 2010-08-19 13:03:12 EDT
(In reply to comment #5)
> (In reply to comment #3)
> > (c) Content is contributed under EPL.
> 
> Do you mean the EDL instead?
> 
> JGit is only licensed under the EDL.  For JGit contributions we cannot accept
> EPL code.  I'm pretty sure you mean EDL, but I'd appreciate it if you could
> clarify here in Bugzilla so the IP folks are happy.

Sorry, I wasn't aware of that difference. Yes, I confirm that content is contributed under EDL.
Comment 7 Robin Rosenberg CLA 2010-08-19 17:13:50 EDT
*** Bug 320519 has been marked as a duplicate of this bug. ***
Comment 8 Chris Aniszczyk CLA 2010-08-20 11:56:01 EDT
FYI, the CQ was approved and we can check this code in now...

https://dev.eclipse.org/ipzilla/show_bug.cgi?id=4419
Comment 9 Shawn Pearce CLA 2010-08-20 15:12:35 EDT
(In reply to comment #8)
> FYI, the CQ was approved and we can check this code in now...
> 
> https://dev.eclipse.org/ipzilla/show_bug.cgi?id=4419

Great!  I'll unblock the change and get it submitted today.
Comment 10 Shawn Pearce CLA 2010-08-20 21:09:13 EDT
(In reply to comment #9)
> (In reply to comment #8)
> > FYI, the CQ was approved and we can check this code in now...
> 
> Great!  I'll unblock the change and get it submitted today.

Submitted.  Leaving the bug open since the feature is still
not yet complete, but this is a good head start.  :-)
Comment 11 Edwin Kempin CLA 2010-09-07 09:06:11 EDT
I understand that the support for core.autocrlf in jgit is not fully 
implemented yet. Just in case you are not aware I want to share the following
stacktrace with you that I'm currently facing when working with Gerrit and 
a repository for which core.autocrlf is set to 'input'. 

Caused by: java.lang.IllegalArgumentException: Invalid boolean value: core.autocrlf=input
	at org.eclipse.jgit.lib.Config.getBoolean(Config.java:331)
	at org.eclipse.jgit.lib.Config.getBoolean(Config.java:304)
	at org.eclipse.jgit.lib.CoreConfig.<init>(CoreConfig.java:76)
	at org.eclipse.jgit.lib.CoreConfig.<init>(CoreConfig.java:56)
	at org.eclipse.jgit.lib.CoreConfig$1.parse(CoreConfig.java:60)
	at org.eclipse.jgit.lib.CoreConfig$1.parse(CoreConfig.java:58)
	at org.eclipse.jgit.lib.Config.get(Config.java:441)
	at org.eclipse.jgit.storage.file.ObjectDirectoryInserter.compress(ObjectDirectoryInserter.java:172)
	at org.eclipse.jgit.storage.file.ObjectDirectoryInserter.toTemp(ObjectDirectoryInserter.java:141)
	at org.eclipse.jgit.storage.file.ObjectDirectoryInserter.insert(ObjectDirectoryInserter.java:84)
	at org.eclipse.jgit.lib.ObjectInserter.insert(ObjectInserter.java:244)
	at org.eclipse.jgit.lib.ObjectInserter.insert(ObjectInserter.java:224)
	at com.google.gerrit.server.patch.PatchListCacheImpl$Loader.emptyTree(PatchListCacheImpl.java:641)
	at com.google.gerrit.server.patch.PatchListCacheImpl$Loader.aFor(PatchListCacheImpl.java:626)
	at com.google.gerrit.server.patch.PatchListCacheImpl$Loader.readPatchList(PatchListCacheImpl.java:216)
	at com.google.gerrit.server.patch.PatchListCacheImpl$Loader.createEntry(PatchListCacheImpl.java:185)
	at com.google.gerrit.server.patch.PatchListCacheImpl$Loader.createEntry(PatchListCacheImpl.java:171)
	at com.google.gerrit.server.cache.PopulatingCache$1.createEntry(PopulatingCache.java:55)
	at net.sf.ehcache.constructs.blocking.SelfPopulatingCache.get(SelfPopulatingCache.java:72)
	... 51 more

The exception above prevents me to browse certain changes in the Gerrit WebUI.
Also the creation of new branches for such a repository through the Gerrit 
WebUI is failing.
Comment 12 Shawn Pearce CLA 2010-09-07 10:56:50 EDT
(In reply to comment #11)
> I understand that the support for core.autocrlf in jgit is not fully 
> implemented yet. Just in case you are not aware I want to share the following
> stacktrace with you that I'm currently facing when working with Gerrit and 
> a repository for which core.autocrlf is set to 'input'. 
> 
> Caused by: java.lang.IllegalArgumentException: Invalid boolean value:
> core.autocrlf=input

Whoops.  core.autocrlf needs to be more like an enum.  I've got code for handling enums from Git config files in Gerrit that I keep forgetting to move to JGit.  I'll move this over today.
Comment 13 Shawn Pearce CLA 2010-09-07 20:17:30 EDT
(In reply to comment #12)
> (In reply to comment #11)
> > I understand that the support for core.autocrlf in jgit is not fully 
> > implemented yet. Just in case you are not aware I want to share the following
> > stacktrace with you that I'm currently facing when working with Gerrit and 
> > a repository for which core.autocrlf is set to 'input'. 
> > 
> > Caused by: java.lang.IllegalArgumentException: Invalid boolean value:
> > core.autocrlf=input
> 
> Whoops.  core.autocrlf needs to be more like an enum.

Enum support added in http://egit.eclipse.org/r/1552
Comment 14 Daniel Trebbien CLA 2010-10-16 13:58:42 EDT
There is a problem with `core.autocrlf` support in EGit/JGit 0.9.3:  When a new repository is initialized via Team -> Share Project... -> Git, the repository's (local) setting of `core.autocrlf` is set to `false` even though I have `core.autocrlf` set to `true` in my global config. In fact, whenever I newly initialize a git repository using EGit/JGit, the repository's `core.autocrlf` setting is always `false`.

I am using Eclipse 3.5.

Steps to reproduce:
1. Select Window -> Preferences. Under Team -> Git -> Configuration, ensure that `core.autocrlf` is set to either `true` or `input`.
2. Select File -> New -> Project.... Under General, select Project, then click "Next >". Enter a project name (e.g. "Git Test") and click "Finish".
3. In "Package Explorer", right click on "Git Test" and select Team -> Share Project.... Select Git, then click "Next >". Select the "Git Test" row in the table of projects, then click "Create Repository". Click Finish.
4. Open a terminal and `cd` into the `Git Test` directory in your workspace directory.
5. Execute `git config core.autocrlf`.
git will say that the repository's `core.autocrlf` configuration is `false`, when it should be the value of `git config --global core.autocrlf`. Compare this with git's behavior:
6. `cd .. && mkdir "Git Test2" && cd "Git Test2" && git init`
7. Execute `git config core.autocrlf`.
git will respond with whatever `git config --global core.autocrlf` is. Change your global `core.autocrlf` and execute `git config core.autocrlf` again. git will still respond with whatever you changed `git config --global core.autocrlf` to---indicating that git does not set the repository's `core.autocrlf` setting when initializing a new repository.
Comment 15 Robin Rosenberg CLA 2011-01-08 11:17:53 EST
(In reply to comment #14)
> There is a problem with `core.autocrlf` support in EGit/JGit 0.9.3:  When a new
> repository is initialized via Team -> Share Project... -> Git, the repository's
> (local) setting of `core.autocrlf` is set to `false` even though I have
> `core.autocrlf` set to `true` in my global config. In fact, whenever I newly
> initialize a git repository using EGit/JGit, the repository's `core.autocrlf`
> setting is always `false`.

Added references to relevant needed features, ie, 333269 333216
Comment 16 Adam Rich CLA 2011-01-12 12:37:03 EST
(In reply to comment #15)
> (In reply to comment #14)
> > There is a problem with `core.autocrlf` support in EGit/JGit 0.9.3:  When a new
> > repository is initialized via Team -> Share Project... -> Git, the repository's
> > (local) setting of `core.autocrlf` is set to `false` even though I have
> > `core.autocrlf` set to `true` in my global config. In fact, whenever I newly
> > initialize a git repository using EGit/JGit, the repository's `core.autocrlf`
> > setting is always `false`.
> 
> Added references to relevant needed features, ie, 333269 333216

Now that both those bugs are fixed, how does that effect the progress on this bug?
Comment 17 Robin Rosenberg CLA 2011-01-15 06:13:40 EST
(In reply to comment #16)
> (In reply to comment #15)
> > (In reply to comment #14)
> > > There is a problem with `core.autocrlf` support in EGit/JGit 0.9.3:  When a new
> > > repository is initialized via Team -> Share Project... -> Git, the repository's
> > > (local) setting of `core.autocrlf` is set to `false` even though I have
> > > `core.autocrlf` set to `true` in my global config. In fact, whenever I newly
> > > initialize a git repository using EGit/JGit, the repository's `core.autocrlf`
> > > setting is always `false`.
> > 
> > Added references to relevant needed features, ie, 333269 333216
> 
> Now that both those bugs are fixed, how does that effect the progress on this
> bug?

You can install the nightly build and test.
Comment 18 Rob Withers CLA 2011-02-28 21:54:20 EST
(In reply to comment #17)
> (In reply to comment #16)
> > (In reply to comment #15)
> > > (In reply to comment #14)
> > > > There is a problem with `core.autocrlf` support in EGit/JGit 0.9.3:  When a new
> > > > repository is initialized via Team -> Share Project... -> Git, the repository's
> > > > (local) setting of `core.autocrlf` is set to `false` even though I have
> > > > `core.autocrlf` set to `true` in my global config. In fact, whenever I newly
> > > > initialize a git repository using EGit/JGit, the repository's `core.autocrlf`
> > > > setting is always `false`.
> > > 
> > > Added references to relevant needed features, ie, 333269 333216
> > 
> > Now that both those bugs are fixed, how does that effect the progress on this
> > bug?
> 
> You can install the nightly build and test.

I just today loaded Helios and tried out EGit.  We develop on Linux, MacOSX and I am on Windows.  We need autocrlf=true, so I was glad to see this fix and I loaded up the latest code.  On the one hand, I changed a line of code and with the latest jgit/egit source, the entire file was NOT marked as changed with the Windows CRLF.  Only the one line was marked.  Unfortunately, that one line had a CRLF, which shows up in msysgit shell as a ^M.  You can see the lines with '+       ^M' below.  Here is the relevant git log -p:





commit 75c1d2e2ddaa9a900d319a9e38f9a7daa5dcddf7
Author: Rob Withers <rob.withers@arpassoc.com>
Date:   Mon Feb 28 21:27:25 2011 -0500

    test1

diff --git a/main/java/com/arpassoc/ibstt/msgProxy/util/Size.java b/main/java/com/arpassoc/ibstt/msgProxy/util/Size.java
index 84b3e23..9465c55 100644
--- a/main/java/com/arpassoc/ibstt/msgProxy/util/Size.java
+++ b/main/java/com/arpassoc/ibstt/msgProxy/util/Size.java
@@ -4,12 +4,15 @@ import java.io.Serializable;

 import com.arpassoc.ibstt.msgProxy.enums.SizeUnits;

-public class Size extends Number implements Serializable {
+public class Size implements Serializable {^M
        private static final long serialVersionUID = -9047430043018949077L;

        public static Size bytes(int value) { return new Size(SizeUnits.BYTES, value); }
        public static Size bits(int value) { return new Size(SizeUnits.BITS, value); }

+       ^M
+       ^M
+       ^M
        private SizeUnits units = SizeUnits.BYTES;
        private int value = 0;
Comment 19 dnd1066 CLA 2011-03-29 06:23:02 EDT
doesn't seem to work on the latest nightly build:   
Eclipse JGit (Incubation) 0.12.0.201103281315

i created a new file with dos line-endings and it merrily committed and pushed this.  when i pulled with auto.crlf set to false the file had dos line endings still.

committing/pushing with eg TortoiseGit or msysgit resulted in a file with unix line endings when pulled.
Comment 20 Tomasz Zarna CLA 2011-08-04 07:08:59 EDT
FWIW, bug 353867 (http://egit.eclipse.org/r/3957) contains a failing test case of file checkout when mixed line endings are involved and aurocrlf is true.
Comment 21 Robin Rosenberg CLA 2011-10-28 09:01:09 EDT
A patch related to /finding/ the core.autocrlf setting on Windows has been posted to Gerrit http://egit.eclipse.org/r/#change,4430
Comment 22 Robin Rosenberg CLA 2011-11-22 16:10:08 EST
RFC Patch at http://egit.eclipse.org/r/#change,4530
Comment 23 Fabian Schmidt CLA 2011-12-05 09:42:07 EST
*** Bug 364708 has been marked as a duplicate of this bug. ***
Comment 24 Matt Jensen CLA 2011-12-07 10:44:25 EST
Ran several tests of this functionality against the 2011-12-05 nightly build of the EGit plugin. It appears that diffs work better with the nightly than with the full release, as only changed lines are highlighted in the diff view. However, I was not able to get the CRLF<->LF changes to occur at the appropriate times.

Here is what I tried.

1) Create a new repository on my Git server with a couple files created on Linux (LF line endings.) core.autocrlf=false here.
2) Clone the repository to a Windows machine using the command line (Git Bash from mSysGit.) core.autocrlf=true here.
3) Verify that mSysGit correctly translated line endings to CRLF. It did.
4) Modify the file on Windows, using an editor that is configured for CRLF line endings.
5) Commit and push from Windows to the remote repository.
6) Pull to the local Linux repo. Modifications from Windows are present; line endings are LF.

-- at this point we know that mSysGit did the LF<->CRLF changes correctly. --

7) Clone the repository on Windows again, this time using EGit.
8) Verify, via Eclipse preferences, that core.autocrlf=true at all three levels: "User Settings," "System Settings," and "Repository Settings." It is true in all three places.
9) Check line endings. Line endings are LF on the Windows repo cloned by EGit--no translation appears to have occurred.
10) Edit a file in the repository. Editor changes line endings to CRLF.
11) Commit and push from Windows EGit to the remote repository.
12) Pull to the local Linux repo. Line endings are CRLF.

-- EGit appears to have ignored the core.autocrlf setting and pushed CRLFs to the remote repository. --

I'm wondering if the problem here is that EGit not picking up my core.autocrlf setting. Changing the setting does not appear to change the way EGit behaves, despite the fact that the setting is shown as "true" in all three panels on Preferences->Team->Git->Configuration.

This really makes things difficult for a shop that develops on Windows and on Unix (both Linux and Mac OS X.) We're hoping that a fix for this is forthcoming. Is there anything I can do to be of assistance on this? Does it appear that I am doing something wrong? (Always possible...)
Comment 25 Robin Rosenberg CLA 2011-12-07 18:24:36 EST
(In reply to comment #24)
> Ran several tests of this functionality against the 2011-12-05 nightly build of

The RFC patch mentioned before your comment  (#22) is NOT included in the nightly build.
To test it you have to pull it down and build it. Considering the large list of Cc's I'd
imagine someone would be interested in doing that or at least commenting on the
patch.
Comment 26 Issac Goldstand CLA 2011-12-15 09:17:39 EST
I'm attempting to pull it and test it.  This is holding back https://jira.atlassian.com/browse/BAM-9591 and I'm trying to patch their libs and give it a try.  I've actually got JGit patched there and am just trying to get it into my running binary, and will post my results here once I've got it set up.
Comment 27 Issac Goldstand CLA 2011-12-28 03:24:38 EST
Not to leave people hanging, the patch applied cleanly but couldn't determine if it works correctly.  Something kept touching my core.autocrlf setting and switching it off, and unfortunately, I ran out of tuits before I could figure out what.
Comment 28 Robin Rosenberg CLA 2012-01-20 10:27:12 EST
76dd9d1d46007fc49639d264631658114f4fbd24 was merged some time ago. It's in the nightly builds.

Note: EGit does not currently set autocrlf by default. Msysgit does set it,
but you need to tell JGit/EGit where it is since the system wide git config file is located where git is located and you can have 0..n git installations.

For JGit you can set the property jgit.gitprefix to git's installation directory, e.g. the one that is one level above the bin directory. You could
do that with eclipse too.

There is a patch http://egit.eclipse.org/r/#change,4430 that makes this easier
to do for EGit.

A better may is probably to copy the content of /etc/gitconfig to ~/.gitconfig
and avoid the problem with the "system-wide" config file.
Comment 29 Matthias Sohn CLA 2012-01-20 19:11:50 EST
merged http://egit.eclipse.org/r/#change,4430 as 20d1d972a5db3caca0036b7bddeb8c3df8e70496
Comment 30 Robin Rosenberg CLA 2012-04-14 10:27:43 EDT
The 2.0 snapshot has been updated regarding core.autocrlf some time ago. The latest commit on the topic is https://git.eclipse.org/r/#/c/5412/
Comment 31 Robin Rosenberg CLA 2012-04-26 19:19:14 EDT
No response. I'm closing this and related issues if noone objects.
Comment 32 Robin Rosenberg CLA 2012-05-31 18:56:26 EDT
See previous comments
Comment 33 Sylvain Laurent CLA 2012-06-07 16:25:04 EDT
I tried eclipse 4.2 RC2 which comes with egit 2.0.0-rc2 and this bug seems still present.
I tried with core.autocrlf=true at the system and/or user level, in all cases when egit clones a repo, it does not convert text files to CRLF.

I found a workaround however : 
- after the clone is finished, I see that core.autocrlf has been set to false on the repository created with eclipse
- I change core.autocrlf to true on the repository itself
- with Widnows Explorer I delete all files/folders from its working directory (except the .git folder of course)
- I do a hard reset in egit
- now all text files are in CRLF

so, all in all it seems that the core.autocrlf=true feature is working, it's just the initial detection of the system or user (global) setting which is still bugged.

I cannot reopen this issue, I don't have enough rights...
Comment 34 Sylvain Laurent CLA 2012-06-07 17:23:15 EDT
In org.eclipse.jgit.storage.file.FileRepository.create(boolean bare) one can see that core.autocrlf is forced to false on the repository config when it is created, thus leading to the behavior I just described.

http://git.eclipse.org/c/jgit/jgit.git/commit/org.eclipse.jgit/src/org/eclipse/jgit/storage/file/FileRepository.java?h=stable-2.0&id=7aeea3b27c86e638dae2328929769703db5f1a89

cfg.setBoolean(ConfigConstants.CONFIG_CORE_SECTION, null,
				ConfigConstants.CONFIG_KEY_AUTOCRLF, false);

It should be forced to the value inherited from the parent configs (user or system).
Comment 35 Robin Rosenberg CLA 2012-06-07 17:36:43 EDT
(In reply to comment #34)
> In org.eclipse.jgit.storage.file.FileRepository.create(boolean bare) one can
> see that core.autocrlf is forced to false on the repository config when it is
> created, thus leading to the behavior I just described.
> 
> http://git.eclipse.org/c/jgit/jgit.git/commit/org.eclipse.jgit/src/org/eclipse/jgit/storage/file/FileRepository.java?h=stable-2.0&id=7aeea3b27c86e638dae2328929769703db5f1a89
> 
> cfg.setBoolean(ConfigConstants.CONFIG_CORE_SECTION, null,
>                 ConfigConstants.CONFIG_KEY_AUTOCRLF, false);
> 
> It should be forced to the value inherited from the parent configs (user or
> system).

You can open a new issue (bug) as this is not about supporting autocrlf per se, but rather the default behaviour. I'm not sure we should change it this late in the cycle, but if we get a patch maybe. It has to come from a committer to
make it into 2.0.
Comment 36 Sylvain Laurent CLA 2012-06-07 17:55:41 EDT
(In reply to comment #35)
> You can open a new issue (bug) as this is not about supporting autocrlf per se,
> but rather the default behaviour. I'm not sure we should change it this late in
> the cycle, but if we get a patch maybe. It has to come from a committer to
> make it into 2.0.

done, see bug 382067
Comment 37 Cristiano Gaviao CLA 2012-12-11 06:45:29 EST
I'm using MacOSX (juno or kepler) and even using core.autocrlf=input I still getting the undesired behavior of get all files changed at JGit but not in another git tool.
Comment 38 Nemanja Nedic CLA 2013-01-28 06:20:06 EST
I'm having the same issue.
Comment 39 Marc Gobeil CLA 2013-05-15 14:02:03 EDT
I'm also still having this issue with EGit 2.3.1.201302201838-r
Comment 40 Andreas Gondek CLA 2016-06-13 05:34:43 EDT
3 years later and still having this problem with EGit 4.1.1
Comment 41 Matthias Sohn CLA 2016-06-13 08:15:32 EDT
(In reply to Andreas Gondek from comment #40)
> 3 years later and still having this problem with EGit 4.1.1

if you think core.autocrlf support is broken, please try to reproduce your problem with latest release 4.3.1 (http://download.eclipse.org/egit/updates) and nightly build (http://download.eclipse.org/egit/updates-nightly). If this doesn't fix your problem open a bug and provide details how to reproduce.