Re: [jgit-dev] FileMode values failing self-tests on NonStop

[Andrey Loskutov <loskutov@xxxxxx>]

@Randall I'm wondering if by patching FileUtils.createNewFile() with something similar to FS_POSIX_Java7.setExecute() you will fix your tests.
There is code added to properly handle umask (JDK doesn't do this right way) for *executable* files, but you can try to apply same code for regular files too.

On Wednesday 25 February 2015 09:58 Shawn Pearce wrote:
> Variance in the JVM we don't expect. :(
> On Feb 25, 2015 9:57 AM, "Randall S. Becker" <rsbecker@xxxxxxxxxxxxx> wrote:
> 
> > The default UMASK for my user profile is 0002.
> >
> > touch file1 results in 664
> >
> > new File(“file2”).createNewFile() results in 775
> >
> > So it does appear that new files are being created from Java with execute
> > bits set. Is this a variance in the test case or is it an assumption in
> > JGit?
> >
> >
> >
> > *From:* Shawn Pearce [mailto:spearce@xxxxxxxxxxx]
> > *Sent:* February 25, 2015 12:47 PM
> > *To:* Randall S. Becker
> > *Cc:* jgit-dev; Matthias Sohn
> > *Subject:* RE: [jgit-dev] FileMode values failing self-tests on NonStop
> >
> >
> >
> > Do files default to executable because of something like a umask?
> >
> > I think the test suite assumes new files are automatically not executable.
> >
> > On Feb 25, 2015 7:51 AM, "Randall S. Becker" <rsbecker@xxxxxxxxxxxxx>
> > wrote:
> >
> > On 25 Feb 2015 at 9:22AM, Matthias Sohn wrote:
> > On Wed, Feb 25, 2015 at 3:14 PM, Randall S. Becker <rsbecker@xxxxxxxxxxxxx>
> > wrote:
> > On 25 Feb 2015 1:24AM Shawn Pearce wrote:
> > > On Tue, Feb 24, 2015 at 9:01 PM, Randall S. Becker
> > > <rsbecker@xxxxxxxxxxxxx> wrote:
> > > > Brief intro - I am involved in porting git, JGit, and Gerrit over to
> > > > the HP NonStop OSS platform. Git went pretty well, so far anyway,
> > > > although we have timeout issues from EGit (not relevant to this). For
> > > > JGit, we are currently building for JDK 1.6, and it is not passing 16
> > > > AddCommand tests, due to file mode mismatches, for example:
> > > >
> > > > testAddExistingSingleBinaryFile(org.eclipse.jgit.api.AddCommandTest)
> > > > Time
> > > > elapsed: 1.936 sec  <<< FAILURE!
> > > > org.junit.ComparisonFailure: expected:<[a.txt, mode:100[644],
> > > > content:row1 row2...> but was:<[a.txt, mode:100[755], content:row1
> > > > row2...>
> > > >         at org.junit.Assert.assertEquals(Assert.java:115)
> > > >         at org.junit.Assert.assertEquals(Assert.java:144)
> > > >         at
> > > > org.eclipse.jgit.api.AddCommandTest.testAddExistingSingleBinaryFile(Ad
> > > > dComma
> > > > ndTest.java:179)
> > > >
> > > > The platform is a Big-Endian Itanium with a POSIXesque file system. A
> > > > visual inspection of the FileMode, AddCommand, NB, IO code were
> > > > unrevealing as to why the mode test would fail. All other
> > > > non-mode-related tests pass, except GitConstructionTest, which I am
> > > > just guessing may be related. This applies from HEAD back to at least
> > commit
> > > 8899006. Has this happened elsewhere?
> > >
> > > I haven't seen this specific failure before.
> > >
> > > It looks like JGit is detecting a file is executable when the test
> > expected it to be
> > > a regular file. On Windows this was always fuzzy because the concept of
> > > "executable" isn't like on UNIX systems. The core.filemode config
> > variable
> > > should be auto-detected during repository creation and used during
> > things like
> > > AddCommand to decide if it should trust the filesystem's concept of
> > > "executable". On UNIX systems, yes, on Windows systems typically no,
> > because
> > > its too likely to give you bad results.
> > >
> > > This smells like the Windows failure mode when core.filemode isn't
> > honored
> > > and the code is just blindly trusting the filesystem's concept of
> > executable, and
> > > the filesystem is falsely claiming everything is executable.
> > >
> > > I know nothing of HP NonStop, so I don't know what "executable" means
> > there,
> > > or how the Java APIs might be showing that bit about a file to JGit.
> > > It is very much UNIX-like and Java usually returns proper file
> > attributes when asked. The File object behaves properly [just verified it,
> > with >this JVM] at least for canRead(),canWrite(),canExecute() and matches
> > 'ls -l' exactly for the user, but as this is Java 6, there is no standard
> > >group/other permission API. I figured this should use FS_POSIX_java
> > (eventually), but os.name="NONSTOP_KERNEL" is not known to JGit. >Can you
> > point me to the class where stuff is get/set in JGit?
> > >FS.detect() is detecting the FS implementation to use
> >
> > Looks like FS.detect() picked FS_POSIX_Java6 correctly resolving:
> > canExecute=public boolean java.io.File.canExecute()
> > setExecute=public boolean java.io.File.setExecutable(boolean)
> >
> > The problem does not appear computational. FileMode appears to work
> > properly, as below:
> > >>FileMode(mode=16384/40000,expType=2)
> > <<octalBytes[52,48,48,48,48]
> > >>FileMode(mode=40960/120000,expType=3)
> > <<octalBytes[49,50,48,48,48,48]
> > >>FileMode(mode=33188/100644,expType=3)
> > <<octalBytes[49,48,48,54,52,52]
> > >>FileMode(mode=33261/100755,expType=3)
> > <<octalBytes[49,48,48,55,53,53]
> > >>FileMode(mode=57344/160000,expType=1)
> > <<octalBytes[49,54,48,48,48,48]
> > >>FileMode(mode=0,expType=-1)
> > <<octalBytes[0]
> >
> > This is likely problem when creating files or querying attributes from the
> > file system. Could you point me at the exact spot where this happens? I can
> > see the createNewFile call in FileUtils, but that does not explicitly set
> > file modes. The platform's File.setExecutable(executable,owner) works
> > correctly - just verified it.
> >
> >

-- 
Kind regards,
google.com/+AndreyLoskutov