Bug 301431 - Ant build fails completely due to filesystem permissions within bundle
Summary: Ant build fails completely due to filesystem permissions within bundle
Status: CLOSED NOT_ECLIPSE
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: Framework (show other bugs)
Version: 3.6   Edit
Hardware: Macintosh Mac OS X - Carbon (unsup.)
: P3 blocker (vote)
Target Milestone: ---   Edit
Assignee: equinox.framework-inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: readme
Depends on:
Blocks:
 
Reported: 2010-02-01 11:32 EST by Timothy Mowlem CLA
Modified: 2010-02-09 15:47 EST (History)
4 users (show)

See Also:


Attachments
Screenshot of example error log (223.23 KB, image/png)
2010-02-01 11:34 EST, Timothy Mowlem CLA
no flags Details
Screenshot of example error log (223.23 KB, image/png)
2010-02-01 11:36 EST, Timothy Mowlem CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Timothy Mowlem CLA 2010-02-01 11:32:54 EST
Build Identifier: 3.6M5 I20100129-1300

When I try to do an Ant build it fails completely with an error and no output at all is visible in console.
Error shows clearly a filesystem permissions error within the Eclipse bundle.

I am installing Eclipse in /Applications as an admin account and trying to use it from a different normal user which is a common usage scenario so it should work in this configuration.  (Apple recommend that admin accounts not be used for normal use if possible).

See attached screenshot for example error. Many error of this type are occurring when Eclipse tries to access various files.

Reproducible: Always
Comment 1 Timothy Mowlem CLA 2010-02-01 11:34:27 EST
Created attachment 157798 [details]
Screenshot of example error log
Comment 2 Timothy Mowlem CLA 2010-02-01 11:36:01 EST
Created attachment 157799 [details]
Screenshot of example error log
Comment 3 Timothy Mowlem CLA 2010-02-01 12:21:22 EST
(In reply to comment #2)
> Created an attachment (id=157799) [details]
> Screenshot of example error log

This is a serious problem for me as I have reverted to 3.6M4 assuming that 3.6M5 has a permissions bug but M4 is showing the same problem. Have therefore raised priority to MAX as Eclipse is completely unusable for normal work at present.

Also there is a forum post by another user running on Linux who has the same problem with no solution.

Some details which may help track down the problem:

As mentioned I install as an admin user and run from a different normal user. I have been operating this way for over a year without any issues so this cannot be the problem per se.

I also edit my eclipse.ini inside the bundle (with TextMate set to unix EOLs) to set some extra memory settings, etc.

I have several versions of Eclipse side by side. All are in /Applications and are in similarly named folders thus:

"eclipse 3.5 64-bit"
"eclipse 3.6M3 64-bit"
"eclipse 3.6M4 64-bit"
"eclipse 3.6M5 64-bit"

When this problem occurred I logged this bug and reverted to 3.6M4 for work assuming that this would work okay until the issue is resolved but 3.6M4 is also not working now for the same reason.

I restarted my machine but the problem persists.
Comment 4 Frederic Fusier CLA 2010-02-01 12:23:13 EST
Move to Platform/Ant
Comment 5 Remy Suen CLA 2010-02-01 12:23:48 EST
(In reply to comment #3)
> Also there is a forum post by another user running on Linux who has the same
> problem with no solution.

Please provide a link to this post.

I presume this can be reproduced on a new workspace. Please attach the log file
as seen in a new workspace.
Comment 6 Timothy Mowlem CLA 2010-02-01 13:14:59 EST
(In reply to comment #5)
> (In reply to comment #3)
> > Also there is a forum post by another user running on Linux who has the same
> > problem with no solution.
> 
> Please provide a link to this post.
> 
> I presume this can be reproduced on a new workspace. Please attach the log file
> as seen in a new workspace.


Forum posts:

Home » Language IDEs » Java Development Tools (JDT) » due to insufficient permissions or insufficient disk space [message #509337]
Home » Language IDEs » Java Development Tools (JDT) » Eclipse unusable on MAcOSX due to permissions issues [message #511487]


Some extra info:

(1) Ran 3.6M5 and found problem described
(2) deleted ~/.eclipse
(3) started 3.6M4 - same problems
(4) started 3.6M3 - works OK, do NOT see errors (a new ~/.eclipse is created)
(5) re-ran 3.6M4 - same problems


Also note that I have definitely seen problems like this before. I reported the issue with the source editor collapse controls appearing as small square red dots instead of the usual 'plus' and 'minus' icons (bug #297727) and the stack trace I think was almost the same.


I switched to a new (non-existing) workspace. I get lots of errors again. Below is the output from one log entry:


eclipse.buildId=I20100129-1300
java.version=1.6.0_17
java.vendor=Apple Inc.
BootLoader constants: OS=macosx, ARCH=x86_64, WS=cocoa, NL=en_US
Framework arguments:  -keyring /Users/xxx/.eclipse_keyring -showlocation
Command-line arguments:  -os macosx -ws cocoa -arch x86_64 -keyring /Users/xxx/.eclipse_keyring -showlocation


Error
Mon Feb 01 18:03:05 GMT 2010
The URL "bundleentry://178.fwk749784719/icons/elcl16/restore_log.gif" could not be extracted probably due to insufficient permissions or insufficient disk space.

java.io.IOException: The URL "bundleentry://178.fwk749784719/icons/elcl16/restore_log.gif" could not be extracted probably due to insufficient permissions or insufficient disk space.
	at org.eclipse.core.runtime.internal.adaptor.URLConverterImpl.toFileURL(URLConverterImpl.java:40)
	at org.eclipse.core.runtime.FileLocator.toFileURL(FileLocator.java:206)
	at org.eclipse.jface.resource.URLImageDescriptor.getFilePath(URLImageDescriptor.java:137)
	at org.eclipse.jface.resource.URLImageDescriptor.createImage(URLImageDescriptor.java:157)
	at org.eclipse.jface.resource.ImageDescriptor.createResource(ImageDescriptor.java:165)
	at org.eclipse.jface.resource.DeviceResourceManager.allocate(DeviceResourceManager.java:56)
	at org.eclipse.jface.resource.AbstractResourceManager.create(AbstractResourceManager.java:88)
	at org.eclipse.jface.resource.LocalResourceManager.allocate(LocalResourceManager.java:82)
	at org.eclipse.jface.resource.AbstractResourceManager.create(AbstractResourceManager.java:88)
	at org.eclipse.jface.resource.ResourceManager.createImageWithDefault(ResourceManager.java:192)
	at org.eclipse.jface.action.ActionContributionItem.updateImages(ActionContributionItem.java:1046)
	at org.eclipse.jface.action.ActionContributionItem.update(ActionContributionItem.java:783)
	at org.eclipse.jface.action.ActionContributionItem.fill(ActionContributionItem.java:342)
	at org.eclipse.jface.action.ToolBarManager.update(ToolBarManager.java:353)
	at org.eclipse.ui.internal.ViewPane.updateActionBars(ViewPane.java:447)
	at org.eclipse.ui.internal.ViewActionBars.updateActionBars(ViewActionBars.java:59)
	at org.eclipse.ui.internal.ViewReference.createPartHelper(ViewReference.java:414)
	at org.eclipse.ui.internal.ViewReference.createPart(ViewReference.java:226)
	at org.eclipse.ui.internal.WorkbenchPartReference.getPart(WorkbenchPartReference.java:595)
	at org.eclipse.ui.internal.Perspective.showView(Perspective.java:2245)
	at org.eclipse.ui.internal.WorkbenchPage.busyShowView(WorkbenchPage.java:1071)
	at org.eclipse.ui.internal.WorkbenchPage$20.run(WorkbenchPage.java:3822)
	at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:70)
	at org.eclipse.ui.internal.WorkbenchPage.showView(WorkbenchPage.java:3819)
	at org.eclipse.ui.internal.WorkbenchPage.showView(WorkbenchPage.java:3795)
	at org.eclipse.ui.handlers.ShowViewHandler.openView(ShowViewHandler.java:162)
	at org.eclipse.ui.handlers.ShowViewHandler.execute(ShowViewHandler.java:77)
	at org.eclipse.ui.internal.handlers.HandlerProxy.execute(HandlerProxy.java:293)
	at org.eclipse.core.commands.Command.executeWithChecks(Command.java:476)
	at org.eclipse.core.commands.ParameterizedCommand.executeWithChecks(ParameterizedCommand.java:508)
	at org.eclipse.ui.internal.handlers.HandlerService.executeCommand(HandlerService.java:169)
	at org.eclipse.ui.internal.handlers.SlaveHandlerService.executeCommand(SlaveHandlerService.java:241)
	at org.eclipse.ui.menus.CommandContributionItem.handleWidgetSelection(CommandContributionItem.java:820)
	at org.eclipse.ui.menus.CommandContributionItem.access$19(CommandContributionItem.java:806)
	at org.eclipse.ui.menus.CommandContributionItem$5.handleEvent(CommandContributionItem.java:796)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84)
	at org.eclipse.swt.widgets.Display.sendEvent(Display.java:3706)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1314)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1337)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1322)
	at org.eclipse.swt.widgets.Widget.notifyListeners(Widget.java:1136)
	at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3566)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3224)
	at org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2407)
	at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2371)
	at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2220)
	at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:500)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:493)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
	at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:115)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:194)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:367)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:611)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:566)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1363)
Comment 7 Remy Suen CLA 2010-02-01 13:21:03 EST
(In reply to comment #6)
> Home » Language IDEs » Java Development Tools (JDT) » due to insufficient
> permissions or insufficient disk space [message #509337]

http://www.eclipse.org/forums/index.php?t=msg&goto=509436

> Home » Language IDEs » Java Development Tools (JDT) » Eclipse unusable on
> MAcOSX due to permissions issues [message #511487]

http://www.eclipse.org/forums/index.php?t=msg&th=161825
Comment 8 Olivier Thomann CLA 2010-02-01 13:39:33 EST
Moving to Equinox/OSGI
Comment 9 Thomas Watson CLA 2010-02-01 17:07:21 EST
On the workspace that is failing, does it come up at all?  If so please attach the configuration details:  About Eclipse SDK -> Installation Details -> Configuration

In particular I am curious what the value is for osgi.configuration.area.
Comment 10 Timothy Mowlem CLA 2010-02-02 04:25:18 EST
(In reply to comment #9)
> On the workspace that is failing, does it come up at all?  If so please attach
> the configuration details:  About Eclipse SDK -> Installation Details ->
> Configuration
> 
> In particular I am curious what the value is for osgi.configuration.area.

Yes it does load the workspace okay. I can see all of the modules in the package explorer.

osgi.configuration.area=file:/Applications/eclipse 3.6M5 64-bit/configuration/

I can paste the whole configuration data if required with a little obfuscation.
Comment 11 Timothy Mowlem CLA 2010-02-02 04:30:09 EST
(In reply to comment #10)
> (In reply to comment #9)
> > On the workspace that is failing, does it come up at all?  If so please attach
> > the configuration details:  About Eclipse SDK -> Installation Details ->
> > Configuration
> > 
> > In particular I am curious what the value is for osgi.configuration.area.
> 
> Yes it does load the workspace okay. I can see all of the modules in the
> package explorer.
> 
> osgi.configuration.area=file:/Applications/eclipse 3.6M5 64-bit/configuration/
> 
> I can paste the whole configuration data if required with a little obfuscation.

By contrast 3.6M3 has a different setting:

osgi.configuration.area=file:/Users/xxx/.eclipse/org.eclipse.platform_3.5.0_307584333/configuration/

where xxx is my username
Comment 12 Thomas Watson CLA 2010-02-02 11:34:32 EST
(In reply to comment #11)
> 
> By contrast 3.6M3 has a different setting:
> 
> osgi.configuration.area=file:/Users/xxx/.eclipse/org.eclipse.platform_3.5.0_307584333/configuration/
> 
> where xxx is my username

This is the root cause of the problem.  For some reason we have miscalculated the configuration area in your environment.  I have yet to be able to reproduce it on the mac.  Would you be willing to debug a few launcher methods for us to figure out where we are going wrong?  It would be great if you could step through the following two methods and let us know what you see:

org.eclipse.equinox.launcher.Main.canWrite(File) 

Should pass in the File object to the location on disk where you have Eclipse installed and this method should return false.

org.eclipse.equinox.launcher.Main.computeDefaultUserAreaLocation(String)

This should get called from computeDefaultConfigurationLocation when we detect that the install area is not writable.  This ultimately should return the ~/.eclipse/... folder in your users home dir.
Comment 13 Thomas Watson CLA 2010-02-03 09:40:38 EST
Tim sent me a note asking how to debug this.  I decided to reply in the open in case others are wondering.

You will need to launch a working eclipse of the same version as the failing one.  Just extract eclipse into your home directory (or where ever you can write as the user you want).  

- Open the PDE perspective and go to the Plug-ins tab.
- Select all plug-ins and right click to "Add to Java Search".
- This should allow you to view and debug all the eclipse code.
- Now "Open Type" <command>-<shift>-<t> and open the org.eclipse.equinox.launcher.Main type

- Add a break point to the methods mentioned in comment 12

Now you will need to launch eclipse from the installation/user that is failing in debug mode.

- Using a terminal cd into the installation folder where eclipse is and run the following command:

./eclipse -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000

This should should cause the process to wait in the console on port 8000 for you to connect a debugger.

- Go back to your PDE workspace and open the debugger: Debug Configurations -> Remote Java Application -> hit the new button.  You should be able to use the defaults.  Make sure host is localhost and port is 8000.  Click the debug button and away you go.
Comment 14 Andrew Niefer CLA 2010-02-03 09:45:38 EST
(In reply to comment #13)
Small correction, you need a -vmargs there
./eclipse -vmargs -Xdebug Xrunjdwp:transport=........
          ^^^^^^^
Comment 15 Thomas Watson CLA 2010-02-03 09:52:21 EST
(In reply to comment #14)
> (In reply to comment #13)
> Small correction, you need a -vmargs there
> ./eclipse -vmargs -Xdebug Xrunjdwp:transport=........
>           ^^^^^^^

Thanks Andrew!  Good catch.
Comment 16 Timothy Mowlem CLA 2010-02-04 05:54:05 EST
(In reply to comment #15)
> (In reply to comment #14)
> > (In reply to comment #13)
> > Small correction, you need a -vmargs there
> > ./eclipse -vmargs -Xdebug Xrunjdwp:transport=........
> >           ^^^^^^^
> 
> Thanks Andrew!  Good catch.


Also for people that haven't done any Eclipse internal development before...

There is no "PDE" perspective, rather "Plug-in Development"
Comment 17 Timothy Mowlem CLA 2010-02-04 06:08:24 EST
(In reply to comment #13)
> Tim sent me a note asking how to debug this.  I decided to reply in the open in
> case others are wondering.
> 
> You will need to launch a working eclipse of the same version as the failing
> one.  Just extract eclipse into your home directory (or where ever you can
> write as the user you want).  
> 
> - Open the PDE perspective and go to the Plug-ins tab.
> - Select all plug-ins and right click to "Add to Java Search".
> - This should allow you to view and debug all the eclipse code.
> - Now "Open Type" <command>-<shift>-<t> and open the
> org.eclipse.equinox.launcher.Main type
> 
> - Add a break point to the methods mentioned in comment 12
> 
> Now you will need to launch eclipse from the installation/user that is failing
> in debug mode.
> 
> - Using a terminal cd into the installation folder where eclipse is and run the
> following command:
> 
> ./eclipse -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000
> 
> This should should cause the process to wait in the console on port 8000 for
> you to connect a debugger.
> 
> - Go back to your PDE workspace and open the debugger: Debug Configurations ->
> Remote Java Application -> hit the new button.  You should be able to use the
> defaults.  Make sure host is localhost and port is 8000.  Click the debug
> button and away you go.


Okay everything seems fine here, the root install directory (which I have named "eclipse 3.6M5 64-bit" so it has spaces in it) is a directory and is writeable. I observed the temporary file created there and deleted. The method returned true. Therefore the second method is never called.
Comment 18 Timothy Mowlem CLA 2010-02-04 06:24:14 EST
(In reply to comment #17)
> (In reply to comment #13)
> > Tim sent me a note asking how to debug this.  I decided to reply in the open in
> > case others are wondering.
> > 
> > You will need to launch a working eclipse of the same version as the failing
> > one.  Just extract eclipse into your home directory (or where ever you can
> > write as the user you want).  
> > 
> > - Open the PDE perspective and go to the Plug-ins tab.
> > - Select all plug-ins and right click to "Add to Java Search".
> > - This should allow you to view and debug all the eclipse code.
> > - Now "Open Type" <command>-<shift>-<t> and open the
> > org.eclipse.equinox.launcher.Main type
> > 
> > - Add a break point to the methods mentioned in comment 12
> > 
> > Now you will need to launch eclipse from the installation/user that is failing
> > in debug mode.
> > 
> > - Using a terminal cd into the installation folder where eclipse is and run the
> > following command:
> > 
> > ./eclipse -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000
> > 
> > This should should cause the process to wait in the console on port 8000 for
> > you to connect a debugger.
> > 
> > - Go back to your PDE workspace and open the debugger: Debug Configurations ->
> > Remote Java Application -> hit the new button.  You should be able to use the
> > defaults.  Make sure host is localhost and port is 8000.  Click the debug
> > button and away you go.
> 
> 
> Okay everything seems fine here, the root install directory (which I have named
> "eclipse 3.6M5 64-bit" so it has spaces in it) is a directory and is writeable.
> I observed the temporary file created there and deleted. The method returned
> true. Therefore the second method is never called.


There may be another issue here. When I run Eclipse 3.6M5 as a process in the debugger of another Eclipse as per your above instructions it seems to run out of memory even if I set max heap space to 800MB.

First thread to die OOM is:

BaseStorage$StateSaver.run() line: 1174
Thread.run() line: 637

It doesn't do this though when run normally. Not sure if this is some side effect of debugging or a pointer to a problem.
Comment 19 Thomas Watson CLA 2010-02-04 10:18:05 EST
(In reply to comment #17)
> 
> Okay everything seems fine here, the root install directory (which I have named
> "eclipse 3.6M5 64-bit" so it has spaces in it) is a directory and is writeable.
> I observed the temporary file created there and deleted. The method returned
> true. Therefore the second method is never called.

I doubt it is a problem with spaces, but to be sure your M3 installation contained spaces?  Can you double check what the permissions are for the root install directory of M3 vs M5?  Nothing has changed between M3 to M5 that should have changed to logic for being able to wrote to the install directory.  Could you also check the permissions of the configuration/ folders also?
Comment 20 Thomas Watson CLA 2010-02-04 10:22:10 EST
(In reply to comment #18)
> First thread to die OOM is:
> 
> BaseStorage$StateSaver.run() line: 1174
> Thread.run() line: 637
> 
> It doesn't do this though when run normally. Not sure if this is some side
> effect of debugging or a pointer to a problem.

I'm not sure what is going on here.  Could be side effect of not being able to persist the framework data but I'm not sure why that would take up all your heap space.
Comment 21 Timothy Mowlem CLA 2010-02-04 12:00:58 EST
(In reply to comment #19)
> (In reply to comment #17)
> > 
> > Okay everything seems fine here, the root install directory (which I have named
> > "eclipse 3.6M5 64-bit" so it has spaces in it) is a directory and is writeable.
> > I observed the temporary file created there and deleted. The method returned
> > true. Therefore the second method is never called.
> 
> I doubt it is a problem with spaces, but to be sure your M3 installation
> contained spaces?  Can you double check what the permissions are for the root
> install directory of M3 vs M5?  Nothing has changed between M3 to M5 that
> should have changed to logic for being able to wrote to the install directory. 
> Could you also check the permissions of the configuration/ folders also?


Yes my M3 installation does also contain spaces in its name (I have been doing this for ages so I don't think is has any relevance).

Both Eclipse install directories are owned by the admin user and are in the staff group.
The user is also in the staff group.

My M3 install has drwxr-xr-x
My M5 install has drwxrwxr-x

So M5 has extra permissions. Unnecessary since M3 seems to work (although M3 also has a few errors in the error log but they don't seem to cause any problems during use).

For the configuration folders:

My M3 install has drwxr-xr-x@
My M5 install has drwxrwxr-x@

Apple uses the @ to signify extended attributes. Using xattr both configuration folders have the same extended attributes which are:

com.apple.quarantine
2 others due to Safari

Apple use the former to mark files downloaded from the internet and its presence causes a dialog to be shown to the user when the app is started warning that the app was a download and asking the user to confirm that it is safe to proceed.

AFAIK opening the app for the first time when logged in as the user who performed the download and confirming the dialog causes the attribute to be removed. For that reason I open the app in the admin user after installation and then quit it.

All of the folders within the install directory in M5 have this quarantine attribute set. I am not sure if there are other ramifications of it being set?
I will try removing them using xattr and re-test M5.

I haven't tested removing the write permission from the M5 install dir to see if that has an effect as I can't see why this would help when M3 works okay without it.

Also be aware that Apple create a new file and replace the original when a changed file is saved (at least in the Finder, not sure whether this is in the API call?
In Cocoa file save methods the atomically flag may control this but I am only guessing here). So therefore directory write permissions are important on MacOSX when saving a file although you might assume that only the file w permission is needed for this.

I raised a bug with Apple about this as I feel it is breaks the user's expectation of Unix file permissions but they don't agree with me and I got the famous "Behaves as designed" fob off.
Comment 22 Timothy Mowlem CLA 2010-02-04 12:01:50 EST
(In reply to comment #20)
> (In reply to comment #18)
> > First thread to die OOM is:
> > 
> > BaseStorage$StateSaver.run() line: 1174
> > Thread.run() line: 637
> > 
> > It doesn't do this though when run normally. Not sure if this is some side
> > effect of debugging or a pointer to a problem.
> 
> I'm not sure what is going on here.  Could be side effect of not being able to
> persist the framework data but I'm not sure why that would take up all your
> heap space.

See comment 21 regarding saving files on MacOSX.
Comment 23 Timothy Mowlem CLA 2010-02-04 12:15:06 EST
(In reply to comment #21)
> (In reply to comment #19)
> > (In reply to comment #17)
> > > 
> > > Okay everything seems fine here, the root install directory (which I have named
> > > "eclipse 3.6M5 64-bit" so it has spaces in it) is a directory and is writeable.
> > > I observed the temporary file created there and deleted. The method returned
> > > true. Therefore the second method is never called.
> > 
> > I doubt it is a problem with spaces, but to be sure your M3 installation
> > contained spaces?  Can you double check what the permissions are for the root
> > install directory of M3 vs M5?  Nothing has changed between M3 to M5 that
> > should have changed to logic for being able to wrote to the install directory. 
> > Could you also check the permissions of the configuration/ folders also?
> 
> 
> Yes my M3 installation does also contain spaces in its name (I have been doing
> this for ages so I don't think is has any relevance).
> 
> Both Eclipse install directories are owned by the admin user and are in the
> staff group.
> The user is also in the staff group.
> 
> My M3 install has drwxr-xr-x
> My M5 install has drwxrwxr-x
> 
> So M5 has extra permissions. Unnecessary since M3 seems to work (although M3
> also has a few errors in the error log but they don't seem to cause any
> problems during use).
> 
> For the configuration folders:
> 
> My M3 install has drwxr-xr-x@
> My M5 install has drwxrwxr-x@
> 
> Apple uses the @ to signify extended attributes. Using xattr both configuration
> folders have the same extended attributes which are:
> 
> com.apple.quarantine
> 2 others due to Safari
> 
> Apple use the former to mark files downloaded from the internet and its
> presence causes a dialog to be shown to the user when the app is started
> warning that the app was a download and asking the user to confirm that it is
> safe to proceed.
> 
> AFAIK opening the app for the first time when logged in as the user who
> performed the download and confirming the dialog causes the attribute to be
> removed. For that reason I open the app in the admin user after installation
> and then quit it.
> 
> All of the folders within the install directory in M5 have this quarantine
> attribute set. I am not sure if there are other ramifications of it being set?
> I will try removing them using xattr and re-test M5.
> 
> I haven't tested removing the write permission from the M5 install dir to see
> if that has an effect as I can't see why this would help when M3 works okay
> without it.
> 
> Also be aware that Apple create a new file and replace the original when a
> changed file is saved (at least in the Finder, not sure whether this is in the
> API call?
> In Cocoa file save methods the atomically flag may control this but I am only
> guessing here). So therefore directory write permissions are important on
> MacOSX when saving a file although you might assume that only the file w
> permission is needed for this.
> 
> I raised a bug with Apple about this as I feel it is breaks the user's
> expectation of Unix file permissions but they don't agree with me and I got the
> famous "Behaves as designed" fob off.


I have tried removing all the extended attributes in "eclipse 3.6M5 64-bit" recursively but the file access problems are still there. Many of them appear when Eclipse is "Initializing Java Tooling..." on startup
Comment 24 Thomas Watson CLA 2010-02-04 12:22:12 EST
(In reply to comment #21)
> Both Eclipse install directories are owned by the admin user and are in the
> staff group.
> The user is also in the staff group.
> 
> My M3 install has drwxr-xr-x
> My M5 install has drwxrwxr-x
> 
> So M5 has extra permissions. Unnecessary since M3 seems to work (although M3
> also has a few errors in the error log but they don't seem to cause any
> problems during use).
> 
> For the configuration folders:
> 
> My M3 install has drwxr-xr-x@
> My M5 install has drwxrwxr-x@
> 

I think this difference is the key.  How do you extract eclipse and where did you download M5 from?  One my Mac the extraction of M3 and M5 give me the same permissions for the installation folder: drwxr-xr-x@

So only the owner has permission to write to the folder.  In your case all users in the staff group have write access.


> Apple uses the @ to signify extended attributes. Using xattr both configuration
> folders have the same extended attributes which are:
> 
> com.apple.quarantine
> 2 others due to Safari
> 
> Apple use the former to mark files downloaded from the internet and its
> presence causes a dialog to be shown to the user when the app is started
> warning that the app was a download and asking the user to confirm that it is
> safe to proceed.
> 
> AFAIK opening the app for the first time when logged in as the user who
> performed the download and confirming the dialog causes the attribute to be
> removed. For that reason I open the app in the admin user after installation
> and then quit it.
> 

This is another thing that is contributing to the failure.  When launching with the admin user new folders get created under the configuration folder (i.e. configuration/org.eclipse.osgi)  I bet these new folder do not give write access to the staff group.  So our check for canWrite for the install folder passes for all staff users but the staff users do not really have write access to all subfolders of the configuration folder.  This can lead to the issues you have been seeing.  The installation folder of eclipse should only give write access to the owner at most.  So the question for me is why does the extraction of M5 on your machine grant write access to the install folder?

Thanks for your patience on this issue.
Comment 25 Timothy Mowlem CLA 2010-02-04 12:49:30 EST
(In reply to comment #24)
> (In reply to comment #21)
> > Both Eclipse install directories are owned by the admin user and are in the
> > staff group.
> > The user is also in the staff group.
> > 
> > My M3 install has drwxr-xr-x
> > My M5 install has drwxrwxr-x
> > 
> > So M5 has extra permissions. Unnecessary since M3 seems to work (although M3
> > also has a few errors in the error log but they don't seem to cause any
> > problems during use).
> > 
> > For the configuration folders:
> > 
> > My M3 install has drwxr-xr-x@
> > My M5 install has drwxrwxr-x@
> > 
> 
> I think this difference is the key.  How do you extract eclipse and where did
> you download M5 from?  One my Mac the extraction of M3 and M5 give me the same
> permissions for the installation folder: drwxr-xr-x@
>
> So only the owner has permission to write to the folder.  In your case all
> users in the staff group have write access.
 

I download from:    http://download.eclipse.org/eclipse/downloads/

I download either as admin or as another user and move it to /Ssers/Shared then copy it to the Desktop from the admin user.

I extract it on the desktop to give a directory named "eclipse".

Sometimes I may install it as my dev (normal) user and in these cases I manually chown -R it to admin:staff

 
> > Apple uses the @ to signify extended attributes. Using xattr both configuration
> > folders have the same extended attributes which are:
> > 
> > com.apple.quarantine
> > 2 others due to Safari
> > 
> > Apple use the former to mark files downloaded from the internet and its
> > presence causes a dialog to be shown to the user when the app is started
> > warning that the app was a download and asking the user to confirm that it is
> > safe to proceed.
> > 
> > AFAIK opening the app for the first time when logged in as the user who
> > performed the download and confirming the dialog causes the attribute to be
> > removed. For that reason I open the app in the admin user after installation
> > and then quit it.
> > 
> 
> This is another thing that is contributing to the failure.  When launching with
> the admin user new folders get created under the configuration folder (i.e.
> configuration/org.eclipse.osgi)  I bet these new folder do not give write
> access to the staff group.  So our check for canWrite for the install folder
> passes for all staff users but the staff users do not really have write access
> to all subfolders of the configuration folder.  This can lead to the issues you
> have been seeing.  The installation folder of eclipse should only give write
> access to the owner at most.  So the question for me is why does the extraction
> of M5 on your machine grant write access to the install folder?


Yes I think you nailed it. Exactly as you describe according to my terminal output. I will confirm by a fresh download and reinstall.

Presumably I should avoid install as normal user, chown to admin, then run as normal user But if I don't run as admin user (assuming I installed as admin) then I don't think the quarantine attributes are removed and the nag dialog comes up every time.

What do you recommend for this issue? Maybe just use xattr which should leave all of the permissions settings the same?

What is the recommended way to install given that I normally install from an admin account and run it from a different (normal) user? (as I said previously this is Apple's recommended MO generally)


> Thanks for your patience on this issue.

I think the thanks are due to you Thomas for your efforts.


Best regards,

Tim Mowlem
Comment 26 Timothy Mowlem CLA 2010-02-05 05:02:54 EST
(In reply to comment #24)
> (In reply to comment #21)
> > Both Eclipse install directories are owned by the admin user and are in the
> > staff group.
> > The user is also in the staff group.
> > 
> > My M3 install has drwxr-xr-x
> > My M5 install has drwxrwxr-x
> > 
> > So M5 has extra permissions. Unnecessary since M3 seems to work (although M3
> > also has a few errors in the error log but they don't seem to cause any
> > problems during use).
> > 
> > For the configuration folders:
> > 
> > My M3 install has drwxr-xr-x@
> > My M5 install has drwxrwxr-x@
> > 
> 
> I think this difference is the key.  How do you extract eclipse and where did
> you download M5 from?  One my Mac the extraction of M3 and M5 give me the same
> permissions for the installation folder: drwxr-xr-x@
> 
> So only the owner has permission to write to the folder.  In your case all
> users in the staff group have write access.
> 
> 
> > Apple uses the @ to signify extended attributes. Using xattr both configuration
> > folders have the same extended attributes which are:
> > 
> > com.apple.quarantine
> > 2 others due to Safari
> > 
> > Apple use the former to mark files downloaded from the internet and its
> > presence causes a dialog to be shown to the user when the app is started
> > warning that the app was a download and asking the user to confirm that it is
> > safe to proceed.
> > 
> > AFAIK opening the app for the first time when logged in as the user who
> > performed the download and confirming the dialog causes the attribute to be
> > removed. For that reason I open the app in the admin user after installation
> > and then quit it.
> > 
> 
> This is another thing that is contributing to the failure.  When launching with
> the admin user new folders get created under the configuration folder (i.e.
> configuration/org.eclipse.osgi)  I bet these new folder do not give write
> access to the staff group.  So our check for canWrite for the install folder
> passes for all staff users but the staff users do not really have write access
> to all subfolders of the configuration folder.  This can lead to the issues you
> have been seeing.  The installation folder of eclipse should only give write
> access to the owner at most.  So the question for me is why does the extraction
> of M5 on your machine grant write access to the install folder?
> 
> Thanks for your patience on this issue.


Thomas,

Okay we now understand the whole thing. Starting with the freshly downloaded archive file on my Desktop logged in as an admin user:

If I unzip and unarchive using "The Unarchiver" (a utility designed to replace the built in Apple equivalent) then the unarchived directory is given the permissions drwxrwxr-x

If I unzip and unarchive using "Archive Utility" (the Apple built-in utility which is used by default) then the unarchived directory is given the permissions drwxr-xr-x

I use "The Unarchiver" by default as it has some extra features.

Then things proceed as per your email.

I have set this bug to CLOSED NOT_ECLIPSE.

Also the red dot issue goes away as well.
I will set bug #297727 to CLOSED NOT_ECLIPSE.

Only further comment is whether you should include slightly stricter checks in the Eclipse startup process?

This issue could easily occur for other users and not necessarily just on MacOSX. There are of course numerous archiving utilities available on all major platforms and a user could innocently use one ans get this problem without ever having consciously done anything out of the usual.

Thank you.
Comment 27 Thomas Watson CLA 2010-02-08 09:58:27 EST
(In reply to comment #26)

Thanks for the information.  We probably should add something to the readme about this.


> Only further comment is whether you should include slightly stricter checks in
> the Eclipse startup process?
> 
> This issue could easily occur for other users and not necessarily just on
> MacOSX. There are of course numerous archiving utilities available on all major
> platforms and a user could innocently use one ans get this problem without ever
> having consciously done anything out of the usual.
> 
> Thank you.

I have thought about this, but the check will have to be pretty thorough and it would likely slow startup performance significantly with large configuration caches.  To do it correctly we would have to recursively check each directory under the configuration folder to make sure the user has write permission.
Comment 28 Timothy Mowlem CLA 2010-02-09 15:08:46 EST
(In reply to comment #27)
> (In reply to comment #26)
> 
> Thanks for the information.  We probably should add something to the readme
> about this.
> 
> 
> > Only further comment is whether you should include slightly stricter checks in
> > the Eclipse startup process?
> > 
> > This issue could easily occur for other users and not necessarily just on
> > MacOSX. There are of course numerous archiving utilities available on all major
> > platforms and a user could innocently use one ans get this problem without ever
> > having consciously done anything out of the usual.
> > 
> > Thank you.
> 
> I have thought about this, but the check will have to be pretty thorough and it
> would likely slow startup performance significantly with large configuration
> caches.  


Is that even possible? ;-))


To do it correctly we would have to recursively check each directory
> under the configuration folder to make sure the user has write permission.


Yes the thin end of the wedge. Although presumably there aren't that many directories involved and subfolders when created would presumably have known permissions on Unix-like platforms via the umask so you may not need to check all recursively.

Also I note from the debug walkthrough that you actually create and destroy a temporary file, but JDK has canRead and canWrite for testing this which I would have thought would be quicker.
Comment 29 Thomas Watson CLA 2010-02-09 15:47:34 EST
(In reply to comment #28)
> Also I note from the debug walkthrough that you actually create and destroy a
> temporary file, but JDK has canRead and canWrite for testing this which I would
> have thought would be quicker.

See bug 168445.