Bug 431116 - Releases for Mac OS X should be bundled as a proper "Mac App" and/or "Library"
Summary: Releases for Mac OS X should be bundled as a proper "Mac App" and/or "Library"
Status: RESOLVED FIXED
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: unspecified   Edit
Hardware: Macintosh Mac OS X
: P3 normal with 1 vote (vote)
Target Milestone: Mars   Edit
Assignee: Pascal Rapicault CLA
QA Contact:
URL:
Whiteboard:
Keywords:
: 446320 (view as bug list)
Depends on: 444201 445050 446390 461467 461512 461517 461518 461653 461673 461674 461678 461725 461728 462183 462186 462224 462282 462401 462691 462905 463670
Blocks: 432932 446351
  Show dependency tree
 
Reported: 2014-03-25 10:12 EDT by Tobias Bertelsen CLA
Modified: 2016-04-11 00:28 EDT (History)
34 users (show)

See Also:


Attachments
A few screenshots showing .app advantages (169.06 KB, image/png)
2014-03-28 05:36 EDT, Tobias Bertelsen CLA
no flags Details
Screenshot of Momentics.app (188.98 KB, image/png)
2014-03-28 10:48 EDT, Doug Schaefer CLA
no flags Details
Shell Script to package Eclipse as an OS X application bundle (8.69 KB, text/plain)
2014-11-21 17:44 EST, James Watkins-Harvey CLA
no flags Details
Bourne shell script to launch eclipse from the .app format (1.83 KB, text/plain)
2015-03-30 17:05 EDT, Brian de Alwis CLA
no flags Details
Bourne shell script to launch eclipse from the .app format (3.12 KB, text/plain)
2015-04-08 12:21 EDT, Brian de Alwis CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Tobias Bertelsen CLA 2014-03-25 10:12:38 EDT
The official distributions of Eclipse to OS X from http://www.eclipse.org/downloads/ is not within a single.app folder as is standard on OS X. Instead folders like plug-ins or features are siblings to the .app folder
The p2 director has have the capability of installing the correct way since may 12. (https://bugs.eclipse.org/57349)

A further improvement will be to package the distribution as a .dmg image, with a sym-link to the application folder. Most apps are distributed this way, and it will make the installation significantly smoother and usable for all the many developers on mac out there.  

To be clear the current folder layout is like this
Eclipse.tar.gz
  -Eclipse.app
    -...
  -plug-ins
  -features
  ...

But it should look like this
Eclipse.dmg\
  -Eclipse.app
    -plug-ins
    -features
    ...


To get the right folder structure all you need to do is to use 'Eclipse.app' as destination instead of 'Eclipse' when calling the p2 director.

Make a proper .dmg requires a little more work, but I am sure the user base is large enough to invest a few hours into that. There are many examples online.
Comment 1 David Williams CLA 2014-03-26 00:21:52 EDT
Did Doug put you up to opening this bug? (Just kidding ... he mentions it occasionally and I believe he packages CDT this way).

And since you've mentioned http://www.eclipse.org/downloads/ specifically, those are actually packaged by the EPP project, so I've added Markus to CC list. 

"Our" stuff would be at http://download.eclipse.org/eclipse/downloads/
but of course is a foundation of the EPP packages ... and ... not sure ... could "we" continue as we are ... and EPP do it differently? I would think so. 

I doubt there's much we can do at this point in development cycle, but perhaps you could educate me on two things I've always wondered about (keep in mind, while I have access to a Mac, I personally am quite the newbie). 

1) When packaged this way, how do users keep different versions on their machines? Even from a "casual user" point of view, I would imagine users that would want a "Juno" install, a "Kepler Install", as well as most recent Luna M6 install ... not to mention perhaps an install of "CDT and Linux Tools", plus a separate install of "Java EE Tools" ... and on and on. As things are now, they are all named "Eclipse.app", if I'm not mistaken. Is it as simple as renaming, one Eclipse.app, before you install the next one? 

2) Besides doing things "the Apple Way" (which, I am not discounting, that is important) what concrete advantages does this have over doing it the way we currently do? Security advantages? Multiuser advantages? Or is the folder structure purely one step to getting to "dmg" packages? (Again, I'm not questioning it ... just wanted to educate myself and thought you might be willing ... ).
Comment 2 David Williams CLA 2014-03-26 00:30:04 EDT
To elaborate on the "too late for this development cycle", I just wanted to mention a) not sure what changes would be needed for our "unit test framework", and similar, b) could have downstream impact to "repackagers or adopters" much like EPP but commercial adopters too, plus, I'm sure there's lot's of tiny things we'd learn/discover as we went, where we have in one way or another assumed current structure. All of those are probably "not a problem" or are "easy to solve" ... but, at this point there is simply so much else to do, that are closer to "blocking" and this is closer to an "enhancement". 

Plus, not enough time, IMHO, for proper "community feedback" ... I'm sure most Apple users would love the change ... but I do know of some "command line users" that might have some ripple effects? 

But, much appreciate you opening the bug to begin to document the concrete discussions.
Comment 3 Doug Schaefer CLA 2014-03-27 19:32:14 EDT
It's just the right thing to do. And to clarify, I package our Momentics IDE that way. Works great and is a much more natural experience on the Mac.
Comment 4 Tobias Bertelsen CLA 2014-03-28 05:13:28 EDT
No I don't know Doug, but he sounds like a clever guy :)

1)
A most valid usecase. I often use have multiple Eclipse's myself.
And yes; you can just rename 'Eclipse.app' to e.g. 'Eclipse Juno.app'. This will also make it show up as 'Eclipse Juno' when you search, in the dock etc. 
If you rename the containing folder in the current setup, both versions will just be 'Eclipse.app', so changing the packaging will actually improve the usability in this usecase.

2)
I am not aware of any significant security advantages or such. The internals are better hidden, though its only takes a right-click to go into the folder. But the primary reason is, that it integrates better into the system both during and after install, and mentioned above.
In short this is just the right thing to do. Just like you assign an icon to the .exe file on windows.


Dev.Cycle)
I just left the milestone and priorities at default, since I'm far from knowledgeable enough of your plans to determine this. I understand your concern about community feedback, so feel free to set the milestone to 4.4.1, or whatever you feel like.

A way that might help with incubating this, could be to have two mac-packages for a short while. One as it is now, and one that with the proper .app structure. The former will be published as usual, while the latter will be available for repackagers and anybody else who would want to give community feedback.
At 4.4.1 the roles could be changed to the proper structure is published. The current is kept a little hidden, but ready for those who where caught unaware, to give them extra time to make whatever changes is required.

As I mentioned before this change is (almost) a oneliner. In Ant the change will look something like this

From:
<property name="dest.folder.name" value="Eclipse"/>
To:
<condition property="dest.folder.name" value="Eclipse.app" else="Eclipse">
  <equals arg1="${target.os}" arg2="macosx"/>
</condition>
Comment 5 Tobias Bertelsen CLA 2014-03-28 05:36:15 EDT
Created attachment 241359 [details]
A few screenshots showing .app advantages

To all of those who do not use mac, I have uploaded some screen shots showing what I explained above. I have compared Eclipse to Gimp (which uses the proper .app structure) and how renamed apps look.
From the top the shots show, Icons in the 'Applications' folder, search through spotlight (usual way of starting apps), and the two versions running in the dock.
Comment 6 Doug Schaefer CLA 2014-03-28 10:48:44 EDT
Created attachment 241387 [details]
Screenshot of Momentics.app
Comment 7 Doug Schaefer CLA 2014-03-28 10:51:21 EDT
From an Eclipse developer point of view, it's even better. I have a Momentics.app on my desktop which I created with Maven, I just dragged and dropped it there. Now I can just double click on it and run the results of my build. And, of course, I have one in /Applications so that it shows up with all my other Mac applications. It just hides all the gory details of app and provides a superior user experience.

Of course, all Mac users know that already.
Comment 8 Doug Schaefer CLA 2014-03-28 10:53:29 EDT
BTW, it it's pretty easy to do this with Maven/Tycho. I'll take some time next week and put together a patch for the platform.aggregator build so that we can do this in an optional profile.
Comment 9 David Williams CLA 2014-04-16 10:38:38 EDT
To follow up, in a Planning Council call, it was decided "too late for Luna". 

But, we all recognized the communities voice here. So I'll clone ot a cross-project bug as well, for wider awareness for Mars release. 

One of the issues raised was that it might be hard to update from "Luna" to "Mars" with this change but that was only technical issue known (and, it might be solvable, just would need more investigation).
Comment 10 Doug Schaefer CLA 2014-04-16 11:21:38 EDT
For me the big issue is that the builds need to support updating for both bundled and unbundled installs (the Mac term for this is app bundle, BTW), from the same p2 repo. I'm not sure that's possible to generate both at the same time with out some tricks. This needs to be tried and bugs raised against p2 and/or tycho.
Comment 11 David Williams CLA 2014-10-12 23:56:24 EDT
I've become concerned about some of the suggested approaches, or -- more exactly -- the one that is "easy" to do, currently, by a small change to 
tycho-p2-director-plugin
= = = = = = = =
That change is to specify a bit more 'configuration' data, so that 
                  <product>
                    <id>org.eclipse.sdk.ide</id>
                    <rootFolder>eclipse</rootFolder>
                  </product>
is instead
                  <product>
                    <id>org.eclipse.sdk.ide</id>
                    <rootFolder>eclipse</rootFolder>
                      <rootFolders>
                         <macosx>Eclipse.app</macosx>
                      </rootFolders>
                  </product>
- - - - - - - -
This does result in a structure as follows: 
$ ls -1 Eclipse.app/
artifacts.xml
configuration
Contents
dropins
epl-v10.html
features
notice.html
p2
plugins
readme

Where the "Contents" folder, is the same as it currently is (that is, when not using the "Eclipse.app" name):

$ find ./Eclipse.app/Contents/
./Eclipse.app/Contents/
./Eclipse.app/Contents/Info.plist
./Eclipse.app/Contents/Resources
./Eclipse.app/Contents/Resources/Eclipse.icns
./Eclipse.app/Contents/MacOS
./Eclipse.app/Contents/MacOS/eclipse.ini
./Eclipse.app/Contents/MacOS/eclipse
= = = = = = = =
My concern is the with the changes to "Apple signing" [1] in 10.9.5 (and planned for 10.10) There should no no peers (siblings) of the "Contents" directory but everything should be under "Contents" in proscribed directories. (And, this part of "new spec" seems true of all 10.9 versions, not just 10.9.5 ... or, perhaps just starting to enforce? Which might be enforced more in 10.10?) 

[1] https://developer.apple.com/library/mac/technotes/tn2206/_index.html#//apple_ref/doc/uid/DTS40007919-CH1-TNTAG205

We are currently having problems changing location of "eclipse.ini"! (bug 446390) and reading their spec, on the surface, would seem to rule out man of the other directories, especially those that "change contents" such as dropins, artifacts.xml, p2, and others, if "update" counts!
Comment 12 Gunnar Wagenknecht CLA 2014-10-13 14:31:23 EDT
(In reply to David Williams from comment #11)
> My concern is the with the changes to "Apple signing" [1] in 10.9.5 (and
> planned for 10.10) There should no no peers (siblings) of the "Contents"
> directory but everything should be under "Contents" in proscribed
> directories. (And, this part of "new spec" seems true of all 10.9 versions,
> not just 10.9.5 ... or, perhaps just starting to enforce? Which might be
> enforced more in 10.10?) 

I share this concern, but Mozilla decided to put things into a MozResources folders at the same level (i.e., a sibling to Contents and Resources). Therefore, I think the approach is valid for now. 

Anyone with a 10.10 master build to test things?
Comment 13 David Williams CLA 2014-10-13 15:36:03 EDT
(In reply to Gunnar Wagenknecht from comment #12)
> (In reply to David Williams from comment #11)
> > My concern is the with the changes to "Apple signing" [1] in 10.9.5 (and
> > planned for 10.10) There should no no peers (siblings) of the "Contents"
> > directory but everything should be under "Contents" in proscribed
> > directories. (And, this part of "new spec" seems true of all 10.9 versions,
> > not just 10.9.5 ... or, perhaps just starting to enforce? Which might be
> > enforced more in 10.10?) 
> 
> I share this concern, but Mozilla decided to put things into a MozResources
> folders at the same level (i.e., a sibling to Contents and Resources).
> Therefore, I think the approach is valid for now. 
> 

Well, I think they've admitted various places it's not exactly valid, but "might work for minimum support". And, some specific cases discussed in their newsgroup, 

https://groups.google.com/forum/#!topic/mozilla.dev.platform/RGHGZuvQn24

And, I'm sure there will be more as more in learned. 

Plus, as I mentioned, with Apple's OSX complicated system, "just because it works" is not a very good test of if it is correct (or will work for others, or in the future). 

= = = = = 

@Doug, yes, I agree, one "model" is that if the App is installed by "root" so users can not change it, then I think users get their own copy of updates and user customizations under ~/.eclipse (much like Linux "multi-user installations"). This might "work" for Mac's too ... but, would not be the "intuitive experience" we are looking for, I don't believe. 

= = = = =

My purpose in posting my concerns was just to say we'd need someone who is a "real" Apple developer to help with some of these issues -- or put another way, I no longer believe there is a "quick fix" by simply changing a few build parameters -- especially if it "worked on the surface", but might result in even more bugs (or even blocked some "extenders or adopters") down the road.
Comment 14 Doug Schaefer CLA 2014-10-13 15:56:52 EDT
+1. I think the requirement is really to structure Eclipse as a standard Mac App bundle. The big thing is understanding what that means. There are other similar apps out there, Mozilla aside. We just need to do the leg work to figure this all out. And, yes, it's not going to be a quick fix. 

Thanks David for all your work on this. I've learned a lot, or at least enough to know that I that there is more to know :)
Comment 15 Doug Schaefer CLA 2014-10-13 23:08:45 EDT
BTW, found this very interesting on my Mac. Firefox fails while Chrome passes. But Chrome keeps everything in the user's ~/Library directory. Very little other than a launcher in the .app dir.

Dougs-Machine:Applications dschaefer$ spctl -a -t exec -vv Firefox.app
Firefox.app: rejected
source=obsolete resource envelope
origin=Developer ID Application: Mozilla Corporation
Dougs-Machine:Applications dschaefer$ spctl -a -t exec -vv Google\ Chrome.app/
Google Chrome.app/: accepted
source=Developer ID
origin=Developer ID Application: Google Inc.

And, hell, NetBeans isn't even signed...

Dougs-Machine:NetBeans dschaefer$ spctl -a -t exec -vv NetBeans\ 8.0.1.app
NetBeans 8.0.1.app: rejected
source=no usable signature
Comment 16 Gunnar Wagenknecht CLA 2014-10-14 13:58:15 EDT
(In reply to Doug Schaefer from comment #15)
> BTW, found this very interesting on my Mac. Firefox fails while Chrome
> passes. But Chrome keeps everything in the user's ~/Library directory. Very
> little other than a launcher in the .app dir.

I had the same thought. It should be possible to use the ~/Library directory for writable stuff.
Comment 17 David Williams CLA 2014-10-15 10:38:31 EDT
Changing title to make clear the goal is a "proper Mac App" not necessarily a "single directory" since that doesn't appear to be the proper "Mac solution". (i.e. simply trying to state the issue, not any particular solution).
Comment 18 Matthias Sohn CLA 2014-10-15 16:32:15 EDT
(In reply to Gunnar Wagenknecht from comment #16)
> (In reply to Doug Schaefer from comment #15)
> > BTW, found this very interesting on my Mac. Firefox fails while Chrome
> > passes. But Chrome keeps everything in the user's ~/Library directory. Very
> > little other than a launcher in the .app dir.
> 
> I had the same thought. It should be possible to use the ~/Library directory
> for writable stuff.

The Apple file system programming guide [1] gives an overview on the Apple filesystem and also gives guidelines for filesystem layout.

Section "The Library Directory Stores App-Specific Files" [2] says that Application configuration files should go under /Library/Application Support for system wide configuration and under ~/Library/Application Support for user specific configuration

[1] https://developer.apple.com/library/mac/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html
[2] https://developer.apple.com/library/mac/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW1
Comment 19 James Watkins-Harvey CLA 2014-11-21 17:42:22 EST
Hi everyone,


I attached a shell script that I'm currently writing to package Eclipse as an all in-one os x application bundle. It notably includes a private copy of Oracle's JSE 8 JRE, and configure all Equinox and P2 path variables so that they satisfy (well, I believe) all of Apple's requirements for a standard application (it won't work in sandboxed mode, ans therefore on the Mac App Store, because I assume the Application Support folder to be at it's non-sandboxed location; sandboxed applications would have a randomized path, which can only be retrieved at runtime, using proper OS X API).

This is work in progress, obviously, and much work still needs to be done. I haven't tried yet signing the resulting application yet, and building a DMG for final distribution is definitely the way to go. I have not tried neither installing new features using P2.

Now, just for the record, the previous assertions are based mainly on my own personal experience (been developing mainly on OS X for the last 10 years), and light reading of Apple's tech notes and guidelines over the years. I have not tested the resulting throughly on multiple computers, have not read thoroughly every single document published by Apple, and have not discussed with anyone else who might have other informations.


-James Watkins-Harvey
Comment 20 James Watkins-Harvey CLA 2014-11-21 17:44:25 EST
Created attachment 248833 [details]
Shell Script to package Eclipse as an OS X application bundle
Comment 21 Gunnar Wagenknecht CLA 2014-11-23 11:35:58 EST
(In reply to James Watkins-Harvey from comment #20)
> Created attachment 248833 [details]
> Shell Script to package Eclipse as an OS X application bundle

Cool! Thanks James for sharing.

(In reply to James Watkins-Harvey from comment #19)
> ... (it won't work in sandboxed mode, ans therefore on the Mac App
> Store, because I assume the Application Support folder to be at it's
> non-sandboxed location; sandboxed applications would have a randomized path,
> which can only be retrieved at runtime, using proper OS X API).

That sounds like a task for the launcher. It's a Mac binary. It could set an environment var or pass it to Eclipse to configure the install location as well as other p2 paths. Do you have more details on this?
Comment 22 James Watkins-Harvey CLA 2014-11-24 10:54:26 EST
(In reply to Gunnar Wagenknecht from comment #21)
> > ... (it won't work in sandboxed mode, ans therefore on the Mac App
> > Store, because I assume the Application Support folder to be at it's
> > non-sandboxed location; sandboxed applications would have a randomized path,
> > which can only be retrieved at runtime, using proper OS X API).
> 
> That sounds like a task for the launcher. It's a Mac binary. It could set an
> environment var or pass it to Eclipse to configure the install location as
> well as other p2 paths. Do you have more details on this?

I absolutely agree: most of this has to be moved to the launcher, and have already begun some work on this. The shell script was only an initial exploration step in that direction. But first, I indeed have more insights to share from what I have learnt up to this point.

But first of all, given that there a lot of open bugs (and confusion, really) related to Eclipse IDE/RCP/Equinox/SWT packaging and behaviour on OS X, let me first recap some of the most critical issues that are to be resolved to achieve a "proper Mac App".

1. Make it possible to package a complete Eclipse installation as a single OS X Application bundle (for example Eclipse.app, without any sibling files or directory). This part is already functional, assuming that correct path variables are provided on as command line arguments (potentially through Info.plist or eclipse.ini).

2. Support the use of a _private_ Java JRE 7 or 8 installation, without requirement for installation Apple's Java JRE 6 package. Requiring a public JRE (no matter the version) is incompatible with the notion of a "proper Mac App", and not using deprecated frameworks is a requirement for apps to be distributed on the Mac App Store. The current launcher links to Apple's OS X Java System Framework, which has been deprecated. Now, for some unexplained reasons, Oracle's JRE (and maybe OpenJDK, haven't verified) does not contains the JNI_CreateJavaVM symbol in expected files, but instead in jre/lib/jli/libjli.dylib. Last think on this point, Oracle's .jre package can't be included as-is inside a Eclipse.app application bundle because the .jre package contains internal symlinks, which appears not be acceptable for the new security-signing process. Doing some cleanup and restructuration inside the .jre package make it safe for embedding purpose. So overall, I think this point is mostly resolved, except for the removal of references to the Java Framework in the launcher native code. 

3. Define an alternative method for user settings that affect the launcher (that is mostly JVM memory and optimization settings). Having users modify themselves the eclipse.ini file (that is Eclipse.app/Contents/MacOS/eclipse.ini) is definitely not the way to go. It could be acceptable to copy that file to the user's application support directory, and then read that file rather than the default one, but since that file contains the exact file name (including the timestamp qualified version number) of the launcher companion bundle, the launcher would fail to launch whenever the application is updated. A better solution would be to store inside a user+application specific location (could be application support, but preferences would be better purposed) only arguments that have been specifically altered by the user (that is, in general, min and max memory size). Anyway, the same should be true on all platforms, since having these settings inside the Eclipse directory means that these configurations are lost every time users do a major update. We should also review the process of concatenating arguments from various sources (eclipse.ini, .plist, command line, and now preferences). Right now, there seems to be a problem when one of those contains the "-vmargs" argument, preventing correct handling of arguments specified in following sources. There again, this is mostly modifications to native launcher's code. I plan to work on this soon. Some private API might also be required to allow modification of these settings from Java code, for example to support a preference panel).

4. Correctly support application updates and new feature installations (for "proper Mac App" and "regular sandboxed app" use cases; not for the "Mac App Store" use case, see point 5). The all-inclusive application bundle must be treated as read only. It doesn't mean though that P2 installation and updates can't be supported. New bundles, as well as configuration and touch points modifications, can all be stored inside the application support directory (either in the user domain or local domain, depending on either the features are to be made available to that single user or all users). Unfortunately, P2 only support a single install area, and can't handle new bundle installation when that location is read only. Also, when the install area point to a directory inside the .app, P2 creates a new Eclipse.app/Contents/MacOS/eclipse.ini structure INSIDE of the existing MacOS directory (that is Eclipse.app/Contents/MacOS/Eclipse.app/Contents/MacOS/eclipse.ini). I have not yet investigated the work to be done related to this.

5. Support the ability to easily remove or totally disable P2 from Eclipse (for the "Mac App Store" use case). Applications distributed on the Mac App Store can't contains mechanisms allow installation of new features, and can't perform self update. In short, it is probable that Apple would deny an app that contains P2, even if it is not accessible through the UI, if it is still possible that P2 might be used to modify in anyway the set of installed features. Consider the fact that P2 can be invoked as an app from the command line, that installation history can be viewed and reverted through  Eclipse's About box, and so on... I think that might be a though one to resolve, and don't plan to personally work on this one.

Well, my time is up for now. I will come back later with a few more issues to be shared, but at this point, let me sum all of this: packaging Eclipse IDE/RCP as a proper Mac application bundle is achievable. Making such an app work correctly with Sandbox constraints might or might not be easy, depending on the nature of a specific Eclipse application (Eclipse IDE would not be a good candidate for being sandboxed). Delivering a Sandboxed Eclipse RCP application through the Mac App Store might eventually be possible, but probably require much more work.
Comment 23 Gunnar Wagenknecht CLA 2014-11-24 12:46:43 EST
(In reply to James Watkins-Harvey from comment #22)
> 1. Make it possible to package a complete Eclipse installation as a single
> OS X Application bundle (for example Eclipse.app, without any sibling files
> or directory). 

Did you do any modifications for this? I was able to make this working by changing a few settings in my Tycho product build. That is a fully app bundle which can be signed and works *pre* Yosemite. 

For Yosemite, a few additional changes are required (as your are writing below).

> Doing some cleanup and restructuration
> inside the .jre package make it safe for embedding purpose. 

That's interesting. It should probably be documented on the wiki and/or discussed with the OpenJDK project.

> 3. Define an alternative method for user settings that affect the launcher
> (that is mostly JVM memory and optimization settings). Having users modify
> themselves the eclipse.ini file (that is
> Eclipse.app/Contents/MacOS/eclipse.ini) is definitely not the way to go. 

I think we can reduce the work here a bit by doing more convention over configuration. The launcher.ini should not be modified manually at all. The memory and optimization settings are a leftover of the past. These days, JREs do all those things automatically. We should therefore not specify those settings anymore.

> Right now, there seems to be a
> problem when one of those contains the "-vmargs" argument, preventing
> correct handling of arguments specified in following sources.

I don't think it's a problem. This is by design. If "-vmargs" is given on the command line it overrides *any* vmargs specified in the launcher ini. I believe the plist thing on Mac is like a "default command line". We should probably not specify any vmargs there.

> Unfortunately, P2 only support a single install area, and can't handle new
> bundle installation when that location is read only.

On Linux, p2 supports a split mode whereas a super user (root) installs Eclipse into /usr/... which is not writable by the regular user. Therefore p2 creates and maintains a private profile for the specific user. It combines that with the base profile. AFAIK they also support updates done by the root user in the read-only shared install location.
Comment 24 James Watkins-Harvey CLA 2014-11-24 15:16:40 EST
(In reply to Gunnar Wagenknecht from comment #23)
> (In reply to James Watkins-Harvey from comment #22)
> > 1. Make it possible to package a complete Eclipse installation as a single
> > OS X Application bundle (for example Eclipse.app, without any sibling files
> > or directory). 
> 
> Did you do any modifications for this? I was able to make this working by
> changing a few settings in my Tycho product build. That is a fully app
> bundle which can be signed and works *pre* Yosemite. 

As I said previously, "this part is already functional". Note that I have not yet tried producing the application bundle directly from Eclipse; at this point, I use my shell script to repackage a regular eclipse directory (as would be generated through either Tycho, PDE Product Export or whatever else mechanism).

> For Yosemite, a few additional changes are required (as your are writing
> below).
> 
> > Doing some cleanup and restructuration
> > inside the .jre package make it safe for embedding purpose. 
> 
> That's interesting. It should probably be documented on the wiki and/or
> discussed with the OpenJDK project.

It should, indeed, but I have the feeling that the issue here is something around the line of: either comply with Apple's Java Framework way of doing things, or comply with the way things work outside. The problem is that Oracle doesn't provide a JRE that is intended specifically to be embedded, even though they recommend doing so. The problematic symbol (JNI_CreateJavaVM) is inside jre/lib/jli/libjli.dylib, instead of the locations where it is expected to be. When installed as a public JRE, that library is symlinked as the main entry point (have a look at /Library/Java/JavaVirtualMachines/*.{jdk,jre}/Contents/MacOS/libjli.dylib). So Apple's framework must be expecting this, and provide the missing part to ensure that that symbol is visible to applications that use the Java framework. Now, when installed in a private location, we are not going through Apple Java Framework, so that symbol is not visible unless we explicitly refer to that library, and the symlink under MacOS/ is not accepted by the new signing process. Simply remove the symlink, point to the libjli.dylib library, and everything works fine.

> > 3. Define an alternative method for user settings that affect the launcher
> > (that is mostly JVM memory and optimization settings). Having users modify
> > themselves the eclipse.ini file (that is
> > Eclipse.app/Contents/MacOS/eclipse.ini) is definitely not the way to go. 
> 
> I think we can reduce the work here a bit by doing more convention over
> configuration. The launcher.ini should not be modified manually at all.

I do agree on this.

> memory and optimization settings are a leftover of the past. These days,
> JREs do all those things automatically. We should therefore not specify
> those settings anymore.

This ain't true, unfortunately. The min and max heap space are particularly problematic in Java. I have clients where these settings needs to be set to values in the range 2gb-4gb only to be able to handle some large data sets. But if I set these values as default, then the garbage collector will tolerate huge waste in heap usage even for customers that have minimal needs. There are also several situations that require passing in some system properties in order to properly access some Java features that turns out to be absolutely required in some specific environment.

The problem here is more about the fact that the "eclipse.ini" is, at the same time, a "product-definition file", a "default user configuration file", and a "customized user configuration file". There ought to be distinct locations for each of these purposes.

> > Right now, there seems to be a
> > problem when one of those contains the "-vmargs" argument, preventing
> > correct handling of arguments specified in following sources.
> 
> I don't think it's a problem. This is by design. If "-vmargs" is given on
> the command line it overrides *any* vmargs specified in the launcher ini. I
> believe the plist thing on Mac is like a "default command line". We should
> probably not specify any vmargs there.

Indeed, the fact that -vmargs specified on the command line "reset" arguments defined in the eclipse.ini makes sense for memory and optimization settings. But what about system properties (think of -Dxxxxx=..., -Xdock:icon, -XstartOnFirstThread, ...)? As for the plist being used as a default command line is not an OS X behaviour, but a mechanism coded in the Laucher's code (see https://git.eclipse.org/c/equinox/rt.equinox.framework.git/tree/features/org.eclipse.equinox.executable.feature/library/carbon/eclipseCarbonMain.c#n88). Finally, my comment was about the fact that specifying -vmargs sometimes prevents some non-vm arguments from other sources from being parsed correctly, preventing Eclipse from launching. I think the following bug is related to this: https://bugs.eclipse.org/bugs/show_bug.cgi?id=427318 ; I also encountered this issue personally several times, but that's something that will need more investigation.

> > Unfortunately, P2 only support a single install area, and can't handle new
> > bundle installation when that location is read only.
> 
> On Linux, p2 supports a split mode whereas a super user (root) installs
> Eclipse into /usr/... which is not writable by the regular user. Therefore
> p2 creates and maintains a private profile for the specific user. It
> combines that with the base profile. AFAIK they also support updates done by
> the root user in the read-only shared install location.

Right, that's exactly the mechanism I used as a base for my packaging. However, users can't update nor install new features themselves, except by manually dropping bundles to a dropins directory. Now, given that the Eclipse.app bundle should not be altered even by an administrator (because doing so will break the bundle's signature), and that it is definitely not acceptable to have users redownload a complete, new application every time they want to install a single new feature, then enabling users to install new features using P2's UI into a private profile is definitely a requirement.
Comment 25 Gunnar Wagenknecht CLA 2014-11-25 02:07:02 EST
James,

Would it make more sense to start collecting requirements for Mac OS bundling and launching rework in a wiki page and then start implementing the new strategy in the launcher instead of trying to find and fix individual issues in the launcher one-by-one?

See here for an example:
https://wiki.eclipse.org/Equinox_p2_Shared_Install

-Gunnar
Comment 26 James Watkins-Harvey CLA 2014-11-26 10:35:23 EST
(In reply to Gunnar Wagenknecht from comment #25)
> Would it make more sense to start collecting requirements for Mac OS
> bundling and launching rework in a wiki page and then start implementing the
> new strategy in the launcher instead of trying to find and fix individual
> issues in the launcher one-by-one?

I agree, it might be better this way. I'll try to draft something on the wiki in the next few days.
Comment 27 Kaloyan Raev CLA 2015-01-16 09:44:52 EST
(In reply to James Watkins-Harvey from comment #20)
> Created attachment 248833 [details]
> Shell Script to package Eclipse as an OS X application bundle

Thanks for this script! It really helped me a lot to bundle a RCP app as a Mac app.

I made one important modification - I packed the JRE under Resources instead of MacOS folder. If it is packed under MacOS then I cannot sign the application, because only the executable file is expected under MacOS.
Comment 28 Marcel Austenfeld CLA 2015-01-17 11:53:22 EST
(In reply to Kaloyan Raev from comment #27)
> (In reply to James Watkins-Harvey from comment #20)
> > Created attachment 248833 [details]
> > Shell Script to package Eclipse as an OS X application bundle
> 
> Thanks for this script! It really helped me a lot to bundle a RCP app as a
> Mac app.
> 
> I made one important modification - I packed the JRE under Resources instead
> of MacOS folder. If it is packed under MacOS then I cannot sign the
> application, because only the executable file is expected under MacOS.



Hello,

is it possible to share the rcp *.ini file here or the adapted script. I still have problems to bundle a jre under Macosx Yosemite. For my rcp product i need to bundle the jre 1.8.05.
Comment 29 Kaloyan Raev CLA 2015-01-19 02:42:37 EST
> is it possible to share the rcp *.ini file here or the adapted script. I
> still have problems to bundle a jre under Macosx Yosemite. For my rcp
> product i need to bundle the jre 1.8.05.

There is no *.ini file. If you examine the script you will see that all info that is usually found in the *.ini file is now in the Info.plist file.

The script attached by James works fine as is on OS X 10.9.5. I just had problems with signing the app afterwards (using a different script) and I resolved it as I explained in comment 27.
Comment 30 Marcel Austenfeld CLA 2015-01-19 14:05:28 EST
(In reply to Kaloyan Raev from comment #29)
> > is it possible to share the rcp *.ini file here or the adapted script. I
> > still have problems to bundle a jre under Macosx Yosemite. For my rcp
> > product i need to bundle the jre 1.8.05.
> 
> There is no *.ini file. If you examine the script you will see that all info
> that is usually found in the *.ini file is now in the Info.plist file.
> 
> The script attached by James works fine as is on OS X 10.9.5. I just had
> problems with signing the app afterwards (using a different script) and I
> resolved it as I explained in comment 27.

Ok. Thanks for the info.
Comment 31 Pascal Rapicault CLA 2015-02-10 21:25:52 EST
I'll be working on this in the coming weeks with the goal of shipping Mars in this new layout.

For this release, my focus is on making sure that:
- Eclipse can be shipped as a proper mac app, 
- plug-ins can be installed / removed (through the UI and the director)
- shared-install work
- make sure that the app can be signed and signature validation works
- produce a .dmg
Comment 32 Marcel Austenfeld CLA 2015-02-11 04:12:23 EST
(In reply to Pascal Rapicault from comment #31)
> I'll be working on this in the coming weeks with the goal of shipping Mars
> in this new layout.
> 
> For this release, my focus is on making sure that:
> - Eclipse can be shipped as a proper mac app, 
> - plug-ins can be installed / removed (through the UI and the director)
> - shared-install work
> - make sure that the app can be signed and signature validation works
> - produce a .dmg

That sounds great.  

In addition it would be nice to bundle a selected JRE with the .dmg.

In my case i have to bundle a JRE because of the licensing issues regarding the use and availability of e.g. tools.jar and the jfxswt.jar inside a(optimal) JRE.

It would require additional efforts to let the users install the required libs (and JRE).
Comment 33 Christian Georgi CLA 2015-02-11 04:36:24 EST
(In reply to Pascal Rapicault from comment #31)
> For this release, my focus is on making sure that:
> - Eclipse can be shipped as a proper mac app, 

Pascal, please also not the dependency to bug 455732 (Eclipse platform) and bug 457234 (P2) that currently block Eclipse itself being shipped in macosx-bundled mode.
Comment 34 Pascal Rapicault CLA 2015-02-11 10:13:57 EST
> In addition it would be nice to bundle a selected JRE with the .dmg.
   Handling of the JRE 
> 
> In my case i have to bundle a JRE because of the licensing issues regarding
> the use and availability of e.g. tools.jar and the jfxswt.jar inside
> a(optimal) JRE.
> 
> It would require additional efforts to let the users install the required
> libs (and JRE).
   The bundling of the JRE is currently not in my scope. If contributions are coming for this while I'm working on the support, then I'll be happy to incorporate those.

> Pascal, please also not the dependency to bug 455732 (Eclipse platform) and
> bug 457234 (P2) that currently block Eclipse itself being shipped in
> macosx-bundled mode.
   Tycho work is implicitly included since the platform uses this.

Also please note that the old layout will no longer be supported.
Comment 35 Doug Schaefer CLA 2015-02-11 11:29:18 EST
Thanks Pascal! I look forward to following and helping test this.
Comment 36 Pascal Rapicault CLA 2015-02-11 12:31:58 EST
Here is the layout that I'm going to try to create:

Eclipse.app/
    Info.plist
    Contents/
        MacOS/
            eclipse
            eclipse.ini
        Resources/
            eclipse.icns
        configuration/
            config.ini
            ...
        plugins/
            ...
        features/
            ...
        artifacts.xml

I'm not sure if the eclipse.ini needs to be moved or not.
What I like about this layout is that it is almost identical to the layout on the other platforms and it will minimize bad surprises for code that walks the filesystem.

@James: The script you provided generates a very different layout. Was this layout driven by technical reasons (e.g. the signature tool) or a desire to separate concerns.
Comment 37 Doug Schaefer CLA 2015-02-11 13:28:33 EST
(In reply to Pascal Rapicault from comment #36)
> Here is the layout that I'm going to try to create:
> 
> Eclipse.app/
>     Info.plist
>     Contents/
>         MacOS/
>             eclipse
>             eclipse.ini
>         Resources/
>             eclipse.icns
>         configuration/
>             config.ini
>             ...
>         plugins/
>             ...
>         features/
>             ...
>         artifacts.xml

Like it. Matches Xcode and pretty much every other app I have installed.

Others though have the Info.plist in the Contents folder as well. My guess is that it actually has to be in there. Why do you have it at the top?
Comment 38 Pascal Rapicault CLA 2015-02-11 13:29:26 EST
> Others though have the Info.plist in the Contents folder as well. My guess
> is that it actually has to be in there. Why do you have it at the top?
   This was a typo, sorry for the confusion.
Comment 39 Justin Dolezy CLA 2015-02-11 16:17:04 EST
Are we not going to have signing problems if the following aren't in the Resources folder?

>         configuration/
>             config.ini
>             ...
>         plugins/
>             ...
>         features/
>             ...
>         artifacts.xml

We structure our .app so that these are in Resources\Java in fact, and have no issues with the 'new' Gatekeeper codesign rules (introduced with Mavericks or Yosemite)
Comment 40 Tobias Bertelsen CLA 2015-02-11 17:21:43 EST
Great you are getting on this Pascal.

I have noticed that your proposed folder layout is different from the one you'll currently get if installing a product through the p2-director. The p2 director puts everything in the root of the .app folder (which is actually not that OSX'y)

I will suggest that you try hard, to make Eclipse just use the default p2 installation. If you find that there are issues with the current way the p2 director install products, I hope you will fix the root problem in the p2 director, and not make a quick-fix for Eclipse only.

We are many people developing RCP apps, that would benefit immensely from the p2-director worked out-of-the-box with osx – including signing as several people have mentioned. It is likely that everyone else are having the same problems you might encounter, and fixing them for everyone would benefit the platform.

It be pretty poor advertising for p2 and OSGi, if it can't even handle its flagship Eclipse on OSX, without a bunch of tweaking.

In other words, go for something that is not much more than:

eclipsec.exe -application org.eclipse.equinox.p2.director 
   -repository http://download.eclipse.org/releases/mars/,...
   -installIU org.eclipse.rcp,...
   -destination ./eclipse.app


Thanks again for giving this a shot.
Comment 41 James Watkins-Harvey CLA 2015-02-11 22:13:58 EST
> @James: The script you provided generates a very different layout. Was this
> layout driven by technical reasons (e.g. the signature tool) or a desire to
> separate concerns.

Well, yes and no. Basically, the OS X bundle singing architectue allows for "nested code", which is an independently signed bundle stored inside a parent bundle. The internal bundle can then be replaced by a different version (either an update or an alternative, as long as it is signed using the same certificate as the original internal bundle), without breaking the outer bundle signature. The layout I used in my original script was a tentative of a possible way to allow Eclipse base installation upgrade... But this is all very incertain, and I have had very little free time recently to push my tests any further on this. So at this point, you'd better go with whatever layout that will be easiest to maintain.

> I'm not sure if the eclipse.ini needs to be moved or not.
> What I like about this layout is that it is almost identical to the layout
> on the other platforms and it will minimize bad surprises for code that
> walks the filesystem.

Apple's rules forbid non-code content inside the MacOS directory. So the .ini file must be moved outside. Anyway, having the .ini file next to the configuration, plugins and features directories is the standard layout on other platforms, isn't it? For some reasons though, I feel that all of these should be put inside a parent directory, lets say "Eclipse.app/Contents/Eclipse" or "Eclipse.app/Contents/Equinox".  Basically, that inner directory contains exactly the files and directories that would otherwise be siblings of the .app bundle (or .exe on Windows). I think grouping things this way would reduce risks from OS X specific files "conflicting" in whatever way with Eclipse-specific files...
Comment 42 Pascal Rapicault CLA 2015-02-13 11:02:10 EST
(In reply to James Watkins-Harvey from comment #41)
> > @James: The script you provided generates a very different layout. Was this
> > layout driven by technical reasons (e.g. the signature tool) or a desire to
> > separate concerns.
> 
> Well, yes and no. Basically, the OS X bundle singing architectue allows for
> "nested code", which is an independently signed bundle stored inside a
> parent bundle. The internal bundle can then be replaced by a different
> version (either an update or an alternative, as long as it is signed using
> the same certificate as the original internal bundle), without breaking the
> outer bundle signature. The layout I used in my original script was a
> tentative of a possible way to allow Eclipse base installation upgrade...
> But this is all very incertain, and I have had very little free time
> recently to push my tests any further on this. So at this point, you'd
> better go with whatever layout that will be easiest to maintain.
    In that direction, you may be interested in looking at what Fedora does to deliver Eclipse. They allow a user to install a feature (e.g. CDT, PDE, etc.) through the OS package manager, and have this feature be directly "hooked" in the main install of Eclipse. The OSGi bundles to run are discovered at runtime by looking for certain files in known places. I don't remember how they keep p2 happy but I know they do keep it happy :)
   The downside to this approach is that of course Tycho does not produce the layout for this, and the provider of the OS packages needs to make sure that the dependencies are handled properly.

> > I'm not sure if the eclipse.ini needs to be moved or not.
> > What I like about this layout is that it is almost identical to the layout
> > on the other platforms and it will minimize bad surprises for code that
> > walks the filesystem.
> 
> Apple's rules forbid non-code content inside the MacOS directory. So the
> .ini file must be moved outside. Anyway, having the .ini file next to the
> configuration, plugins and features directories is the standard layout on
> other platforms, isn't it? For some reasons though, I feel that all of these
> should be put inside a parent directory, lets say
> "Eclipse.app/Contents/Eclipse" or "Eclipse.app/Contents/Equinox". 
> Basically, that inner directory contains exactly the files and directories
> that would otherwise be siblings of the .app bundle (or .exe on Windows). I
> think grouping things this way would reduce risks from OS X specific files
> "conflicting" in whatever way with Eclipse-specific files...
    I realized the issue in the eclipse.ini shortly after sending my note :)
I'll try to deliver Eclipse in a subfolder of the Contents folder.
Comment 43 Eclipse Genie CLA 2015-03-03 10:18:28 EST
New Gerrit change created: https://git.eclipse.org/r/43083
Comment 45 Pascal Rapicault CLA 2015-03-04 14:36:16 EST
Here is an update of the progress so far:
I've just pushed the first batch of changes to p2. With these changes (and some hackeries), I have been able to produce a build of the Platform with the following layout.
 /Eclipse.app/
    Contents/
        Info.plist
        Eclipse/
            eclipse.ini
            configuration/
            plugins/
                ...
        MacOS/
            eclipse

This layout supports the installation of new plug-ins, supports the installation of new plug-in when the Bundle is read-only, PDE can find plug-ins from the running instance, etc.

It will probably take another couple days before we are able to make this initial set of goodness available to everybody (so you don't have to do the hackeries) since we need to do changes in Tycho, and then spin an Eclipse build.
Comment 46 David Williams CLA 2015-03-04 17:28:30 EST
(In reply to Pascal Rapicault from comment #45)
> Here is an update of the progress so far:
> I've just pushed the first batch of changes to p2. With these changes (and
> some hackeries), I have been able to produce a build of the Platform with
> the following layout.
>  /Eclipse.app/
>     Contents/
>         Info.plist
>         Eclipse/
>             eclipse.ini
>             configuration/
>             plugins/
>                 ...
>         MacOS/
>             eclipse
> 
> This layout supports the installation of new plug-ins, supports the
> installation of new plug-in when the Bundle is read-only, PDE can find
> plug-ins from the running instance, etc.

And do we (i.e. you :) have a plan for "modifying eclipse.ini"? 
Or, is there a well known way to do that, in a "shared install" too?
Comment 47 Pascal Rapicault CLA 2015-03-04 20:08:12 EST
> And do we (i.e. you :) have a plan for "modifying eclipse.ini"? 
> Or, is there a well known way to do that, in a "shared install" too?
   By default shared installs always write a modified eclipse.ini in the configuration folder (normally located in ~/.eclipse/<someId>_<someVersion>_<hashcode>) but it is never read because the launcher does not have enough information to figure the location of the configuration folder.
For mac only, I'm thinking about defaulting the configuration location to a known place (~/System/Application Support/<BundleName>/) which will then make this location computable by the launcher, and thus it will be able to load the eclipse.ini.

The downsides to this approach are:
- if there are multiple eclipse installs then they will share the same configuration folder which will cause interferences (whereas with the generated folder name this would not have happened).
- when the installation is renamed (e.g. from Eclipse.app to Eclipse2.app) then the configurations stored in BundleName (which include plug-ins) won't be migrated.
Comment 48 Markus Keller CLA 2015-03-05 09:55:50 EST
(In reply to Eclipse Genie from comment #44)
> Gerrit change https://git.eclipse.org/r/43083 was merged to [master].
> Commit:
> http://git.eclipse.org/c/equinox/rt.equinox.p2.git/commit/?id=1b96ce896c49151b0e20fa49ba680d08415cca8f

This causes a compile error in master of InfoPListEditor.java:
The constructor IOException(String, Exception) is undefined

It probably doesn't show up in the N-build because Tycho only looks at the BREE. But the Eclipse project still has 1.5 on the build path and the project's Java Compiler properties are still set to 1.5. Please click "Update the classpath settings" on the Overview tab of the MANIFEST.MF editor.
Comment 49 James Watkins-Harvey CLA 2015-03-05 12:27:54 EST
Pascal,

These files really ought to be stored inside the Application Support directory; more over, the path _has_ to be determined through the proper OS X semantic (see https://developer.apple.com/library/mac/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/AccessingFilesandDirectories/AccessingFilesandDirectories.html#//apple_ref/doc/uid/TP40010672-CH3-SW7). Following Apple’s specification, its the App Bundle identifier that should be used inside the Application Support directory, so renaming the application should not present any issue. As for the possibility of installing multiple copies of an application, Apple recommends that applications assign themselves a UUID, attached to a specific extended file system attribute on the .app directory, and that this UUID be then used in whatever way that might be appropriate when manipulating preferences and application support files. See here: https://developer.apple.com/library/mac/technotes/tn2206/_index.html#//apple_ref/doc/uid/DTS40007919-CH1-TNTAG401. Both of these could « easily » be done by the native launcher; the computed paths would then need to be provided to the Java counterpart in some way.

The main downside of this approach is that reinstalling Eclipse (either the same version or a newer one) would generate a new instance UUID, so any files stored in the application support directory would then be ignored, and worst, might then never be cleaned up. It might therefore be worth investigating eventually the possibility of « sharing » bundle files inside the application support directory, P2 repositories caches, and some other files, so that « instance » specific directories contains very little. But let’s not think too far away: that’s not something that needs be considered right now.


James
Comment 50 Pascal Rapicault CLA 2015-03-09 11:12:22 EDT
Hi James, thanks for the specific pointers. I started looking at these and I agree that using those should not be too hard. The more complicated part will be to decide whether user specific values are stored in a specific files, or in the eclipse.ini and how the merge should be handled if we decide to do it in the eclipse.ini.
The other thing that I'm not a fan of is that it adds a lot more logic (e.g. specifying a configuration, checking if the user specified one, etc) to the launcher which we always tried to shy away from. I've opened bug #461728 and #461725.
At this point, I'm not sure how much of this I will be able to do for the release so help is definitely welcome.
Comment 51 Pascal Rapicault CLA 2015-03-09 11:27:16 EDT
Thanks to David Williams' help, I'm happy to report that a first experimental build is available here:
http://build.eclipse.org/eclipse/builds/4X/siteDir/eclipse/downloads/drops4/X20150308-1647/

Since the build is not yet signed, you will need to right click on the bundle, select open and ignore the security warning.

Let me know what you think.
Comment 52 Gunnar Wagenknecht CLA 2015-03-09 14:42:06 EDT
Pascal, the build works for me. Now signing is probably the harder part.

I was wondering why it's displayed as "eclipse.app" (lower case e) in the Finder but as "Eclipse.app" in terminal. Could it be that the application name is lower case "eclipse"?
Comment 53 Pascal Rapicault CLA 2015-03-09 15:55:30 EDT
Thanks for checking Gunnar.
(In reply to Gunnar Wagenknecht from comment #52)
> Pascal, the build works for me. Now signing is probably the harder part.
  It is hard because the infrastructure does not make it easy, otherwise it is just a couple commands :)

> I was wondering why it's displayed as "eclipse.app" (lower case e) in the
> Finder but as "Eclipse.app" in terminal. Could it be that the application
> name is lower case "eclipse"?
   I noticed that as well. I need to investigate but I think this is related to the casing of a key in the info.plist file.
Comment 54 James Watkins-Harvey CLA 2015-03-09 16:06:40 EDT
Have a look at the value of key "CFBundleDisplayName" in plist.info. It most certainly contains the text "eclipse", with lowercase "e". The Finder use that value (which support localization), where as the terminal use the actual file name in the filesystem.


James


(In reply to Pascal Rapicault from comment #53)
> Thanks for checking Gunnar.
> (In reply to Gunnar Wagenknecht from comment #52)
> > Pascal, the build works for me. Now signing is probably the harder part.
>   It is hard because the infrastructure does not make it easy, otherwise it
> is just a couple commands :)
> 
> > I was wondering why it's displayed as "eclipse.app" (lower case e) in the
> > Finder but as "Eclipse.app" in terminal. Could it be that the application
> > name is lower case "eclipse"?
>    I noticed that as well. I need to investigate but I think this is related
> to the casing of a key in the info.plist file.
Comment 55 Pascal Rapicault CLA 2015-03-09 16:21:53 EDT
Bingo! Thanks James. In fact the key is not even in the plist file.
James, would you mind checking if we are missing any other key from the plist file?
Comment 57 David Williams CLA 2015-03-09 16:27:35 EDT
(In reply to Pascal Rapicault from comment #56)
> Here is a link to the repo for the plist file
> http://git.eclipse.org/c/equinox/rt.equinox.framework.git/tree/features/org.
> eclipse.equinox.executable.feature/bin/cocoa/macosx/x86_64/Eclipse.app/
> Contents/Info.plist

And ... we do not actually use that one during build of Eclipse SDK (etc). 
(We used to, but Tycho does a better job of just creating one, without out out interference, so anything "missing" should be proposed to them ... that's my understanding, unless you mean they ask p2 for it, and that's where p2 gets it from?)
Comment 58 Pascal Rapicault CLA 2015-03-09 16:30:10 EDT
(In reply to David Williams from comment #57)
> (In reply to Pascal Rapicault from comment #56)
> > Here is a link to the repo for the plist file
> > http://git.eclipse.org/c/equinox/rt.equinox.framework.git/tree/features/org.
> > eclipse.equinox.executable.feature/bin/cocoa/macosx/x86_64/Eclipse.app/
> > Contents/Info.plist
> 
> And ... we do not actually use that one during build of Eclipse SDK (etc). 
> (We used to, but Tycho does a better job of just creating one, without out
> out interference, so anything "missing" should be proposed to them ...
> that's my understanding, unless you mean they ask p2 for it, and that's
> where p2 gets it from?)
  Tycho gets the executable.launcher feature (or whatever the name is) which includes a sample info.plist which is then branded by p2's BrandingIron.
It is my understanding that this launcher feature is created as part of the platform build and the sample info.plist is indeed the one I pointed at.
Comment 59 David Williams CLA 2015-03-09 16:38:25 EDT
(In reply to Pascal Rapicault from comment #58)

>   Tycho gets the executable.launcher feature (or whatever the name is) which
> includes a sample info.plist which is then branded by p2's BrandingIron.
> It is my understanding that this launcher feature is created as part of the
> platform build and the sample info.plist is indeed the one I pointed at.

You might be right, but suggest you confirm, as much has changed :) and ... it used to be "in" the "configuration feature" ... but, that was causing other problems, so I removed that, from our "products" (even though we still create it, for legacy reasons).
Comment 60 James Watkins-Harvey CLA 2015-03-10 09:32:58 EDT
Ok, after verification, the .plist (the one in yesterday's build) indeed doesn't contains key "CFBundleDisplayName", but contains "CFBundleName" which is used in the absence of the former. And it's value is indeed "eclipse" with a lower case "e". This value appears to origin from BrandingIron, around line 505: "infoPListEditor.setKey(InfoPListEditor.BUNDLE_KEY, name);". The variable "name" at that location is also used as the executable name, which is indeed lowercased.
Comment 61 David Williams CLA 2015-03-12 10:09:36 EDT
(In reply to James Watkins-Harvey from comment #60)
> Ok, after verification, the .plist (the one in yesterday's build) indeed
> doesn't contains key "CFBundleDisplayName", but contains "CFBundleName"
> which is used in the absence of the former. And it's value is indeed
> "eclipse" with a lower case "e". This value appears to origin from
> BrandingIron, around line 505:
> "infoPListEditor.setKey(InfoPListEditor.BUNDLE_KEY, name);". The variable
> "name" at that location is also used as the executable name, which is indeed
> lowercased.

See also bug 458484 for partially related issues about default values in .plist. 
(That is, if any of these is changed in BrandingIron, would make sense to fix it at same time)
Comment 62 James Watkins-Harvey CLA 2015-03-12 12:57:49 EDT
> See also bug 458484 for partially related issues about default values in
> .plist. 
> (That is, if any of these is changed in BrandingIron, would make sense to
> fix it at same time)

Indeed, this ought to be fixed at the same time, and it should not require much efforts. It seems at first that the main product identifier, or the EPP identifier when applicable, should be used as bundle identifier in the plist file. The application name however might be a little more complex. Let me consider these questions later today.

One a side note, though, I get the feeling that we might want to consider slowly introducing more OS-specific configurations in the process of assembling native applications; this is true not only for OS X, but also on Windows (where RCP application developpers would certainly appreciate that their applications be better adapted to latest evolutions such as the Metro styled application launcher), and Linux where extra files might be required to better deal with various package distribution mechanisms and various window managers.

It is obvious that most of these can be dealt with by RCP developpers themselves by adding whatever post-actions to their product export process, and I would definitely not push to develop whatever one might want to add later on, but it seems to me that asking some more specific questions to users, related to each OS, might help reduce the difficulty of properly "deducing" these configuration items based some of the current "generic" settings. A proper "application name" for example is not subject to the same "standards" on OS X, Windows or Linux, and the actual uses of that text might also be very different on each target. Just thinking aloud, anyway.
Comment 63 DJ Houghton CLA 2015-03-12 15:04:58 EDT
Sorry, I'm late to the party here... We talked about this at lunch and wondered does this effect the File -> Export behaviour? Is that still using PDE Build? and therefore would get still get the old structure?
Comment 64 Pascal Rapicault CLA 2015-03-12 15:10:32 EDT
(In reply to DJ Houghton from comment #63)
> Sorry, I'm late to the party here... We talked about this at lunch and
> wondered does this effect the File -> Export behaviour? Is that still using
> PDE Build? and therefore would get still get the old structure?
    Whoa! I did not expect to see you here :) The File > Export has been changed to produce the new layout. It was just so much fun ;) You can see the commits here: http://git.eclipse.org/c/pde/eclipse.pde.ui.git/commit/?id=fdf7dbec451ce7324595c9170d0a08c995655e73 and http://git.eclipse.org/c/pde/eclipse.pde.build.git/commit/?id=39bf543a0d5d6c0bd14fd23d15a5a41062276897.
Note though that because the I-build being produced does not have the new layout, the product export on mac may not be working.
Comment 65 DJ Houghton CLA 2015-03-12 15:37:10 EDT
Thanks, Pascal. I am everywhere. :)
Comment 66 Brian de Alwis CLA 2015-03-16 15:25:22 EDT
Thanks to Pascal for taking this further!

The symlink for the command-line eclipse is incorrect.  The symlink itself is found at:

   Eclipse.app/Contents/Eclipse/eclipse

and is a link to

   $ cd Eclipse.app/Contents/Eclipse
   $ ls -l eclipse
   lrwxrwxrwx    1 bsd  wheel      34 31 Dec  1969 eclipse@ -> Eclipse.app/Contents/MacOS/eclipse

whereas it should be to ../MacOS/eclipse (or at least ../../Contents/MacOS/eclipse).  Otherwise the user can't rename the app.

But even with that corrected, invoking the command-line still fails with an alert dialog with:

   The Eclipse executable launcher was unable to locate
   its
   companion shared library.

I've tried running this from the top-level directory (containing Eclipse.app), and Eclipse.app/Contents/Eclipse to no avail.

I am able to launch the command-line by invoking the launcher directly from Eclipse.app/Contents/Eclipse:

   java -XstartOnFirstThread -jar plugins/org.eclipse.equinox.launcher_*.jar etc.etc.


I'm unable to figure out the right incantation to invoke the command-line eclipse.
Comment 67 Pascal Rapicault CLA 2015-03-16 15:53:47 EDT
Thanks for trying this out Brian. 

The symlink should be deleted altogether.

> I'm unable to figure out the right incantation to invoke the command-line
> eclipse.

The launcher issue is known and fixed but it will not make it for M6 (We would need to recompile the launcher (bug #461674) and then update the p2 included in Tycho.
Comment 68 Brian de Alwis CLA 2015-03-16 16:16:38 EDT
I hit an additional problem when trying to update from within an existing pre-.app installation:

   An error occurred while installing the items
   session context was:(profile=SDKProfile, \
     phase=org.eclipse.equinox.internal.p2.engine.phases.Install, \
     operand=null —> [R]org.eclipse.sdk.ide.executable.cocoa.macosx.x86_64 \
     4.5.0.I20150316-0800, action=org.eclipse.equinox.internal.p2.touchpoint.natives.actions.ChmodAction).
   The action chmod failed - file /usr/local/installs/e4-dev/../MacOS/eclipse does not exist

(line wrapped)
Comment 69 Pascal Rapicault CLA 2015-03-16 16:20:17 EDT
(In reply to Brian de Alwis from comment #68)
> I hit an additional problem when trying to update from within an existing
> pre-.app installation:
> 
>    An error occurred while installing the items
>    session context was:(profile=SDKProfile, \
>      phase=org.eclipse.equinox.internal.p2.engine.phases.Install, \
>      operand=null —> [R]org.eclipse.sdk.ide.executable.cocoa.macosx.x86_64 \
>      4.5.0.I20150316-0800,
> action=org.eclipse.equinox.internal.p2.touchpoint.natives.actions.
> ChmodAction).
>    The action chmod failed - file
> /usr/local/installs/e4-dev/../MacOS/eclipse does not exist
> 
> (line wrapped)
    Update from the old layout to the new one is and will not be supported. I still need to work on the metadata that will prevent users to run into this situation.
Comment 70 David Williams CLA 2015-03-17 01:35:42 EDT
(In reply to Brian de Alwis from comment #66)

> 
> I'm unable to figure out the right incantation to invoke the command-line
> eclipse.

Did you try 

open ./Eclipse.app 

That worked for me. (Careful though, if you just type open Eclipse.app, it will launch the one in the Applications folder.
Comment 71 Brian de Alwis CLA 2015-03-17 10:02:27 EDT
Sorry, that last line was meant to be deleted.

By command-line, I meant being able to launch a console version of Eclipse.  The OS X builds used to include a symlink to the launcher binary (XXX.app/Contents/MacOS/XXX).  The launcher loads the .ini file, processes the command-line arguments and then starts the JVM.  It also handles the EXIT_RESTART and EXIT_RELAUNCH handling to restart the JVM.

For now I invoke the org.eclipse.equinox.launcher_*.jar, which does most of the same thing.

   $ cd Eclipse.app/Contents/Eclipse
   $ java -XstartOnFirstThread -Dorg.eclipse.swt.internal.carbon.smallFonts \
      -jar plugins/org.eclipse.equinox.launcher_*.jar
Comment 72 James Watkins-Harvey CLA 2015-03-17 17:23:11 EDT
@Pascal and David: Considering the construction of DMG disk images for exported applications, "where" were you planing to introduce this mechanism? Were you considering having P2 perform that step? Tycho? Neither, but simply introduce the DMG packaging step in Eclipse's official release procedure?

I have a shell script here we use for that purpose on our own applications; I'll need to clean it up a little though before I can contribute it. The script directly invoke Apple's command line utilities to assemble the DMG image, which sounds reasonable to me, but seems to be very absolutely innapropriate for P2's purpose, and mostly "out of place" for Tycho, considering that that code would then be nothing more than a very generic plugin for Maven, with nothing that is OSGi/Equinox specific.

There are however alternatives, that would allow DMG construction on other platforms: the 'hfsutils' package on Linux, some 100% Java-based implementations of the DMG file format, etc. What it more something along that line that you were considering?

To sumarize, what kind of solutions/information/sample/examples would be helpful in moving toward that objective?
Comment 73 James Watkins-Harvey CLA 2015-03-17 17:40:45 EDT
(In reply to Brian de Alwis from comment #71)
> Sorry, that last line was meant to be deleted.
> 
> By command-line, I meant being able to launch a console version of Eclipse. 
> The OS X builds used to include a symlink to the launcher binary
> (XXX.app/Contents/MacOS/XXX).  The launcher loads the .ini file, processes
> the command-line arguments and then starts the JVM.  It also handles the
> EXIT_RESTART and EXIT_RELAUNCH handling to restart the JVM.
> 
> For now I invoke the org.eclipse.equinox.launcher_*.jar, which does most of
> the same thing.
> 
>    $ cd Eclipse.app/Contents/Eclipse
>    $ java -XstartOnFirstThread -Dorg.eclipse.swt.internal.carbon.smallFonts \
>       -jar plugins/org.eclipse.equinox.launcher_*.jar


This works fine:

  $ cd /Applications
  $ Eclipse.app/Contents/MacOS/eclipse

It seems however that the behavior is incorrect if current working directory is anything but the directory that contains the Eclipse.app bundle. It might indicates that some file paths (that is at least the "launcher companion library", but probably others too) are not converted to their absolute paths before passing them to Java. This will have to be fixed in the native launcher, though.
Comment 74 Pascal Rapicault CLA 2015-03-17 17:58:56 EDT
> To sumarize, what kind of solutions/information/sample/examples would be
> helpful in moving toward that objective?
  I'm planning on using the script provided at https://github.com/andreyvit/create-dmg   I tried it locally and it produced a decent looking dmg. Of course it is possible to do fancier things like the installer for Adium but I think it will suffice to match the Mac user's expectations.

At this point no specific integration is planned. I'm planning to modify either the build script itself, or do it as part of the code signature process since it will save some back-and-forth between servers.
Comment 75 Beth Tibbitts CLA 2015-03-27 23:35:34 EDT
(In reply to James Watkins-Harvey from comment #73)

> This works fine:
> 
>   $ cd /Applications
>   $ Eclipse.app/Contents/MacOS/eclipse

Thanks. I almost always launch Eclipse (on my Mac of course) from the command line
and tried to do cd eclipse;./eclipse -showlocation -data my/workspace/location
but the eclipse directory another stuff doesn't exist.  
The above works ok for me, but my fingers type the old way from muscle memory
Comment 76 Brian de Alwis CLA 2015-03-30 17:05:02 EDT
Created attachment 252015 [details]
Bourne shell script to launch eclipse from the .app format

I've attached my eclipse script; it should behave the same as the previous launcher executable: it will load the options from the .ini file, handles -vmargs, respects —launcher.overrideVmargs and —launcher.appendVmargs, knows —launcher.XXMaxPermSize (well, it rewrites it as -XX:MaxPermSize=xxx).  Download it and place it alongside the Eclipse.app.
Comment 77 Tobias Oberlies CLA 2015-03-31 03:48:58 EDT
*** Bug 446320 has been marked as a duplicate of this bug. ***
Comment 78 Doug Schaefer CLA 2015-03-31 19:40:42 EDT
Has anyone figure out how to get a bundled JRE to work in this new layout.

The old app layout worked fine. I bundled my JRE in a feature and did a setJVM to .../bin/java.

Now i'm getting the missing JNI_CreateJavaVM symbol message.
Comment 79 Christian Georgi CLA 2015-04-01 02:10:04 EDT
(In reply to Doug Schaefer from comment #78)
> Has anyone figure out how to get a bundled JRE to work in this new layout.
> 
> The old app layout worked fine. I bundled my JRE in a feature and did a
> setJVM to .../bin/java.
> 
> Now i'm getting the missing JNI_CreateJavaVM symbol message.

This is working again with my patch for the native launcher in bug 444201.  It's just waiting for review...
Comment 80 Eclipse Genie CLA 2015-04-02 10:38:47 EDT
New Gerrit change created: https://git.eclipse.org/r/45151
Comment 81 David Williams CLA 2015-04-02 12:00:35 EDT
I think to be "most accurate", this bug belongs in equinox/p2 component, since that's where most of the work needs to be done. (And, don't worry, I'll still help in anyway needed :)
Comment 82 Brian de Alwis CLA 2015-04-08 12:21:24 EDT
Created attachment 252232 [details]
Bourne shell script to launch eclipse from the .app format

Here's an updated script that rewrites local file paths within the app bundle.  This is necessary for properly setting the dock icon.
Comment 83 David Williams CLA 2015-04-23 11:49:38 EDT
Pascal, 

I do not have a concrete example to try (yet) but, I have been asked to confirm that this new "Mac App Layout" still support "bundle pooling". (https://wiki.eclipse.org/Equinox/p2/Getting_Started#Bundle_pooling) 

So ... does it? should it? ... Thought I'd start with verbal confirmation :) 

Do we (i.e. you) have any unit test cases for this? (For maxosx).

Thanks,
Comment 84 Pascal Rapicault CLA 2015-04-23 11:56:45 EDT
> should it? ... 
   Yes

> does it? 
   This needs to be tested.

> Do we (i.e. you) have any unit test cases for this? (For maxosx).
   Not that I remember.
Comment 85 Eike Stepper CLA 2015-04-23 12:46:41 EDT
(In reply to David Williams from comment #83)
> does it?

In Oomph we use the new layout with bundle pooling and for us it works fine.
Comment 87 Brian de Alwis CLA 2015-05-14 15:54:41 EDT
Comment on attachment 252232 [details]
Bourne shell script to launch eclipse from the .app format

Obsoleted the script since the launcher was rebuild (bug 461674) and so can now be invoked from outside (comment 66).

Interesting aside: launching using 'java -jar plugins/org.eclipse.equinox.launcher_XXX.jar' causes JDT/Debug inspect hovers to appear blank.  It doesn't happen when using the launcher binary.  But this behaviour happens with Luna and possibly before and is unrelated to any of these changes.
Comment 88 Peter Severin CLA 2015-05-24 05:09:18 EDT
I am trying to build my RCP product using the latest release of 4.5 (4.5RC2a). I am using headless PDE build (via org.eclipse.pde.build_*/scripts/productBuild/productBuild.xml). I see a problem with Mac OS X packages being produced as they do not contain the executable launcher. I don't get anything related to .app folder, neither the old layout nor the new one. Here's the contents of the tar.gz file:

configuration/
dropins/
features/
jre/
p2/
plugins/
.eclipseproduct
about.html
artifacts.xml
WireframeSketcher.ini

Is this a known issue? Is there a solution?
Comment 89 Peter Severin CLA 2015-05-24 11:28:25 EDT
I also have another concern regarding the layout of Eclipse.app. Up to this point I was assembling a similar layout myself in my RCP application. Here's the final layout I came up with that does work well with Yosemite's code signing prerequisites:

/WireframeSketcher.app/
    Contents/
        Info.plist
        Resources/
            WireframeSketcher.ini
            configuration/
            plugins/
            jre/
                ...
        MacOS/
            WireframeSketcher

Note that it's similar to the layout shown in https://bugs.eclipse.org/bugs/show_bug.cgi?id=431116#c45 with the difference that I've used a more generic Contents/Resources instead of Contents/Eclipse. I think this convention is more generic and works both for Eclipse and RCP apps. Please consider using it instead.
Comment 90 Pascal Rapicault CLA 2015-05-24 11:43:52 EDT
> Is this a known issue? Is there a solution?
  This is now a know issue. Please open a bug report against PDE: https://bugs.eclipse.org/bugs/enter_bug.cgi?product=PDE 
The best solution is to use Tycho 0.23 instead.
Comment 91 Peter Severin CLA 2015-05-24 11:51:04 EDT
I opened Bug 468131. I know that Tycho is the preferred method now, but I would like to avoid migrating my build from headless PDE to Tycho if possible.
Comment 92 Peter Severin CLA 2015-05-26 09:35:13 EDT
As a followup to https://bugs.eclipse.org/bugs/show_bug.cgi?id=431116#c89, here's the Mac layout that I get for my RCP app using Tycho:

WireframeSketcher.app
 Contents
  Resources
    sketcher.icns
  MacOS
    WireframeSketcher
  Eclipse
    readme
    plugins
    p2
    jre
    features
    dropins
    configuration
    WireframeSketcher.ini
    artifacts.xml
    about.html
  Info.plist

Note that there is an Eclipse folder. Although it's mostly an aesthetic issue, I find that it would look better if Eclipse and Resources folders were merged together into a single Resources folder, or if Eclipse folder renamed to something more generic.
Comment 93 Eike Stepper CLA 2015-05-26 10:05:43 EDT
(In reply to Peter Severin from comment #92)
> Note that there is an Eclipse folder. Although it's mostly an aesthetic
> issue, I find that it would look better if Eclipse and Resources folders
> were merged together into a single Resources folder, or if Eclipse folder
> renamed to something more generic.

If this is just a matter of personal taste I would prefer to avoid the cascade of downstream problems that any change in this area would cause.
Comment 94 Pascal Rapicault CLA 2015-05-26 10:20:15 EDT
> Note that it's similar to the layout shown in
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=431116#c45 with the difference
> that I've used a more generic Contents/Resources instead of
> Contents/Eclipse. I think this convention is more generic and works both for
> Eclipse and RCP apps. Please consider using it instead.
   We are way passed the point where we can do that kind of changes.
Comment 95 Peter Severin CLA 2015-05-26 17:28:16 EDT
I understand. I know I am late with my suggestion, but if this change to be ever made then it should be made now, considering how it has the potential to break p2 updates.
Comment 96 Thomas Watson CLA 2015-06-25 08:13:53 EDT
I assume this bug is fixed for Mars.  If not then reopen and target for Neon.