Bug 230384 - [shared] Position of p2 folder seems inconsistent when running -configuration
Summary: [shared] Position of p2 folder seems inconsistent when running -configuration
Status: CLOSED WONTFIX
Alias: None
Product: Equinox
Classification: Eclipse Project
Component: p2 (show other bugs)
Version: 3.4   Edit
Hardware: PC Mac OS X - Carbon (unsup.)
: P3 normal with 3 votes (vote)
Target Milestone: ---   Edit
Assignee: P2 Inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords: helpwanted, readme
: 238200 (view as bug list)
Depends on:
Blocks:
 
Reported: 2008-05-06 08:55 EDT by Kristoffer Peterhänsel CLA
Modified: 2019-10-19 17:41 EDT (History)
28 users (show)

See Also:


Attachments
-configuration patch (2.09 KB, patch)
2008-05-14 14:26 EDT, Simon Kaegi CLA
no flags Details | Diff
-configuration patch v2 (6.68 KB, patch)
2010-03-02 19:25 EST, Martin Oberhuber CLA
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Kristoffer Peterhänsel CLA 2008-05-06 08:55:45 EDT
Build ID: I20080502-0100

Steps To Reproduce:
1. start eclipse with a -configuration parameter
2. See that the 'p2 folder' is made in parent of the location described in the -configuration parameter


More information:
Not a whole lot to say really. It seems very inconsistent that the p2 folder don't either stay in the eclipse installation. Or move inside of the configuration folder.

If the configuration and p2 folder needs to be separate - and I presume so as they are by default. Perhaps the -configuration parameter needs to be the parent of both. As it is rather counterintuitive that you need to make a -configuration location end in two new folders (as in '<someloc>/mynewconfig/configuration' instead of just '<someloc>/mynewconfig' , if you happen to store several configuration in the same place.
Comment 1 Simon Kaegi CLA 2008-05-14 00:35:43 EDT
Although we can run everything from inside the configuration folder, I'm a little hesitant to act on this in RC1. The SDK by default provides a data area setting in config.ini as follows:

eclipse.p2.data.area=@config.dir/../p2

In the shared install use-case we're currently honoring this location for creating the users p2 data area. Ideally we might change this to always put p2 inside the configuration folder (e.g. something like @config.dir/.p2) however it seems too risky to make a change like this right now.

Another alternative might be to recognize when we're in shared mode and effectively take control and always place the user's p2 data area in the configuration folder. eclipse.p2.data.area in that case would identify the shared data area (using the shared config.dir) Not really pretty but probably workable.

Any thoughts on this?
Comment 2 Kristoffer Peterhänsel CLA 2008-05-14 03:39:28 EDT
I suppose it depends on what exactly is stored in the p2 folder. Previously you could create new configurations by using the -configuration parameter. While you made new workspaces with the -data parameter (or in the GUI).

Now that might seem obvious. But to me a configuration is basically what plugins I am loading. So I could have one base Eclipse install and then a mix of the extension locations I wanted.

On the other hand I might want to use the same configuration with more than one workspace. Having too much unrelated matter in one workspace can become confusing. And while you can open and close projects. It is often much easier if they are not there at all.

As I understand it the data location would be part of defining the configuration, as I see it, if you moved the p2 folder into. So I'd be against that from how I have been using these parameters before.
Comment 3 Pascal Rapicault CLA 2008-05-14 09:40:52 EDT
configuration and workspace locations are two distinct things and I don't think Simon is proposing to merge them.

The p2 folder contains the description of your configuration in terms of p2 constructs. To that extent when you are flushing the configuration you really want to flush the p2 folder as well.

With this folder not located into the configuration folder, now the difficulty comes when you enter:
 eclipse -configuration /foo/bar

and then use
 eclipse -configuration /foo/zoo

since with the current logic this will create a p2 folder in the foo/ folder which will obviously cause problems since then we will have 2 configurations sharing the same p2 area.

There are several ways to address this:
1) in shared installs, move the p2 folder into the configuration folder. But then where do plugins and features that have been installed by the user go?
2) move the p2 folder inside the configuration folder all the time, but we moved out of this model in January
3) use the folder created as the parent folder for the configuration folder (e.g. specifying /foo/bar will lead to /foo/bar/configuration)
4) reuse the same logic than when eclipse runs from a read-only instance and no configuration folder is specified. For background when eclipse is run in shared mode and no configuration folder is specified, the configuration is stored under:  <user-home>/org.eclipse.platform_3.4.0.xxxxx/configuration. So in the discussed case that would give us: /foo/zoo/org.eclipse.platform_3.4.0.xxxx/configuration.

I personally like 3 and 4, since they allow for a nice symmetry with what we have in a regular eclipse install. 
Comment 4 Thomas Watson CLA 2008-05-14 10:21:26 EDT
(In reply to comment #3)
> configuration and workspace locations are two distinct things and I don't think
> Simon is proposing to merge them.
> 
> The p2 folder contains the description of your configuration in terms of p2
> constructs. To that extent when you are flushing the configuration you really
> want to flush the p2 folder as well.

If p2 folder is tied to the configuration then I am not sure why it is not under the configuration location by default.

> 
> With this folder not located into the configuration folder, now the difficulty
> comes when you enter:
>  eclipse -configuration /foo/bar
> 
> and then use
>  eclipse -configuration /foo/zoo
> 
> since with the current logic this will create a p2 folder in the foo/ folder
> which will obviously cause problems since then we will have 2 configurations
> sharing the same p2 area.

If more than one configuration is not supposed to share the same p2 area then shouldn't p2 area be under configuration?

> 
> There are several ways to address this:
> 1) in shared installs, move the p2 folder into the configuration folder. But
> then where do plugins and features that have been installed by the user go?

Where do they go under the current model?  Isn't there a user configuration area cascaded to the user home under <user-home>/org.eclipse.platform_3.4.0.xxxxx/configuration and the bundles go somewhere under there?  is there also a p2/ folder under <user-home>/org.eclipse.platform_3.4.0.xxxxx/ in this case?

> 2) move the p2 folder inside the configuration folder all the time, but we
> moved out of this model in January

What was the reason for the move?

> 3) use the folder created as the parent folder for the configuration folder
> (e.g. specifying /foo/bar will lead to /foo/bar/configuration)

Are you suggesting a change to the behavior of -configuration command option?  Today -configuration /foo/bar tells us to use /foo/bar as the actual configuration area where the config.ini goes.  Would the new proposed behavior append "configuration" to any path specified by the -configuration option?  This seems like a breaking change to the behavior of -configuration.

> 4) reuse the same logic than when eclipse runs from a read-only instance and no
> configuration folder is specified. For background when eclipse is run in shared
> mode and no configuration folder is specified, the configuration is stored
> under:  <user-home>/org.eclipse.platform_3.4.0.xxxxx/configuration. So in the
> discussed case that would give us:
> /foo/zoo/org.eclipse.platform_3.4.0.xxxx/configuration.

To make sure I understand you correctly, you are proposing that "-configuration /foo/zoo" would yield an osgi.configuration.area = file:/foo/zoo/org.eclipse.platform_3.4.0.xxxx/configuration

Again this seems like a breaking change.  I would not be in favor of this type of change this far after API freeze.

> 
> I personally like 3 and 4, since they allow for a nice symmetry with what we
> have in a regular eclipse install. 
> 

Comment 5 Pascal Rapicault CLA 2008-05-14 10:38:39 EDT
> If p2 folder is tied to the configuration then I am not sure why it is not under the configuration location by default.
  This is because the p2 folder is representing the agent data area and it is capable of managing multiple configurations. For example with some of the p2 API one could cause another installation to be created.
  Obviously in the self-contained scenario of an eclipse install, then the mapping from p2 to configuration is a one to one mapping.

>This seems like a breaking change to the behavior of -configuration.
  It is a different behavior but given that the current behavior we have to do something the question is why? Also even though I understand that we are way passed API freeze, the question is do we want to not change things just because of that and have to put hacks everywhere that we will have to maintain forever.

Comment 6 Thomas Watson CLA 2008-05-14 10:54:10 EDT
(In reply to comment #5)
> > If p2 folder is tied to the configuration then I am not sure why it is not under the configuration location by default.
>   This is because the p2 folder is representing the agent data area and it is
> capable of managing multiple configurations. For example with some of the p2
> API one could cause another installation to be created.
>   Obviously in the self-contained scenario of an eclipse install, then the
> mapping from p2 to configuration is a one to one mapping.

In the case where the p2 folder manages more than one configuration would you not use the eclipse.p2.data.area option to point the p2 data area to a central folder for each configuration managed?  Currently we set this to @config.dir/../p2 with the SDK config.ini.  That seems wrong for a default.  What is the most common case when you download the SDK?  A one to one mapping between the configuration and p2 folder?

> 
> >This seems like a breaking change to the behavior of -configuration.
>   It is a different behavior but given that the current behavior we have to do
> something the question is why? Also even though I understand that we are way
> passed API freeze, the question is do we want to not change things just because
> of that and have to put hacks everywhere that we will have to maintain forever.
> 

My fear is unintended consequences this late in the cycle.  I still am not convinced the change is needed.  For cases where p2 is shared between more than one configuration then the proposed changes do not seem to solve the issue.  Wouldn't you still need to set the eclipse.p2.data.area property to the shared p2 area.
Comment 7 Kristoffer Peterhänsel CLA 2008-05-14 12:49:07 EDT
(In reply to comment #3)
> 3) use the folder created as the parent folder for the configuration folder
> (e.g. specifying /foo/bar will lead to /foo/bar/configuration)
> 4) reuse the same logic than when eclipse runs from a read-only instance and no
> configuration folder is specified. For background when eclipse is run in shared
> mode and no configuration folder is specified, the configuration is stored
> under:  <user-home>/org.eclipse.platform_3.4.0.xxxxx/configuration. So in the
> discussed case that would give us:
> /foo/zoo/org.eclipse.platform_3.4.0.xxxx/configuration.

Wouldn't option 4 mean that your configuration would be reset every time you installed a platform update?

Option 3 was pretty much what I suggested I think. And it seems like the workable middle ground. I does seems like a major change to how -configuration works tho. The question here being how big of a problem that would be.


(In reply to comment #5)
> > If p2 folder is tied to the configuration then I am not sure why it is not under the configuration location by default.
>   This is because the p2 folder is representing the agent data area and it is
> capable of managing multiple configurations. For example with some of the p2
> API one could cause another installation to be created.
>   Obviously in the self-contained scenario of an eclipse install, then the
> mapping from p2 to configuration is a one to one mapping.

It seems a little odd that the Eclipse Platform is starting to store configurations in several locations. I realize it is different things. But in the end it also needs to be workable for the user. And AFAIK the configuration and p2 folders are both products of the platform. So it's not even the IDE projects that are 'doing it wrong'.

From how I see it the p2 folder can only be part of one of two things in an Eclipse IDE installation: The installation or the configuration. From what I read here it could be either, as it is supposed to be able to support several configurations. But there need to be support for this scenario as the primary user of the framework is Eclipse IDE projects. 
Comment 8 Simon Kaegi CLA 2008-05-14 14:26:09 EDT
Created attachment 100264 [details]
-configuration patch

This is the framework patch I've been using that changes how -configuration is interpreted and appends a "configuration" folder. Obviously we need to think about this some more but at least the patch is available to try.
Comment 9 Simon Kaegi CLA 2008-05-14 16:42:37 EDT
Un-targetting for now.

I'm going to suggest deferring doing anything here as I think by reacting at this stage we're going to dig our selves into a deeper hole if we either do a hack in p2 or else make a last second framework modification. The net result is a behaviour change in that we are writing into directories that are siblings of osgi.configuration.area. This primarily impacts "-configuration" users in that these siblings must also be writeable. This should not affect the more typical case where a default configuration area is created in the user.home because we will end up writing in $(user.home)/.eclipse/org.eclipse.platform_3.4.0_xxx folder which should always be writeable.

The use-case Pascal mentions with "-configuration" /foo/bar and /foo/zoo will result in problems as they'll likely be sharing the same profile entry. We talked briefly about a way to create independent location based profile ids to handle this case however we're not very well set up for this at the moment. In the end, this would likely be a rare case and as alluded to by Kristoffer the workaround would be to specify /foo/bar/configuration and /foo/zoo/configuration.

Longer term it might be interesting to make the p2.data.area independent from the osgi.configuration.area. We could investigate following similar logic to what we're doing for confiuration e.g using $(user.home)/.eclipse/org.eclipse.platform_3.4.0_xxx/p2. This might be a good step towards providing a writeable per user bundlepool.
Comment 10 Jason R. Coombs CLA 2008-06-02 10:08:50 EDT
I expect this problem may cause users some extended problems, particularly in Windows Vista, where the -configuration option is the suggested way to relocate the configuration to address User Access Control restrictions (programs directory not writable).

I encountered an issue with respect to this behavior in my environment:

---
Last week, I tried RC2, but ran into problems installing plugins, so punted.  Now, I'm trying to install clean, but references to the plugins linger around and can't be uninstalled.

I'm using Windows Vista, so am starting eclipse with -configuration c:\users\myaccount\appdata\roaming\eclipse-3.4

I've completely removed that configuration directory, in an attempt to get rid of the lingering plugin references.

But when I restart, the plugins appear in the Software Updates... dialog.

If I specify a different directory for the configuration, such as c:\eclipse-settings, the lingering plugins go away.

However, if I specify a different directory in the appdata folder (such as c:\users\myaccount\appdata\roaming\eclipse-new-foo), the lingering plugins reappear.

Okay.  I figured out the problem.  It seems that even though I specify a directory c:\x for the configuration, eclipse-3.4 assumes that the sub-directory is where settings go (i.e. c:\), so it created "p2", "features", and "plugins" in the parent of the configuration directory.  This was very confusing.  I fixed the problem by using a different configuration spec: %appdata%\eclipse-3.4\configuration

Then, plugins, features, p2 (and possibly others) end up in %appdata\eclipse-3.4.

I don't know if this is a bug, or by design, but at the very least, it's going to cause users problems.  At the very least, there should be a note in the release notes to this effect.
---
Comment 11 John Arthorne CLA 2008-06-02 10:18:42 EDT
Note the recommended approach on Vista under UAC is to not specify any special arguments to deal with this. The framework automatically detects this condition and selects a configuration location that is writable for the current user. If there is doc that says otherwise, can you give a pointer to it... it may be out of date documentation that needs updating.
Comment 12 Jason R. Coombs CLA 2008-06-02 10:35:36 EDT
I have not seen the Vista UAC recommendation in recent documentation.  I noted it once prior to the 3.3 release.  I personally don't use UAC, but I relocate my configuration for consistency sake.

My mistake assuming the -configuration parameter is the recommended workaround for UAC.

I do note that in the eclipse-3.4RC3 readme, it does suggest using the -configuration parameter for environments where the JVM cannot detect writability, which is probably still pertinent to this bug, though admittedly less widespread.
Comment 13 John Arthorne CLA 2008-06-02 10:59:21 EDT
Thanks, I'll make sure the readme is up to date.
Comment 14 Simon Kaegi CLA 2008-06-23 23:37:35 EDT
*** Bug 238200 has been marked as a duplicate of this bug. ***
Comment 15 Martin Oberhuber CLA 2009-11-30 08:41:59 EST
See Michael Scharf's Blog about the problem:
http://michaelscharf.blogspot.com/2009/09/never-put-things-above-user-specified.html
Comment 16 Martin Oberhuber CLA 2010-01-26 13:12:27 EST
(In reply to comment #9)
> Un-targetting for now.
> Longer term it might be interesting to make the p2.data.area independent from
> the osgi.configuration.area.

I'm wondering whether the new public p2 API in Eclipse 3.6 might be a chance (probably the last chance?) to get this right.

Writing into the parent of the -configuration folder specified by the user has been a problem for many -- big enough for us to patch Eclipse in our commercial product to avoid this.

Have we become wiser since Eclipse 3.4? Does anybody see a chance getting this fixed? Or have things even got worse such that this needs to be closed as WONTFIX?
Comment 17 John Arthorne CLA 2010-01-26 13:53:55 EST
(In reply to comment #16)

I don't really see any connection to the p2 API work here. Nowhere in p2 API or internals does it assume the location of the p2 folder as far as I know. This is specified by the "eclipse.p2.data.area" system property. I guess what you're asking for is changing the value of this variable in the case of the Eclipse SDK, which currently specifies this location in config.ini:

eclipse.p2.data.area=@config.dir/../p2/

Note a product can already put something different here if you want the p2 metadata folder to be elsewhere. Do you have a specific suggestion for a different value here?
Comment 18 Pascal Rapicault CLA 2010-01-26 14:08:13 EST
We are already overcommitted for this release. If someone from your team who is already looking at the p2 code wanted to take a look, this would help toward resolution of this problem. CC'ing Meng and Helmut
Comment 19 Martin Oberhuber CLA 2010-01-26 14:12:05 EST
(In reply to comment #17)
Looking at John's comment, I am not so sure whether this is an issue in p2 or an issue in how the Eclipse SDK leverages p2. If the only problem is the default setting from Eclipse's config.ini, what is the problem with doing

eclipse.p2.data.area=@config.dir/.p2/

which gives a 1-to-1 mapping between configuration and p2 data by default -- I
think this is what most people want. Those (few?) experts who want a shared p2
data area for multiple configurations could override the default. Perhaps they
would want a commandline argument to do so (but then, I think they could
-vmargs -Declipse.p2.data.area=/foo)
Comment 20 Ian Bull CLA 2010-01-28 18:42:04 EST
I don't think it's just the p2 folder that gets set 1 directory up.  I've read that you can use the -configuration option to set a local configuration for an eclipse install that exists elsewhere.

If you do this and set -configuration ~me/configuration, I actually get
~me/configuration
~me/artifacts.xml
~me/plugins

If I ever install something using this install, I will also get 
~me/plugins
~me/features
Comment 21 Ian Bull CLA 2010-01-28 18:43:12 EST
(In reply to comment #20)
> I don't think it's just the p2 folder that gets set 1 directory up.  I've read
> that you can use the -configuration option to set a local configuration for an
> eclipse install that exists elsewhere.
> 
> If you do this and set -configuration ~me/configuration, I actually get
> ~me/configuration
> ~me/artifacts.xml
> ~me/plugins
> 
> If I ever install something using this install, I will also get 
> ~me/plugins
> ~me/features

Sorry, that first list should have been:
~me/configuration
~me/artifacts.xml
~me/p2
Comment 22 Martin Oberhuber CLA 2010-01-29 05:04:41 EST
(In reply to comment #21)
Thanks Ian -- yes, this is exactly the problem. It looks like changing the eclipse.p2.data.area is not sufficient in order to avoid Eclipse writing above the specified folder.

FYI, the patch we applied to our commercial product is that in our version of org.eclipse.equinox.launcher, we inspect the commandline and if a -configuration argument has been given, we patch the commandline arg to append a "/config" path segment. As a result, when you do

    -configuration ~me/configuration

in Wind River Workbench, you actually get

    ~me/configuration/config
    ~me/configuration/artifacts.xml
    ~me/configuration/p2
    ~me/configuration/plugins
    ~me/configuration/features

which is what our users expect. Our users do not understand or need the internals of artifacts.xml, p2 data area and the like -- they want to specify a local configuration where they can install stuff beyond the read-only shared install easily. In other words:

   As a user of a read-only shared Eclipse installation, I want to be able
   to specify a custom local configuration area on the commandline, such 
   that I can control where my local additions are configured. I expect 
   the product to write only into the location I specified and nowhere else.

I do understand that our solution is not universally valid and it has issues. For instance, existing scripts which have been written against the standard Eclipse semantics do not expect the /config path segment to be appended and thus won't work as expected. We address this by using a System Property (-vmargs -DUSE_ECLIPSE_CONFIG_DIR_STYLE) in order to turn off our custom behavior.

I am interested in this issue because (a) I want to avoid patching Eclipse in our product, and (b) I understand that this problem not only affects Wind River customers so I want to help working towards a solution that works for all. 

What could be done to 
- avoid writing into the parent dir by default,
- but still provide all available functionality if needed,
- and be sufficiently backward compatible to not break the world?
Comment 23 Martin Oberhuber CLA 2010-01-29 05:09:25 EST
I just noticed that our current solution does exactly what Simon's patch from
comment 8 does. As mentioned in my previous comment, this has issues such as not being backward compatible, and is thus not good enough as a general solution.

From a very high level, I could imagine the following approach:
* Allow some persisted settings in the config area to "redirect" the files
  which are now written into the parent dir
* When reading an existing config area, these settings are missing, so
  look for the files in the parent dir as today
* When creating a new config area, write the redirection settings
  such that files are created inside the config area
* Have explicit system properties or commandline args to override the
  redirection settings in the config area.

Does that make sense?
What current settings beyond eclipse.p2.data.area control the location 
of files such as artifacts.xml / p2 / plugins / features / links / dropins?
Comment 24 James Blackburn CLA 2010-01-29 05:48:43 EST
(In reply to comment #22)
> FYI, the patch we applied to our commercial product is that in our version of
> org.eclipse.equinox.launcher, we inspect the commandline and if a
> -configuration argument has been given, we patch the commandline arg to append
> a "/config" path segment. As a result, when you do

My user's ran their Eclipse from a shell script, and it needs to do exactly the same thing to give the users what they expect.  Unfortunately we also need to check the version of Eclipse before munging the -configuration line - as pre-p2 Eclipse does what you expect.

I consider this a breaking change from the old-UM world. Whereas users could previously do
  -configuration config1
  -configuration config2
  etc..
and expect to have Eclipse instances with separate configurations, with p2 some of the black-box configuration metadata is shared -- not the intention of using the -configuration switch in the first place!
Comment 25 Martin Oberhuber CLA 2010-01-29 07:22:33 EST
(In reply to comment #24)
> with p2 some of the black-box configuration metadata is shared -- not the
> intention of using the -configuration switch in the first place!

Well that's my point, does the black-box metadata need to be shared by default?
I would say no, what people intuitively expect is a 1:1 relationship between configuration and blackbox metadata, so this should be the default. People who want the blackbox shared / redirected should say so through specific options.

If I get your point right, my suggestion from comment 23 should work for you because it would create -configuration as it always did (non-breaking), but put the blackbox metadata into the configuration rather than keeping it outside. Unless it detects an old config so it would run in "compatibility mode".
Comment 26 James Blackburn CLA 2010-01-29 07:27:51 EST
(In reply to comment #25)
> If I get your point right, my suggestion from comment 23 should work for you

Yep, +1 to that. It just needs someone to provide a patch and for the p2ers to accept it...
Comment 27 Martin Oberhuber CLA 2010-01-29 09:47:26 EST
I should also mention a scenario where the current behavior is particularly
severe, and what caused us to end up patching Eclipse in our product:

* On a shared Linux host, user A does -configuration /tmp/aconf in order to
  just try something out, while user B does -configuration /tmp/bconf
  --> Their p2,artifacts etc files will be shared, leading to odd behavior
      which the users won't understand

* When the parent dir of the configuration dir is not writable, (like
  eclipse -configuration /tmp), Eclipse terminates with error messages that
  do not help understanding the problem -- users are doomed. This "error
  messages" problem is slightly related to bug 301226.

We can help out providing patches for these issues (and we do, like for
bug 301226), but for the concrete issues of redirecting files as discussed here we do not feel like we understand the big picture well enough. I think that we all need to agree on a strategy for resolving the issue (by discussing it here on the bug) before contributing a patch would make sense.

That being said, what do the p2 champions think about comment 23?
What components other than p2 are potentially affected?
Comment 28 John Arthorne CLA 2010-02-04 11:19:24 EST
(In reply to comment #27)
> * On a shared Linux host, user A does -configuration /tmp/aconf in order to
>   just try something out, while user B does -configuration /tmp/bconf
>   --> Their p2,artifacts etc files will be shared, leading to odd behavior
>       which the users won't understand

How has the configuration data been crafted in this case? p2 assumes responsibility for managing several files in the configuration folder (config.ini, bundles.info), and the p2 metadata in the p2/ folder describes what that configuration data looks like. If someone is crafting a configuration by hand it's never going to be consistent with the p2 metadata. If that configuration folder has been created by p2, then it's valid for multiple configuration folders to map to a single p2 folder (assuming both configurations were provisioned with the same agent location). Of course using a p2 folder that is writeable by multiple users is not recommended - it should be made read only in this case.


> * When the parent dir of the configuration dir is not writable, (like
>   eclipse -configuration /tmp), Eclipse terminates with error messages that
>   do not help understanding the problem -- users are doomed. This "error
>   messages" problem is slightly related to bug 301226.

This is definitely an invalid configuration. I assume /tmp/config.ini specifies the line "eclipse.p2.data.area=@config.dir/../p2", so it is pointing to an invalid location for the p2 metadata. We should improve the error reporting in this case, but we are just using the paths specified (incorrectly) by the user here.
Comment 29 Martin Oberhuber CLA 2010-02-04 12:12:04 EST
Thanks for your comments, John, there is one point where I disagree:

> but we are just using the paths specified (incorrectly) by the user here.

I'm not sure what you mean by "the user" in this context. Nobody specified anything special, I just ran the Eclipse 3.6M5 SDK as I can download today. The config.ini file that's being shipped with the SDK does specify 
   "eclipse.p2.data.area=@config.dir/../p2"

So, when I do 
   eclipse -configuration /tmp/aconf
the SDK will copy it's original config.ini into my /tmp/aconf, and therefore it will create /tmp/p2 (which is going to be shared with user B doing the same).

I did not fully understand your comments about "How has the configuration data been crafted". Nobody crafted anything, I just launched the Eclipse SDK with a -configuration argument because I'm running Eclipse from a shared read-only location and I don't want it to dump files anywhere in my home directory. 

As a (truly dumb) user in this case I'm surprised by the behavior, and I'd like the default behavior to be more intuitive, i.e. have a 1-to-1 mapping between the -configuration that I specified and the data being used by p2 (i.e. not writing into a parent of what I specified).

Perhaps this bug is not properly categorized here in Equinox/p2 since it may be more of an issue how p2 is being configured by the SDK rather than what p2 does. On the other hand, here in p2 I hope to have knowledgeable people who can help getting the configuration right.




(In reply to comment #28)
> (In reply to comment #27)
> > * On a shared Linux host, user A does -configuration /tmp/aconf in order to
> >   just try something out, while user B does -configuration /tmp/bconf
> >   --> Their p2,artifacts etc files will be shared, leading to odd behavior
> >       which the users won't understand
> 
> How has the configuration data been crafted in this case? p2 assumes
> responsibility for managing several files in the configuration folder
> (config.ini, bundles.info), and the p2 metadata in the p2/ folder describes
> what that configuration data looks like. If someone is crafting a configuration
> by hand it's never going to be consistent with the p2 metadata. If that
> configuration folder has been created by p2, then it's valid for multiple
> configuration folders to map to a single p2 folder (assuming both
> configurations were provisioned with the same agent location). Of course using
> a p2 folder that is writeable by multiple users is not recommended - it should
> be made read only in this case.
> 
> 
> > * When the parent dir of the configuration dir is not writable, (like
> >   eclipse -configuration /tmp), Eclipse terminates with error messages that
> >   do not help understanding the problem -- users are doomed. This "error
> >   messages" problem is slightly related to bug 301226.
> 
> This is definitely an invalid configuration. I assume /tmp/config.ini specifies
> the line , so it is pointing to an
> invalid location for the p2 metadata. We should improve the error reporting in
> this case, but we are just using the paths specified (incorrectly) by the user
> here.
Comment 30 Martin Oberhuber CLA 2010-02-25 19:31:57 EST
I noticed today that when I launch an Eclipse Application as a PDE Debuggee, the generated config.ini, which resides in
   .metadata\.plugins\org.eclipse.pde.core\launchName
actually reads
   eclipse.p2.data.area=@config.dir/.p2

so apparently, it is possible to have the p2 folder inside the configuration area. I think that the config.ini which is generated when launching Eclipse with the -configuration option pointing to an empty folder should also use this setting.

Having the p2.data.area _inside_ the config area for a freshly generated configuration should be independent of where the p2.data.area is located in the installation or shared configuration area. Is this possible today?
Comment 31 Martin Oberhuber CLA 2010-02-25 19:49:20 EST
(In reply to comment #1)
> Although we can run everything from inside the configuration folder, [...]
> The SDK by default provides a config.ini as
>    eclipse.p2.data.area=@config.dir/../p2
> In the shared install use-case we're currently honoring this location [...]
> Another alternative might be to recognize when we're in shared mode and
> effectively take control and always place the user's p2 data area in the
> configuration folder.

It looks like Simon had actually quite exactly analyzed the situation already in 2008, and as of today the real problem is still that the
   eclipse.p2.data.area
property is used for both the shared config area as well as the user config area. But it seems incorrect that the "product setup" (in the shared config area) must match the "user setup" (in the user config area).

In case that assumption is right, a good solution might be properties like
   eclipse.p2.data.area.user=@config.dir/.p2
   eclipse.p2.data.area.default=@config.dir/.p2
to indicate the desired location of the user's p2 data area when a new config.ini file is generated in a new, empty directory. Products could still modify that default for new config areas in their own config.ini; for backward compability, existing config.ini files would be still understood correctly.

Any comments before I start trying a patch on an area that I haven't touched before? - This issue is really important for us, and I'd love to see it addressed in Eclipse 3.6.
Comment 32 Martin Oberhuber CLA 2010-03-02 14:10:58 EST
It looks like the place where this can be fixed is in

  org.eclipse.equinox.p2.reconciler.dropins
    Activator#checkConfigIni()

This seems to be the place where the config.ini file is written, which holds the "Linked configuration" pointer. It seems that at this point, a "local p2.data.area" could override the "shared" one.

I've been trying to work out a patch, but I'm having trouble compiling the bundle when self-hosting with I20100217, and also when getting all of p2 from HEAD.
 
Also, I'm a unsure how to best debug my code that early during startup - "deploy into running platform" with printf debugging was all I could come up with so far, since I cannot see how I could reproduce a shared config area scenario as a debuggee.

Tips, ideas, suggestions are welcome.
Comment 33 Martin Oberhuber CLA 2010-03-02 19:25:35 EST
Created attachment 160716 [details]
-configuration patch v2

Attached is a 2nd try fixing this by massaging the argument to -configuration, as suggested by Simon in comment 8. This patch tries a bit harder to stay backward compatible. It correctly handles aliases like @user.home, and only appends the extra configuration/ folder if

 1/ a config.ini file does not yet exist in the selected location
 2/ the parent folder is not a .eclipseproduct or .eclipseextension
 3/ the osgi.configuration.area had not been set as a vm argument.

Point 3/ is an exit strategy: in case the patch causes issues, just add an extra -vmargs -Dorg.eclipse.configuration.area=osgi to fall back to the standard osgi semantics.

I still think that a more proper place for fixing the issue would be in
   org.eclipse.equinox.p2.reconciler.dropins/Activator
because this is where the problem of writing into ".." got introduced, and it feels odd that the -configuration workaround affects OSGi apps even if they are not using p2. But after seeing that a lot more locations are hardcoded as "../plugins" and similar, and not finding a way to redirect the p2.data.area in there, I eventually backed off to the -configuration approach.

Perhaps a more initiated person would need to look at this class.

The -configuration approach does make some sense by arguing that the argument to -configuration is meant to be a "black box" kind of location where anything related to configuration is to be stored. This is also in line with the end-user documentation of -configuration in
  http://help.eclipse.org/galileo/topic/org.eclipse.platform.doc.user/tasks/running_eclipse.htm
whereas the developer docs would need updating in
  http://help.eclipse.org/galileo/topic/org.eclipse.platform.doc.isv/reference/misc/runtime-options.html

Does this patch look acceptable?
Comment 34 Martin Oberhuber CLA 2010-03-08 04:47:37 EST
After digging a little deeper, here is an outline for a better patch. Please let me know if there are any obvious flaws, before I start implementing it.

1. Create a new System Property
        eclipse.p2.install.dir 
   with following implicit default value:
   1a) osgi.install.area if not shared
   1b) @config.dir/.. if shared AND that folder contains .eclipseextension
   1c) @config.dir/p2.install otherwise (i.e. inside the config area)

   When set from the outside, semantics of this System Property are "the
   location into which p2 may install downloaded software". Just like the
   old .eclipseextension. This could also be seen as the local bundlepool.

2. Compute this property in org.eclipse.p2.core, before setting the agent 
   location. Create a new alias ("@install.dir"), and express the 
   p2.data.area as a sibling of it:
     eclipse.p2.data.area=@install.dir/p2
   Set the computed System Property for other bundles on top of p2.core to use.

3. Similarly, in org.eclipse.p2.reconciler.dropins, the other locations which
   are currently hardcoded as ../ can now be expressed relative to
   @install.dir:
     @install.dir/plugins @install.dir/features
     @install.dir/links @install.dir/dropins 

The trick with these changes is that it allows us encapsulating the odd selection of a ".." directory in one place only (1b above), and have all other folders expressed as siblings of that directory. The default setting as per 1c) addresses the needs of this bug, whereas 1b) is for backward compatibility when .eclipseextension exists.

Furthermore, there is an exit strategy in case anything goes wrong: Just setting 
    -vmargs -Declipse.p2.install.dir=@config.dir/..
makes everything behave exactly as it does today, i.e. gets rid of the new fallback from (1c).

I'm not yet exactly happy about the name of the System Property and aliases, but these can be changed easily later.
Comment 35 Martin Oberhuber CLA 2012-09-13 08:25:20 EDT
CQ:WIND00180740
Comment 36 Michael Osipov CLA 2013-03-24 10:06:09 EDT
Just found this nuisance with Eclipse 4.2.2. config.ini says: eclipse.p2.data.area=@config.dir/../p2

Eclipse Help does not contain any information on the property nor @config.dir. This has be to, at least, documented clearly. It's quite confusing why p2 creates a directory outside of the config dir. Since plugin directories a fully qualified, why can't it be eclipse.p2.data.area=@config.dir?
Comment 37 Eclipse Genie CLA 2019-10-19 17:41:52 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.