Bug 442858 - [target] [editor] editor does not fetch latest versions of plugins - even after pressing update
Summary: [target] [editor] editor does not fetch latest versions of plugins - even aft...
Status: NEW
Alias: None
Product: PDE
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 4.4   Edit
Hardware: PC Mac OS X
: P3 normal with 2 votes (vote)
Target Milestone: ---   Edit
Assignee: PDE-UI-Inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords: investigate
: 182373 (view as bug list)
Depends on: 565091
Blocks:
  Show dependency tree
 
Reported: 2014-08-29 04:45 EDT by Marcel Bruch CLA
Modified: 2022-09-27 01:26 EDT (History)
7 users (show)

See Also:


Attachments
A hello world app that uses a plugin found in a P2 (8.09 KB, application/zip)
2020-06-01 09:25 EDT, Philippe Craeye CLA
no flags Details
A simple P2, in version 1.0.0 (11.50 KB, application/zip)
2020-06-01 09:26 EDT, Philippe Craeye CLA
no flags Details
A simple P2, promoted to version 1.1.0 (11.50 KB, application/zip)
2020-06-01 09:26 EDT, Philippe Craeye CLA
no flags Details
Source code to build the P2, using Maven/Tycho (10.77 KB, application/zip)
2020-06-01 09:27 EDT, Philippe Craeye CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Marcel Bruch CLA 2014-08-29 04:45:51 EDT
We have plugins on various update sites. We use 0.0.0 for several features that change often. When I open the editor, it shows me the old version from 3 days ago without looking for newer versions. Even when pressing on the "update" button, it still shows me the 3 days old versions.

Closing the editor, open the target definition with a text editor, increasing the sequenceNumber in the xml file manually, saving and opening the target platform editor again fixes the issue. Now, I see todays' features with current versions.

I'd expect that at least by pressing update, it would update the versions?
Comment 1 Simon F. Eggler CLA 2016-02-02 04:00:35 EST
doesn't work with windows 7 either.
Comment 2 Vikas Chandra CLA 2016-02-02 04:21:32 EST
Can you try reload button?
Comment 3 Simon F. Eggler CLA 2016-02-02 11:59:03 EST
the reload button doesn't do anything. only creating a new target with the same contents helps.
Comment 4 Vikas Chandra CLA 2016-02-03 00:19:28 EST
Did you try an update after doing reload? Does that work?
Comment 5 Simon F. Eggler CLA 2016-02-03 01:31:15 EST
nope, the target-p doesn't seem to change anymore, after it has been created.
Comment 6 Vikas Chandra CLA 2016-02-03 01:38:29 EST
Can you attach a small repeatable scenario?
Comment 7 Philippe Craeye CLA 2020-06-01 09:25:30 EDT
Created attachment 283110 [details]
A hello world app that uses a plugin found in a P2
Comment 8 Philippe Craeye CLA 2020-06-01 09:26:03 EDT
Created attachment 283111 [details]
A simple P2, in version 1.0.0
Comment 9 Philippe Craeye CLA 2020-06-01 09:26:46 EDT
Created attachment 283112 [details]
A simple P2, promoted to version 1.1.0
Comment 10 Philippe Craeye CLA 2020-06-01 09:27:23 EDT
Created attachment 283113 [details]
Source code to build the P2, using Maven/Tycho
Comment 11 Philippe Craeye CLA 2020-06-01 09:36:13 EDT
Hello there

This is the bugzilla I've found that's the closest to my problem (also this one: 394788)

We're developing some RCP applications and use a target platform that points to several P2 sites that we automatically build using a custom process; we're currently in an active development phase and these P2 are generated quite often, so the users of these P2 expect to get the latest versions of the features by just pressing "Update" on the target editor... but it does not work.

Retrieving the latest features in Eclipse is actually quite a nightmare for us. Usually, it's all about closing Eclipse, deleting the .bundle_pool folder, restarting Eclipse, pressing Update, restart sometimes several times, until we finally get the platform updated.

We started to use "0.0.0" in the feature version specified in the our .target files, like the initial bug reporter, hoping this would help.
At the beginning, we were not using 0.0.0 (I can't remember where I found the info this is supposed to mean "get the latest" version) but I also don't really understand we could do differently: when the complete feature version (included the date set in the qualifier) is somehow "hard-coded" in the .target file, as it is the first time the target definition is created - and the feature downloaded in the bundle pool for the first time-, how is supposed to work the Update button ?

Anyway, maybe the bug is in our process or the way we use the target editor, and not in Eclipse: so I tried to come up with a minimal scenario to contribute to this bugzilla as requested on comment 6.

We are currently using Eclipse Oxygen 2, linux version (running on Ubuntu 18.04).

I've just been doing some tests and tried the latest 2020-03 release and it looks like things have improved: the update button still does not work for us, but when relaunching Eclipse the target platform gets automatically updated.

I've attached several things:

- A small P2, in 2 versions: 1.0.0 and 1.1.0 (code inside is the same, only the versions for the plugin and its feature differ)
- An application plugin that uses this P2
- The source code to build the P2. 

The source code for the P2 is a set of 3 Eclipse projects + all the Maven/Tycho pom-less related files
	- org.eclipse.p2tests.plugin (which exports its main package and provides a class that prints to stdout its plugin version)
	- org.eclipse.p2tests.feature (only includes the p2tests plugin)
	- org.eclipse.p2tests.updatesite

To "upgrade" the feature, I only change the version in the MANIFEST.MF of the plugin and in the feature.xml of the feature. In the udpatesite project, I also use 0.0.0.
Just doing a "mvn clean verify" builds the P2. I then copy the result (located in the subfolder target/repository of the "updatesite" project) in "/var/www/html/p2" for my local webserver.



To quickly reproduce the bug, you can use the P2 binaries provided but I preferred to ship the source code, just in case.
I use my local web server ("http://127.0.0.1/p2/")

- Extract "442858-P2-bin-1.0.0.zip" in /var/www/html/p2 to endup with "/var/www/html/p2/features" and "/var/www/html/p2/plugins" (as well as all the index/artifacts/content files at the root of /var/www/html/p2/)
- Start Eclipse 2020-03 with a fresh, empty workspace
- "Import/General/Existing projects into Workspace/Select archive file" and choose the zip file "442858-P2-usage-application.zip"
- The project does not compile yet as the dependency against org.eclipse.p2tests.plugin is not found
- Open the ".target" file at the root of the plugin: the target editor opens. The platform is using the ${eclipse_home} variable + the site found at 127.0.0.1/p2: if the p2 has been put in the local webserver correctly, the feature "P2tests" 1.0.0 is found
- Set as active target platform
- The application now compiles. Right click on the product and "Run as eclipse application": version 1.0.0 is printed.

Everything is fine for now.

Now, on the command line, delete the /var/www/html/p2 content and replace it with the new version of the P2 1.1.0

- back to Eclipse: select the http://127.0.0.1/p2 line in the target editor and click the "Update" button: a dialog appears for one second, showing something is going on, but still the 1.0.0 version. Update/Reload does nothing.
- restart Eclipse via File/Restart
- Eclipse is back with the target editor still open: the version is now 1.1.0: OK

Same test in Eclipse Oxygen 2: when restarting Eclipse, still the version 1.0.0.

This already shows that if there is a bug in the way I generate the P2, 2 different versions of Eclipse don't behave the same way with the same generated P2.

I also found this bug: https://bugs.eclipse.org/bugs/show_bug.cgi?id=394788 which looks very similar to what I get so I tried the same test with Eclipse Mars but pressing Update and restarting Eclipse does not show the new feature.

In my research for existing bugs in the eclipse buzilla database, I've also found those:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=182373
 "[target] [p2] Have target update when remote update site changes/updates"

https://bugs.eclipse.org/bugs/show_bug.cgi?id=324344 
 "version 0.0.0 for .target resolves to '0 plug-ins available' when newer plug-ins are available in other software sites"


I finally tried to debug the Eclipse source code: I downloaded the git repositories "eclipse.pde.ui.git" and "rt.equinox.p2.git" and did some debugging, hoping to find where http requests were done using "sudo tcpdump -i lo -n port 80" and some breakpoints in the code.

When starting Eclipse 2020-03 I managed to find where the first http requests are done; this is the loadIndexFile method which eventually causes a HTTP request "HTTP: GET /p2/p2.index HTTP/1.1":

MetadataRepositoryManager(AbstractRepositoryManager<T>).loadRepository(URI, IProgressMonitor, String, int) line: 665
MetadataRepositoryManager.loadRepository(URI, int, IProgressMonitor) line: 110	
MetadataRepositoryManager.loadRepository(URI, IProgressMonitor) line: 105	
P2TargetUtils.getQueryableMetadata(URI[], IProgressMonitor) line: 976	
IUBundleContainer.getRootIUs(ITargetDefinition, IProgressMonitor) line: 763	
P2TargetUtils.getRootIUs(ITargetDefinition, IProgressMonitor) line: 1439	
P2TargetUtils.resolveWithSlicer(ITargetDefinition, IProgressMonitor) line: 1200	
P2TargetUtils.synchronize(ITargetDefinition, IProgressMonitor) line: 829	
TargetDefinition.resolve(IProgressMonitor) line: 403	
TargetEditor$TargetChangedListener$1.run(IProgressMonitor) line: 611	
Worker.run() line: 63	

On the other hand, when I change the content of the P2 repo and press the "Update" button, I get:

MetadataRepositoryManager(AbstractRepositoryManager<T>).loadRepository(URI, IProgressMonitor, String, int) line: 659	
MetadataRepositoryManager.loadRepository(URI, int, IProgressMonitor) line: 110	
MetadataRepositoryManager.loadRepository(URI, IProgressMonitor) line: 105	
P2TargetUtils.getQueryableMetadata(URI[], IProgressMonitor) line: 976	
IUBundleContainer.update(Set<Object>, IProgressMonitor) line: 385	
IUFactory.update(ITargetDefinition, ITargetLocation, IProgressMonitor) line: 85	
UpdateTargetJob.run(IProgressMonitor) line: 111	
Worker.run() line: 63	

basicGetRepository, on line 657, returns a LocalMetadataRepository non null instance and just return. No HTTP request...

I may not look at the right place so I'm sorry if my analysis is wrong but I'm willing to help more to get this problem finally solved and help my team to be happier in our day-to-day work.

Kind regards
Philippe
Comment 12 Julian Honnen CLA 2020-06-02 02:50:31 EDT
Thanks for the extensive analysis, Philippe! I'll try to reproduce your findings soon.
Comment 13 Philippe Craeye CLA 2020-06-04 01:37:56 EDT
Thank you Julian
Comment 14 Julian Honnen CLA 2020-06-16 11:06:01 EDT
As you noticed, both Reload and Update always hit the cached p2 repository and never query the remote repository again.

That cache refresh is only triggered by manually reloading the software site via the 'Available Software Sites' preference page (or restarting eclipse). PDE never does that on its own.

I think the Reload button should always include a cache refresh, given its tooltip "Reload will clear any cached target data and resolve target".
Comment 15 Julian Honnen CLA 2020-06-16 11:07:38 EDT
*** Bug 182373 has been marked as a duplicate of this bug. ***
Comment 16 Eclipse Genie CLA 2020-06-19 06:56:05 EDT
New Gerrit change created: https://git.eclipse.org/r/165209
Comment 17 Vikas Chandra CLA 2020-07-05 05:09:55 EDT
Lets take this up in next iteration.
Comment 18 Julian Honnen CLA 2020-07-06 02:31:32 EDT
(In reply to Vikas Chandra from comment #17)
> Lets take this up in next iteration.
Please check the gerrit, it needs feedback from you.
Comment 19 Vikas Chandra CLA 2020-08-27 05:53:46 EDT
(In reply to Julian Honnen from comment #18)
> (In reply to Vikas Chandra from comment #17)
> > Lets take this up in next iteration.
> Please check the gerrit, it needs feedback from you.

Lets take this up in early 4.18
Comment 20 Julian Honnen CLA 2020-10-06 02:17:07 EDT
This bug has been moved long enough. I don't see a clean solution without support from p2.
Comment 21 Eclipse Genie CLA 2022-09-27 01:26:35 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.