Bug 579503 - Update sites continue adding http:// URIs instead of https://
Summary: Update sites continue adding http:// URIs instead of https://
Status: NEW
Alias: None
Product: Platform
Classification: Eclipse Project
Component: IDE (show other bugs)
Version: 4.23   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Platform-UI-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-03-30 07:37 EDT by Martijn Dashorst CLA
Modified: 2022-04-05 02:50 EDT (History)
2 users (show)

See Also:


Attachments
Screen shot of the update sites (104.35 KB, image/png)
2022-03-30 07:37 EDT, Martijn Dashorst CLA
no flags Details
Only https sites (57.33 KB, image/png)
2022-04-05 02:46 EDT, Ed Merks CLA
no flags Details
Sites after doing a check for updates (66.08 KB, image/png)
2022-04-05 02:50 EDT, Ed Merks CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Martijn Dashorst CLA 2022-03-30 07:37:47 EDT
Created attachment 288333 [details]
Screen shot of the update sites

I've modified all my update sites to use HTTPS instead of HTTP, but somehow the plugins keep adding http URIs to the update sites, without any interaction from my part.

See attached screen shot.

- notice the progress view constantly trying to load updates from http: sites
- modify the update sites to remove/disable/update all http: entries to https:
- restart IDE

...

- notice the progress view constantly trying to load updates from http: sites

Also, this will continue to spam the log with entries that ask to switch to HTTPS.
Comment 1 Rolf Theunissen CLA 2022-03-30 11:16:00 EDT
@Ed you must be able to shed some light on this topic.
Comment 2 Ed Merks CLA 2022-03-30 11:30:51 EDT
Plugins can have touchpoints to add update sites.  E.g., like this

      <touchpointData size='1'>
        <instructions size='1'>
          <instruction key='configure'>
            addRepository(type:0,location:https${#58}//download.eclipse.org/eclipse/updates/4.23,name:The Eclipse Project Updates);addRepository(type:1,location:https${#58}//download.eclipse.org/eclipse/updates/4.23,name:The Eclipse Project Updates);addRepository(type:0,location:https${#58}//download.eclipse.org/releases/2022-03,name:2022-03);addRepository(type:1,location:https${#58}//download.eclipse.org/releases/2022-03,name:2022-03);mkdir(path:${installFolder}/dropins);
          </instruction>
        </instructions>
      </touchpointData>

So if those use http instead of https, they'll add such things when they are installed/updated.


Some of the eclipse sites listed in the picture look VERY old...
Comment 3 Martijn Dashorst CLA 2022-04-01 08:48:00 EDT
I have not installed nor updated any plugins in this reproduction scenario.

I start eclipse. Notice the continuous downloads using http:// in the Progress view. I go to update sites settings, remove all http:// entries. Restart Eclipse, and boom they are back again.
Comment 4 Ed Merks CLA 2022-04-03 03:39:38 EDT
I tried adding an update site to an installation and it remembers all my changes:

I created my installation with the installer with D:\Users\test9 as my user home and where the installation is created in D:\Users\test9\swt-master\eclipse

The bundle pool is in the default location D:\Users\test9\.p2 so the p2 "profile" preference for that installation are stored at this location.

D:\Users\test9\.p2\org.eclipse.equinox.p2.engine\profileRegistry\D__Users_test9_swt-master_eclipse.profile\.data\.settings\org.eclipse.equinox.p2.metadata.repository.prefs

The segment D__Users_test9_swt-master_eclipse.profile is derived from the installation location.

If you didn't install using the installer, the prefs will be in the installation folder here:

eclipse/p2/org.eclipse.equinox.p2.engine/profileRegistry/SDKProfile.profile/.data/.settings/org.eclipse.equinox.p2.metadata.repository.prefs

So you should be able to check that your preference changes are in fact being saved properly; the *.prefs file should be updated already when you say okay on the dialog after making changes.

If they aren't being saved, is there something in the Error log view that might shed light on what's failing.   If they are saved, does restarting really modify the preference back to the old values?  It's hard to imagine from where it could get the old values in that case...
Comment 5 Martijn Dashorst CLA 2022-04-04 05:49:19 EDT
I couldn't discover what caused the issue, so I completely nuked my local installation including any metadata residing on my computer and added our plugins necessary for developement.

It appears that jboss tools contains those http:// repositories, but I don't know why they get added to the Eclipse config.

```
<repository name='jbosstools-multiple.target' type='org.eclipse.equinox.internal.p2.metadata.repository.LocalMetadataRepository' version='1.0.0'>
  <properties size='2'>
    <property name='p2.timestamp' value='1643225615703'/>
    <property name='p2.compressed' value='true'/>
  </properties>
  <references size='4'>
    <repository uri='https://www.mihai-nita.net/eclipse' url='https://www.mihai-nita.net/eclipse' type='0' options='0'/>
    <repository uri='https://www.mihai-nita.net/eclipse' url='https://www.mihai-nita.net/eclipse' type='1' options='0'/>
    <repository uri='http://download.eclipse.org/technology/m2e/releases/' url='http://download.eclipse.org/technology/m2e/releases/' type='0' options='1'/>
    <repository uri='http://testng.org/testng-p2-update-site/' url='http://testng.org/testng-p2-update-site/' type='0' options='1'/>
  </references>

```
Comment 6 Martijn Dashorst CLA 2022-04-04 05:50:31 EDT
I had to peform this on my Eclipse.app installation to figure out what caused the inclusion:

find . -type f -name "*.jar" -exec ~/banaan.sh {} \;


Contents of banaan.sh:

unzip -q -c $1 | grep -q "http://testng.org/testng-p2-update-site/" && echo "$1 bevat http://testng.org/testng-p2-update-site/"
Comment 7 Ed Merks CLA 2022-04-04 06:09:02 EDT
Where (which path) did you locate the files with this information?

I ask because p2 downloads repository metadata files from the internet and caches them locally; what you show looks like such a cached repository's metadata.  So you can see that even if you specify https for all your repos at the top level, those repos can compose (or reference) other repos using http such that you cannot really/easily eliminate the warning messages about http being used...

I don't expect all these things (composed and reference repos) to be automatically added to your preferences though, and I can't reproduce any behavior which does that...
Comment 8 Martijn Dashorst CLA 2022-04-04 06:54:01 EDT
It was found in:

Eclipse.app/Contents/Eclipse/p2/org.eclipse.equinox.p2.repository/cache/content2075110845.jar
Comment 9 Martijn Dashorst CLA 2022-04-04 06:55:25 EDT
If you want to see the plugin list that gets imported, it is available here:

https://github.com/topicusonderwijs/topicusonderwijs/tree/master/eclipse
Comment 10 Ed Merks CLA 2022-04-04 07:01:45 EDT
(In reply to Martijn Dashorst from comment #8)
> It was found in:
> 
> Eclipse.app/Contents/Eclipse/p2/org.eclipse.equinox.p2.repository/cache/
> content2075110845.jar

Yes, that's p2's cache repository cache and any URI in one of these files is a URI that someone put in their p2 repository and then made it available on the internet.
Comment 11 Ed Merks CLA 2022-04-04 07:03:19 EDT
(In reply to Martijn Dashorst from comment #9)
> If you want to see the plugin list that gets imported, it is available here:
> 
> https://github.com/topicusonderwijs/topicusonderwijs/tree/master/eclipse

What we need is a set of steps that we could follow to reproduce the problem; it can't be fixed if we can't reproduce it...
Comment 12 Martijn Dashorst CLA 2022-04-04 07:27:21 EDT
- Download Eclipse 2022.3 for Java Developers zip package
- Go to your downloads folder using your file explorer
- Double click the Eclipse zip file
- Copy the Eclipse package to your application installation folder
- Double click the Eclipse application in the Eclipse folder in your application folder
- Open the preferences
- Go to the update sites (search for updates)
- Resize the window because
- Notice no http: locations in the update sites URIs
- Switch to your browser
- Go to the https://github.com/topicusonderwijs/topicusonderwijs/tree/master/eclipse website
- Download p2f file of your choosing, e.g. https://raw.githubusercontent.com/topicusonderwijs/topicusonderwijs/master/eclipse/eclipse-202112.p2f
- Switch back to Eclipse
- Open File -> Import... -> Installation -> Install Software Items from File
- Click Browse
- Navigate to your Downloads folder where the p2f file was downloaded
- Select the p2f file
- Click Open
- Select all plugins in the p2f file (click "Select All")
[VARIATION:] 
    - select "Contact all update sites during install to find required software"
- Click Finish
- Trust unsigned sources (click trust in the sign popup dialog)
- Wait a god awful time for the installation to complete
- Restart Eclipse
- Open the preferences
- Go to the update sites (search for updates)
- Resize the window because
- Notice the http: locations in the update sites URIs

[VARIATION:]
    - remove all http: update sites
    - restart eclipse
    - Open the preferences
    - Go to the update sites (search for updates)
    - Resize the window because
    - Notice the http: locations in the update sites URIs
Comment 13 Ed Merks CLA 2022-04-05 02:46:11 EDT
Created attachment 288373 [details]
Only https sites

I can certainly edit the sites and a restart does not loose my settings.
Comment 14 Ed Merks CLA 2022-04-05 02:50:46 EDT
Created attachment 288374 [details]
Sites after doing a check for updates

So it seems to me that my edits/preferences are never lost (you can see the "Modified to use https" Names are still there) but the other non-http versions that come from the composite sites' composed sites and referenced sites do come back as you noticed.


I don't think that's actually wrong and in the end kind of irrelevant because even if they did not come back in the preferences (where you can even disable them so they aren't actually used, directly), they will still be used indirectly because the composites use them.