Bug 559925 - With EGit 5.7, missing constraint org.apache.commons.codec.binary
Summary: With EGit 5.7, missing constraint org.apache.commons.codec.binary
Status: NEW
Alias: None
Product: PDE
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 4.15   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: PDE-UI-Inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords: investigate
Depends on:
Blocks:
 
Reported: 2020-02-07 11:28 EST by Simeon Andreev CLA
Modified: 2022-11-23 13:44 EST (History)
10 users (show)

See Also:


Attachments
Screenshot of the error. (59.07 KB, image/png)
2020-02-07 11:28 EST, Simeon Andreev CLA
no flags Details
Screenshot of the Plugin Registry view contents, from the started Eclipse application launch. (167.71 KB, image/png)
2020-02-07 11:30 EST, Simeon Andreev CLA
no flags Details
screenshot with all bundles started via plug in registry (59.66 KB, image/png)
2020-02-07 11:57 EST, Andrey Loskutov CLA
no flags Details
screenshot with all bundles started via plug in registry (104.09 KB, image/png)
2020-02-07 11:58 EST, Andrey Loskutov CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Simeon Andreev CLA 2020-02-07 11:28:57 EST
Created attachment 281740 [details]
Screenshot of the error.

We see the following problem in our product, when we install EGit 5.7: "egit_error.png"

To reproduce, I downloaded:

https://download.eclipse.org/eclipse/downloads/drops4/I20200206-1805/download.php?dropFile=eclipse-SDK-I20200206-1805-linux-gtk-x86_64.tar.gz

Then I installed EGit 5.7 via the nightly update site:

https://download.eclipse.org/egit/updates-nightly/

When I launch an Eclipse application launch, I see the error from the screenshot.

org.apache.commons.codec.binary import for [1.10.0,1.11) fails

We see 2 versions of that plug-in (1.10 and 1.13), and 2 versions of the plug-ins that require those: org.apache.httpcommoponents.httpclient. 4.5.10 and 4.5.6. Those are not singletons, so no idea what the problem is.

We do notice that the plug-in org.apache.commons.codec both exports and imports package "
org.apache.commons.codec.binary". Maybe the plug-in org.apache.commons.codec.1.10 somehow imports the .binary package from org.apache.commons.codec.1.13?
Comment 1 Simeon Andreev CLA 2020-02-07 11:30:40 EST
Created attachment 281741 [details]
Screenshot of the Plugin Registry view contents, from the started Eclipse application launch.

See the plug-ins in Eclipse, marked in red boxes.
Comment 2 Simeon Andreev CLA 2020-02-07 11:49:06 EST
In the Apache codec plug-in manifests, we see:

Export-Package: ... org.apache.commons.codec.binary;version="1.10.0"
Import-Package: ... org.apache.commons.codec.binary;resolution:=optional;version="[1.10,2)"

Export-Package: ... org.apache.commons.codec.binary;version="1.13.0"
Import-Package: ... org.apache.commons.codec.binary;resolution:=optional;version="[1.13,2)"
Comment 3 Andrey Loskutov CLA 2020-02-07 11:56:13 EST
Just to make clear who needs what and how that is all wired:

org.apache.commons.codec 1.13.0.v20200108-0001
exports org.apache.commons.codec.binary 1.13.0
that is imported by org.apache.httpcomponents.httpclient 4.5.10.v20200114-1512
that is included in org.eclipse.jgit.http.apache 5.7.0.202002052335

org.apache.commons.codec 1.10.0.v20180409-1845
??? imports optional org.apache.commons.codec.binary [1.10,2)
exports org.apache.commons.codec.binary 1.10.0
that is imported by org.apache.httpcomponents.httpclient 4.5.6.v20190503-0009
that is included in org.eclipse.ecf.filetransfer.httpclient45.feature 1.0.100.v20191024-1546

Not sure if this optional import of org.apache.commons.codec.binary [1.10,2) in the org.apache.commons.codec 1.10.0 somehow makes PDE think there is now no 1.10 version of org.apache.commons.codec.binary (because this import will import 1.13 package). I also have no idea why this import needed at all and what does it do from OSGI point of view (bundle that imports and exports "same" package).

Both org.apache.commons.codec bundles are shown in the Plug-In registry view, can be started and I see no errors in SDK.

Interesting, if I right-click on the org.apache.commons.codec bundles in Plug-In registry view, and say "Diagnose", in both cases "Diagnosis" dialog says that the *imported* "binary" packages are missing.

So is this PDE validation error, or is this real OSGI problem we have?
I believe this something more on PDE validation side.

@Thomas: WDYT?
Comment 4 Andrey Loskutov CLA 2020-02-07 11:57:00 EST
Created attachment 281742 [details]
screenshot with all bundles started via plug in registry
Comment 5 Andrey Loskutov CLA 2020-02-07 11:58:51 EST
Created attachment 281743 [details]
screenshot with all bundles started via plug in registry
Comment 6 Andrey Loskutov CLA 2020-02-10 07:35:23 EST
@Matthias, could you please give an insight, what was the reason for the original EGit change https://git.eclipse.org/r/#/c/156003/ ?

Just updating to the latest Orbit, or were there some changes in org.apache.httpcomponents.httpclient / org.apache.commons.codec that were needed by jgit or egit?

Seeing the log below I'm starting to worry if the new version will cause problems for all clients still running "old" Eclipse platforms.

So in our product that is Eclipse 4.12 based we may have troubles now also at runtime because of the new EGit?

Note: 4.15 SDK itself still includes "old" org.eclipse.ecf.filetransfer.httpclient45.feature 1.0.100.v20191024-1546 that contains "old" versions of org.apache.commons.codec 1.10.0.v20180409-1845 and org.apache.httpcomponents.httpclient 4.5.6.v20190503-0009.

@Platfrom  releng: do we plan to update org.eclipse.ecf.filetransfer.httpclient45.feature in 4.15? This will not solve the problem for older platforms however.

@All: 

please  check attachment 281750 [details] from bug 483982 that contains *runtime* error where same package is involved.

Not sure if this exact our problem here or not, here excerpt from the log:

!ENTRY org.eclipse.ecf.provider.filetransfer.httpclient45.win32 4 0 2020-02-08 09:28:55.780
!MESSAGE FrameworkEvent ERROR
!STACK 0
org.osgi.framework.BundleException: Could not resolve module: org.eclipse.ecf.provider.filetransfer.httpclient45.win32 [569]
  Unresolved requirement: Import-Package: org.apache.http.auth; version="[4.5.0,5.0.0)"
    -> Export-Package: org.apache.http.auth; bundle-symbolic-name="org.apache.httpcomponents.httpclient"; bundle-version="4.5.6.v20190503-0009"; version="4.5.6"; uses:="org.apache.http,org.apache.http.config,org.apache.http.params,org.apache.http.protocol,org.ietf.jgss"
       org.apache.httpcomponents.httpclient [135]
         Unresolved requirement: Import-Package: org.apache.commons.codec.binary; version="[1.10.0,1.11.0)"
  Unresolved requirement: Import-Package: org.apache.http.client; version="[4.5.0,5.0.0)"
    -> Export-Package: org.apache.http.client; bundle-symbolic-name="org.apache.httpcomponents.httpclient"; bundle-version="4.5.6.v20190503-0009"; version="4.5.6"; uses:="org.apache.http,org.apache.http.auth,org.apache.http.client.methods,org.apache.http.conn,org.apache.http.conn.routing,org.apache.http.cookie,org.apache.http.params,org.apache.http.protocol"
  Unresolved requirement: Import-Package: org.apache.http.impl.client; version="[4.5.0,5.0.0)"
    -> Export-Package: org.apache.http.impl.client; bundle-symbolic-name="org.apache.httpcomponents.httpclient"; bundle-version="4.5.6.v20190503-0009"; version="4.5.6"; uses:="javax.net.ssl,org.apache.commons.logging,org.apache.http,org.apache.http.auth,org.apache.http.client,org.apache.http.client.config,org.apache.http.client.entity,org.apache.http.client.methods,org.apache.http.concurrent,org.apache.http.config,org.apache.http.conn,org.apache.http.conn.routing,org.apache.http.conn.socket,org.apache.http.conn.ssl,org.apache.http.conn.util,org.apache.http.cookie,org.apache.http.impl,org.apache.http.impl.auth,org.apache.http.impl.execchain,org.apache.http.message,org.apache.http.params,org.apache.http.pool,org.apache.http.protocol"
  Unresolved requirement: Import-Package: org.apache.http.client.protocol; version="[4.5.0,5.0.0)"
    -> Export-Package: org.apache.http.client.protocol; bundle-symbolic-name="org.apache.httpcomponents.httpclient"; bundle-version="4.5.6.v20190503-0009"; version="4.5.6"; uses:="org.apache.http,org.apache.http.auth,org.apache.http.client,org.apache.http.client.config,org.apache.http.client.entity,org.apache.http.config,org.apache.http.conn.routing,org.apache.http.cookie,org.apache.http.protocol"
  Unresolved requirement: Import-Package: org.apache.http.impl.auth; version="[4.5.0,5.0.0)"
    -> Export-Package: org.apache.http.impl.auth; bundle-symbolic-name="org.apache.httpcomponents.httpclient"; bundle-version="4.5.6.v20190503-0009"; version="4.5.6"; uses:="org.apache.commons.logging,org.apache.http,org.apache.http.auth,org.apache.http.client,org.apache.http.params,org.apache.http.protocol,org.apache.http.util,org.ietf.jgss"
  Unresolved requirement: Import-Package: org.apache.http.impl.auth.win; version="[4.5.0,5.0.0)"
    -> Export-Package: org.apache.http.impl.auth.win; bundle-symbolic-name="org.apache.httpcomponents.httpclient.win"; bundle-version="4.5.6.v20190213-1947"; version="4.5.6"; x-internal:="true"; uses:="org.apache.http,org.apache.http.auth,org.apache.http.client,org.apache.http.impl.auth,org.apache.http.protocol,org.apache.http.util"
       org.apache.httpcomponents.httpclient.win [136]
         Unresolved requirement: Import-Package: org.apache.commons.codec.binary; version="[1.10.0,1.11.0)"
         Unresolved requirement: Import-Package: org.apache.http.conn.routing; version="[4.5.6,4.6.0)"
           -> Export-Package: org.apache.http.conn.routing; bundle-symbolic-name="org.apache.httpcomponents.httpclient"; bundle-version="4.5.6.v20190503-0009"; version="4.5.6"; uses:="org.apache.http,org.apache.http.protocol"
         Unresolved requirement: Import-Package: org.apache.http.client.protocol; version="[4.5.6,4.6.0)"
           -> Export-Package: org.apache.http.client.protocol; bundle-symbolic-name="org.apache.httpcomponents.httpclient"; bundle-version="4.5.6.v20190503-0009"; version="4.5.6"; uses:="org.apache.http,org.apache.http.auth,org.apache.http.client,org.apache.http.client.config,org.apache.http.client.entity,org.apache.http.config,org.apache.http.conn.routing,org.apache.http.cookie,org.apache.http.protocol"
         Unresolved requirement: Import-Package: org.apache.http.auth; version="[4.5.6,4.6.0)"
           -> Export-Package: org.apache.http.auth; bundle-symbolic-name="org.apache.httpcomponents.httpclient"; bundle-version="4.5.6.v20190503-0009"; version="4.5.6"; uses:="org.apache.http,org.apache.http.config,org.apache.http.params,org.apache.http.protocol,org.ietf.jgss"
         Unresolved requirement: Import-Package: org.apache.http.impl.auth; version="[4.5.6,4.6.0)"
           -> Export-Package: org.apache.http.impl.auth; bundle-symbolic-name="org.apache.httpcomponents.httpclient"; bundle-version="4.5.6.v20190503-0009"; version="4.5.6"; uses:="org.apache.commons.logging,org.apache.http,org.apache.http.auth,org.apache.http.client,org.apache.http.params,org.apache.http.protocol,org.apache.http.util,org.ietf.jgss"
         Unresolved requirement: Import-Package: org.apache.http.client; version="[4.5.6,4.6.0)"
           -> Export-Package: org.apache.http.client; bundle-symbolic-name="org.apache.httpcomponents.httpclient"; bundle-version="4.5.6.v20190503-0009"; version="4.5.6"; uses:="org.apache.http,org.apache.http.auth,org.apache.http.client.methods,org.apache.http.conn,org.apache.http.conn.routing,org.apache.http.cookie,org.apache.http.params,org.apache.http.protocol"
  Unresolved requirement: Import-Package: org.apache.http.client.config; version="[4.5.0,5.0.0)"
    -> Export-Package: org.apache.http.client.config; bundle-symbolic-name="org.apache.httpcomponents.httpclient"; bundle-version="4.5.6.v20190503-0009"; version="4.5.6"; uses:="org.apache.http"

	at org.eclipse.osgi.container.Module.start(Module.java:462)
	at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel$2.run(ModuleContainer.java:1844)
	at org.eclipse.osgi.internal.framework.EquinoxContainerAdaptor$1$1.execute(EquinoxContainerAdaptor.java:136)
	at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.incStartLevel(ModuleContainer.java:1837)
	at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.incStartLevel(ModuleContainer.java:1778)
	at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.doContainerStartLevel(ModuleContainer.java:1742)
	at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1664)
	at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1)
	at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:234)
	at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:345)
Comment 7 Thomas Watson CLA 2020-02-10 09:19:37 EST
(In reply to Andrey Loskutov from comment #3)
> Just to make clear who needs what and how that is all wired:
> 
> org.apache.commons.codec 1.13.0.v20200108-0001
> exports org.apache.commons.codec.binary 1.13.0
> that is imported by org.apache.httpcomponents.httpclient
> 4.5.10.v20200114-1512
> that is included in org.eclipse.jgit.http.apache 5.7.0.202002052335
> 
> org.apache.commons.codec 1.10.0.v20180409-1845
> ??? imports optional org.apache.commons.codec.binary [1.10,2)
> exports org.apache.commons.codec.binary 1.10.0
> that is imported by org.apache.httpcomponents.httpclient 4.5.6.v20190503-0009
> that is included in org.eclipse.ecf.filetransfer.httpclient45.feature
> 1.0.100.v20191024-1546
> 
> Not sure if this optional import of org.apache.commons.codec.binary [1.10,2)
> in the org.apache.commons.codec 1.10.0 somehow makes PDE think there is now
> no 1.10 version of org.apache.commons.codec.binary (because this import will
> import 1.13 package). I also have no idea why this import needed at all and
> what does it do from OSGI point of view (bundle that imports and exports
> "same" package).

This is called substitutable exports, it allows the framework to wire the import to another version that is available, this can be useful if trying to maximize the number of bundles wired to the same package version.  When the import is wired then the export will be "dropped" by the framework.

> 
> Both org.apache.commons.codec bundles are shown in the Plug-In registry
> view, can be started and I see no errors in SDK.
> 
> Interesting, if I right-click on the org.apache.commons.codec bundles in
> Plug-In registry view, and say "Diagnose", in both cases "Diagnosis" dialog
> says that the *imported* "binary" packages are missing.
> 
> So is this PDE validation error, or is this real OSGI problem we have?

I don't think this is an issue with the framework.  The framework seems to cope with the multiple versions and wires up a consistent class space both when installing the new bundles to an existing install and resolving the delta and when running with -clean to force a re-resolve of the complete installation.

> I believe this something more on PDE validation side.
> 
> @Thomas: WDYT?

Yes, this appears to be an issue with PDE and its usage of the old Equinox resolver. Back in the Luna release the Equinox Framework stopped using the equinox resolver from the package org.eclipse.osgi.service.resolver and now uses the standard OSGi Resolver implementation from Felix.  It could be that the older Equinox resolver is not handling this situation as well as the current OSGi Resolver used by the framework.  This is not surprising because the Equinox resolver has remained largely untouched since the Luna release, but has been kept around for PDE (and a small part of p2) to use.

What I find most odd about the error reported by PDE is that the import should be optional and I would expect it to not complain at all if it detects that a provider is missing.
Comment 8 Andrey Loskutov CLA 2020-02-19 04:58:17 EST
With latest nightly SDK build (I20200218-1800) ECF was updated, so the error is gone. But all old SDK target platforms are still affected of course.
Comment 9 Thomas Wolf CLA 2020-02-23 12:11:37 EST
(In reply to Thomas Watson from comment #7)
> (In reply to Andrey Loskutov from comment #3)
> > org.apache.commons.codec 1.10.0.v20180409-1845
> > ??? imports optional org.apache.commons.codec.binary [1.10,2)
> > exports org.apache.commons.codec.binary 1.10.0
> > that is imported by org.apache.httpcomponents.httpclient 4.5.6.v20190503-0009
> > that is included in org.eclipse.ecf.filetransfer.httpclient45.feature
> > 1.0.100.v20191024-1546
> 
> This is called substitutable exports, it allows the framework to wire the
> import to another version that is available, this can be useful if trying to
> maximize the number of bundles wired to the same package version.  When the
> import is wired then the export will be "dropped" by the framework.

And that of course means that if something provides commons.codec 1.13, anything that needs httpcomponents.httpclient 4.5.6 (and can't use 4.5.10) gets into trouble, since httpcomponent.httpclient 4.5.6 requires commons.codec.binary [1.10, 1.11). :-(
Comment 10 Yolian Ignatov CLA 2020-12-02 17:09:02 EST
> And that of course means that if something provides commons.codec 1.13,
> anything that needs httpcomponents.httpclient 4.5.6 (and can't use 4.5.10)
> gets into trouble, since httpcomponent.httpclient 4.5.6 requires
> commons.codec.binary [1.10, 1.11). :-(

@Thomas - a project I'm a part of has that very problem. Would you, or anyone else on this thread, have an advice for a possible solution?
Comment 11 Eclipse Genie CLA 2022-11-23 13:44:53 EST
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.