Re: [tycho-user] [EXTERNAL] Re: Need info about mirroring maven artifact

[Justin Georgeson <JGeorgeson@xxxxxxx>]

Top-posting now because it's getting harder to find the chunk of new text in the middle. 

I was under the impression that nested jars could not be loaded without unpacking the plugin, hence the Eclipse-BundleShape header. Maybe I'm wrong about it being required.

The folder structure of target/libs/ vs libs/ being in your bundle has no functional impact. It just subjectively felt more correct to not package the transient build-tool folder structure into the packaged jar. 

> -----Original Message-----
> From: tycho-user-bounces@xxxxxxxxxxx [mailto:tycho-user-
> bounces@xxxxxxxxxxx] On Behalf Of David M. Karr
> Sent: Wednesday, March 09, 2016 12:56 PM
> To: Tycho user list
> Subject: Re: [tycho-user] [EXTERNAL] Re: Need info about mirroring maven
> artifacts in a p2 mirror with Nexus
> 
> On 03/09/2016 07:51 AM, Justin Georgeson wrote:
> >
> >> -----Original Message-----
> >> From: tycho-user-bounces@xxxxxxxxxxx [mailto:tycho-user-
> >> bounces@xxxxxxxxxxx] On Behalf Of David M. Karr
> >> Sent: Wednesday, March 09, 2016 9:05 AM
> >> To: Tycho user list
> >> Subject: Re: [tycho-user] [EXTERNAL] Re: Need info about mirroring
> >> maven artifacts in a p2 mirror with Nexus
> >>
> >> On 03/08/2016 11:36 AM, Justin Georgeson wrote:
> >>>> -----Original Message-----
> >>>> From: tycho-user-bounces@xxxxxxxxxxx [mailto:tycho-user-
> >>>> bounces@xxxxxxxxxxx] On Behalf Of David M. Karr
> >>>> Sent: Tuesday, March 08, 2016 1:24 PM
> >>>> To: Tycho user list
> >>>> Subject: Re: [tycho-user] [EXTERNAL] Re: Need info about mirroring
> >>>> maven artifacts in a p2 mirror with Nexus
> >>>>
> >>>> On 03/08/2016 11:20 AM, Justin Georgeson wrote:
> >>>>> You could use the maven-dependency-plugin to dynamically pull
> >>>>> those
> >>>> dependencies from a standard M2 repo, and then package them into
> >>>> your bundle jar with a Bundle-Classpath header to include those extra
> jar files.
> >>>> This also requires setting Eclipse-BundleShape header to "dir" in
> >>>> your manifest so your plugin is extracted into a folder when
> >>>> installed into a product.
> >>>>
> >>>> I've heard of this option before, but I've never fully understood
> >>>> the details well enough to move forward with it.  Are there
> >>>> existing simple (but
> >>>> complete) examples that I can reference for this?
> >>> Don't know about public examples, but the Bundle-Classpath header is
> >>> documented here
> >>>
> >>> https://urldefense.proofpoint.com/v2/url?u=http-3A__wiki.osgi.org_wi
> >>> ki
> >>> _Bundle-2DClassPath&d=CwICAg&c=PskvixtEUDK7wuWU-
> >> tIg6oKuGYBRbrMXk2FZvF0
> >>
> UfTo&r=dLxYM3PBhAqFnkH7uKz_OVZL1uyui4QoEmBCjCmEiTk&m=4I8tMYY2
> >> k40ggyQRL
> >>> -
> >>
> _8ChDDwhTU1CPSSkR9NeONVlE&s=IuB_2BnZ09zY4B_sDxEIhhi4U6U2RXhAP
> >> Rm5DoTgM
> >>> wM&e=
> >>>
> >>> The dependency plugin I would suggest using the copy goal
> >>>
> >>> https://urldefense.proofpoint.com/v2/url?u=https-
> >> 3A__maven.apache.org_
> >>> plugins_maven-2Ddependency-2Dplugin_copy-
> >> 2Dmojo.html&d=CwICAg&c=Pskvix
> >>> tEUDK7wuWU-
> >> tIg6oKuGYBRbrMXk2FZvF0UfTo&r=dLxYM3PBhAqFnkH7uKz_OVZL1uyui4
> >>> QoEmBCjCmEiTk&m=4I8tMYY2k40ggyQRL-
> >> _8ChDDwhTU1CPSSkR9NeONVlE&s=gQb0B-Wx
> >>> rhgh7Z0Dob1NxoCpI6f-j_3EY7RNChZhXVw&e=
> >>> https://urldefense.proofpoint.com/v2/url?u=https-
> >> 3A__maven.apache.org_
> >>> plugins_maven-2Ddependency-
> >> 2Dplugin_usage.html&d=CwICAg&c=PskvixtEUDK7
> >>> wuWU-
> >>
> tIg6oKuGYBRbrMXk2FZvF0UfTo&r=dLxYM3PBhAqFnkH7uKz_OVZL1uyui4QoE
> >> mBC
> >>> jCmEiTk&m=4I8tMYY2k40ggyQRL-
> >> _8ChDDwhTU1CPSSkR9NeONVlE&s=9LrgEVu4vE3zWk
> >>> OmQRiirzRMmkbmjjNshGzM2rl4hwo&e=
> >>>
> >>> Put all the jar files in a folder like "libs", at the root instead
> >>> of the target
> >> folder. This is because of how the bin.includes entries in the
> >> build.properties file are handled. In the bin.includes property you'll need
> to include the "libs"
> >> folder. Then in your Manifest you can have something like 'Bundle-
> Classpath:
> >> libs/*' and 'Eclipse-Bundle-Shape: dir'.
> >>
> >> In addition to my pending question about the purpose and need for
> >> "Eclipse- Bundle-Shape", I could use some additional clarification on
> >> the need to put this "libs" folder outside of the "target" folder.
> >> You indicate this  is because of how "bin.includes" works, but I'd
> appreciate a full explanation.
> > The Eclipse-BundleShape header tells P2 that when it installs the plugin jar
> into a product, it should expand the contents. You build produces a jar, your
> software site holds it as a jar, but at runtime (open Eclipse IDE, Help -> Install
> New Software, install your feature) it will be a folder instead of jar
> (${ECLIPSE}/plugins/com.example.plugin_1.2.3.4444/ instead of
> ${ECLIPSE}/plugins/com.example.plugin_1.2.3.4444.jar).
> >
> > https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__eclipsesource.com_
> > blogs_2009_01_20_tip-2Declipse-
> 2Dbundleshape_&d=CwICAg&c=PskvixtEUDK7w
> > uWU-
> tIg6oKuGYBRbrMXk2FZvF0UfTo&r=dLxYM3PBhAqFnkH7uKz_OVZL1uyui4QoE
> mBCj
> >
> CmEiTk&m=PWDjkoTlzkl4TAo_MaNS9v1DltsaUCYCZpwH3yRpd_w&s=XGaDb
> xkvAaGxoa-
> > H1wzRKwSMqOscFiiEenj0w3g3fl0&e=
> >
> > In regards to the bin.includes, with the exception of the compiled classes
> (the '.' in bin.includes) I believe all the other entries are relative to
> ${project.basedir}. So if you put the jar files in target/libs and added
> "target/libs" to your bin.includes, your bundle will have "target/libs" instead
> of "libs".
> 
> Well, ok, but I still have no understanding of why these things matter.
> 
> For Eclipse-Bundle-Shape, I first built and installed it without it, and I saw no
> obvious issues, and then I did the same with that property, and the only
> difference I saw was seeing that property set in the manifest.
> What is the actual impact of setting or not setting that property?
> 
> Similarly, for the location of where the jars are copied to, why does it matter
> that my "bin.includes" will have "target/libs" instead of "libs"?
> Is the point that that former path will not exist at runtime?
> >
> >>> Your bundle jar should end up with the "libs" folder at the root,
> >>> and P2 will
> >> install your bundle as an expanded folder instead of the jar file. It
> >> needs to be expanded for the jar file(s) in the "libs" folder to be
> >> loaded at runtime when the bundle is started.
> >>>>> The Nexus P2 metadata feature is useful if you're doing a 'mvn
> >>>>> deploy' of
> >>>> an OSGi compliant bundle or feature jar. It reads the metadata in
> >>>> the jar files and publishes them to a P2 repo inside the Maven
> >>>> repo. It doesn't support extra p2.inf requirements though (at least
> >>>> it didn't in my case), and the bug I field was closed as WONTFIX
> >>>> because that functionality was only beta and the comment implied a
> >>>> big internal rewrite that would obviate it. However they wouldn't
> >>>> provide any further
> >> information.
> >>>>> To use pomDependencies=consider the dependencies in the Maven
> >> repo
> >>>> have to already be OSGi bundle jars. You can have a separate Maven
> >>>> project which depends on the non-OSGI jar, uses bundle-maven-plugin
> >>>> to produce an OSGi version, 'mvn deploy' that, and then your
> >>>> eclipse-plugin project can list the modified version as a standard
> >>>> maven pom dependency in conjunction with
> >> pomDependencies=consider.
> >>>> There is no transitive resolution of these dependencies, and you'll
> >>>> still have to list the dependency in your manifest too.
> >>>>>> -----Original Message-----
> >>>>>> From: tycho-user-bounces@xxxxxxxxxxx [mailto:tycho-user-
> >>>>>> bounces@xxxxxxxxxxx] On Behalf Of David M. Karr
> >>>>>> Sent: Tuesday, March 08, 2016 12:49 PM
> >>>>>> To: Tycho user list
> >>>>>> Subject: [EXTERNAL] Re: [tycho-user] Need info about mirroring
> >>>>>> maven artifacts in a p2 mirror with Nexus
> >>>>>>
> >>>>>> On 03/08/2016 10:28 AM, Tom Bryan (tombry) wrote:
> >>>>>>> I don't remember your original use case, but is the problem
> >>>>>>> you're hitting similar to this user's situation?
> >>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-
> >>>>>> 3A__stackoverflow.com_
> >>>>>>> questions_35346113_using-2Dthird-2Dparty-2Dlibraries-2Din-
> >>>>>> 2D&d=CwICAg&
> >>>>>>> c=PskvixtEUDK7wuWU-
> >>>>>> tIg6oKuGYBRbrMXk2FZvF0UfTo&r=dLxYM3PBhAqFnkH7uKz_OV
> >>>>>>>
> ZL1uyui4QoEmBCjCmEiTk&m=5KhxhnQhnyctf3V1EaF89bva189AfbJ-
> >>>>>> cixX8Rplgy4&s=
> >>>>>>> lXkou9wjJu-b6MRCwKJbKLpLF0QCXw3WlFT0f-iiuto&e=
> >>>>>>> eclipse-rcp-tycho-app/
> >>>>>>> I think that my reply on that thread explains the current state
> >>>>>>> of affairs and the problem bridging between Eclipse and Tycho on
> >>>>>>> one hand and normal Maven artifacts and dependencies on the
> other.
> >>>>>> I guess my situation is similar to that, yes.
> >>>>>>
> >>>>>> The present build just "hard-stores" the jars in the project and
> >>>>>> references them in the classpath in the manifest. I need to
> >>>>>> remove them from the project itself and replace them with some
> >>>>>> sort of declarative dependencies, as I can't store jars in our
> >>>>>> git repo (it's a project
> >>>> policy, but I agree with it).
> >>>>>>> The problem is that even if you could automatically convert a
> >>>>>>> JAR to an Eclipse plug-in and add it to the Maven repository,
> >>>>>>> you're still going to have to reference that specific plug-in in
> >>>>>>> your Eclipse plug-in that uses that library, right?  Are you
> >>>>>>> just hoping to create some automation that follows the Maven
> >> dependency
> >>>>>>> tree for some specific artifact and then creates an equivalent
> >>>>>>> set of Eclipse "library plug-ins" with the correct transitive
> >>>>>>> references?  Once you do that, your developers will still need
> >>>>>>> all of those plug-ins in their local
> >>>>>> development environments, right?
> >>>>>>
> >>>>>> I had assumed that once the jars I need are properly
> >>>>>> OSGi-configured into the Nexus p2 repo, I can specify that repo
> >>>>>> in my target platform, along with the dependencies I need, and
> >>>>>> then my code will be
> >>>> able to use those classes.
> >>>>>> Is there anything else I need?
> >>>>>>
> >>>>>>> Tycho is great for handling Eclipse bundles, but as soon as you
> >>>>>>> try to include a complex third party (JAR) library into your RCP
> >>>>>>> application, you hit this problem.  It would be great if Tycho
> >>>>>>> users could help articulate what a workable solution would look
> like.
> >>>>>>> That way, maybe someone can solve that problem so that we don't
> >>>>>>> have a bunch of different software teams having to work around
> >>>>>>> this disconnect between the Maven world and the Eclipse-tycho
> >> world.
> >>>>>>> ---Tom
> >>>>>>>
> >>>>>>> On 3/8/16, 12:53 PM, "tycho-user-bounces@xxxxxxxxxxx on behalf
> >>>>>>> of David
> >>>>>> M.
> >>>>>>> Karr" <tycho-user-bounces@xxxxxxxxxxx on behalf of
> >>>>>>> davidmichaelkarr@xxxxxxxxx> wrote:
> >>>>>>>
> >>>>>>>> A while ago, I asked questions in here (and other channels)
> >>>>>>>> about my Eclipse plugin build, using Tycho, which was "hard-
> storing"
> >>>>>>>> some maven artifacts in the project instead of specifying them
> >>>>>>>> "declaratively".  I tried for quite a while to figure out how
> >>>>>>>> to specify them as Maven dependencies, but I never got it to
> work.
> >>>>>>>>
> >>>>>>>> I started to understand that the only way to get this to work
> >>>>>>>> was to be able to access those artifacts in a p2 repository, so
> >>>>>>>> I can specify those dependencies in my target platform.  The
> >>>>>>>> administrator of the Nexus server that I use has been working
> >>>>>>>> on this, but he's telling me that the artifacts mirrored in
> >>>>>>>> this repository have to be OSGi
> >>>>>> bundles.
> >>>>>>>> A "plain" Maven artifact jar won't work.
> >>>>>>>>
> >>>>>>>> I'm now understanding that it's straightforward to "augment" an
> >>>>>>>> existing Maven artifact with the appropriate META-
> >> INF/MANIFEST.MF
> >>>>>>>> file to have it be recognized and properly used as an OSGi bundle.
> >>>>>>>> This can be done at build time with the "maven-bundle-plugin",
> >>>>>>>> but Nexus also has the ability (I don't know the details yet)
> >>>>>>>> to semi-automatically add manifest info to artifacts so they
> >>>>>>>> can be used in a
> >>>>>> p2 repo.
> >>>>>>>> However, I'm also hearing that even if the artifact is a valid
> >>>>>>>> OSGi bundle, it won't properly work in a p2 repo unless it can
> >>>>>>>> somehow see "features".  What do I have to do to plain Maven
> >>>>>>>> artifact jars to make them usable in an Eclipse plugin build as
> >>>>>>>> repository targets in a target platform specification?
> >>>>>>>>
> >>>>>>>> _______________________________________________
> >>>>>>>> tycho-user mailing list
> >>>>>>>> tycho-user@xxxxxxxxxxx
> >>>>>>>> To change your delivery options, retrieve your password, or
> >>>>>>>> unsubscribe
> >>>>>>> >from this list, visit
> >>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-
> >> 3A__dev.eclipse.
> >>>>>>>> or
> >>>>>>>> g_
> >>>>>>>> mailman_listinfo_tycho-
> 2Duser&d=CwICAg&c=PskvixtEUDK7wuWU-
> >>>>>> tIg6oKuGYBR
> >>>>>>
> >>
> brMXk2FZvF0UfTo&r=dLxYM3PBhAqFnkH7uKz_OVZL1uyui4QoEmBCjCmEiTk
> >>>>>> &m=5Khxh
> >>>>>>>> nQhnyctf3V1EaF89bva189AfbJ-
> >>>>>> cixX8Rplgy4&s=ni2u4HPqgrBZTJ7T0YDLFXCOOBsZ
> >>>>>>>> vWVu6ULTBdsWOeg&e=
> >>>>>>>>
> >>>>>>> _______________________________________________
> >>>>>>> tycho-user mailing list
> >>>>>>> tycho-user@xxxxxxxxxxx
> >>>>>>> To change your delivery options, retrieve your password, or
> >>>>>>> unsubscribe from this list, visit
> >>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-
> >>>>>> 3A__dev.eclipse.org_m
> >>>>>>> ailman_listinfo_tycho-2Duser&d=CwICAg&c=PskvixtEUDK7wuWU-
> >>>>>> tIg6oKuGYBRbr
> >>>>>>
> >>
> MXk2FZvF0UfTo&r=dLxYM3PBhAqFnkH7uKz_OVZL1uyui4QoEmBCjCmEiTk&
> >>>>>> m=5KhxhnQh
> >>>>>>> nyctf3V1EaF89bva189AfbJ-
> >>>>>> cixX8Rplgy4&s=ni2u4HPqgrBZTJ7T0YDLFXCOOBsZvWVu
> >>>>>>> 6ULTBdsWOeg&e=
> >>>>>> _______________________________________________
> >>>>>> tycho-user mailing list
> >>>>>> tycho-user@xxxxxxxxxxx
> >>>>>> To change your delivery options, retrieve your password, or
> >>>>>> unsubscribe from this list, visit
> >>>>>> https://urldefense.proofpoint.com/v2/url?u=https-
> >>>>>> 3A__dev.eclipse.org_mailman_listinfo_tycho-
> >>>>>> 2Duser&d=CwICAg&c=PskvixtEUDK7wuWU-
> >>>>>>
> >>
> tIg6oKuGYBRbrMXk2FZvF0UfTo&r=dLxYM3PBhAqFnkH7uKz_OVZL1uyui4QoE
> >>>>>> mBCjCmEiTk&m=5KhxhnQhnyctf3V1EaF89bva189AfbJ-
> >>>>>>
> >>
> cixX8Rplgy4&s=ni2u4HPqgrBZTJ7T0YDLFXCOOBsZvWVu6ULTBdsWOeg&e=
> >>>>> _______________________________________________
> >>>>> tycho-user mailing list
> >>>>> tycho-user@xxxxxxxxxxx
> >>>>> To change your delivery options, retrieve your password, or
> >>>>> unsubscribe from this list, visit
> >>>>> https://urldefense.proofpoint.com/v2/url?u=https-
> >>>> 3A__dev.eclipse.org_m
> >>>>> ailman_listinfo_tycho-2Duser&d=CwICAg&c=PskvixtEUDK7wuWU-
> >>>> tIg6oKuGYBRbr
> >>>>
> >>
> MXk2FZvF0UfTo&r=dLxYM3PBhAqFnkH7uKz_OVZL1uyui4QoEmBCjCmEiTk&
> >>>> m=MJuWCGW7
> >>>>> LT6lrT2I3SN1QD4asTdVy-
> >>>> ZnrfCPp5OnJG0&s=w5vtyzy3eMOAszEoz1mdV4rO6_hU662m
> >>>>> FjIoDYpWWWY&e=
> >>>> _______________________________________________
> >>>> tycho-user mailing list
> >>>> tycho-user@xxxxxxxxxxx
> >>>> To change your delivery options, retrieve your password, or
> >>>> unsubscribe from this list, visit
> >>>> https://urldefense.proofpoint.com/v2/url?u=https-
> >>>> 3A__dev.eclipse.org_mailman_listinfo_tycho-
> >>>> 2Duser&d=CwICAg&c=PskvixtEUDK7wuWU-
> >>>>
> >>
> tIg6oKuGYBRbrMXk2FZvF0UfTo&r=dLxYM3PBhAqFnkH7uKz_OVZL1uyui4QoE
> >>>> mBCjCmEiTk&m=MJuWCGW7LT6lrT2I3SN1QD4asTdVy-
> >>>>
> >>
> ZnrfCPp5OnJG0&s=w5vtyzy3eMOAszEoz1mdV4rO6_hU662mFjIoDYpWWWY
> >>>> &e=
> >>> _______________________________________________
> >>> tycho-user mailing list
> >>> tycho-user@xxxxxxxxxxx
> >>> To change your delivery options, retrieve your password, or
> >>> unsubscribe from this list, visit
> >>> https://urldefense.proofpoint.com/v2/url?u=https-
> >> 3A__dev.eclipse.org_m
> >>> ailman_listinfo_tycho-2Duser&d=CwICAg&c=PskvixtEUDK7wuWU-
> >> tIg6oKuGYBRbr
> >>
> MXk2FZvF0UfTo&r=dLxYM3PBhAqFnkH7uKz_OVZL1uyui4QoEmBCjCmEiTk&
> >> m=4I8tMYY2
> >>> k40ggyQRL-
> >> _8ChDDwhTU1CPSSkR9NeONVlE&s=igbqvgJ6FWIlM36UrlL5tCha7yrpBhLr
> >>> cGIFv2fmKIE&e=
> >> _______________________________________________
> >> tycho-user mailing list
> >> tycho-user@xxxxxxxxxxx
> >> To change your delivery options, retrieve your password, or
> >> unsubscribe from this list, visit
> >> https://urldefense.proofpoint.com/v2/url?u=https-
> >> 3A__dev.eclipse.org_mailman_listinfo_tycho-
> >> 2Duser&d=CwICAg&c=PskvixtEUDK7wuWU-
> >>
> tIg6oKuGYBRbrMXk2FZvF0UfTo&r=dLxYM3PBhAqFnkH7uKz_OVZL1uyui4QoE
> >> mBCjCmEiTk&m=4I8tMYY2k40ggyQRL-
> >>
> _8ChDDwhTU1CPSSkR9NeONVlE&s=igbqvgJ6FWIlM36UrlL5tCha7yrpBhLrcGI
> >> Fv2fmKIE&e=
> > _______________________________________________
> > tycho-user mailing list
> > tycho-user@xxxxxxxxxxx
> > To change your delivery options, retrieve your password, or
> > unsubscribe from this list, visit
> > https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__dev.eclipse.org_m
> > ailman_listinfo_tycho-2Duser&d=CwICAg&c=PskvixtEUDK7wuWU-
> tIg6oKuGYBRbr
> >
> MXk2FZvF0UfTo&r=dLxYM3PBhAqFnkH7uKz_OVZL1uyui4QoEmBCjCmEiTk&
> m=PWDjkoTl
> >
> zkl4TAo_MaNS9v1DltsaUCYCZpwH3yRpd_w&s=9tQa9t0dDKghFvCDIG_wBaZ
> _GfAsO8EQ
> > GC5sW_1c_qk&e=
> 
> _______________________________________________
> tycho-user mailing list
> tycho-user@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe
> from this list, visit https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__dev.eclipse.org_mailman_listinfo_tycho-
> 2Duser&d=CwICAg&c=PskvixtEUDK7wuWU-
> tIg6oKuGYBRbrMXk2FZvF0UfTo&r=dLxYM3PBhAqFnkH7uKz_OVZL1uyui4QoE
> mBCjCmEiTk&m=PWDjkoTlzkl4TAo_MaNS9v1DltsaUCYCZpwH3yRpd_w&s=9t
> Qa9t0dDKghFvCDIG_wBaZ_GfAsO8EQGC5sW_1c_qk&e=