> > Not sure if anyone is interested but we can support building bundles
> > with Tycho and deploying the bundle to the remote Nexus.
> When would the p2 metadata generation happen?
> As the bundles are deployed.
> Also what happens to the metadata in case of a copy from repos, I
> assume that Nexus stores it too?
> If by copy you mean a physically host P2 repository, we would
> regenerate the metadata to make sure it's intact.
This is a major problem. Metadata should/can not be regenerated. The
main reasons are:
1) An IU should be treated immutably. Id + version == immutable content.
The IU is for a particular artifact is the responsibility of the owner
of the artifact.
2) The binary can not always be reverse engineered to produce the IU
that the owner of the artifact would have created. It happens that you
have been successful so far because orbit metadata is a straight
transcription of the bundle manifest. However this breaks when a) the
generator is changed, b) the owner of the artifact used a p2.inf to
specify more information (e.g. overwrite capability, requirement, etc).
c) the artifact is not something that p2 metadata gen recognize.
3) Also some IUs are pure metadata, there is no artifact associated with
it that needs to be stored and deployed. An example of such a thing is
the top leve IU of a product, or the configuration units.
> for Maven repositories as well in Nexus. In the case of the Orbit
> zip we downloaded it didn't seem to have any so we just generated
> it. Right now we are using the standard P2 metadata generator but
> we're finding it uses a _lot_ of memory so we're not sure if it's
> something that's known and will be fixed. We might take a crack at
> writing one as while we are combining many repositories it's
> doubling the normal memory requirements of Nexus.
Most of the memory consumption is caused by the vast amount of strings
that result from the generation because we do not try to intern at
generation time like we do at runtime. But the core issue is the
SimpleMetadataRepository implementation that loads the complete repo. If
you are really worried about memory consumption rather than changing the
generator I recommend you to look at an alternative implementation of
the MetadataRepository. But in reality the best solution would be to
simply reuse metadata when it is provided.
[...]
> > have a "lockdown list" of sorts which is really to pick the root set
> > of IUs which basically hides any other versions for stability. A
> > profile essentially.
> In p2, a profile is the result of an installation. Do you mean that
> this is software made to work together like a "distro" (or repo
> controlled line up), e.g. all the milestones sites for a set of repos?
> Yes, it is essentially a versioned distribution. I might have 4
> versions of Subclipse in a hosted repository but I don't want the
> resolver to "see" it. Basically locking down the first level of IUs
> so that you can definitively control what the client is going to
> consume. Our use case is that for a Eclipse installation you need
> something definitive, and for server side your QA people are going
> to test a very specific set of bundles and say they are OK. You need
> to make sure that same set of bundles shows up on the server. If you
> use version ranges in your bundles to allow for dynamic updating you
> cannot point a resolver at a boundless repository. So the lock down
> lists are essentially a constraint to deal with the dichotomy of "I
> must know that what I use is what I tested, but I still must be able
> to incrementally update bundles. So this lock down (or whatever we
> end up calling it) is something that a QA engineer could manage. The
> QA person tests a new bundle and updates the lock down list and
> creates a new version which might reveal 1..N new versions of
> bundles. The client on the other side when resolving will now see
> the new versions and update. We didn't really see a large managed P2
> repository being very useful for production purposes without this
> constraining layer. The case where you are continually deploying to
> a P2 repository and updating, you definitely do not want a resolver
> picking up new versions just because they happen to be in the
> repository. We figured this provides the flexibility so that you
> have content you can actively manage as opposed to producing static
> repositories with one-time generated metadata which will soon be out
> of date. Especially in cases where you are repeatedly building new
> bundles test over and over again.
I understand the need and see where you are coming from. This is
definitely valuable when you have the complete control over the set of
repositories that any user can see at anytime. However, it is a bit
fuzzy to me on how this will work for people who are adding random repos
that they found on web. Eventually would not they be exposed to content
they are not supposed to see?
At eclipse, bundle providers usually maintain several sites for
different purpose (N build, M build, Release) with Galileo as the main
integration hub for the community. This is our cheap version of this
lockdown :) Also in the eclipse eco-system, this problem has not been
encountered as much because people are still using features which
provide a lock down aspect from the producer point of view thus limiting
the set of choices and drastically reducing the search space. That said,
we are not super happy about this and we will be looking at addressing
this in 3.6.
