Re: [tycho-dev] [discuss] tycho repository layout and metadata format

[Igor Fedorenko <igor@xxxxxxxxxxxxxx>]

I think we should discuss Tycho repository format and reproducible
dependency resolution separately. We need to deal with reproducible
builds even if we decide to only support "standard" p2 repository
layouts in tycho, so no need to couple the two discussions/decisions.


Also see my commends about reproducible resolution inline

--
Regards,
Igor

On 11-06-01 8:03 AM, Sievers, Jan wrote:
> see inline.
>
> > How do you plan to define and manage the list?
> [...]
> > * as a list of IUs that is then fed as p2 resolver input. how do
> > you plan to produce and manage the list in this case? target
> > platforms can easily have 1000x of IUs, so I don't think it is
> > feasible to manage by hand. if the list is initially resolved from
> > some top-level constraints (eclipse sdk 3.6.2, wtp 3.0.2, m2e
> > 0.12.1, etc) than re-resolving this list for a single dependency
> > change may result in arbitrary large changes to the resolution
> > result.
>
> This will need tooling support.
>
> How about starting with a small set of (top-level) features to
> define the target platform in a file. (transitive) feature includes
> have perfect version matches and using slicer mode this would mean
> that there is no "wiggle room".

Heh. This was exactly my plan when I started to integrate p2 into tycho
few years back. I actually had working prototype that took top-level
features, calculated resolved dependency set and reported on loose IUs.
There was also a way to provide additional resolution constrains to
further lock down the resolution. I think this is the right approach,
but as I mentioned earlier, I am not positive this is possible to
implement in p2 3.6+.

> Later when doing a patch, I have to make sure the feature IU list is
> translated into the equivalent fine-grained (bundle) IU list. E.g.
> the .target editor does this for me if I switch to the "Content" tab
> (I know the editor is crappy but the point is here that there could
> be tooling that does the resolution for me). Now I have control on
> bundle level.
>
> My main point for having a whitelist for defining the target
> platform resolution scope is:
>
> If we don't restrict resolution to a whitelist, any bundle with
> matching Export-Package for one of your Import-Package requirements
> can ruin your build when deployed on the global repo you resolve
> against.

Yes, I completely understand the problem. Btw, the problem is not
limited to Import-Package, but affects all requirements with version
ranges and even requirements with strict versions as p2 does not force
uniqueness of provided capability (ns,id,version) 3-tuple.

> I also see that you don't always want to have this strict
> reproducibility mode, but we should offer it e.g. for release
> builds.

Agree. This only makes sense during release process or as part of
pre-release lock down. Forcing strict build reproducibility during
development is likely impossible and certainly counter-productive.

> Regards, Jan