[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [p2-dev] Are there order effects in composite repositories?

Thanks everyone for your helpful replies. I have opened an enhancement request to document the requirement and continue the discussion, for anyone interested.

Bug 337790 - Should be able to specify order or priority of artifact repositories

As for two of the questions ...

> Why don't we have mirroring support for things served from archive?

I'm not sure I understand the question, but will answer anyway :) ... the definition of 'archive' is that they are not mirrored. That's sort of the point. A place to put old stuff that might rarely be needed, but not needed enough to justify "filling up" all the mirroring space.

> Also rather than changing the repos, why can't we have http-level redirect for these URLs?

Not sure what problems this might solve, or if feasible, but probably similar point, is why not just take some locations out of the mirroring rules. Such as, given our fictional URLs,

   <child location='http://download.eclipse.org/webtools/R3.2.0/' />
   <child location='http://download.eclipse.org/webtools/R3.2.1/' />
   <child location='http://download.eclipse.org/webtools/R3.2.2/' />

Once "3.2.0" is considered old, we could add a mirror rule to "do not mirror /webtools/R3.2.0/ content". (I don't think there is a way to specify rules to remove stuff, once mirrored, other than to literally remove it ... but, not relevant to the crux of the issue).

But, I'm not sure what would solve the problem ... the problem could better be understood, probably, with more fictional examples: For many bundles (say) each release will have a different version of the artifact ... so, not too much of a problem here ... what is eventually matched is probably uniquely required, so will come from where ever it needs to come from. The problem (the inefficiency) is if there is a bundle, say bundleA, version 1.0.0.a .. and it stays the same each release. So, now, that bundle exists in all three artifact repos, and would be nice to say "give preference to <certain> repositories" while getting bundleA, version 1.0.0.a. Of course, one bundle is no big deal ... but over thousands of bundles, over many releases, seems likely we could end up with a lot of "dups" and probably a lot of unnecessary traffic to 'archive.eclipse.org'.

[Now that I think about it ... perhaps I should be more explicit ... the (composite) repo that people would be using would be only "http://download.eclipse.org/webtools" ... so to some extent, changing the list in the composite file _is_ a type of http redirection already.]

Hope that helps clarify ... without too much reading .... but, let's move the discussion to Bug 337790 so issues are captured there.

Thanks again,







From:        Pascal Rapicault <pascal@xxxxxxxxxxxx>
To:        P2 developer discussions <p2-dev@xxxxxxxxxxx>
Date:        02/21/2011 08:23 PM
Subject:        Re: [p2-dev] Are there order effects in composite repositories?
Sent by:        p2-dev-bounces@xxxxxxxxxxx




Though it is true that children repos are consulted in order, this is a not a contract.
Why don't we have mirroring support for things served from archive?
Also rather than changing the repos, why can't we have http-level redirect for these URLs?

On 2011-02-19, at 3:23 AM, Thomas Hallgren wrote:

Hi David,

For the meta-data, order doesn't matter since that would break the whole idea of using a SAT algorithm to come up with a resolution. p2 will build a set of all UI's in the repository (composite or not), and then feed that to the SAT solver. The solver then comes up with a plan. The order of children doesn't affect the content of the set.

For the artifacts, I know that url's with 'file:' are consulted first so that unnecessary download traffic is avoided. I'm almost certain that with the 'file' priority out of the way, the artifact repositories are consulted in the order they are listed.

Regards,
Thomas Hallgren

On 2011-02-19 08:01, David M Williams wrote:

In particular, is it part of the "spec" or API? (That is, I'm not just asking about current implementation).

I think this issue will become more relevant, especially for
eclipse.org, now that p2 is getting "old" :)  because some repos that were on 'downloads' should at some point move to 'archives'.

Let me elaborate. In a composite repo, many of us might have child repos that might be similar to the following fictional URLs ...


<children size='3'>

  <child location='
http://archive.eclipse.org/webtools/R3.2.0/' />
  <child location='
http://archive.eclipse.org/webtools/R3.2.1/' />
  <child location='
http://download.eclipse.org/webtools/R3.2.2/' />
</children>


Is that order, above, treated differently than the order, below? Which repo is searched (or matched) first for artifacts?


<children size='3'>

  <child location='
http://download.eclipse.org/webtools/R3.2.2/' />
  <child location='
http://archive.eclipse.org/webtools/R3.2.1/' />
  <child location='
http://archive.eclipse.org/webtools/R3.2.0/' />
</children>


This can become important, for example, if some artifact is the same in all three repositories .... that is, the version/qualifier is unchanged. If the "archive" repos are tried first, then the result would not be mirrored, and always come from
eclipse.org directly, it seems. The opposite would be desired, that 'downloads' repos would be "searched/matched" first, and if found there, then, via the magic of the mirrors URL, would have a chance of coming from a mirror, thus being faster for many users (and off-loading some bandwidth from eclipse.org).

So, is there a way to make sure 'download' URLs are matched first? Should they come first in list, or last ... or is the outcome indeterminable?


Any advice?


Feel free to comment in bug 330134 ... which was opened some time ago to "investigate" this issue ... but, I'm hoping some p2 expert knows the answer, and I won't have to investigate :)


https://bugs.eclipse.org/bugs/show_bug.cgi?id=330134

 
Thanks,




_______________________________________________
p2-dev mailing list
p2-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/p2-dev


_______________________________________________
p2-dev mailing list

p2-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/p2-dev
_______________________________________________
p2-dev mailing list
p2-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/p2-dev