Re: [p2-dev] Are there order effects in composite repositories?

[David M Williams <david_williams@xxxxxxxxxx>]

Thanks everyone for your helpful replies.
I have opened an enhancement request to document the requirement and continue
the discussion, for anyone interested. 

Bug 337790
- Should be able to specify order or priority of artifact repositories


As for two of the questions ... 

> Why don't we have mirroring support for things served
from archive?

I'm not sure I understand the question, but will answer
anyway :) ... the definition of 'archive' is that they are not mirrored.
That's sort of the point. A place to put old stuff that might rarely be
needed, but not needed enough to justify "filling up" all the
mirroring space. 

> Also rather than changing the repos, why can't we
have http-level redirect for these URLs?

Not sure what problems this might solve, or if feasible,
but probably similar point, is why not just take some locations out of
the mirroring rules. Such as, given our fictional URLs, 

   <child location='http://download.eclipse.org/webtools/R3.2.0/'
/> 
   <child location='http://download.eclipse.org/webtools/R3.2.1/'
/> 
   <child location='http://download.eclipse.org/webtools/R3.2.2/'
/> 

Once "3.2.0" is considered old, we could add
a mirror rule to "do not mirror /webtools/R3.2.0/ content". (I
don't think there is a way to specify rules to remove stuff, once mirrored,
other than to literally remove it ... but, not relevant to the crux of
the issue). 

But, I'm not sure what would solve the problem ... the
problem could better be understood, probably, with more fictional examples:
For many bundles (say) each release will have a different version of the
artifact ... so, not too much of a problem here ... what is eventually
matched is probably uniquely required, so will come from where ever it
needs to come from. The problem (the inefficiency) is if there is a bundle,
say bundleA, version 1.0.0.a .. and it stays the same each release. So,
now, that bundle exists in all three artifact repos, and would be nice
to say "give preference to <certain> repositories" while
getting bundleA, version 1.0.0.a. Of course, one bundle is no big deal
... but over thousands of bundles, over many releases, seems likely we
could end up with a lot of "dups" and probably a lot of unnecessary
traffic to 'archive.eclipse.org'. 

[Now that I think about it ... perhaps I should be more
explicit ... the (composite) repo that people would be using would be only
"http://download.eclipse.org/webtools"
... so to some extent, changing the list in the composite file _is_ a type
of http redirection already.]

Hope that helps clarify ... without too much reading ....
but, let's move the discussion to Bug 337790
so issues are captured there. 

Thanks again, 







From:      
 Pascal Rapicault <pascal@xxxxxxxxxxxx>
To:      
 P2 developer discussions
<p2-dev@xxxxxxxxxxx>
Date:      
 02/21/2011 08:23 PM
Subject:    
   Re: [p2-dev]
Are there order effects in composite repositories?
Sent by:    
   p2-dev-bounces@xxxxxxxxxxx

---

Though it is true that children repos are consulted in
order, this is a not a contract.
Why don't we have mirroring support for things served
from archive?
Also rather than changing the repos, why can't we have
http-level redirect for these URLs?

On 2011-02-19, at 3:23 AM, Thomas Hallgren wrote:

Hi David,

For the meta-data, order doesn't matter since that would break the whole
idea of using a SAT algorithm to come up with a resolution. p2 will build
a set of all UI's in the repository (composite or not), and then feed that
to the SAT solver. The solver then comes up with a plan. The order of children
doesn't affect the content of the set.

For the artifacts, I know that url's with 'file:' are consulted first so
that unnecessary download traffic is avoided. I'm almost certain that with
the 'file' priority out of the way, the artifact repositories are consulted
in the order they are listed.

Regards,
Thomas Hallgren

On 2011-02-19 08:01, David M Williams wrote: 
In particular, is it part of the "spec"
or API? (That is, I'm not just asking about current implementation). 

I think this issue will become more relevant, especially for eclipse.org,
now that p2 is getting "old" :)  because some repos that
were on 'downloads' should at some point move to 'archives'. 

Let me elaborate. In a composite repo, many of us might have child repos
that might be similar to the following fictional URLs ... 

<children size='3'> 
   <child location='http://archive.eclipse.org/webtools/R3.2.0/'
/> 
   <child location='http://archive.eclipse.org/webtools/R3.2.1/'
/> 
   <child location='http://download.eclipse.org/webtools/R3.2.2/'
/> 
</children> 

Is that order, above, treated differently than the order, below? Which
repo is searched (or matched) first for artifacts? 

<children size='3'> 
   <child location='http://download.eclipse.org/webtools/R3.2.2/'
/> 
   <child location='http://archive.eclipse.org/webtools/R3.2.1/'
/> 
   <child location='http://archive.eclipse.org/webtools/R3.2.0/'
/> 
</children> 

This can become important, for example, if some artifact is the same in
all three repositories .... that is, the version/qualifier is unchanged.
If the "archive" repos are tried first, then the result would
not be mirrored, and always come from eclipse.org
directly, it seems. The opposite would be desired, that 'downloads' repos
would be "searched/matched" first, and if found there, then,
via the magic of the mirrors URL, would have a chance of coming from a
mirror, thus being faster for many users (and off-loading some bandwidth
from eclipse.org).

So, is there a way to make sure 'download' URLs are matched first? Should
they come first in list, or last ... or is the outcome indeterminable?

Any advice? 

Feel free to comment in bug 330134 ... which was opened some time ago to
"investigate" this issue ... but, I'm hoping some p2 expert knows
the answer, and I won't have to investigate :) 

https://bugs.eclipse.org/bugs/show_bug.cgi?id=330134

  
Thanks, 



_______________________________________________
p2-dev mailing list
p2-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/p2-dev


_______________________________________________
p2-dev mailing list
p2-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/p2-dev
_______________________________________________
p2-dev mailing list
p2-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/p2-dev