Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [eclipse-mirrors] What is the best way to do infrequent internal mirroring?
Thanks David for your help.  Responses inline.

On Fri, Oct 21, 2011 at 11:49 AM, David Williams <davidw@xxxxxxxxxxxxxxxxxx> wrote:
On 10/20/2011 01:59 PM, Terry Parker wrote:
I just discovered this mailing list and the eclipse.org-rsync.sh script, but I'm not sure if the standard Eclipse mirroring can be used meet our needs.  Ideally we would only mirror the most recent release (e.g., 3.7.1 without 3.7.0).  At the top level we use composite repositories to tie together offerings from Eclipse, our internally generated plug-ins, and other third party plugins, so only getting an exact release works well.  Does anyone have suggestions on the best way for us to grab the 3.7.1 and future releases?  Is that the "releases" part of the mirror?

[apologies in advance if my reply shows up multiple times ... this is my third attempt to send the message, using my correct email id.]

I would stick with the b3 aggregator approach ... but, I'm biased :) (Since I use it to produce the common repo). Using it, you'd have a full-fledge internal mirror that would be used by your internal requests. That is, some of the p2 metadata is "converted" to be its own self contained repository. If you use rsync, you end up with a copy of some of (but not, easily, all) of the files you need, but your users could still be directed back to "eclipse.org" to get some of the (meta) data, even if ended up getting some of the "large" artifacts (jars) from your internal mirror.

Yes the b3 aggregator creates a self-consistent repo--I really like that.  The problem we have with the b3 aggregator is that trying to mirror something as simple as "http://download.eclipse.org/releases/indigo/" and "http://download.eclipse.org/webtools/indigo/", the RepositoryVerifier reports "Cannot complete the install because of a conflicting dependency" for multiple versions of the same plug-in.

What does the mirrored setup look like after it is mirrored?  I'd assume there is a top-level composite repository that aggregates the original 3.N.N releases.  If that is the case, mirroring the entire repo, modifying the composite repository to cull out older Eclipse versions, and then using the b3 aggregator might be the easiest approach for us.

This rsync script (and this newsgroup list) is more for those providing mirrors for _everyone_ to use. Some eclipse.org traffic, for some artifacts, is diverted to these mirrors, but, as far as I know, there's no provision for "internal mirrors" .... there could easily be, the webmaster could answer some of that more specifically, but, I do know for sure it would not produce an "internal" p2 repository. I am sure with lots of tweaking you could end up with that ... but, easier just to use the b2 aggregator to start with, rather than using rsync and tweaks, IMHO.

http://www.eclipse.org/downloads/mir_request.php does specifically mention internal mirrors.  I'll fill in that form if it looks like running the script 3x per year will really help us. 

The fact that you get "version conflicts" sounds like a bug, somewhere ... perhaps in the way we use composite repos to combine releases and service releases ... but ... could be many other things. And, this list wouldn't be the place to discuss that ... you could open a "cross project" bug, or discuss on b3 aggregator list.

I'll open a bug about this on the b3 aggregator list after making sure my versions are all up-to-date.

In any case, I'm curious as to whether my assumption about the structure of the mirrored repos is true--do you get a top-level composite repository that aggregates 3.7.0 and 3.7.1 releases?

Thanks,
Terry

HTH


_______________________________________________
eclipse-mirrors mailing list
eclipse-mirrors@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/eclipse-mirrors


Back to the top