Re: [cbi-dev] Best practices for setting p2.mirrorsURL? (long)

["David M Williams" <david_williams@xxxxxxxxxx>]

I don't have a tool "magic-copy"
(but, it is a good idea :) but agree with you that "promotion"
it the right time to set the p2.mirrorsURL because we too "move them
around" without rebuilding.  

What I do is use the org.eclipse.wtp.releng.tools.addRepoPropertiesto modify the value, 
then just recreate the xz version of
contents.jar using a small bash file and the native xz Linux support. 

I can't say I'd have time to work on
it any time soon, but a "move repository" function seems worth
a CBI feature request. 




From:      
 Andreas Sewe <andreas.sewe@xxxxxxxxxxxxxx>
To:      
 Common-build Developers
discussion <cbi-dev@xxxxxxxxxxx>, 
Date:      
 07/21/2016 04:59 AM
Subject:    
   [cbi-dev] Best
practices for setting p2.mirrorsURL? (long)
Sent by:    
   cbi-dev-bounces@xxxxxxxxxxx

---

Hi,

I re-read the recent "writing to my download area" thread but
couldn't
find any reference to mirror servers, so I guess I have to ask in a new
thread:

We (Eclipse Code Recommenders) try to be good Eclipse citizens and imbue
our update sites with a p2.mirrorsURL [1]. ATM, we use the
tycho-eclipserun-plugin launching
org.eclipse.wtp.releng.tools.addRepoProperties [2] for that.

Alas, we publish several update sites: /recommenders/updates/stable,
/recommenders/updates/milestones, etc. Thus, adding a p2.mirrorsURL
during the build requires that we build several nearly identical
eclipse-repository projects with Tycho. This increases build time (~10
seconds/eclipse-repository) and consumes a lot of disk space on our HIPP
(~33 MB/eclipse-repository), as we archive the target/repository folders
for a couple of milestones.

Last but not least, due to the recent introduction of XZ-compressed
metadata, it has become hard to keep artifacts.jar and artifacts.xml.xz
in sync in a Tycho-based build (see Bug 497546 [3]).

Thus, I have come to the conclusion that setting the final p2.mirrorsURL
of an update site during the *build* is the wrong approach.

A better approach would IMHO be to set the final p2.mirrorsURL only
during *promotion* of the built update site to its final location under
/home/data/httpd/download.eclipse.org. It things were done this way, it
would also be easy to perform a move
/home/data/httpd/download.eclipse.org [4]; you would simply not set an
p2.mirrorsURL during copying.

Ideally, there exists a tool (let's call it "magic-copy" ;-)
that can be
invoked like this from a Hudson promotion job:

  magic-copy

"${HUDSON_HOME}/jobs/${PROMOTED_JOB_NAME}/builds/${PROMOTED_ID}/archive/update-site/target/repository"
    "/home/data/httpd/download.eclipse.org/recommenders/updates/stable"

--p2MirrorURL="http://www.eclipse.org/downloads/download.php?file=/recommenders/updates/stable&format=xml"

  magic-copy

"${HUDSON_HOME}/jobs/${PROMOTED_JOB_NAME}/builds/${PROMOTED_ID}/archive/update-site/target/repository"

"/home/data/httpd/archive.eclipse.org/recommenders/updates/stable/${VERSION}"


Does anyone has put together such a tool (using shell scripts,
maven-antrun-plugin, whatever)? If so, I would love to hear about this
(and other Eclipse projects probably as well), as using p2.mirrorsURL is
something we all should do, but which is currently more painful than it
should be.

Best wishes,

Andreas

[1] <https://wiki.eclipse.org/Equinox/p2/p2.mirrorsURL>
[2] <https://wiki.eclipse.org/WTP/Releng/Tools/addRepoProperties>
[3] <https://bugs.eclipse.org/bugs/show_bug.cgi?id=497546>
[4]
<https://wiki.eclipse.org/Equinox/p2/p2.mirrorsURL#Moving_a_repo_to_archive.eclipse.org>

-- 
Codetrails GmbH
The knowledge transfer company

Robert-Bosch-Str. 7, 64293 Darmstadt
Phone: +49-6151-276-7092
Mobile: +49-170-811-3791
http://www.codetrails.com/

Managing Director: Dr. Marcel Bruch
Handelsregister: Darmstadt HRB 91940
_______________________________________________
cbi-dev mailing list
cbi-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe
from this list, visit
https://dev.eclipse.org/mailman/listinfo/cbi-dev