[platform-update-dev] RE: [eclipse-mirrors] Errors when using the standa

Hi Kim,

Thank you for your reply. It seems to work fine now, although I have yet to begin testing the mirrored sites.

I have created an ant script to automate the steps you mention (see attached). I would attach this to the wiki faq but do not have login permissions.

FYI my problems were mostly due to a buggy proxy server. I switched proxies and the strange errors went away.

Best regards,

Miles

From: eclipse-mirrors-bounces@xxxxxxxxxxx [mailto:eclipse-mirrors-bounces@xxxxxxxxxxx] On Behalf Of Kim Moir
Sent: 27 July 2006 16:27
To: Eclipse download mirror administrators discussion and announcements
Cc: Cena, Bernard (IT); Eclipse Platform Update component developers list.
Subject: Re: [eclipse-mirrors] Errors when using the standalone updaterto mirror Callisto

Hi Miles

If you are interested in maintaining a local mirror, I suggest submitting a request to become a mirror to the webmaster.

http://www.eclipse.org/downloads/mir_request.php

This page is geared more toward public mirrors but there are certainly private mirrors. Rsync is much simpler approach, you just add a cronjob and updates automatically appear :-)

That being said, here is how I was able to mirror Caliisto using the update manager. Please use Eclipse 3.2, not 3.1 because there are are a number of important update bugs that have fixed since 3.1

Create a blank directory for your update site, for example
mkdir /home/kmoir/site

From your eclipse3.2 directory
/home/kmoir/jdk1.4.2_03/jre/bin/java -cp startup.jar org.eclipse.core.launcher.Main -application org.eclipse.update.core.standaloneUpdate -command mirror -from http://download.eclipse.org/callisto/releases -to /home/kmoir/site -ignoreMissingPlugins true
The mirroring command won't copy the packed jars. So, after the update site is ready, you will have to recreate the packed jars using the following command ensuring you use a 1.5 vm
/home/kmoir/jdk1.5.0_06/jre/bin/java -jar startup.jar -application org.eclipse.update.core.siteOptimizer -jarProcessor -outputDir /home/kmoir/site -processAll -pack /home/kmoir/site

Then generate the digest
/home/kmoir/IBMJava2-142/jre/bin/java -jar /home/kmoir/eclipse3.2rc6/eclipse/startup.jar -application org.eclipse.update.core.siteOptimizer -digestBuilder -digestOutputDir=/home/kmoir/site -siteXML=/home/kmoir/site/site.xml
Ensure that your site.xml specifies that it uses pack200 and the specifies the location of the digest...for instance
<site pack200="true" digestURL="http://localmirror/updatesite/">

These steps are summarized here...
http://wiki.eclipse.org/index.php/Platform-releng-faq#How_can_I_run_the_update_manager_from_a_command_line_to_mirror_a_remote_site.3F

For information on how to controlling update policies, refer to this document
http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.user/tasks/tasks-37.htm

Kim

"Daffin, Miles (IT)" <Miles.Daffin@xxxxxxxxxxxxxxxxx>
Sent by: eclipse-mirrors-bounces@xxxxxxxxxxx
07/27/2006 07:36 AM

Please respond to
Eclipse download mirror administrators discussion and announcements <eclipse-mirrors@xxxxxxxxxxx>

To
<eclipse-mirrors@xxxxxxxxxxx>

cc
"Cena, Bernard (IT)" <Bernard.Cena@xxxxxxxxxxxxxxxxx>, "Eclipse Platform Update component developers list." <platform-update-dev@xxxxxxxxxxx>

Subject
[eclipse-mirrors] Errors when using the standalone updater to mirror Callisto

Dear All,

I have been trying to mirror the Callisto Discovery and Eclipse project update sites using the standalone updater (versions 3.1 and 3.2):

Eclipse: http://update2.eclipse.org/updates/3.2
Callisto: http://download3.eclipse.org/callisto/releases

This morning I managed to successfully mirror Eclipse using the 3.2 updater (not tested it yet) but *cannot* successfully mirror Callisto using either the 3.1 or 3.2 updater. I have attached the scripts I used plus the error logs from the various runs to this email. Can anyone tell me what is causing these errors and what I/we can do about them?
Best regards,

Miles
----------------------------------------------
Miles Daffin
Morgan Stanley | EAI | MSDE
20 Cabot Square | Canary Wharf | London E14 4QA | UK
Tel: +44 (0) 20 767 75119
Fax: +44 (0) 20 705 64572
miles.daffin@xxxxxxxxxxxxxxxxx

NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error._______________________________________________ eclipse-mirrors mailing list eclipse-mirrors@xxxxxxxxxxx https://dev.eclipse.org/mailman/listinfo/eclipse-mirrors

NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.

<project name="MSDE Mirror Maker" default="none" basedir="."> <description>Build file for creating mirrors of various external Eclipse IDE Update Sites</description> <property file="build.properties"/> <property name="eclipse.main.class" value="org.eclipse.core.launcher.Main"/> <property name="eclipse.application.updater" value="org.eclipse.update.core.standaloneUpdate"/> <property name="eclipse.application.optimizer" value="org.eclipse.update.core.siteOptimizer"/> <path id="main.classpath"> <pathelement location="${eclipse.home}/startup.jar"/> </path> <fail> </fail> <target name="none"> <fail> <condition> </condition> </fail> <echo message="No default target defined. Please run one of the public targets."/> </target> <target name="mirror-all" depends="mirror-eclipse, mirror-callisto, mirror-checkstyle, mirror-p4wsad" description="Runs all mirror-site targets in sequence. (Might run in parallel later.)" /> <target name="mirror-eclipse" description="Creates a mirror of the Eclipse 3.2 update site" > <mirror-update-site site.name="eclipse"/> </target> <target name="mirror-callisto" description="Creates a mirror of the Callisto Discovery update site"> <mirror-update-site site.name="callisto"/> </target> <target name="mirror-checkstyle" description="Creates a mirror of the Checkstyle update site" > <mirror-update-site site.name="checkstyle"/> </target> <target name="mirror-p4wsad" description="Creates a mirror of the Checkstyle update site" > <mirror-update-site site.name="p4wsad"/> </target> <macrodef name="mirror-update-site" description="Uses eclipse standalone updater to mirror an external update site"> <attribute name="site.name" description="The name of the site to be mirrored, used as a prefix for finding the other props"/> <sequential> <echo message="*** Mirroring remote update site: @{site.name}"/> <echo message="- remote.site.url: ${@{site.name}.remote.site.url}"/> <echo message="- mirror.site.dir: ${@{site.name}.mirror.site.dir}"/> <echo message="- mirror.site.url: ${@{site.name}.mirror.site.url}"/> <fail unless="@{site.name}.remote.site.url" message="Property @{site.name}.remote.site.url not set!"/> <fail unless="@{site.name}.mirror.site.dir" message="Property @{site.name}.mirror.site.dir not set!"/> <fail unless="@{site.name}.mirror.site.url" message="Property @{site.name}.mirror.site.url not set!"/> <echo message="*** Step 1: Running mirror command"/> <run-mirror-command remote.site.url="${@{site.name}.remote.site.url}" mirror.site.dir="${@{site.name}.mirror.site.dir}" mirror.site.url="${@{site.name}.mirror.site.url}"/> <echo message="*** Step 2: Generating packed jars (1.5 vm required)"/> <generate-packed-jars mirror.site.dir="${@{site.name}.mirror.site.dir}"/> <echo message="*** Step 3: Generating site digest"/> <generate-digest mirror.site.dir="${@{site.name}.mirror.site.dir}"/> <echo message="*** Remote site mirrored here: ${@{site.name}.mirror.site.dir}"/> </sequential> </macrodef>  <macrodef name="run-mirror-command"> <attribute name="remote.site.url" description="The url of the remote site to be mirrored (base dir containing site.xml)"/> <attribute name="mirror.site.dir" description="Local directory where the mirror will be created"/> <attribute name="mirror.site.url" description="URL of the mirror site (used in the generated update policy.xml file)"/> <sequential> <mkdir dir="@{mirror.site.dir}" description="Create mirror site dir if it does not already exist." /> <java classname="${eclipse.main.class}" fork="true" classpathref="main.classpath" failonerror="true"> <sysproperty key="proxyHost" value="${proxy.host}"/> <sysproperty key="proxyPort" value="${proxy.port}"/> <arg line="-application ${eclipse.application.updater} -command mirror -from @{remote.site.url} -to @{mirror.site.dir} -mirrorURL @{mirror.site.url} -data ${basedir} -ignoreMissingPlugins true"/> </java> </sequential> </macrodef>  <macrodef name="generate-packed-jars"> <attribute name="mirror.site.dir" description="Local directory where the mirror will be created"/> <sequential> <java classname="${eclipse.main.class}" fork="true" classpathref="main.classpath" failonerror="true">   <arg line="-application ${eclipse.application.optimizer} -jarProcessor -outputDir @{mirror.site.dir} -processAll -pack @{mirror.site.dir}"/> </java> </sequential> </macrodef>  <macrodef name="generate-digest"> <attribute name="mirror.site.dir" description="Local directory where the mirror will be created"/> <sequential> <java classname="${eclipse.main.class}" fork="true" classpathref="main.classpath" failonerror="true">   <arg line="-application ${eclipse.application.optimizer} -digestBuilder -digestOutputDir=@{mirror.site.dir} -siteXML=@{mirror.site.dir}/site.xml"/> </java> </sequential> </macrodef> </project>