[platform-update-dev] Using standalone updater to mirror Callisto - miss

["Daffin, Miles (IT)" <Miles.Daffin@xxxxxxxxxxxxxxxxx>]

Hi All,

 

I do not seem to be able to create a faithful mirror of callisto using the standalone updater: I am missing some plugins. I ran the following:

 

java org.eclipse.core.launcher.Main
    -application org.eclipse.update.core.standaloneUpdate
    -command mirror
    -from http://download.eclipse.org/callisto/releases
    -to C:\tmp\eclipseide\mirrors\callisto
    -mirrorURL http://localhost/eclipseide/mirrors/callisto
    -ignoreMissingPlugins true

 

This threw no errors and seemed to result in a viable Callisto mirror. I put this on a test server, added the site to the update manager's list, visited the site and attempted to install the C/C++ Dev tools from it. I got the following error:

 

---

org.eclipse.update.core^1245e75 Downloading...:http://afsweb.ms.com/ms/user/d/daffinm/san/home/workshop/eclipseide/mirror-callisto/3.2.0/callisto/features/org.eclipse.cdt_3.1.0.200606261600.jar

!ENTRY org.eclipse.update.core 4 0 2006-08-31 16:05:41.685
!MESSAGE Error retrieving "plugins/org.eclipse.cdt.core_3.1.0.200606261600.jar". [Server returned HTTP response code: "404 Not Found" for URL: http://afsweb.ms.com/ms/user/d/daffinm/san/home/workshop/eclipseide/mirror-callisto/3.2.0/callisto/plugins/org.eclipse.cdt.core_3.1.0.200606261600.jar.]
---

 

I checked the Callisto site and the plugin jar is there:

 

http://download.eclipse.org/callisto/releases/plugins/org.eclipse.cdt.core_3.1.0.200606261600.jar

 

Can someone please tell me what I have to do in order to get the standalone updater to create a faithful Callisto mirror?

 

Many thanks,

 

Miles

 

---

From: platform-update-dev-bounces@xxxxxxxxxxx [mailto:platform-update-dev-bounces@xxxxxxxxxxx] On Behalf Of Dorian Birsan
Sent: Monday 31 July 2006 16:29
To: Eclipse Platform Update component developers list.
Cc: platform-update-dev-bounces@xxxxxxxxxxx; Eclipse download mirror administrators discussion and announcements; Cena, Bernard (IT); Eclipse Platform Update component developers list.
Subject: RE: [platform-update-dev] RE: [eclipse-mirrors] Errors when usingthestandalone updaterto mirror Callisto

Good work,Miles.

See some answers below for questions 1 and 2.

Regards,
Dorian

platform-update-dev-bounces@xxxxxxxxxxx wrote on 07/31/2006 11:00:49 AM:

> Dear All,
>  
> With Kim's help I can now create mirrors of external update sites using the
> standalone updater. This is marvellous and I hope the script I contributed to
> automate this process will save future generations much pain and trouble. I
> now have some additional questions on this topic. I hope you can help. (If
> anyone knows a better list to send them to please tell me and I will bother
> them instead.)
>  
> Thanks in advance,
>  
> Miles
>  
> ---
>  
> QU 1:
>  
> As you all know, the 3.2 install comes with 2 update sites pre-installed: The
> Eclipse Project Updates & Callisto Discovery Site. I want to configure these
> to point to my internal mirrors. How can I do this? (Both the edit and remove
> buttons are disabled in the Update Manager dialog).

Does this help?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=75045

>  
> (Why do I want to do this? Because I want to make sure that users cannot
> update directly from the outside world. I have created a master update policy
> by merging the site.xml files created by the standalone updater BUT I believe
> that this will not prevent the update manager bypassing my mirrors if the
> external site publishes new updates that are not matched by my policy. Yes, I
> could add a wild card catch all at the bottom of the policy but the simplest
> thing would be to remove all references to the Callisto and Eclipse external sites.)
>  
> QU 2:
>  
> Can anyone explain (or give me a URL to the docs - I have searched in vain)
> the distinction between an 'Update' and a 'Discovery' site?

http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-update-home/doc/eclipse_update_packaging.html
In particular, this short description:

• <url> - optional URL specifying site(s) contain feature updates, or new features
  • <update> - URL to go to for updates to this feature
  • <discovery> - URL to go to for new features. In general, a provider can use this element to reference its own site(s), or site(s) of partners that offer complementary features. Eclipse uses this element simply as a way to distribute new site URLs to the clients
  
  
  >  
  > QU 3:
  >  
  > I created a mirror of the checkstyle update site (http://eclipse-cs.
  > sourceforge.net/update) and added it to the update manager's list. I then used
  > this to install the checkstyle feature. When the IDE restarted I noticed a
  > second update site had been added for checkstyle (I have seen this before with
  > some of the Callisto features). What is going on here, and what (if anything)
  > I can do to prevent it happening?
  >
  > From: platform-update-dev-bounces@xxxxxxxxxxx [mailto:platform-update-dev-
  > bounces@xxxxxxxxxxx] On Behalf Of Daffin, Miles (IT)
  > Sent: 28 July 2006 16:11
  > To: Eclipse download mirror administrators discussion and announcements
  > Cc: Cena, Bernard (IT); Eclipse Platform Update component developers list.
  > Subject: [platform-update-dev] RE: [eclipse-mirrors] Errors when using
  > thestandalone updaterto mirror Callisto
  
  > Hi Kim,
  >  
  > Thank you for your reply. It seems to work fine now, although I have yet to
  > begin testing the mirrored sites.
  >  
  > I have created an ant script to automate the steps you mention (see attached).
  > I would attach this to the wiki faq but do not have login permissions.
  >  
  > FYI my problems were mostly due to a buggy proxy server. I switched proxies
  > and the strange errors went away.
  >  
  > Best regards,
  >  
  > Miles
  >
  > From: eclipse-mirrors-bounces@xxxxxxxxxxx [mailto:eclipse-mirrors-bounces@xxxxxxxxxxx]
  > On Behalf Of Kim Moir
  > Sent: 27 July 2006 16:27
  > To: Eclipse download mirror administrators discussion and announcements
  > Cc: Cena, Bernard (IT); Eclipse Platform Update component developers list.
  > Subject: Re: [eclipse-mirrors] Errors when using the standalone updaterto
  > mirror Callisto
  
  >
  > Hi Miles
  >
  > If you are interested in maintaining a local mirror, I suggest submitting a
  > request to become a mirror to the webmaster.
  >
  > http://www.eclipse.org/downloads/mir_request.php
  >
  > This page is geared more toward public mirrors but there are certainly private
  > mirrors. Rsync is much simpler approach, you just add a cronjob and updates
  > automatically appear :-)
  >
  > That being said, here is how I was able to mirror Caliisto using the update
  > manager.  Please use Eclipse 3.2, not 3.1 because there are are a number of
  > important update bugs that have fixed since 3.1
  >
  > Create a blank directory for your update site, for example
  > mkdir /home/kmoir/site
  >
  > From your eclipse3.2 directory
  > /home/kmoir/jdk1.4.2_03/jre/bin/java -cp startup.jar org.eclipse.core.
  > launcher.Main -application org.eclipse.update.core.standaloneUpdate -command
  > mirror -from http://download.eclipse.org/callisto/releases  -to
  > /home/kmoir/site -ignoreMissingPlugins true
  >
  > The mirroring command won't copy the packed jars.  So, after the update site
  > is ready,  you will have to recreate the packed jars using the following
  > command ensuring you use a 1.5 vm
  > /home/kmoir/jdk1.5.0_06/jre/bin/java -jar startup.jar -application org.
  > eclipse.update.core.siteOptimizer -jarProcessor -outputDir /home/kmoir/site -
  > processAll -pack /home/kmoir/site
  >
  > Then generate the digest
  > /home/kmoir/IBMJava2-142/jre/bin/java -jar /home/kmoir/eclipse3.
  > 2rc6/eclipse/startup.jar -application org.eclipse.update.core.siteOptimizer -
  > digestBuilder -digestOutputDir=/home/kmoir/site -siteXML=/home/kmoir/site/site.xml
  > Ensure that your site.xml specifies that it uses pack200 and the specifies the
  > location of the digest...for instance
  > <site pack200="true" digestURL="http://localmirror/updatesite/">
  >
  > These steps are summarized here...
  > http://wiki.eclipse.org/index.php/Platform-releng-
  > faq#How_can_I_run_the_update_manager_from_a_command_line_to_mirror_a_remote_site.3F
  >
  > For information on how to controlling update policies, refer to this document
  > http://help.eclipse.org/help32/index.jsp?topic=/org.eclipse.platform.doc.
  > user/tasks/tasks-37.htm
  >
  > Kim
  >
  >
  >
  >
  
  >
  > "Daffin, Miles (IT)" <Miles.Daffin@xxxxxxxxxxxxxxxxx>
  > Sent by: eclipse-mirrors-bounces@xxxxxxxxxxx
  > 07/27/2006 07:36 AM
  >
  > Please respond to
  > Eclipse download mirror administrators discussion and announcements <eclipse-
  > mirrors@xxxxxxxxxxx>
  >
  > To
  >
  > <eclipse-mirrors@xxxxxxxxxxx>
  >
  > cc
  >
  > "Cena, Bernard (IT)" <Bernard.Cena@xxxxxxxxxxxxxxxxx>, "Eclipse Platform
  > Update component developers list." <platform-update-dev@xxxxxxxxxxx>
  >
  > Subject
  >
  > [eclipse-mirrors] Errors when using the standalone updater to        mirror Callisto
  >
  >
  >
  >
  > Dear All,
  >  
  > I have been trying to mirror the Callisto Discovery and Eclipse project update
  > sites using the standalone updater (versions 3.1 and 3.2):
  >  
  > Eclipse: http://update2.eclipse.org/updates/3.2
  > Callisto: http://download3.eclipse.org/callisto/releases
  >  
  > This morning I managed to successfully mirror Eclipse using the 3.2 updater
  > (not tested it yet) but *cannot* successfully mirror Callisto using either the
  > 3.1 or 3.2 updater. I have attached the scripts I used plus the error logs
  > from the various runs to this email. Can anyone tell me what is causing these
  > errors and what I/we can do about them?
  > Best regards,
  >  
  > Miles
  > ----------------------------------------------
  > Miles Daffin
  > Morgan Stanley | EAI | MSDE
  > 20 Cabot Square | Canary Wharf | London E14 4QA | UK
  > Tel: +44 (0) 20 767 75119
  > Fax: +44 (0) 20 705 64572
  > miles.daffin@xxxxxxxxxxxxxxxxx
  >  
  >
  > NOTICE: If received in error, please destroy and notify sender. Sender does
  > not intend to waive confidentiality or privilege. Use of this email is
  > prohibited when received in error._______________________________________________
  > eclipse-mirrors mailing list
  > eclipse-mirrors@xxxxxxxxxxx
  > https://dev.eclipse.org/mailman/listinfo/eclipse-mirrors
  
  >
  > NOTICE: If received in error, please destroy and notify sender. Sender does
  > not intend to waive confidentiality or privilege. Use of this email is
  > prohibited when received in error.
  >
  > NOTICE: If received in error, please destroy and notify sender. Sender does
  > not intend to waive confidentiality or privilege. Use of this email is
  > prohibited when received in error._______________________________________________
  > platform-update-dev mailing list
  > platform-update-dev@xxxxxxxxxxx
  > https://dev.eclipse.org/mailman/listinfo/platform-update-dev
  

---

NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.