The server-side 'exclude' list
is this for public mirrors. This exclude list is *not* applied to
eclipseFullMirror
exclude = drops/I* drops/N* drops/M* webtools/committers/ *.jpg *.gif
callisto/staging/ callisto/testUpdates/ eclipse/testUpdates/
eclipse/updates/3.2milestones dev/TPTP* tools/cdt/builds/*
modeling/gmf/downloads/drops/B*
Perhaps you could add portions of this exclude list on your client and see how
that impacts disk space and page links. The easiest way to add this to
your script would be to 'cheat' the RSYNC variable:
Jim,
The website's 170G is based on public mirrors, not full internal mirrors.
You're getting all the nightly builds and a bunch of other stuff that gets
excluded from public mirrors.
If your users don't need nightly/integration builds, you can change this line:
RSYNC_PATH=eclipseFullMirror
to
RSYNC_PATH=eclipseMirror
If they *do* need the nightly/integration builds, we'd have to investigate
other ways of cutting down on size. I will launch a "nag"
campaign against the Eclipse developers to ensure their disk usage is not
excessive.
Thanks,
Denis
Jim Cooper wrote:
Been having problems for a few day trying to do a “full”
internal mirror of eclipse.org. Any and all help appreciated.
The web site says that the max space required is 255G, and
current used is 170G.
However, in trying to sync, we’ve kept bumping up the size,
and currently with 270G of space, it’s filling up before completion. :-\
Here’s the output from ‘df –h’:
maytag:/vol/vol10/eclipse.org
270G 270G 41M 100% /nfs/maytag/vol/vol10/eclipse.org
And here are the last few lines from the rsync:
tptp/4.5.0/TPTP-4.5.0-200802170400/tptpdc.win_ipf.sdk-TPTP-4.5.0-200802170400.zip
tptp/4.5.0/TPTP-4.5.0-200802170400/versionComparison.html
tptp/4.5.0/TPTP-4.5.0M3/tptp.runtime.allInOne.linux.gtk.x86-TPTP-4.5.0M3.zip
rsync:
close failed on
"/nfs/maytag/vol/vol10/eclipse.org/tptp/4.5.0/TPTP-4.5.0M3/.tptp.runtime.allInOne.linux.gtk.x86-TPTP-4.5.0M3.zip.wVyYoL":
No space left on device (28)
rsync
error: error in file IO (code 11) at receiver.c(555)
rsync:
connection unexpectedly closed (26243779 bytes received so far) [generator]
rsync
error: error in rsync protocol data stream (code 12) at io.c(359)
$
The rsync file is the latest from the eclipse.org site, as
evidenced by this line:
##
Last updated: 2007-05-03 by Denis Roy (webmaster@xxxxxxxxxxx)
These are my options set in eclipse.org-rsync.sh (comments
removed for brevity):
mirror_path="/nfs/maytag/vol/vol10/eclipse.org"
RSYNC=/usr/bin/rsync
LOCKFILE="/tmp/eclipse_lockfile"
RSYNC_HOST=download.eclipse.org
RSYNC_PATH=eclipseFullMirror
full_eclipse=yes
full_platform=no
platform_release=no
platform_stable=no
form
or full_eclipse.
platform_update=no
callisto=no
release_trains=no
full_birt=no
full_technology=no
full_tools=no
full_tptp=no
full_webtools=no
full_datatools=no
_______________________________________________
eclipse-mirrors mailing list
eclipse-mirrors@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/eclipse-mirrors
--
Eclipse WebMaster - webmaster@xxxxxxxxxxx
Questions? Consult the WebMaster FAQ at http://wiki.eclipse.org/index.php/Webmaster_FAQ
View my status at http://wiki.eclipse.org/index.php/WebMaster
