Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [eclipse-mirrors] ftp.acc.umu.se never chosen by eclipse.org download site

On Tue, 28 Mar 2017, Niklas Edmundsson wrote:

On Mon, 27 Mar 2017, Denis Roy wrote:

Our most recent error was:
2017-03-26 04:16:23 Error 999 Response size: 1
http://saimei.acc.umu.se/mirror/eclipse.org/tm/downloads/drops/R-3.3.2-201202061000/RSE-runtime-3.3.2.zip


Yeah, that's likely because
http://ftp.acc.umu.se/mirror/eclipse.org/tm/downloads/drops/R-3.3.2-201202061000/RSE-runtime-3.3.2.zip
is the correct URL... You should have gotten a redirect to the correct
offloader though, not an error, but I guess 999 is something that your
script emits as it's not a HTTP error code...

But it could also happen if you expect us to sync more often than we
do (two times per day, your docco requires once a day), and punish us
for it...

Our sampling uses files >= 2 days old to avoid such penalties.  The
error code could indeed be a redirect, so we should not see any of those
anymore, as we've updated the hostname of your mirror.

If your script barfs on redirects it will always report errors for us now, as only small files are served directly by the frontend(s) responding at the ftp.acc.umu.se name.

My recommendation would be to fix the script to allow redirects, but also check that the filename component of the final redirect target is the same as the requested file (to avoid sites with mod_speling or equivalent messing things up). Comparing Content-Length with your expected file length could also be an extra safeguard.

This still seems to be an issue. Our mirror, ftp.acc.umu.se, never seems to be chosen nowadays. Likely because your mirror-status-check-thing treats redirects as errors.

I checked https://www.eclipse.org/downloads/download.php?file=/oomph/epp/neon/R3/eclipse-inst-linux64.tar.gz&format=xml where we should show up.

http://ftp.acc.umu.se/mirror/eclipse.org/oomph/epp/neon/R3/eclipse-inst-linux64.tar.gz seems correct and present on our mirror.

An interesting note though: http://eclipse.stu.edu.tw/oomph/epp/neon/R3/eclipse-inst-linux64.tar.gz shows up twice ;)

By the way, I've noticed that eclipse.org seems to prefer https nowadays. If you want, feel free to enable using https for ftp.acc.umu.se download URL:s as well. With the current hardware we can easily saturate our 20 Gbps connectivity with https traffic.

PS.
If you want to avoid maintaining your own code/stuff for mirror status/redirects I can really recommend investigating Mirrorbits https://github.com/etix/mirrorbits which comes highly recommended by various projects we mirror.
DS.


/Nikke
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
 Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se      |     nikke@xxxxxxxxxx
---------------------------------------------------------------------------
 "Oh I just suck at undercover." - Buffy
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=


Back to the top