Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cross-project-issues-dev] What's up with download.eclipse.org?

We are trialing S3 too and have run into network issues with connections dropping. Is that what you are seeing here too?

On Tue, Sep 29, 2009 at 3:03 PM, Markus Knauer <mknauer@xxxxxxxxxxxxxxxxx> wrote:
How is the download content mirrored to Amazon S3?
E.g. For our cloudfront offerings I am using s3sync for doing this with settings like this:

  s3sync.rb --recursive --public-read --progress --make-dirs ... bucket-name:...

To me it looks like there are too many "access denied" and maybe the public read access bit is not set. Maybe this is the root cause for the other failures?

Regards, Markus

2009/9/29 Eric Gwin <eric.gwin@xxxxxxxxxx>

Denis,

I don't know if the main proxy is just at capacity or if there is a bigger issue. I also don't know if this will help any (but I'm hoping so).
I'm seeing similar results when just trying to download from:
http://www.eclipse.org/downloads/download.php?file=/rt/eclipselink/nightly/2.0.0/20090929/eclipselink-2.0.0.v20090929-r5327.zip

using the Main Download Site (Canada):
http://www.eclipse.org/downloads/download.php?file=/rt/eclipselink/nightly/2.0.0/20090929/eclipselink-2.0.0.v20090929-r5327.zip&url="">
which remaps to:
http://d2u376ub0heus3.cloudfront.net/rt/eclipselink/nightly/2.0.0/20090929/eclipselink-2.0.0.v20090929-r5327.zip

Result:

<?xml version="1.0" encoding="UTF-8" ?>
- <Error>
 <Code>AccessDenied</Code>
 <Message>Access Denied</Message>
 <RequestId>A2590BCCE1EAAAC1</RequestId>
 <HostId>oVCBsBuJBZVk9imBU6k85PMPENGDq4TV/93oIp/CxFNPP3lwB4RMMr0Jg6xS0Ahi</HostId>
 </Error>

-Eric

Denis Roy wrote:
On 09/29/2009 11:14 AM, Thomas Hallgren wrote:
If this list is sorted, I think the top 40 files or so (down to the first zip) fit well under 50Mb so uploading them should be easy. Perhaps P2 meta-data should be prioritized for upload to cloudfront given the request frequency for those files?

p2 metadata seems to be scattered across our directory tree, making it difficult to strategically redirect content.  It's also easy to draw up a plan 5 days after the release.  I mean, would you have expected /tools/mylyn/update/e3.4/content.jar to be a top-3 downloaded file?

But all in all, as I said earlier, I think the process when we release needs to be improved. We can't open the taps until we are absolutely sure that the infrastructure is ready to take the hit.

Sorry, but you are stating the obvious.  I would not deliberately open the taps if I didn't think we were ready.  Really.  We did mirror tests.  But then we had one of those unforeseen problems (see bug 289408 comment 37 <https://bugs.eclipse.org/bugs/show_bug.cgi?id=289408#c37>).  The problem is that there are many moving parts to these releases, and we seem unable to foresee those unforeseen problems.


I know this is frustrating.  I've spent the last five years trying to make these releases as painless as possible, but as long as our servers and your code keep evolving, there are always going to be some unexpected results.

Denis
------------------------------------------------------------------------


_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
 
_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev



--
Markus Knauer
EclipseSource
###   phone: +49 721 664 733 0  (GMT +2)
###     fax: +49 721 664 733 29
###     web: www.eclipsesource.com

Innoopract Informationssysteme GmbH
Stephanienstrasse 20, 76133 Karlsruhe Germany
General Manager: Jochen Krause
Registered Office: Karlsruhe, Commercial Register Mannheim HRB 107883


_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev



Back to the top