s3sync.rb -v -r -p --delete ~/downloads/releases eclipse.foundation:
The Access denied response seems to be because the file isn't there
yet. Judging by the name of the file, I think that's a fair statement.
I have essentially three issues with Amazon:
1. Upload speed is quite slow. I can't seem to upload much faster than
3-4 megabits/sec
2. s3sync.rb wastes lots of time with retries, timeouts, dropped
connections, server error 500's etc.
3. Since s3sync seems to rely on md5 checksums to determine if a file
has changed, a sync of eclipse.org takes an insane amount of time even
if there are no changes.
Denis
On 09/29/2009 03:03 PM, Markus Knauer wrote:
How is the download content mirrored to Amazon S3?
E.g. For our cloudfront offerings I am using s3sync for doing this with
settings like this:
s3sync.rb --recursive --public-read --progress --make-dirs ...
bucket-name:...
To me it looks like there are too many "access denied" and maybe the
public read access bit is not set. Maybe this is the root cause for the
other failures?
Regards, Markus
2009/9/29 Eric Gwin <eric.gwin@xxxxxxxxxx>
Denis,
I don't know if the main proxy is just at capacity or if there is a
bigger issue. I also don't know if this will help any (but I'm hoping
so).
I'm seeing similar results when just trying to download from:
http://www.eclipse.org/downloads/download.php?file=/rt/eclipselink/nightly/2.0.0/20090929/eclipselink-2.0.0.v20090929-r5327.zip
using the Main Download Site (Canada):
http://www.eclipse.org/downloads/download.php?file=/rt/eclipselink/nightly/2.0.0/20090929/eclipselink-2.0.0.v20090929-r5327.zip&url="">
which remaps to:
http://d2u376ub0heus3.cloudfront.net/rt/eclipselink/nightly/2.0.0/20090929/eclipselink-2.0.0.v20090929-r5327.zip
Result:
<?xml version="1.0" encoding="UTF-8" ?>
- <Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>A2590BCCE1EAAAC1</RequestId>
<HostId>oVCBsBuJBZVk9imBU6k85PMPENGDq4TV/93oIp/CxFNPP3lwB4RMMr0Jg6xS0Ahi</HostId>
</Error>
-Eric
Denis Roy wrote:
On 09/29/2009 11:14 AM, Thomas Hallgren wrote:
If this list is sorted, I think the top 40 files or so (down to the
first zip) fit well under 50Mb so uploading them should be easy.
Perhaps P2 meta-data should be prioritized for upload to cloudfront
given the request frequency for those files?
p2 metadata seems to be scattered across our directory tree, making it
difficult to strategically redirect content. It's also easy to draw up
a plan 5 days after the release. I mean, would you have expected
/tools/mylyn/update/e3.4/content.jar to be a top-3 downloaded file?
But all in all, as I said earlier, I think the process when we release
needs to be improved. We can't open the taps until we are absolutely
sure that the infrastructure is ready to take the hit.
I know this is frustrating. I've spent the last five years trying to
make these releases as painless as possible, but as long as our servers
and your code keep evolving, there are always going to be some
unexpected results.
Denis
------------------------------------------------------------------------
--
Markus Knauer
EclipseSource
### phone: +49 721 664 733 0 (GMT +2)
### fax: +49 721 664 733 29
### web: www.eclipsesource.com
Innoopract Informationssysteme GmbH
Stephanienstrasse 20, 76133 Karlsruhe Germany
General Manager: Jochen Krause
Registered Office: Karlsruhe, Commercial Register Mannheim HRB 107883
_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
|
