User-agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.6.0
Jonah,
I've been able to install the C/C++ IDE with the released
installer and the current repositories. Mirrors are in place for
both the train and EPP repos.
I am looking at this now to see if I can
understand. Denis/Webmaster if you can help that would
be great as I don't understand how different clients
are getting different responses.
$curl -I http://download.eclipse.org/technology/epp/packages/2020-03/compositeContent.jar
% Total % Received % Xferd Average Speed
Time Time Time Current
Dload Upload
Total Spent Left Speed
0 429 0 0 0 0 0 0
--:--:-- --:--:-- --:--:-- 0HTTP/1.1 200 OK
Server: nginx
Date: Wed, 18 Mar 2020 13:06:13 GMT
Content-Type: application/x-java-archive
Content-Length: 429
Connection: keep-alive
Last-Modified: Fri, 06 Mar 2020 14:22:22 GMT
ETag: "1ad-5a03061c70fb5"
X-NodeID: download2
X-Proxy-Cache: STALE
Accept-Ranges: bytes
When I do this it bypasses the cache:
$curl -I --header "X-Cache-Bypass: true" http://download.eclipse.org/technology/epp/packages/2020-03/compositeContent.jar
% Total % Received % Xferd Average Speed
Time Time Time Current
Dload Upload
Total Spent Left Speed
0 0 0 0 0 0 0 0
--:--:-- --:--:-- --:--:-- 0HTTP/1.1 404 Not
Found
Server: nginx
Date: Wed, 18 Mar 2020 13:09:15 GMT
Content-Type: text/html
Connection: keep-alive
Vary: Accept-Encoding
X-NodeID: download2
But a subsequent call still returns it as
existing so it does not flush the cache:
$curl -I http://download.eclipse.org/technology/epp/packages/2020-03/compositeContent.jar
% Total % Received % Xferd Average Speed
Time Time Time Current
Dload Upload
Total Spent Left Speed
0 429 0 0 0 0 0 0
--:--:-- --:--:-- --:--:-- 0HTTP/1.1 200 OK
Server: nginx
Date: Wed, 18 Mar 2020 13:09:21 GMT
Content-Type: application/x-java-archive
Content-Length: 429
Connection: keep-alive
Last-Modified: Fri, 06 Mar 2020 14:22:22 GMT
ETag: "1ad-5a03061c70fb5"
X-NodeID: download2
X-Proxy-Cache: STALE
Accept-Ranges: bytes
Both compositeContent.jar and
compositeArtifacts.jar need to yield 404 for a
HEAD request...
Note how it says "X-Proxy-Cache: STALE" in each
case.
Several calls like this:
$curl -I http://download.eclipse.org/technology/epp/packages/2020-03/content.jar
% Total % Received % Xferd Average Speed
Time Time Time Current
Dload Upload
Total Spent Left Speed
0 77058 0 0 0 0 0 0
--:--:-- --:--:-- --:--:-- 0HTTP/1.1 200 OK
Server: nginx
Date: Wed, 18 Mar 2020 13:15:15 GMT
Content-Type: application/x-java-archive
Content-Length: 77058
Connection: keep-alive
Last-Modified: Tue, 17 Mar 2020 11:19:04 GMT
ETag: "12d02-5a10b1a79ee91"
X-NodeID: download2
X-Proxy-Cache: HIT
Accept-Ranges: bytes
Show "X-Proxy-Cache: HIT".
I think both those files were deleted two hours
ago.
Regards,
Ed
On 18.03.2020 13:22, Jonah Graham wrote:
I believe everything is now in
place for the release except that I and
CI user cannot touch the directory so that
mirrors are working for p2. See Bug 561215
For
completeness... Ed's assessment of file
timestamps vs. directory is spot on,
since p2 typically uses the directory as
the mirrorsURL parameter.
Denis
On 2020-03-17 1:20 a.m., Ed Merks
wrote:
Markus,
I believe that the server's general
approach via download.php is to
consider the time stamp of the local
file versus its timestamp on the
mirror. If they match, the mirror is a
good one and will be returned in the
list. For the download folder, I expect
the timestamp is not so important
because only individual files will be
downloaded from it. E.g., for this link,
the timestamp of the file is important,
but not the timestamps of any of the
parent folders:
And here the timestamp of the 2020-03
folder itself is definitely important.
That all being said, an install will
only need to download 4 artifacts from
the EPP repo, while needing to download
100s from the train repo, so if there
were no mirrors for the EPP repo, it
would not cause such a huge problem as
it does when there are no mirrors for
the train repo. That repo already exists
and has many mirrors:
You are right. I used to be very
careful with the modification times
of files and directories, especially
the download directories are
important. I got used to using my
TIMESTAMP files that can be found
here and there on the download
server, usually in commands like
touch -r . TIMESTAMP
touch -r TIMESTAMP .
to "store" the modification times
of the current working directory,
and to use it afterwards to copy the
very same modification time back to
it.
Of course it's not clear over
which time period that
happened, but there's always
quite a bit of activity the
day of and the days after...
The mirror failures right
after the most 2019-12 release
certainly brought that
download server to its knees
enough to take note that there
was an ongoing problem. And
that's in December, close to
Christmas time, when you
wouldn't expect quite so much
either.
That's all to say that I very
much appreciate Jonah being
conservative on this front.
Note too that I see that the
mirrors are back now!
Now that I'm thinking about
it though, we should be
careful what happens to the
timestamp of the EPP folder
and the train folder when the
content/artifacts change just
minutes before the release.
If the folder's timestamp
changes, I believe all the
mirrors will be disabled again
for a while...
Regards,
Ed
On 16.03.2020 16:42, Markus
Knauer wrote:
Full agreement... 24h
is typically enough.
Sometimes, if it was
really urgent in the past,
I double-checked with
Denis and the webmasters
in order to make them
aware that there may be a
download peak. On the
other hand, the peaks are
not that high (especially
for a March release) than
they used to be.
And yes, I would
definitely vote for the
RC2a version (without
aeri).
I believe
24 hours is
normally more
than enough,
though the
problem in https://bugs.eclipse.org/bugs/show_bug.cgi?id=561125
is still a
problem so
sooner is
probably
better than
later. I
feel that it's
safe to assume
that RC2a
versus RC2
will be good
because the
difference is
really very
trivial and
I've confirmed
that the Java,
JEE and RCP
packages
installed from
this basis via
the installer
start up
fine...
Thanks
Ed for confirming. I
will do the
downloads this
evening (Ottawa
time)
Note that if
you're going
to transform
the EPP repo
into a simple
repository,
you should get
the binary/features/plugins folders
copied up into
the 2020-03
folder in
preparation as
well.
I will
place all the binary/features/plugins files in place
too, leaving just
the artifacts and
content files to
be updated Wed
morning.
I will ensure that the composites are
retuning 404 in
time for the
release too.
Thanks,
Jonah
Thanks for
all the heavy
lifting!!
Regards,
Ed
On
16.03.2020
15:42, Jonah
Graham wrote:
Hi
folks,
When
do the files
need to be in
place for
release day? I
assume today
is better than
tomorrow.
In
that case we
need to have
RC2a vs RC2
decision ASAP.
If I don't
hear anything
further I will
be making the
RC2a files the
R files.