Community
Participate
Working Groups
One symptom of this problem was seen yesterday in the following Bugzilla: https://bugs.eclipse.org/bugs/show_bug.cgi?id=539280 I ran into the problem this morning again. Looking into issues from the newcomers forum about problems with the latest installer on the Mac, I did a new nightly build which created new installers. Our wiki uses the appropriate URL for using mirrors to download the latest: https://www.eclipse.org/downloads/download.php?file=/oomph/products/latest/eclipse-inst-mac64.tar.gz The mirror page indicates there are currently (as I write this note) no mirrors. But when I download the contents from "the main site" I get the wrong contents. I also get the wrong contents using this URL: http://download.eclipse.org/oomph/products/latest/eclipse-inst-mac64.tar.gz I know for sure the contents are wrong; the downloaded file is not the correct size, i.e., not the size I see on the server looking at it with WinSCP. So it seems clear to me that the transparent mirroring can and does return stale content. This is a particularly bad problem when transparent mirroring is used for serving up p2 metadata at a location with changing contents, which happens with http://download.eclipse.org/releases/latest but also with http://download.eclipse.org/releases/2018-09 just minutes before the release is made available. I get the impression that transparent mirroring is not doing proper checking that its returning the proper content. I'm sure it's expensive/difficult to check the timestamp of the local version and that of the mirror to which the server redirects, so perhaps a simple check of the local timestamp to see if the content is less than a day old, and in that case return the local content would suffice...
I suspect that the server behavior can also lead to circular redirects as is suggested by this problem report: https://www.eclipse.org/forums/index.php/mv/msg/1095347/1795677/#msg_1795677
Will look at this right after EclipseCon.
If I understand correctly, the problem happens when you upload a new version of an existing file? You are sent to a mirror that still has the old version?
Exactly.
Note that in https://bugs.eclipse.org/bugs/show_bug.cgi?id=515596 it's also evident that being transparently redirected to a mirror can result in being redirected to a mirror that's unresponsive. This can make it next to impossible to download the resource you need or to access a p2 update site you need, depending on one's physical/logical location in the network. Let least if there were some query string that the server would accept to indicate "please really get this resource from the main site", or perhaps more generally a query string that allowed one to specify the mirror to be used, with the main site being one such possible choice...
(In reply to Ed Merks from comment #5) > Let least if there were some query string that the server would accept to > indicate "please really get this resource from the main site", or perhaps > more generally a query string that allowed one to specify the mirror to be > used, with the main site being one such possible choice... See bug 541186. It's impossible for some folks to even get to all the possible mirrors due to network restrictions. A query string might be too easy to abuse. However, in case the Friends of Eclipse mirror is still available this one could be a possibility in combination with authentication.
Yet more problems reported on the forum, in this case not stale content, but truncated contents: https://www.eclipse.org/forums/index.php/mv/msg/1097082/1801211/#msg_1801211
*** Bug 543464 has been marked as a duplicate of this bug. ***
FWIW, we are no longer using transparent mirroring for JAR and XZ files, so that should cover the most important content.
Ed, I'll close this as fixed. Please let me know if there are other improvements that can be made.
*** Bug 549671 has been marked as a duplicate of this bug. ***
Reopening as there is another case with bug 549671.
Thanks for the explanation in Bug 549671 and pointer here. I'm wondering, did you mean to also change the status (it still shows RESOLVED FIXED, FYI..)?
yes, reopening.
I'm considering turning this off. It's a great way to save bandwidth, but after having it on for some while it's caused many issues. And from a "are we doing the right thing" it doesn't feel so.
(In reply to Denis Roy from comment #15) > I'm considering turning this off. > > It's a great way to save bandwidth, but after having it on for some while > it's caused many issues. And from a "are we doing the right thing" it > doesn't feel so. +1
+1 I run into problems from this all the time so I can never feel sure I've really downloaded something that I'm just put in place.
Eike and I ran into what appears to be this problem yet again today. For a composite update site, p2 ended up downloading inconsistent versions of compositeArtifacts.jar and compositeContent.jar, so it could resolve the IUs to the expected latest versions, but then p2 couldn't find the artifacts because the compositeArtifacts pointed at a different build than did the compontentContent.jar. It's just super error prone and confusing that one cannot ever be really sure that one is getting the latest version of anything as opposed to some not-yet-properly-and-still-out-of-date mirrored version. And I see exactly the symptoms of this same type of problem in the newcomers forum on a semi-regular basis.
Patch is pending review.
It's been turned off.
*** Bug 541186 has been marked as a duplicate of this bug. ***
