Community
Participate
Working Groups
Mirrors may have temporary and fairly short outages. They may be incomplete in some respect, or just be under very heavy load for a short period of time. In some cases an artifact may be missing physically although it's listed in the artifacts.xml (same on all mirrors). Any of those failures will increase the failure count for the mirror in question and that is somewhat fatal, since this failure count has higher precedence than download speed and geographic location. Once the count is increased, the mirror is forever deemed bad throughout the download (which can be thousands of artifacts). I think the algorithm could be improved by adding a periodic retry on mirrors with an initialRank value that indicates that it is geographically close. I also think that we should have a ratio between high transfer rate and failure count. Let's say that 5 times higher transfer rate is worth one failure. Perhaps a successful transfer should reset the failure count, or at least cut it in half so that failures are forgiven by subsequent good behavior. The fact that a mirror gets punished for problems that exists on all mirrors (the out of sync) is exceptionally bad since it will hit the top ranked mirrors first and thus, put them out of commission. I think that is what's causing bug 297408.
The text bug 297408 was supposed to show up as a link. Perhaps bugzilla fail to recognize that when it ends up with a line break.
If I remember correctly the MayInstall project had some interesting ranking (and even abort and restart) strategies. We may be able to get some code from there. Jed, could you point us at the code if it is available in the open? Thomas, Henrik do you want to take ownership of this bug?
I'll spend some time on this now.
Created attachment 158083 [details] Patch to improve mirror selection This patch changes the mirror selection in the following ways. 1. A timer is registered that decrements the failure count after some period of time. The time is dependent on the number of failures. The first failure is reset after 30 seconds, the second after 5 minutes. If the total failure count reaches above 2, the timer is not reset. 2. When comparing speed and failure counts, the comparison is using ratios. A failure ratio is considered twice as important then a speed ratio. 3. FileNotFoundExceptions are treated separately. Some files cannot be found on any mirror and it's bad to punish the most popular mirror with a failure. At least 5 FileNotFoundExceptions are needed before the failure count is increased. This is also influenced by the fact that a not found is generally very quick in comparison with a connection failure (most often a socket timeout).
I committed this to HEAD.
This new mirror ranking implementation causes Bug 317785 and breaks compatibility with JRE7. See Bug 317785 for more details. I will commit Unit-Test and Patch soon.
