Everyday-use client-side map-aware mirror ranking tool (Arch Linux; Manjaro; custom ones)
former Rate Arch Mirrors (changed in v0.4.0) - previous README
This is a tool, which tests mirror speed for:
It uses info about submarine cables and internet exchanges (kudos to TeleGeography for data) to jump between countries and find fast mirrors. And it's fast enough to run it before each system update (~30 seconds with default options).
yay -S rate-mirrors-bin
- pre-built binary with statically linked musl
yay -S rate-mirrors
- build binary from sources, linking glibc
dynamicallyor build manually:
cargo build --release --locked
rate-mirrors {base options} subcommand {subcommand options}
rate-mirrors help
to see base options, which go before subcommand--allow-root
option.Each subcommand has its own options, so run rate-mirrors arch --help
to see
arch
specific options, which should go after arch sub-command.
rate-mirrors arch
— fetches Arch Linux mirrors, skips outdated/syncing
ones and tests them.
To backup /etc/pacman.d/mirrorlist
file and update it with the rated mirrors run the command below:
export TMPFILE="$(mktemp)"; \
sudo true; \
rate-mirrors --save=$TMPFILE arch --max-delay=43200 \
&& sudo mv /etc/pacman.d/mirrorlist /etc/pacman.d/mirrorlist-backup \
&& sudo mv $TMPFILE /etc/pacman.d/mirrorlist
Or if you don't need a backup: rate-mirrors arch | sudo tee /etc/pacman.d/mirrorlist
.
rate-mirrors archlinuxcn
- fetches Arch Linux CN mirrors and tests them
rate-mirrors archarm
— fetches Arch Linux ARM mirrors and tests them
rate-mirrors artix
— fetches Artix Linux mirrors and tests them
rate-mirrors blackarch
- fetches BlackArch mirrors and tests them
rate-mirrors cachyos
— fetches CachyOS mirrors and tests them
rate-mirrors chaotic-aur
- fetches Arch Linux Chaotic-AUR mirrors and tests them
rate-mirrors endeavouros
— fetches/reads EndeavourOS mirrors, skips outdated ones and tests them
rate-mirrors manjaro
— fetches Manjaro mirrors, skips outdated ones and tests them
rate-mirrors rebornos
— fetches RebornOS mirrors and tests them
rate-mirrors stdin
— takes mirrors from stdin
Each string should comply with one of two supported formats:
Urls should be what --path-to-test
and --path-to-return
are joined to.
e.g. we have a file with mirrors (countries are required for country-hopping):
https://mirror-a.mirrors.org/best-linux-distro/
US\thttps://mirror-b.mirrors.org/best-linux-distro/
https://mirror-c.mirrors.org/best-linux-distro/\tDE
https://mirror-d.mirrors.org/best-linux-distro/\tAustria
and we'd like to test it & format output for Arch:
cat mirrors_by_country.txt | \
rate-mirrors --concurrency=40 stdin \
--path-to-test="extra/os/x86_64/extra.files" \
--path-to-return='$repo/os/$arch' \
--comment-prefix="# " \
--output-prefix="Server = "
The tool uses the following info:
fetch mirrors from Arch Linux - Mirror status as json
skip ones, which haven’t completed syncing (--completion=1
option)
skip ones with delays-since-the-last-sync longer than 1 day (--max-delay
option)
sort mirrors by “Arch Linux - Mirror Status” score - the lower the better (--sort-mirrors-by=score_asc
option)
take the next country to explore (or --entry-country
option, US
by default -- no need to change)
find neighbor countries --country-neighbors-per-country=3
, using multiple strategies:
take --country-test-mirrors-per-country=2
mirrors per country, selected at step 6, test speed and find 2 mirrors: 1 fastest and 1 with shortest connection time
take countries of mirrors from step 7 and go to step 5
after --max-jumps=7
jumps are done, take top M mirrors by speed (--top-mirrors-number-to-retest=5
), test them with no concurrency, sort by speed and prepend to the resulting list
alias ua-drop-caches='sudo paccache -rk3; yay -Sc --aur --noconfirm'
alias ua-update-all='export TMPFILE="$(mktemp)"; \
sudo true; \
rate-mirrors --save=$TMPFILE arch --max-delay=21600 \
&& sudo mv /etc/pacman.d/mirrorlist /etc/pacman.d/mirrorlist-backup \
&& sudo mv $TMPFILE /etc/pacman.d/mirrorlist \
&& ua-drop-caches \
&& yay -Syyu --noconfirm'
Few notes:
ua-
prefix means "user alias"paccache
from pacman-contrib
packageyay
is an AUR helpersudo true
forces password prompt in the very beginningTo persist aliases, add them to ~/.zshrc
or ~/.bashrc
(based on the shell you use)
Once done, just launch a new terminal and run:
ua-update-all
Here is an example of running the arch mode from Belarus (output truncated):
# STARTED AT: 2021-06-23 21:44:49.694758291 +03:00
# ARGS: rate-mirrors arch
# FETCHED MIRRORS: 845
# MIRRORS LEFT AFTER FILTERING: 471
# JUMP #1
# EXPLORING US
# VISITED US
# + NEIGHBOR ID (by HubsFirst)
# + NEIGHBOR UK (by HubsFirst)
# + NEIGHBOR CN (by HubsFirst)
# + NEIGHBOR DE (by DistanceFirst)
# + NEIGHBOR CA (by DistanceFirst)
# + NEIGHBOR FR (by DistanceFirst)
# [US] SpeedTestResult { speed: 561.60 KB/s; elapsed: 1.087839405s; connection_time: 381.733267ms}
# [US] SpeedTestResult { speed: 1.32 MB/s; elapsed: 1.157819036s; connection_time: 276.577298ms}
# [UK] SpeedTestResult { speed: 2.65 MB/s; elapsed: 1.224126693s; connection_time: 261.353312ms}
# [UK] SpeedTestResult { speed: 5.48 MB/s; elapsed: 1.327602061s; connection_time: 94.138156ms}
# [ID] SpeedTestResult { speed: 179.68 KB/s; elapsed: 745.86963ms; connection_time: 670.100893ms}
# [DE] SpeedTestResult { speed: 1.98 MB/s; elapsed: 1.392069748s; connection_time: 106.878831ms}
# [DE] SpeedTestResult { speed: 1.76 MB/s; elapsed: 1.26812617s; connection_time: 223.358858ms}
# [CA] SpeedTestResult { speed: 1.04 MB/s; elapsed: 962.017229ms; connection_time: 517.499369ms}
# [CA] SpeedTestResult { speed: 1.67 MB/s; elapsed: 1.20327262s; connection_time: 296.884889ms}
# [FR] SpeedTestResult { speed: 2.50 MB/s; elapsed: 1.154907178s; connection_time: 341.534506ms}
# [FR] SpeedTestResult { speed: 2.25 MB/s; elapsed: 1.374341411s; connection_time: 118.696039ms}
# TOP NEIGHBOR - CONNECTION TIME: UK - 94.138156ms
# TOP NEIGHBOR - SPEED: UK - 5.48 MB/s
#
# JUMP #2
# EXPLORING UK
# + NEIGHBOR NL (by HubsFirst)
# + NEIGHBOR NO (by HubsFirst)
# + NEIGHBOR AU (by HubsFirst)
# + NEIGHBOR CZ (by DistanceFirst)
# + NEIGHBOR CH (by DistanceFirst)
# + NEIGHBOR SE (by DistanceFirst)
# ...
# JUMP #7
# ...
# RE-TESTING TOP MIRRORS
# [EE] SpeedTestResult { speed: 4.92 MB/s; elapsed: 1.320800025s; connection_time: 178.606272ms}
# [UK] SpeedTestResult { speed: 6.95 MB/s; elapsed: 791.741365ms; connection_time: 709.525808ms}
# [FI] SpeedTestResult { speed: 8.94 MB/s; elapsed: 1.376742398s; connection_time: 123.530685ms}
# [TR] SpeedTestResult { speed: 5.57 MB/s; elapsed: 1.243114863s; connection_time: 257.162963ms}
# [EE] SpeedTestResult { speed: 9.68 MB/s; elapsed: 1.41574022s; connection_time: 84.436077ms}
# ==== RESULTS (top re-tested) ====
# 1. [EE] SpeedTestResult { speed: 9.68 MB/s; elapsed: 1.41574022s; connection_time: 84.436077ms} -> http://mirror.cspacehostings.com/archlinux/
# 2. [FI] SpeedTestResult { speed: 8.94 MB/s; elapsed: 1.376742398s; connection_time: 123.530685ms} -> http://arch.mirror.far.fi/
# 3. [UK] SpeedTestResult { speed: 6.95 MB/s; elapsed: 791.741365ms; connection_time: 709.525808ms} -> http://archlinux.uk.mirror.allworldit.com/archlinux/
# 4. [TR] SpeedTestResult { speed: 5.57 MB/s; elapsed: 1.243114863s; connection_time: 257.162963ms} -> http://mirror.veriteknik.net.tr/archlinux/
# 5. [EE] SpeedTestResult { speed: 4.92 MB/s; elapsed: 1.320800025s; connection_time: 178.606272ms} -> https://mirror.cspacehostings.com/archlinux/
# 6. [BY] SpeedTestResult { speed: 3.66 MB/s; elapsed: 1.455269308s; connection_time: 44.256906ms} -> http://ftp.byfly.by/pub/archlinux/
# 7. [SE] SpeedTestResult { speed: 3.40 MB/s; elapsed: 1.270306507s; connection_time: 230.675741ms} -> https://mirror.osbeck.com/archlinux/
# 8. [BY] SpeedTestResult { speed: 3.40 MB/s; elapsed: 1.467606681s; connection_time: 32.547398ms} -> http://mirror.datacenter.by/pub/archlinux/
# ...
# FINISHED AT: 2021-06-23 21:45:15.555642390 +03:00
Server = http://mirror.cspacehostings.com/archlinux/$repo/os/$arch
Server = http://arch.mirror.far.fi/$repo/os/$arch
Server = http://archlinux.uk.mirror.allworldit.com/archlinux/$repo/os/$arch
Server = http://mirror.veriteknik.net.tr/archlinux/$repo/os/$arch
Server = https://mirror.cspacehostings.com/archlinux/$repo/os/$arch
Server = http://ftp.byfly.by/pub/archlinux/$repo/os/$arch
Server = https://mirror.osbeck.com/archlinux/$repo/os/$arch
Server = http://mirror.datacenter.by/pub/archlinux/$repo/os/$arch
The tool is made available under the following Creative Commons License: Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0).