gokrazy rsync
Package rsync contains a native Go rsync implementation.
This repository currently contains:
gokr-rsyncd
, a read-only rsync daemon sender-only Go implementation of
rsyncd. It implements the rsync daemon network protocol (port 873/tcp by
default), but can be used over SSH or locally as well.gokr-rsync
is an rsync receiver implementation that can download files via
rsync (daemon protocol or SSH).The following known improvements are not yet implemented:
gokr-rsync
also implement an rsync sender so that it can push
(upload) files to a remote rsync server (daemon protocol or SSH).gokr-rsync
chroot (and/or Linux mount namespaces when available?)
into the destination directory to reduce chances of accidental file system
manipulation in case of bugs.gokr-rsyncd
also implement an rsync receiver, so that it can accept
files.gokr-rsyncd
and gokr-rsync
into a single binary.This project accepts contributions as time permits to merge them (best effort).
This rsync implementation is very fresh. It was started in 2021 and doesn’t have many users yet.
With that warning out of the way, the rsync protocol uses MD4 checksums over file contents, so at least your file contents should never be able to be corrupted.
There is enough other functionality (delta transfers, file metadata, special files like symlinks or devices, directory structures, etc.) in the rsync protocol that provides opportunities for bugs to hide.
I recommend you carefully check that your transfers work, and please do report any issues you run into!
Language | URL | Note | Max Protocol | Server mode? |
---|---|---|---|---|
C | WayneD/rsync | original “tridge” implementation; I found older versions easier to study | 31 | ✔ yes |
C | kristapsdz/openrsync | OpenBSD, good docs | 27 | ✔ yes |
Go | gokrazy/rsync | → you are here ← | 27 | ✔ yes 🎉 |
Go | jbreiding/rsync-go | rsync algorithm | ❌ no | |
Go | kaiakz/rsync-os | only client/receiver | 27 | ❌ no |
Go | knight42 | proxy | ❌ no | |
Go | c4milo/gsync | ❌ no | ||
Java | APNIC-net/repositoryd | archived | ✔ yes | |
Java | JohannesBuchner/Jarsync | archived, internet draft RFC “The rsync Network Protocol” | ✔ yes | |
Java | perlundq/yajsync | ✔ yes | ||
C++ | gilbertchen/acrosync-library | commercial | ❌ no | |
Rust | sourcefrog/rsyn | client, “rsyn is rsync with no c” | 27 | ❌ no |
To serve the current directory via rsync on localhost:8730
, use:
go install github.com/gokrazy/rsync/cmd/gokr-rsyncd@latest
gokr-rsyncd --daemon --gokr.listen=localhost:8730 --gokr.modulemap=pwd=$PWD
You can then copy the contents of the current directory with clients such as
rsync(1)
:
% rsync -v --archive --port 8730 rsync://localhost/pwd/ quine
receiving file list ... done
created directory quine
./
.git/
[…]
.github/workflows/main.yml
LICENSE
Makefile
README.md
cmd/gokr-rsyncd/rsyncd.go
doc.go
go.mod
go.sum
internal/rsyncd/connection.go
internal/rsyncd/rsyncd.go
interop_test.go
sent 1,234 bytes received 5,678 bytes 13,824.00 bytes/sec
total size is 666 speedup is 0.10
…or openrsync(1)
, shown doing a
differential update:
% openrsync -v --archive --port 8730 rsync://localhost/pwd/ quine
socket.c:109: warning: connect refused: ::1, localhost
Transfer starting: 369 files
.git/index (1.1 KB, 100.0% downloaded)
Transfer complete: 5.5 KB sent, 1.2 KB read, 666 B file size
setup | encrypted | authenticated | private files? | privileges | protocol version | config required |
---|---|---|---|---|---|---|
1. rsync daemon protocol (TCP port 873) | ❌ no | ⚠ rsync (insecure) | ❌ only world-readable | ✔ dropped + namespace | ✔ negotiated | config required |
2. anon SSH (daemon) | ✔ yes | ✔ rsync | ❌ only world-readable | ✔ dropped + namespace | ✔ negotiated | config required |
3. SSH (command) | ✔ yes | ✔ SSH | ✔ yes | ⚠ full user | ⚠ assumed | no config |
4. SSH (daemon) | ✔ yes | ✔ SSH (+ rsync) | ✔ yes | ⚠ full user | ✔ negotiated | ~/.config/gokr-rsyncd.toml required |
Regarding protocol version “assumed”: the flags to send over the network are
computed before starting SSH and hence the remote rsync process. You might
need to specify --protocol=27
explicitly on the client. Once the connection is
established, both sides do negotiate the protocol, though.
Serving rsync daemon protocol on TCP port 873 is only safe where the network layer ensures trusted communication, e.g. in a local network (LAN), or when using Tailscale or similar. In untrusted networks, attackers can eavesdrop on file transfers and possibly even modify file contents.
Prefer setup 2 instead.
Example:
gokr-rsyncd --daemon --gokr.modulemap=module=/srv/rsync-module
rsync rsync://webserver/module/path
This setup is well suited for serving world-readable files without authentication.
Example:
gokr-rsyncd --daemon --gokr.modulemap=module=/srv/rsync-module --gokr.anonssh_listen=:22873
rsync -e ssh rsync://webserver/module/path
This setup is well suited for interactive one-off transfers or regular backups, and uses SSH for both encryption and authentication.
Note that because gokr-rsyncd
is invoked with user privileges (not root
privileges), it cannot do namespacing
and hence retains more privileges. When serving public data, it is generally
preferable to use setup 2 instead.
Note that rsync(1)
assumes the server process understands all flags that it
sends, i.e. is running the same version on client and server, or at least a
compatible-enough version. You can either specify --protocol=27
on the client,
or use setup 4, which negotiates the protocol version, side-stepping possible
compatibility gaps between rsync clients and gokr-rsyncd
.
Example:
rsync --rsync-path=gokr-rsyncd webserver:path
This setup is more reliable than setup 3 because the rsync protocol version will
be negotiated between client and server. This setup is slightly inconvenient
because it requires a config file to be present on the server in
~/.config/gokr-rsyncd.toml
.
Example:
rsync -e ssh --rsync-path=gokr-rsyncd rsync://webserver/module/path
In my tests, gokr-rsyncd
can easily transfer data at > 6 Gbit/s. The current
bottleneck is the MD4 algorithm itself (not sure whether in the “tridge” rsync
client, or in gokr-rsyncd
). Implementing support for more recent protocol
versions would help here, as these include hash algorithm negotiation with more
recent choices.
Supported environments:
In all environments, the default instructions will take care that:
gokr-rsyncd
, to guard
against data modification and data exfiltration.gokr-rsyncd
is running without privileges, as user nobody
, to limit the
scope of what an attacker can do when exploiting a vulnerability.Known gaps:
gokr-rsyncd
does not guard against denial of service attacks, i.e. consuming
too many resources (connections, bandwidth, CPU, …).
We provide a gokr-rsyncd.socket
and gokr-rsyncd.service
file for systemd. These
files enables most of systemd’s security features. You can check by running
systemd-analyze security gokr-rsyncd.service
, which should result in an
exposure level of “0.2 SAFE” as of systemd 249 (September 2021).
First, configure your server flags by creating a systemd service override file:
systemctl edit gokr-rsyncd.service
In the opened editor, change the file to:
[Service]
ExecStart=
ExecStart=/usr/bin/gokr-rsyncd --gokr.modulemap=pwd=/etc/tmpfiles.d
Close the editor and install the service using:
systemctl enable --now gokr-rsyncd.socket
Additional hardening recommendations:
gokr-rsyncd
’s built-in IP allow/deny mechanism (once implemented)IPAddressDeny
and IPAddressAllow
in gokr-rsyncd.socket
TemporaryFileSystem=
and
BindReadOnlyPaths=
directives
as described in Use TemporaryFileSystem to hide files or directories from
systemd
services. Note
that you may need to disable ProtectSystem=strict
due to a
bug.We provide a Dockerfile
for
gokr-rsyncd
.
docker run \
--read-only \
-p 127.0.0.1:8730:8730 \
-v /etc/tmpfiles.d:/srv/rsync:ro \
stapelberg/gokrazy-rsync:latest \
--gokr.modulemap=pwd=/srv/rsync
Additional hardening recommendations:
gokr-rsyncd
’s built-in IP allow/deny mechanism (once implemented)
When started as root
on Linux, gokr-rsyncd
will create a Linux mount
namespace, mount all configured
rsync modules read-only into the namespace, then change into the namespace using
chroot(2)
and drop privileges using
setuid(2)
.
Tip: you can verify which file system objects the daemon process can see by
using ls -l /proc/$(pidof gokr-rsyncd)/root/
.
Additional hardening recommendations:
gokr-rsyncd
’s built-in IP allow/deny mechanism (once implemented)When started as root
on non-Linux (e.g. Mac), gokr-rsyncd
will drop
privileges using setuid(2)
.
To prevent accidental misconfiguration, gokr-rsyncd
refuses to start when it
detects that it has write permission in any configured rsync module.