Hermes: a fault-tolerant replication protocol, implemented over RDMA, guaranteeing linearizability and achieving low latency and high throughput.
This is the publicly available artifact repository supporting the ASPLOS'20 paper "Hermes: A Fast, Fault-Tolerant and Linearizable Replication Protocol". The repository contains both code to experimentally evaluate Hermes(KV) and complete Hermes TLA+ specifications which can be used to verify Hermes correctness via model-checking.
@inbook{Katsarakis:20,
author = {Katsarakis, Antonios and Gavrielatos, Vasilis and Katebzadeh, M.R. Siavash and Joshi, Arpit and Dragojevic, Aleksandar and Grot, Boris and Nagarajan, Vijay},
title = {Hermes: A Fast, Fault-Tolerant and Linearizable Replication Protocol},
year = {2020},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
booktitle = {Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems},
pages = {201–217},
numpages = {17}
}
Linearizable reads, writes and RMWs with the following properties:
Coupling Invalidations with per-key logical timestamps (i.e., Lamport clocks) and propagating the value to be updated with the invalidation message (early value propagation), Hermes allows any replica blocked by an update (write or RMW) to safely replay the update and unblock it self and the rest of followers.
A homogeneous cluster of x86_64 nodes interconnected via RDMA network cards and switched (tested on "Mellanox ConnectX-4" Infiniband infrastructure).
Linux OS (tested on Ubuntu 18.04 4.15.0-55-generic) with root access.
The software is tested using the following version of Mellanox OFED RDMA drivers
MLNX_OFED_LINUX-4.4-2.0.7.0
.
Third-party libraries that you will require to run the experiments include:
On every node:
./hermes/bin/setup.sh
On manager (just pick on node in the cluster):
/hermes/exec/hosts.sh
/hermes/include/hermes/config.h
/hermes/exec/
compile hermesKV through make/hermes/exec/
directory of all other nodes in the cluster.cd hermes/exec; make
Warning: Do not compile through cmake; instead use the Makefile in exec/ directory.
Run first on manager:
./run-hermesKV.sh <experiment_parameters>
Then run on all other member nodes
./run-hermesKV.sh <experiment_parameters>
Note that some members will eagerly terminate if experiment uses smaller number of nodes than specified in hosts.sh
An experiment example for three nodes 12 worker threads and 35% write ratio would be as follows:
./run-hermesKV.sh -W 12 -w 350 -M 3
Supported command-line arguments for the experiments are detailed in the run-hermesKV.sh script.
Hermes is based on HERD/MICA design as an underlying KVS, the code of which we have adapted to implement HermesKV.
Antonios Katsarakis: antonis.io
| [email protected]