ReSeC- Redis Service Consul
Resec is a successor to Redis Sentinel and redishappy for handling high availability failover for Redis.
It avoids Redis Sentinel problems of remembering all the sentinels and all the redis servers that ever appeared in the replication cluster.
Resec master election is based on Consul Locks to provide single redis master instance.
Resec continuously monitors the status of redis instance and if it's alive, It starts 2 following processes:
Master.Address
Resec registers service with TTL health check with TTL twice as big as HEALTHCHECK_INTERVAL
and updates consul every HEALTHCHECK_INTERVAL
to maintain service in passing state
There are 2 options to work with services:
Use CONSUL_SERVICE_NAME
for tag based master/slave discovery
MASTER_TAGS
must be provided for ability to watch master instance for changes.Use CONSUL_SERVICE_PREFIX
for service name only based discovery
CONSUL_SERVICE_PREFIX
-Replication.Role
If ANNOUNCE_ADDR
is set it will be used for registration in consul, if it's not provided REDIS_ADDR
will be used for registration in consul.
REDIS_ADDR
is localhost, only port will be announced to the consul.Environment Variables | Default | Description |
---|---|---|
ANNOUNCE_ADDR | IP:Port of Redis to be announced, by default service will be registered wi | |
CONSUL_SERVICE_NAME | Consul service name for tag based service discovery | |
CONSUL_SERVICE_PREFIX | redis | Name Prefix, will be followed by "-(master/slave)", ignored if CONSUL_SERVICE_NAME is used |
CONSUL_LOCK_KEY | resec/.lock | KV lock location, should be overriden if multiple instances running in the same consul DC |
CONSUL_LOCK_SESSION_NAME | resec | Lock session Name to distinguish multiple resec masters on one host |
CONSUL_LOCK_MONITOR_RETRIES | 3 | Number of retries of lock receives 500 Error from Consul |
CONSUL_LOCK_MONITOR_RETRY_INTERVAL | 1s | Retry interval if lock receives 500 Error from Consul |
CONSUL_DEREGISTER_SERVICE_AFTER | 72h | |
CONSUL_LOCK_TTL | 15s | |
MASTER_TAGS | Comma separated list of tags to be added to master instance. The first tag (index 0) is used to configure the role of the Redis/resec task, and must be different from index 0 in SLAVE_TAGS . |
|
SLAVE_TAGS | Comma separated list of tags to be added to slave instance. The first tag (index 0) is used to configure the role of the Redis/resec task, and must be different from index 0 in MASTER_TAGS . |
|
HEALTHCHECK_INTERVAL | 5s | |
HEALTHCHECK_TIMEOUT | 2s | |
REDIS_ADDR | 127.0.0.1:6379 | |
REDIS_PASSWORD | ||
LOG_LEVEL | INFO | Options are "DEBUG", "INFO", "WARN", "ERROR" |
Resec requires permissions for Consul in order to function correctly.
The Consul ACL token is passed as the environment variable CONSUL_HTTP_TOKEN
.
If the Consul cluster being used is running ACLs; the following ACL policy will allow Replicator the required access to perform all functions based on its default configuration:
key "resec/" {
policy = "write"
}
session "" {
policy = "write"
}
service "" {
policy = "write"
}
job "resec" {
datacenters = ["dc1"]
type = "service"
update {
max_parallel = 1
stagger = "10s"
}
group "cache" {
count = 3
task "redis" {
driver = "docker"
config {
image = "redis:alpine"
command = "redis-server"
args = [
"/local/redis.conf"
]
port_map {
db = 6379
}
}
// Let Redis know how much memory he can use not to be killed by OOM
template {
data = <<EORC
maxmemory {{ env "NOMAD_MEMORY_LIMIT" | parseInt | subtract 16 }}mb
EORC
destination = "local/redis.conf"
}
resources {
cpu = 500
memory = 256
network {
mbits = 10
port "db" {}
}
}
}
task "resec" {
driver = "docker"
config {
image = "yotpo/resec"
}
env {
CONSUL_HTTP_ADDR = "http://${attr.unique.network.ip-address}:8500"
REDIS_ADDR = "${NOMAD_ADDR_redis_db}"
}
resources {
cpu = 100
memory = 64
network {
mbits = 10
}
}
}
}
}
resec:
image: yotpo/resec
environment:
- CONSUL_HTTP_ADDR=1.2.3.4:8500
- REDIS_ADDR=redis:6379
container_name: resec
[Unit]
Description=resec - redis ha replication daemon
Requires=network-online.target
After=network-online.target
[Service]
EnvironmentFile=-/etc/default/resec
ExecStart=/usr/local/bin/resec
KillSignal=SIGQUIT
Restart=on-failure
RestartSec=5s
[Install]
WantedBy=multi-user.target
Code released under the MIT license.