Dynamically scale kubernetes resources using length of an AMQP queue
Dynamically scale kubernetes resources using length of an AMQP queue (number of messages available for retrieval from the queue) to determine the load on an application/Kubernetes pod.
NOTICE If your application load is not queue-bound but rather CPU-sensitive, make sure to use built-in Kubernetes Horizontal Pod Autoscaling instead of this project.
This repository contains Dockerfile of KUBE-AMQP-AUTOSCALE for Docker's automated build published to the public Docker Hub Registry.
Install Docker.
Download automated build from public Docker Hub Registry: docker pull mbogus/kube-amqp-autoscale
(alternatively, you can build an image from Dockerfile: docker build -t="mbogus/kube-amqp-autoscale" github.com/mbogus/docker-kube-amqp-autoscale
)
autoscale
docker run -d -e AUTOSCALE_NAME=pod_to_scale -e AUTOSCALE_THRESHOLD=50 -e AUTOSCALE_MAX=10 -e RABBITMQ_URI=amqp://guest:[email protected]:5672// -e RABBITMQ_QUEUE=queue_to_watch -e KUBERNETES_SERVICE_URL=http://127.0.0.1:8080 mbogus/kube-amqp-autoscale
autoscale
w/ persistent shared directories.docker run -d -v <db-dir>:/data/db -v <conf-dir>:/etc/default -e AUTOSCALE_NAME=pod_to_scale -e AUTOSCALE_THRESHOLD=50 -e AUTOSCALE_MAX=10 -e RABBITMQ_URI=amqp://guest:[email protected]:5672// -e RABBITMQ_QUEUE=queue_to_watch -e KUBERNETES_SERVICE_URL=http://127.0.0.1:8080 mbogus/kube-amqp-autoscale
AUTOSCALE_NS
Kubernetes namespace (default default
)AUTOSCALE_KIND
type of the Kubernetes resource to autoscale, one of Deployment
, ReplicationController
, ReplicaSet
(default Deployment
)AUTOSCALE_NAME
name of the Kubernetes resource to autoscaleAUTOSCALE_THRESHOLD
number of messages on a queue representing maximum load on the autoscaled Kubernetes resourceAUTOSCALE_MIN
lower limit for the number of replicas for a Kubernetes pod that can be set by the autoscaler (default 1
)AUTOSCALE_MAX
upper limit for the number of replicate for a Kubernetes pod that can be set by the autoscalerAUTOSCALE_INTERVAL
time interval between Kubernetes resource scale runs in secs (default 30
)AUTOSCALE_INCREASE_LIMIT
limit number of Kubernetes pods to be provisioned in a single scale iteration to max of the value, set to a number greater than 0, default unbounded
AUTOSCALE_DECREASE_LIMIT
limit number of Kubernetes pods to be terminated in a single scale iteration to max of the value, set to a number greater than 0, default unbounded
AUTOSCALE_STATS_COVERAGE
required percentage of statistics to calculate average queue length (default 0.75
)AUTOSCALE_STATS_INTERVAL
time interval between metrics gathering runs in seconds (default 5
)AUTOSCALE_EVAL_INTERVALS
number of autoscale intervals used to calculate average queue length (default 2
)AUTOSCALE_DB
sqlite3 database filename for storing queue length statistics (default :memory:
)Recommendations:
RABBITMQ_URI
RABBITMQ_HOST
KUBERNETES_RABBITMQ_SERVICE_NAME
Variables:
RABBITMQ_URI
required, RabbitMQ broker URI, e.g. amqp://guest:[email protected]:5672//
or:
RABBITMQ_HOST
RabbitMQ broker hostname (default 127.0.0.1
)
RABBITMQ_PORT
port number (default 5672
)
KUBERNETES_RABBITMQ_SERVICE_NAME
name of Kubernetes service exposing RabbitMQ broker
RABBITMQ_PROTO
use amqps
for secure connections (default amqp
)
RABBITMQ_USER
user name (default guest
)
RABBITMQ_PASS
password (default guest
)
RABBITMQ_VHOST
virtual host (default /
)
RABBITMQ_QUEUE
RabbitMQ queue name to measure load on an application
KUBERNETES_SERVICE_URL
Kubernetes API URL, e.g. http://127.0.0.1:8080
or
KUBERNETES_SERVICE_PROTO
(default https
)
KUBERNETES_SERVICE_HOST
(default 127.0.0.1
)
KUBERNETES_SERVICE_PORT
(default 443
)
KUBERNETES_SERVICE_INSECURE
Set to true
for connecting to Kubernetes API
without verifying TLS certificate; unsafe, use for development only (default false
)
Basic authentication
KUBERNETES_SERVICE_USERNAME
username for basic authentication on Kubernetes APIKUBERNETES_SERVICE_PASSWORD
password for basic authentication on Kubernetes APIOAuth2 authentication (inside Kubernetes pod)
Path to a bearer token file for OAuth authentication, on a Kubernetes pod
/var/run/secrets/kubernetes.io/serviceaccount/token
TSL certificate (inside Kubernetes pod)
Path to CA certificate file for HTTPS connections to Kubernetes API from
within a cluster /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
An all-in-one deployment that comprises of a RabbitMQ broker (rabbitmq-broker
),
a worker node (echo-node
) to be scaled according to number of unacknowledged
messages on EchoQueue
queue and the autoscaler deployed as autoscaler
adding
one node for each waiting message on the queue capped at 5.
---
apiVersion: v1
kind: Service
metadata:
labels:
app: autoscale-example
component: broker
name: rabbitmq-broker
spec:
ports:
- port: 5672
name: main-port
nodePort: 30672
- port: 15672
name: admin-port
nodePort: 30080
type: NodePort
selector:
app: autoscale-example
component: broker
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: rabbitmq-broker
spec:
replicas: 1
template:
metadata:
labels:
app: autoscale-example
component: broker
spec:
containers:
- name: rabbitmq
image: rabbitmq:3-management
imagePullPolicy: Always
env:
- name: RABBITMQ_HIPE_COMPILE
value: "1"
resources:
requests:
cpu: 200m
memory: 200Mi
limits:
cpu: 500m
memory: 500Mi
ports:
- containerPort: 5672
hostPort: 5672
- containerPort: 15672
hostPort: 15672
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: echo-node
spec:
replicas: 1
template:
metadata:
labels:
app: autoscale-example
component: worker
spec:
containers:
- name: echo
image: mbogus/amqp-echo
imagePullPolicy: Always
env:
- name: KUBERNETES_RABBITMQ_SERVICE_NAME
value: RABBITMQ_BROKER
- name: RABBITMQ_QUEUE
value: EchoQueue
- name: ECHO_DELAY
value: "15"
resources:
requests:
cpu: 100m
memory: 100Mi
limits:
cpu: 200m
memory: 200Mi
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: autoscaler
spec:
replicas: 1
template:
metadata:
labels:
app: autoscale-example
component: scaler
spec:
containers:
- name: autoscale
image: mbogus/kube-amqp-autoscale
imagePullPolicy: Always
env:
- name: AUTOSCALE_NAME
value: echo-node
- name: AUTOSCALE_THRESHOLD
value: "1"
- name: AUTOSCALE_MAX
value: "5"
- name: RABBITMQ_QUEUE
value: EchoQueue
- name: KUBERNETES_RABBITMQ_SERVICE_NAME
value: RABBITMQ_BROKER
resources:
requests:
cpu: 100m
memory: 100Mi
limits:
cpu: 200m
memory: 200Mi