Deploy a full-fledged Thanos-based setup build on top of Prometheus-Operator locally on k3s and Minio.
Deploy a full-fledged Thanos-based setup build on top of Prometheus-Operator locally on k3s and Minio.
Features:
Ruler
)cp
) has access to the metrics of the customer-cluster (cc
)cp
docker-compose
, docker
, helm
, kubectl
)This work is meant for testing only, in order to understand each component, and not at all production-ready.
Note that the current setup only deploys one Prometheus per cluster, but the architecture diagram pictures multiple ones. I wanted to show that Thanos can also deal with that kind of setup, having multiple Prometheis running in parallel but also as a HA setup using Thanos deduplication features.
Make sure you have docker
, docker-compose
, kubectl
and helm
installed and set up in your PATH
.
local-setup/
run ./startup.sh
../setup-all
.You can use kubectl
and helm
as follows (e.g. for cc
cluster:):
kubectl --kubeconfig=local-setup/kubeconfig-cc.yaml get pods --all-namespaces
helm --kubeconfig=local-setup/kubeconfig-cc.yaml --home=local-setup/.helm-cc list
The setup exposes the following UIs to the host, which you can access in your browser:
http://localhost:21090
http://localhost:22090
http://localhost:21190
http://localhost:22190
http://localhost:21300
(username: admin
, password: admin123
)http://localhost:19000
(username: admin
, password: admin123
)For example, open Grafana http://localhost:21300
and go to the Explore window, select the Thanos
datasource and browse happily across both your clusters' metrics with unlimited retention utilizing S3 storage.
Note that Grafana Dashboards for Thanos components are not yet automatically deployed. Feel free to import them in Grafana (see dashboards/
).
Within local-setup/
simply run ./cleanup.sh
.