k8s-sentry

k8s-sentry is a simple tool to monitor a Kubernetes cluster and report all operational issues to Sentry.

There are two alternatives implementations:

• getsentry/sentry-kubernetes: The official Sentry kubernetes reporter. This is not actively maintained and suffers from a major memory leak.
• stevelacy/go-sentry-kubernetes: An alternative go implementation. This watches for Pod status changes only. This causes it to several event types (missing volumes, ingress errors, etc.).

k8s-sentry watches for several things:

• All warning and error events
• Pod containers terminating with a non-zero exit code
• Pods failing completely

Deployment

See deploy for Kubernetes manifests and installation instructions.

Configuration

Configuration is done completely via environment variables.

Issue grouping

k8s-sentry tries to be smart about grouping issues. To handle that several strategies are used:

• all issues use the event type, event reason and event message as part of the fingerprint
• events related to controlled Pods (for example Pods created through a ReplicaSet (which is automatically done if you use a StatefulSet or Deployment) are grouped by the ReplicateSet.
• other events are grouped by the the involved object

Building

This project uses Go modules and requires Go 1.13 or later. From a git checkout you can build the binary using go build:

$ go build
go: downloading k8s.io/apimachinery v0.0.0-20191020214737-6c8691705fc5
go: downloading k8s.io/client-go v0.0.0-20191016111102-bec269661e48
go: downloading k8s.io/api v0.0.0-20191016110408-35e52d86657a
...

You can then run k8s-sentry directly (assuming you have a valid kubectl configuration):

$ ./k8s-sentry
2019/10/22 15:55:41 Warning: DSN environment variable not set. Can not report to Sentry
2019/10/22 15:55:41 Warning HorizontalPodAutoscaler/istio-ingressgateway: unable to get metrics for resource cpu: no metrics returned from resource metrics API
2019/10/22 15:55:41 Warning HorizontalPodAutoscaler/istio-pilot: unable to get metrics for resource cpu: no metrics returned from resource metrics API