Geras Save

Geras provides a Thanos Store API for the OpenTSDB HTTP API. This makes it possible to query OpenTSDB via PromQL, through Thanos.

Since Thanos's StoreAPI is designed for unified data access and is not too Prometheus specific, Geras is able to provide an implementation which proxies onto the OpenTSDB HTTP API, giving the ability to query OpenTSDB using PromQL, and even enabling unified queries (including joins) over Prometheus and OpenTSDB.

Build

go get github.com/G-Research/geras/cmd/geras

After the build you will have a self-contained binary (geras). It writes logs to stdout.

A Dockerfile is also provided (see docker-compose.yaml for an example of using it).

Deployment

At a high level:

• Run Geras somewhere and point it to OpenTSDB: -opentsdb-address opentsdb:4242;
• Configure a Thanos query instance with --store=geras:19000 (i.e. the gRPC listen address).

Geras additionally listens on a HTTP port for Prometheus /metrics queries and some debug details (using x/net/trace, see for example /debug/requests and /debug/events.

Usage

  -grpc-listen string
        Service will expose the Store API on this address (default "localhost:19000")
  -http-listen string
        Where to serve HTTP debugging endpoints (like /metrics) (default "localhost:19001")
  -trace-enabled
        Enable tracing of requests, which is shown at /debug/requests (default true)
  -trace-dumpbody
        Include TSDB request and response bodies in traces (can be expensive) (default false)
  -label value
        Label to expose on the Store API, of the form '<key>=<value>'. May be repeated.
  -log.format string
        Log format. One of [logfmt, json] (default "logfmt")
  -log.level string
        Log filtering level. One of [debug, info, warn, error] (default "error")
  -healthcheck-metric
        A metric to query as a readiness health check (default "tsd.rpc.recieved")
  -metrics-refresh-interval duration
        Time between metric name refreshes. Use negative duration to disable refreshes. (default 15m0s)
  -metrics-refresh-timeout
        Timeout for metric refreshes (default 2m0s)
  -metrics-suggestions
        Enable metric suggestions (can be expensive) (default true)
  -opentsdb-address string
        <host>:<port>
  -metrics-allowed-regexp regexp
        A regular expression specifying the allowed metrics. Default is `.*`,
        i.e. everything. A good value if your metric names all match OpenTSDB
        style of `service.metric.name` could be `^\w+\..*$`. Disallowed metrics
        are simply not queried and non error is returned -- the purpose is to
        not send traffic to OpenTSDB when the metric source is Prometheus.
  -metrics-blocked-regexp regexp
        A regular expression of metrics to block. Default is empty and means to
        not block anything. The expected use of this is to block problematic
        queries as a fast mitigation therefore an error is returned when a
        metric is blocked.
  -metrics-name-response-rewriting
        Rewrite '.' to a defined character and other bad characters to '_' in all responses (Prometheus
        remote_read won't accept these, while Thanos will) (default true)
  -period-character-replace
		Rewrite '.' to a defined charater that Prometheus will handle better. (default ':')

When specifying multiple labels, you will need to repeat the argument name, e.g:

./geras -label label1=value1 -label label2=value2

Limitations

• PromQL supports queries without __name__. This is not possible in OpenTSDB and no results will be returned if the query doesn't match on a metric name.
• Geras periodically loads metric names from OpenTSDB and keeps them in memory to support queries like {__name__=~"regexp"}.
• Thanos' primary timeseries backend is Prometheus, which doesn't support unquoted dots in metric names. However OpenTSDB metrics generally use . as a seperator within names. In order to query names containing a . you will need to either:
  • Replace all . with another character (we like :).
  • Use the __name__ label to specify the metric name, e.g. {__name__="cpu.percent"}
  • Also watch out for - (dashes) in your metric names