Dynamic flag management for Go.
Dynamic, thread-safe flag
variables that can be modified at runtime through etcd
or Kubernetes.
For a similar project for JVM languages (Java, scala) see java-flagz
File-based or command-line configuration can only be changed when a service restarts. Dynamic flags provide flexibility in normal operations and emergencies. Two examples:
All of this can be done simultaneously across a whole shard of your services.
flag
replacement spf13/pflag
(e.g. ones using spf13/cobra
)flag
that are thread-safe and efficient:
DynInt64
DynFloat64
DynString
DynDuration
DynStringSlice
DynJSON
- a flag
that takes an arbitrary JSON structDynProto3
- a flag
that takes a proto3
struct in JSONpb or binary formvalidator
functions for each flag
, allows the user to provide checks for newly set valuesnotifier
functions allow user code to be subscribed to flag
changesConfigMap
watcher, see configmap/README.md.etcd
based watcher that syncs values from a distributed Key-Value store into the program's memory/debug/flagz
HandlerFunc endpoint that allows for easy inspection of the service's runtime configurationHere's a teaser of the debug endpoint:
Declare a single pflag.FlagSet
in some public package (e.g. common.SharedFlagSet
) that you'll use throughout your server.
var (
limitsConfigFlag = flagz.DynJSON(
common.SharedFlagSet,
"rate_limiting_config",
&rateLimitConfig{ DefaultRate: 10, Policy: "allow"},
"Config for service's rate limit",
).WithValidator(rateLimitConfigValidator).WithNotifier(onRateLimitChange)
)
This declares a JSON flag of type rateLimitConfig
with a default value. Whenever the config changes (statically or dynamically) the rateLimitConfigValidator
will be called. If it returns no errors, the flag will be updated and onRateLimitChange
will be called with both old and new, allowing the rate-limit mechanism to re-tune.
var (
featuresFlag = flagz.DynStringSlice(common.SharedFlagSet, "enabled_features", []string{"fast_index"}, "list of enabled feature markers")
)
...
func MyHandler(resp http.ResponseWriter, req *http.Request) {
...
if existsInStringSlice("fast_index", featuresFlag.Get()) {
doFastIndex(req)
}
...
}
All access to featuresFlag
, which is a []string
flag, is synchronised across go-routines using atomic
pointer swaps.
// First parse the flags from the command line, as normal.
common.SharedFlagSet.Parse(os.Args[1:])
w, err := watcher.New(common.SharedFlagSet, etcdClient, "/my_service/flagz", logger)
if err != nil {
logger.Fatalf("failed setting up %v", err)
}
// Read flagz from etcd and update their values in common.SharedFlagSet
if err := w.Initialize(); err != nil {
log.Fatalf("failed setting up %v", err)
}
// Start listening of dynamic flags from etcd.
w.Start()
The watcher
's go-routine will watch for etcd
value changes and synchronise them with values in memory. In case a value fails parsing or the user-specified validator
, the key in etcd
will be atomically rolled back.
This code is production quality. It's been running happily in production at Improbable for a few months.
Features planned:
FlagSet
checksus using a Prometheus handlerflag
(requires changes in spf13/pflag
interfaces)go-flagz
is released under the Apache 2.0 license. See the LICENSE file for details.