Automatically manage image digests in Dockerfiles, docker-compose files, and Kubernetes manifests by tracking them in a separate Lockfile
docker-lock
is a cli tool that automates managing image digests by tracking
them in a separate Lockfile (think package-lock.json or Pipfile.lock). With
docker-lock
, you can refer to images in Dockerfiles,
docker-compose V3 files, and Kubernetes manifests by
mutable tags (as in python:3.6
) yet receive the same
benefits as if you had specified immutable digests (as in python:3.6@sha256:25a189a536ae4d7c77dd5d0929da73057b85555d6b6f8a66bfbcc1a7a7de094b
).
Note: If you are unsure about the differences between tags and digests, refer to this quick summary.
docker-lock
ships with 3 commands that take you from development
to production:
docker lock generate
finds images in your Dockerfiles
,
docker-compose
files, and Kubernetes
manifests and generates
a Lockfile containing digests that correspond to their tags.docker lock verify
lets you know if there are more recent digests
than those last recorded in the Lockfile.docker lock rewrite
rewrites Dockerfiles
, docker-compose
files,
and Kubernetes
manifests to include digests.docker-lock
is most commonly used as a
cli-plugin for docker
so lock
can be used as subcommand of docker
as in docker lock
. However,
docker-lock
does not require docker
at all. Instead, it can be called
manually as a standalone executable as in docker-lock lock
.
This is especially convenient if the proper version of docker
is unavailable
or you would prefer to use another container technology such as
podman.
Consider a project with a multi-stage build Dockerfile
at its root:
FROM ubuntu AS base
# ...
FROM mperel/log:v1
# ...
FROM python:3.6
# ...
Running docker lock generate
from the root queries each images'
registry to produce a Lockfile, docker-lock.json
.
Note that the Lockfile records image digests so you do not have to manually specify them.
Running docker lock verify
ensures that the image digests are the
same as those on the registry for the same tags.
Now, assume that a change to mperel/log:v1
has been pushed to the registry.
Running docker lock verify
shows that the image digest in the Lockfile
is out-of-date because it differs from the newer image's digest on the registry.
While developing, it can be useful to generate a Lockfile, commit it to source control, and verify it periodically (for instance on PR merges). In this way, developers can be notified when images change, and if a bug related to a change in an image crops up, it will be easy to identify.
Finally, lets assume the Dockerfile
is ready to be built and shared.
Running docker lock rewrite
will add digests from the Lockfile
to all of the images.
At this point, the Dockerfile
will contain all of the digest information
from the Lockfile, so it will always maintain the same, known behavior
in the future.
docker-lock
can be run as a
docker
docker
Ensure docker
cli version >= 19.03 is installed by running docker --version
.
$ mkdir -p "${HOME}/.docker/cli-plugins"
$ curl -fsSL "https://github.com/safe-waters/docker-lock/releases/download/v${VERSION}/docker-lock_${VERSION}_${OS}_${ARCH}.tar.gz" | tar -xz -C "${HOME}/.docker/cli-plugins" "docker-lock"
$ chmod +x "${HOME}/.docker/cli-plugins/docker-lock"
%USERPROFILE%\.docker\cli-plugins
docker-lock.exe
into %USERPROFILE%\.docker\cli-plugins
docker-lock
executable in
your PATH
.docker-lock
, replace any docker
command such as docker lock
with
the name of the executable, docker-lock
, as in docker-lock lock
.docker-lock
was installed, run:$ docker-lock lock --help
docker-lock
can be run in a docker
container, as below. If you leave off
the ${VERSION}
tag, you will use the latest, nightly build from the master branch.
Note: If your host machine uses a credential helper such as
osxkeychain
,wincred
, orpass
, the credentials will not be available to the container even if you pass in yourdocker
config.
docker
config:$ docker run -v "${PWD}":/run safewaters/docker-lock:${VERSION} [commands]
docker
config:$ docker run -v "${HOME}/.docker/config.json":/.docker/config.json:ro -v "${PWD}":/run safewaters/docker-lock:${VERSION} [commands]
docker
config:$ docker run -v "%cd%":/run safewaters/docker-lock:${VERSION} [commands]
docker
config:$ docker run -v "%USERPROFILE%\.docker\config.json":/.docker/config.json:ro -v "%cd%":/run safewaters/docker-lock:${VERSION} [commands]
scratch
. These images only contain
the docker-lock
executable and are tagged as follows:
safewaters/docker-lock:${VERSION}
safewaters/docker-lock
alpine
are provided. They
are tagged as follows:
safewaters/docker-lock:${VERSION}-alpine
safewaters/docker-lock:alpine
docker-lock
supports public and private registries. If necessary, login to
docker before using docker-lock
.
docker-lock
supports options via cli flags or a configuration file,
.docker-lock.yml
.
The root of this repo has an example,
.docker-lock.yml.example.
To see available options, run commands with --help
. For instance:
$ docker lock --help
$ docker lock generate --help
$ docker lock verify --help
$ docker lock rewrite --help
$ docker lock version --help
Note: You can mix and match cli flags to get the output that you want.
docker lock generate
will collect all default files (Dockerfile
,
compose.yml
, compose.yaml
, docker-compose.yaml
, docker-compose.yml
,
pod.yml
, pod.yaml
, deployment.yml
, deployment.yaml
, job.yml
,
and job.yaml
in the default base directory, the directory from which
the command is run) and generate a Lockfile.
docker lock generate --lockfile-name=[file name]
will generate a Lockfile with the
file name as the output, instead of the default docker-lock.json
.
docker lock generate --update-existing-digests
will generate a Lockfile,
querying for all digests, even those that are hardcoded in the files. Normally,
if a digest is hardcoded, it would be used in the Lockfile.
docker lock generate --ignore-missing-digests
will generate a Lockfile,
recording images for which a digest could not be found as not having a digest.
Normally, if a digest cannot be found, docker-lock
would print an error.
docker lock generate --base-dir=[sub directory]
will collect all default
files in a sub directory and generate a Lockfile.
docker lock generate --dockerfiles=[file1,file2,file3]
will collect all
files from a comma separated list ("file1,file2,file3") as well as default
docker-compose files and Kubernetes manifests and generate a Lockfile.
docker lock generate --exclude-all-dockerfiles
will generate a Lockfile,
excluding all Dockerfiles.
docker lock generate --dockerfile-recursive
will collect all default
Dockerfiles (Dockerfile
) in subdirectories from the base directory as well
as default docker-compose files and Kubernetes manifests in the base directory
and generate a Lockfile.
docker lock generate --dockerfile-globs='[glob pattern]'
will collect all
Dockerfiles that match the glob pattern relative to the base directory as well
as default docker-compose files and Kubernetes manifests in the base directory
and generate a Lockfile. Use '**' to recursively search directories.
Remember to quote using single quotes so that the glob is not expanded
before docker-lock
uses it.
docker lock generate --composefiles=[file1,file2,file3]
will collect all
files from a comma separated list ("file1,file2,file3") as well as default
Dockerfiles files and Kubernetes manifests and generate a Lockfile.
docker lock generate --exclude-all-composefiles
will generate a Lockfile,
excluding all docker-compose files.
docker lock generate --composefile-recursive
will collect all default
docker-compose files (compose.yml
, compose.yaml
,
docker-compose.yaml
, docker-compose.yml
) in subdirectories from the
base directory as well as default Dockerfiles and Kubernetes manifests in
the base directory and generate a Lockfile.
docker lock generate --composefile-globs='[glob pattern]'
will collect all
docker-compose files that match the glob pattern relative to the base directory as well
as default Dockerfiles and Kubernetes manifests in the base directory
and generate a Lockfile. Use '**' to recursively search directories.
Remember to quote using single quotes so that the glob is not expanded
before docker-lock
uses it.
docker lock generate --kubernetesfiles=[file1,file2,file3]
will collect all
files from a comma separated list ("file1,file2,file3") as well as default
Dockerfiles files and docker-compose files and generate a Lockfile.
docker lock generate --exclude-all-kubernetesfiles
will generate a Lockfile,
excluding all Kubernetes manifests.
docker lock generate --kubernetesfile-recursive
will collect all default
Kubernetes manifests (pod.yaml
, pod.yml
) in
subdirectories from the base directory as well as default Dockerfiles
and docker-compose files in the base directory and generate a Lockfile.
docker lock generate --kubernetesfile-globs='[glob pattern]'
will collect all
Kubernetes manifests that match the glob pattern relative to the base directory as well
as default Dockerfiles and docker-compose files in the base directory
and generate a Lockfile. Use '**' to recursively search directories.
Remember to quote using single quotes so that the glob is not expanded
before docker-lock
uses it.
docker lock verify
will take an existing Lockfile, with the default name,
docker-lock.json
, generate a new Lockfile and report differences between
the new and existing Lockfiles.
docker lock verify --lockfile-name=[file name]
will use another file, instead
of the default docker-lock.json
, as the Lockfile.
docker lock verify --exclude-tags
will check for differences between a newly
generated Lockfile and the existing Lockfile, ignoring if tags are different.
docker lock verify --ignore-missing-digests
will verify, but when generating
the new Lockfile to compare against, will assume that digests that cannot be
found are empty. Normally, if a digest could not be found, an error would be
reported.
docker lock verify --update-existing-digests
will verify, but when generating
the new Lockfile to compare against, will query for digests even if they are hardcoded.
Normally, the new Lockfile would use the hardcoded digests, instead of querying
for the most recent one.
docker lock rewrite
will write the image names, tags, and digests
from the Lockfile into the referenced Dockerfiles, docker-compose files,
and Kubernetes manifests.
docker lock rewrite --lockfile-name=[file name]
will use another file, instead
of the default docker-lock.json
, as the Lockfile.
docker lock rewrite --exclude-tags
will write image names and digests,
but not the tags, from the Lockfile into the referenced Dockerfiles,
docker-compose files, and Kubernetes manifests.
docker lock rewrite --tempdir=[directory]
will create a temporary directory in the [directory]
and
write all files into it. Afterwards, the files are renamed to the appropriate
location and the temporary directory is deleted. Normally, this occurs in the
current directory. In general, this 2 step process happens to ensure that
either all rewrites succeed, or none of them do. There are also other rollback
measures in docker-lock
to ensure this transaction happens and you are not
left with some files rewritten if a failure occurs.
docker lock generate
to create a Lockfile, docker-lock.json
,
and commit it.docker-lock
in a CI/CD
pipeline. Specifically:
docker lock verify
to make sure that the
Lockfile is up-to-date. If docker lock verify
fails, the developer can
locally rerun docker lock generate
to update the Lockfile. This has
the benefit that digest changes will be explicitly tracked in git.docker lock verify
step in the pipeline passes, the pipeline
should run docker lock rewrite
so all files have correct digests
hardcoded in them.A development container based on ubuntu:bionic
has been provided,
so ensure docker
is installed and the docker
daemon is running.
docker
daemon is mapped from the host into the dev container,
so you can use docker
and docker-compose
commands from within the container
as if they were run on the host.To build and install docker-lock
in docker
's cli-plugins directory,
from the root of the project, run:
$ make install
To clean, format, lint, install, generate a new Lockfile, and run unit tests:
make
The CI pipeline will additionally run integration tests on pull requests.
You can run any step individually.
make clean
docker
's cli-plugins directory: make install
make lock
make format
make lint
make unittest
To view the coverage report after running unit tests, open coverage.html
in
your browser.
Note: While there exists a target in the Makefile for integration tests, these cannot run locally because they require credentials that are only available in the CI pipeline.