Tool to handle dependencies between nomad jobs
Nomad doesn't have its own dependency management plugin between the jobs. Especially with rise in the microservices pattern, it becomes difficult to maintain order of deployment using Nomad. For eg. Database service must be up before the api service is started. The order of deployment of the jobs has to be handled separately, using some external tool like Rundeck.
Nomad also lacks feature of post/pre hook. (Check issue: https://github.com/hashicorp/nomad/issues/419 and https://github.com/hashicorp/nomad/issues/2851). Many of services a pre-setup step or a post setup. For example, when we run a postgres or any db container, we need to create a user, db and grant permissions. Another example, on bringing up a rabbitmq cluster we might need to create vhost, user etc.
nomad-dtree is an attempt to address these problems.
nomad-dtree expects everything to be a nomad job. Even a post/pre hook should be a nomad jobs. It also expects a JSON where the dependency is mentioned.
Consider a small service which has three components:
The order of dependency is same as above. The postgres service has a post hook, a small step of creating a user and db which is used by the api service. So we have nomad jobs:
The dependency file for the above servcies is as follows:
{
"dependencies": {
"nginx": {
"wait_cond": 5,
"pre": {
"job": "api"
}
},
"api": {
"wait_cond": 7,
"pre": {
"job": "postgres"
}
},
"postgres": {
"post": {
"job": "postgres_setup",
"wait_cond": 3
},
"wait_cond": 5
}
}
}
Now, each block in the dependencies
json denotes a nomad job. It has three
parts, consider api
:
"api": {
"wait_cond": 7,
"pre": {
"job": "postgres"
}
},
wait_cond
seconds).Now, when we run the following command:
nomad-dtree \
run \
--job nginx \ (note we are submitting nginx job here)
--server-addr <nomad-address> \
--store filesystem \
--fs-depfile-path <path to dependency json file> \
--fs-jobs-path <path to directory containing all nomad jobs>
The order of execution will be as follows:
postgres.nomad -> postgres_setup.nomad -> api.nomad -> nginx.nomad
Similar to run
you can also stop
the jobs.
Stop command has two flags:
nomad-dtree \
stop --purge --deep \
--server-addr <nomad-address> \
--store filesystem \
--fs-depfile-path <path to dependency json file> \
--fs-jobs-path <path to directory containing all nomad jobs>
For more examples check Examples