A user-friendly and efficient tool for the effortless deployment and configuration of resources on remote machines.
Spot is a powerful and easy-to-use tool for effortless deployment and configuration management. It allows users to define a playbook with a list of tasks and targets, where each task consists of a series of commands that can be executed on remote hosts concurrently. Spot supports running scripts, copying files, syncing directories, and deleting files or directories, as well as custom inventory files or inventory URLs.
spot
with playbook spot.yml
: spot -p spot.yml -t prod
spot
with the same playbook in dry mode: spot -p spot.yml -t prod -v
spot
ad-hoc command on given hosts: spot "ls -la /tmp -t dev1.umputun.com -t dev2.umputun.com
spot.yml
.spot
. This will execute all the tasks defined in the default spot.yml
file for the default
target with a concurrency of 1.--task
flag: spot --task=deploy-things
. This will execute only the deploy-things
task.--task
and --target
flags: spot --task=deploy-things --target=prod
. This will execute only the deploy-things
task for the prod
target.Install from homebrew (macOS)
brew tap umputun/apps
brew install umputun/apps/spot
This will install both spot
and spot-secrets
binaries.
Install from deb package (Ubuntu/Debian)
wget https://github.com/umputun/spot/releases/download/<versiom>/spot_<version>_linux_<arch>.deb
(replace <version>
and <arch>
with the actual values).sudo dpkg -i spot_<version>_linux_<arch>.deb
Example for the version 0.14.6 and amd64 architecture:
wget https://github.com/umputun/spot/releases/download/v0.14.6/spot_v0.14.6_linux_<arch>.deb
sudo dpkg -i spot_v0.14.6_linux_<arch>.deb
Install from rpm package (CentOS/RHEL/Fedora/AWS Linux)
wget https://github.com/umputun/spot/releases/download/v<version>/spot_v<version>_linux_<arch>.rpm
sudo rpm -i spot_v<version>_linux_<arch>.rpm
Install from apk package (Alpine)
wget https://github.com/umputun/spot/releases/download/<versiom>/spot_<version>_linux_<arch>.apk
sudo apk add spot_<version>_linux_<arch>.apk
Universal installation for Linux and macOS
Spot provides a universal installation script that can be used to install the latest version of the tool on Linux and macOS.
wget https://raw.githubusercontent.com/umputun/spot/master/install.sh
sudo sh install.sh
The script will detect the OS and architecture and download the correct binary for the latest version of Spot.
If you are brave enough, you can run the script directly from the web, but I'd recommend downloading it first and reviewing it:
curl -sSfL https://raw.githubusercontent.com/umputun/spot/master/install.sh | sudo sh
Install with go install
This method requires Go to be installed on your system.
go install github.com/umputun/spot/cmd/spot@latest
go install github.com/umputun/spot/cmd/spot-secrets@latest
Spot supports the following command-line options:
-p
, --playbook=
: Specifies the playbook file for use. Defaults to spot.yml
. You can also set the environment
variable $SPOT_PLAYBOOK
to define the playbook file path.-n
, --task=
: Specifies task names to execute. The task should be defined in the playbook file. Several tasks can be executed by providing the --task
flag multiple times, e.g., -n copy_files -n warmup_cache
.
If not specified all the tasks will be executed.-t
, --target=
: Specifies the target name to use for the task execution. The target should be defined in the playbook file and can represent remote hosts, inventory files, or inventory URLs. If not specified, the default
target will be used. User can pass a hostname, group name, tag or IP instead of the target name for a quick override. Providing the -t
, --target
flag multiple times with different targets sets multiple destination targets or multiple hosts, e.g., -t prod -t dev
or -t example1.com -t example2.com
.-c
, --concurrent=
: Sets the number of concurrent hosts to execute tasks. Defaults to 1
, which means hosts will be handled sequentially.--timeout
: Sets the SSH timeout. Defaults to 30s
. User can also set the environment variable $SPOT_TIMEOUT
to define the SSH timeout.--ssh-agent
: Enables using the SSH agent for authentication. Defaults to false
. Users can also set the environment variable SPOT_SSH_AGENT
to define the value.--forward-ssh-agent
: Enables forwarding of connections from an authentication agent. Defaults to false
. Users can also set the environment variable SPOT_FORWARD_SSH_AGENT
to define the value.--shell
- shell for remote ssh execution, default is /bin/sh
. Users can also set the environment variable SPOT_SHELL
to define the value.-i
, --inventory=
: Specifies the inventory file or URL to use for the task execution. Overrides the inventory file defined in the
playbook file. Users can also set the environment variable $SPOT_INVENTORY
to define the default inventory file path or url.-u
, --user=
: Specifies the SSH user to use when connecting to remote hosts. Overrides the user defined in the playbook file .-k
, --key=
: Specifies the SSH key for connecting to remote hosts. Overrides the key defined in the playbook file.-s
, --skip=
: Skips the specified commands during the task execution. Providing the -s
flag multiple times with different command names skips multiple commands.-o
, --only=
: Runs only the specified commands during the task execution. Providing the -o
flag multiple times with different command names runs only multiple commands.-e
, --env=
: Sets the environment variables to be used during the task execution. Providing the -e
flag multiple times with different environment variables sets multiple environment variables, e.g., -e VAR1:VALUE1 -e VAR2:VALUE2
. Values could be taken from the OS environment variables as well, e.g., -e VAR1:$ENV_VAR1
or -e VAR1:${ENV_VAR1}
.-E
, --env-file=
: Sets the environment variables from the file to be used during the task execution. The file can have values from the OS environment variables as well. The default is env.yml. Can also be set with the environment variable SPOT_ENV_FILE
.--no-color
: disable the colorized output. It can also be set with the environment variable SPOT_NO_COLOR
.--dry
: Enables dry-run mode, which prints out the commands to be executed without actually executing them.-v
, --verbose
: Enables verbose mode, providing more detailed output and error messages during the task execution. Setting this flag multiple times increases the verbosity level, i.e., -vv
.--dbg
: Enables debug mode, providing even more detailed output and error messages during the task execution and diagnostic messages.-h
--help
: Displays the help message, listing all available command-line options.Playbook is a YAML or TOML file that defines a list of tasks to be executed on one or more target hosts. Each task consists of a series of commands that can be executed on the target hosts. Playbooks can be used to automate deployment and configuration management tasks.
Task is a named set of commands that can be executed on one or more target hosts. Tasks can be defined in a playbook and executed concurrently on multiple hosts.
Command is an action that can be executed on a target host. Spot supports several built-in commands, including copy, sync, delete, script, echo, and wait.
Target is a host or group of hosts on which a task can be executed. Targets can be specified directly in a playbook or defined in an inventory file. Spot supports several inventory file formats.
Inventory is a list of targets that can be used to define the hosts and groups of hosts on which a task can be executed.
user: umputun # default ssh user. Can be overridden by -u flag or by inventory or host definition
ssh_key: keys/id_rsa # ssh key
ssh_shell: /bin/bash # shell to use for remote ssh execution, default is /bin/sh
local_shell: /bin/bash # shell to use for local execution, default is os shell
inventory: /etc/spot/inventory.yml # default inventory file. Can be overridden by --inventory flag
# list of targets, i.e. hosts, inventory files or inventory URLs
targets:
prod:
hosts: # list of hosts, user, name and port optional.
- {host: "h1.example.com", user: "user2", name: "h1"}
- {host: "h2.example.com", port: 2222}
staging:
groups: ["dev", "staging"] # list of groups from inventory file
dev:
names: ["devbox1", "devbox2"] # list of server names from inventory file
all-boxes:
groups: ["all"] # all hosts from all groups from inventory file
# list of tasks, i.e. commands to execute
tasks:
- name: deploy-things
on_error: "curl -s localhost:8080/error?msg={SPOT_ERROR}" # call hook on error
commands:
- name: wait
script: sleep 5s
- name: copy configuration
copy: {"src": "testdata/conf.yml", "dst": "/tmp/conf.yml", "mkdir": true}
- name: copy other files
copy:
- {"src": "testdata/f1.csv", "dst": "/tmp/things/f1.csv", "recur": true}
- {"src": "testdata/f2.csv", "dst": "/tmp/things/f2.csv", "recur": true}
- name: sync things
sync: {"src": "testdata", "dst": "/tmp/things"}
- name: some command
script: |
ls -laR /tmp
du -hcs /srv
cat /tmp/conf.yml
echo all good, 123
- name: delete things
delete: {"path": "/tmp/things", "recur": true}
- name: show content
script: ls -laR /tmp
- name: docker
commands:
- name: docker pull and start
script: |
docker pull umputun/remark42:latest
docker stop remark42 || true
docker rm remark42 || true
docker run -d --name remark42 -p 8080:8080 umputun/remark42:latest
env: {FOO: bar, BAR: qux} # set environment variables for the command
- wait: {cmd: "curl -s localhost:8080/health", timeout: "10s", interval: "1s"} # wait for health check to pass
Alternatively, the playbook can be represented using the TOML format.
In some cases, the rich syntax of the full playbook is not needed and can feel over-engineered and even overwhelming. For those situations, Spot supports a simplified playbook format, which is easier to read and write, but also more limited in its capabilities.
user: umputun # default ssh user. Can be overridden by -u flag or by inventory or host definition
ssh_key: keys/id_rsa # ssh key
ssh_shell: /bin/bash # shell to use for remote ssh execution, default is /bin/sh
local_shell: /bin/bash # shell to use for local execution, default is os shell
inventory: /etc/spot/inventory.yml # default inventory file. Can be overridden by --inventory flag
targets: ["devbox1", "devbox2", "h1.example.com:2222", "h2.example.com"] # list of host names from inventory and direct host ips
# the actual list of commands to execute
task:
- name: wait
script: sleep 5s
- name: copy configuration
copy: {"src": "testdata/conf.yml", "dst": "/tmp/conf.yml", "mkdir": true}
- name: copy other files
copy:
- {"src": "testdata/f1.csv", "dst": "/tmp/things/f1.csv", "recur": true}
- {"src": "testdata/f2.csv", "dst": "/tmp/things/f2.csv", "recur": true}
- name: sync things
sync: {"src": "testdata", "dst": "/tmp/things"}
- name: some command
script: |
ls -laR /tmp
du -hcs /srv
cat /tmp/conf.yml
echo all good, 123
- name: delete things
delete: {"path": "/tmp/things", "recur": true}
- name: show content
script: ls -laR /tmp
- name: docker pull and start
script: |
docker pull umputun/remark42:latest
docker stop remark42 || true
docker rm remark42 || true
docker run -d --name remark42 -p 8080:8080 umputun/remark42:latest
env: {FOO: bar, BAR: qux} # set environment variables for the command
- wait: {cmd: "curl -s localhost:8080/health", timeout: "10s", interval: "1s"} # wait for health check to pass
For more examples see .examples directory.
Spot supports two types of playbooks: full and simplified. Both can be represented in either YAML or TOML format. The full playbook is more powerful and flexible but also more verbose and complex. The simplified playbook, on the other hand, is easier to read and write but has more limited capabilities.
Here are the main differences between the two types of playbooks:
hosts
, groups
, and names
, while the simplified playbook only supports a single type, which is a list of names or host addresses. See the Targets section for more details.on_error
field, while the full playbook does. See the Task details section for more information.user
field, which allows setting the SSH user to use when connecting to remote hosts for the particular task.target
field (in addition to targets
) that allows setting a single host/name only. This is useful when users want to run the playbook on a single host only. The full playbook does not have this field.Both types of playbooks support the remaining fields and options.
Each task consists of a list of commands that will be executed on the remote host(s). The task can also define the following optional fields:
on_error
: specifies the command to execute on the local host (the one running the spot
command) in case of an error. The command can use the {SPOT_ERROR}
variable to access the last error message. Example: on_error: "curl -s localhost:8080/error?msg={SPOT_ERROR}"
user
: specifies the SSH user to use when connecting to remote hosts. Overrides the user defined in the top section of the playbook file for the specified task.targets
- list of target names, groups, tags, or host addresses to execute the task on. Command line -t
flag can be used to override this field. The targets
field may include variables. For more details see Dynamic targets section.Note: these fields are supported in the full playbook type only
All tasks are executed sequentially on a given host, one after another. If a task fails, the execution of the playbook will stop and the on_error
command will be executed on the local host, if defined. Every task has to have name
field defined, which is used to identify the task everywhere. Playbook with a missing name
field will fail to execute immediately. Duplicate task names are not allowed either.
Relative path resolution is a frequent issue in systems that involve file references or inclusion. Different systems handle this in various ways. Spot uses a widely-adopted method of resolving relative paths based on the current working directory of the process. This means that if you run Spot from different directories, the way relative paths are resolved will change. In simpler terms, Spot doesn't resolve relative paths according to the location of the playbook file itself.
This approach is intentional to prevent confusion and make it easier to comprehend relative path resolution. Generally, it's a good practice to run Spot from the same directory where the playbook file is located when using relative paths. Alternatively, you can use absolute paths for even better results.
Spot supports the following command types:
script
Can be any valid shell script. The script will be executed on the remote host(s) using SSH, inside a shell.
script: |
ls -laR /tmp
du -hcs /srv
cat /tmp/conf.yml
echo all good, 123
Read more about YAML multiline string formatting on yaml-multiline.info and this stackoverflow post.
copy
Copies a file from the local machine to the remote host(s). If mkdir
is set to true
the command will create the destination directory if it doesn't exist, the same as mkdir -p
in bash. The command also supports glob patterns in src
field.
Copy command performs a quick check to see if the file already exists on the remote host(s) with the same size and modification time,
and skips the copy if it does. This option can be disabled by setting force: true
flag. Another option is exclude
which allows to specify a list of files to exclude to be copied.
- name: copy file with mkdir
copy: {"src": "testdata/conf.yml", "dst": "/tmp/conf.yml", "mkdir": true}
- name: copy files with glob
copy: {"src": "testdata/*.csv", "dst": "/tmp/things"}
- name: copy files with glob and exclude
copy: {"src": "testdata/*.yml", "dst": "/tmp/things", "exclude": ["conf.dist.yml"]}
- name: copy files with force flag
copy: {"src": "testdata/*.csv", "dst": "/tmp/things", "force": true}
Copy also supports list format to copy multiple files at once:
- name: copy files with glob
copy:
- {"src": "testdata/*.csv", "dst": "/tmp/things"}
- {"src": "testdata/*.yml", "dst": "/tmp/things"}
Copy file and making it executable is also supported:
- name: copy file and make it executable
copy:
- {"src": "testdata/script.sh", "dst": "/tmp/script.sh", "chmod+x": true}
sync
Synchronises directory from the local machine to the remote host(s). Optionally supports deleting files on the remote host(s) that don't exist locally with "delete": true
flag. Another option is exclude
which allows to specify a list of files to exclude from the sync.
- name: sync directory
sync: {"src": "testdata", "dst": "/tmp/things"}
- name: sync directory with delete
sync: {"src": "testdata", "dst": "/tmp/things", "delete": true}
- name: sync directory with exclude
sync: {"src": "testdata", "dst": "/tmp/things", "exclude": ["*.txt", "*.yml"]}
Sync also supports list format to sync multiple paths at once.
delete
Deletes a file or directory on the remote host(s), optionally can remove recursively.
- name: delete file
delete: {"path": "/tmp/things.csv"}
- name: delete directory recursively
delete: {"path": "/tmp/things", "recur": true}
- name: delete directory recursively with exclude
delete: {"path": "/tmp/things", "recur": true, "exclude": ["*.txt", "*.yml"]}
Delete also supports a list format to remove multiple paths at once.
wait
Waits for the specified command to finish on the remote host(s) with 0 error code. This command is useful when a user needs to wait for a service to start before executing the next command. Allows to specify the timeout as well as check interval.
- name: wait for service to start
wait: {"cmd": "curl -s --fail localhost:8080", "timeout": "30s", "interval": "1s"}
echo
Prints the specified message to the console. This command is useful for debugging purposes and also to print the value of variables to the console.
- name: print message
echo: "hello world"
- name: print variable
echo: $some_var
Each command type supports the following options:
ignore_errors
: if set to true
the command will not fail the task in case of an error.no_auto
: if set to true
the command will not be executed automatically, but can be executed manually using the --only
flag.local
: if set to true
the command will be executed on the local host (the one running the spot
command) instead of the remote host(s).sudo
: if set to true
the command will be executed with sudo
privileges. This option is not supported for sync
command type but can be used with any other command type.only_on
: allows to set a list of host names or addresses where the command will be executed. For example, only_on: [host1, host2]
will execute a command on host1
and host2
only. This option also supports reversed conditions, so if a user wants to execute a command on all hosts except some, !
prefix can be used. For example, only_on: [!host1, !host2]
will execute a command on all hosts except host1
and host2
.example setting ignore_errors
, no_auto
and only_on
options:
commands:
- name: wait
script: sleep 5s
options: {ignore_errors: true, no_auto: true, only_on: [host1, host2]}
The same options can be set for the whole task as well. In this case, the options will be applied to all commands in the task but can be overridden for a specific command. Pls note: the command option cannot reset the boolean options that were set for the task. This limitation is due to the way the default values are set.
- name: deploy-things
on_error: "curl -s localhost:8080/error?msg={SPOT_ERROR}" # call hook on error
options: {ignore_errors: true, no_auto: true, only_on: [host1, host2]}
commands:
- name: wait
script: sleep 5s
cond
: defines a condition for the command to be executed. The condition is a valid shell command that will be executed on the remote host(s) and if it returns 0, the primary command will be executed. For example, cond: "test -f /tmp/foo"
will execute the primary script command only if the file /tmp/foo
exists. The condition can be reversed by adding !
prefix, i.e. ! test -f /tmp/foo
will pass only if the file /tmp/foo
doesn't exist.
example installing curl package if not installed already:
- name: "install curl"
script: "yum install curl -y"
options: {sudo: true}
cond: "! command -v curl"
on_exit
)Each command may have on_exit
parameter defined. It allows executing a command on the remote host after the task with all commands is completed. The command is called regardless of the task's exit code.
This is useful in several scenarios:
on_exit: "rm -fv /tmp/script.sh"
on_exit: "systemctl restart myservice"
- name: "copy script"
copy: {src: "testdata/script.sh", "dst": "/tmp/script.sh", "chmod+x": true}
on_exit: "rm -fv /tmp/script.sh" # register deferred action to remove script.sh after execution
- name: "run script"
script: "/tmp/script.sh"
In the example above, the script.sh
is copied to the remote host, executed, and removed after completion of the task.
Spot allows executing scripts on remote hosts, or locally if options.local
is set to true. Scripts can be executed in two different ways, depending on whether they are single-line or multi-line scripts.
Single-line Script Execution
For single-line scripts, they are executed directly inside the shell with the optional parameters set to the command line. For example:
commands:
- name: some command
script: ls -laR /tmp
env: {FOO: bar, BAR: qux}
this will be executed as: FOO='bar' BAR='qux'ls -laR /tmp FOO=bar BAR=qux
inside the shell on the remote host(s), i.e. sh -c "FOO='bar' BAR='qux'ls -laR /tmp FOO=bar BAR=qux"
.
Multi-line Script Execution
For multi-line scripts, Spot creates a temporary script containing all the commands, uploads it to the remote host (or keeps it locally if options.local
is set to true), and executes the script. Environment variables are set inside the script, allowing the user to create complex scripts that include setting variables, conditionals, loops, and other advanced functionality. Scripts run with "set -e" to fail on error. For example:
commands:
- name: multi_line_script
script: |
touch /tmp/file1
echo "Hello World" > /tmp/file2
echo "Executing loop..."
for i in {1..5}; do
echo "Iteration $i"
done
echo "All done! $FOO $BAR
env: {FOO: bar, BAR: qux}
this will create a temporary script on the remote host(s) with the following content and execute it:
#!/bin/sh
set -e
export FOO='bar'
export BAR='qux'
touch /tmp/file1
echo "Hello World" > /tmp/file2
echo "Executing loop..."
for i in {1..5}; do
echo "Iteration $i"
done
echo "All done! $FOO $BAR"
By using this approach, Spot enables users to write and execute more complex scripts, providing greater flexibility and power in managing remote hosts or local environments.
Users can also set any custom shebang for the script by adding #!
at the beginning of the script. For example:
commands:
- name: multi_line_script
script: |
#!/bin/bash
touch /tmp/file1
echo "Hello World" > /tmp/file2
The spot allows passing variables from one command to another. This feature is especially useful when a command, often a script, sets a variable, and the subsequent command requires this variable. For instance, if one command creates a file and the file name is needed in another command. To pass these variables, a user must use the conventional shell's export directive in the initial script command. Subsequently, all variables exported in this initial command will be accessible in the following commands.
For example:
commands:
- name: first command
script: |
export FILE_NAME=/tmp/file1
touch $FILE_NAME
- name: second command
script: |
echo "File name is $FILE_NAME"
- name: third command
copy: {src: $FILE_NAME, dest: /tmp/file2}
Sometimes, exporting variables is not possible or not desired. For such cases, Spot allows to register variables explicitly using the register
option.
For example:
commands:
- name: first command
script: |
FILE_NAME=/tmp/file1
touch $FILE_NAME
ANOTHER_VAR=foo
register: [FILE_NAME, ANOTHER_VAR]
- name: second command
script: |
echo "File name is $FILE_NAME, var is $ANOTHER_VAR"
- name: third command
copy: {src: $FILE_NAME, dest: /tmp/file2}
Environment variables can be set with --env
/ -e
cli option. For example: -e VAR1:VALUE1 -e VAR2:VALUE2
. Environment variables can also be set in the environment file (default env.yml
can be changed with --env-file
/ -E
cli flag). For example:
vars:
VAR1: VALUE1
VAR2: VALUE2
Environment variables can be used in the playbook file with the expected syntax: $VAR_NAME
or ${VAR_NAME}
. For example:
commands:
- name: some command
script: echo $VAR1
- name: another command
copy: {"src": "testdata/*.csv", "dst": "$VAR2"}
In case of a conflict between environment variables set in the environment file and the cli, the cli variables will take precedence.
Targets are used to define the remote hosts to execute the tasks on. Targets can be defined in the playbook file or passed as a command-line argument. The following target types are supported:
hosts
: a list of destination host names or IP addresses, with optional port and username, to execute the tasks on. Example: hosts: [{host: "h1.example.com", user: "test", name: "h1}, {host: "h2.example.com", "port": 2222}]
. If no user is specified, the user defined in the top section of the playbook file (or override) will be used. If no port is specified, port 22 will be used.groups
: a list of groups from inventory to use. Example: groups: ["dev", "staging"}
. A special group all
combines all the groups.tags
: a list of tags from inventory to use. Example: tags: ["tag1", "tag2"}
.names
: a list of host names from inventory to use. Example: names: ["host1", "host2"}
.All the target types can be combined, i.e. hosts
, groups
, tags
, hosts
and names
all can be used together in the same target. To avoid possible duplicates, the final list of hosts is deduplicated by the host+ip+user.
example of targets set in the playbook file:
targets:
prod:
hosts: [{host: "h1.example.com", user: "test"}, {"h2.example.com", "port": 2222, name: "h2"}]
staging:
groups: ["staging"]
dev:
groups: ["dev", "staging"]
names: ["host1", "host2"]
all-servers:
groups: ["all"]
tasks:
- name: task1
targets: ["dev", "host3.example.com:2222"]
commands:
- name: command1
script: echo "Hello World"
Note: All the target types are available in the full playbook file only. The simplified playbook file only supports a single, anonymous target type combining hosts
and names
together.
targets: ["host1", "host2", "host3.example.com", "host4.example.com:2222"]
in this example, the playbook will be executed on hosts named host1
and host2
from the inventory and on hosts host3.example.com
with port 22
and host4.example.com
with port 2222
.
There are several ways to override or alter the target defined in the playbook file via command-line arguments:
--inventory
set hosts from the provided inventory file or url. Example: --inventory=inventory.yml
or --inventory=http://localhost:8080/inventory
.--target
set groups, names, tags from inventory or direct hosts to run the playbook on. Example: --target=prod
(will run on all hosts in group prod
) or --target=example.com:2222
(will run on host example.com
with port 2222
). User name can be provided as a part of the direct target address as well, i.e. [email protected]:2222
--user
set the ssh user to run the playbook on remote hosts. Example: --user=test
.--key
set the ssh key to run the playbook on remote hosts. Example: --key=/path/to/key
.The target selection is done in the following order:
--target
is set, it will be used.
--target
is not set, Spot will try to check it targets
list for the task. If set, it will use it following the same logic as above.default
target.Spot offers support for dynamic targets, allowing the list of targets to be defined dynamically using variables. This feature becomes particularly useful when users need to ascertain a destination address within one task, and subsequently use it in another task. Here is an illustrative example:
tasks:
- name: get host
targets: ["default"]
script: |
export thehost=$(curl -s http://example.com/next-host)
options: {local: true}
- name: run on host
targets: ["$thehost"]
script: |
echo "doing something on $thehost"
In this example, the host address is initially fetched from http://example.com/next-host. Following this, the task "run on host" is executed on the host that was just identified. This ability to use dynamic targets proves beneficial in a variety of scenarios, especially when the list of hosts is not predetermined.
A practical use case for dynamic targets arises during the provisioning of a new host, followed by the execution of commands on it. Since the IP address of the new host isn't known beforehand, dynamic retrieval becomes essential.
The reason the first task specifies targets: ["default"]
is because Spot requires some target to execute a task. In this case, all commands in "get host" tasks are local and won't be invoked on a remote host. The default
target is utilized by Spot if no alternative target is specified via the command line.
The inventory file is a simple yml (or toml) that can represent a list of hosts or a list of groups with hosts. In case if both groups and hosts are defined, the hosts will be merged with groups and will add a new group named hosts
.
By default, inventory is loaded from the file/url set in SPOT_INVENTORY
environment variable. This is the lowest priority and can be overridden by inventory
from the playbook (next priority) and --inventory
flag (highest priority)
.
This is an example of the inventory file with groups
groups:
dev:
- {host: "h1.example.com", name: "h1", tags:["us-east1", "vpc-1234567"]}
- {host: "h2.example.com", port: 2233, name: "h2"}
- {host: "h3.example.com", user: "user1"}
- {host: "h4.example.com", user: "user2", name: "h4"}
staging:
- {host: "h5.example.com", port: 2233, name: "h5"}
- {host: "h6.example.com", user: "user3", name: "h6"}
--user
flag.In case if port is not defined, the default port 22 will be used. If the user is not defined, the playbook's user will be used.
This is an example of the inventory file with hosts only (no groups)
hosts:
- {host: "hh1.example.com", name: "hh1"}
- {host: "hh2.example.com", port: 2233, name: "hh2", user: "user1"}
- {host: "h2.example.com", port: 2233, name: "h2", tags:["us-east1", "vpc-1234567"]}
- {host: "h3.example.com", user: "user1", name: "h3"}
- {host: "h4.example.com", user: "user2", name: "h4"}
This format is useful when you want to define a list of hosts without groups.
In each case, inventory is automatically merged and a special group all
will be created that contains all the hosts.
Alternatively, the inventory can be represented using the TOML format.
Spot supports exporting all the destinations from selected/matched targets to the file or stdout. This is useful when users want to use the same hosts/ports/server-names/etc in other systems. By default, with --gen
option, Spot will export to stdout in json format. To export to the file, --gen.output=/path/to/file
option can be used.
This exported list of destinations can be consumed by another system, but practically it will require some conversion from the spot's json to the format that is supported by the system. This can be addressed by injecting jq
into the mix but spot also offers a better solution - templating with the standard go templates. To turn this feature on, --gen.template=/path/to/template
option can be used.
Example of the template file, showing all the fields that can be used:
{{- range .}}
"Name": "{{.Name}}"
"Host:Port": "{{.Host}}:{{.Port}}"
"User": "{{.User}}"
"Tags": [{{range .Tags}}"{{.}}"{{end}}]
{{- end -}}
for more info see go templates
Spot supports runtime variables that can be used in the playbook file. The following variables are supported:
{SPOT_REMOTE_HOST}
: The remote hostname or IP address with port.{SPOT_REMOTE_ADDR}
: The remote hostname or IP address.{SPOT_REMOTE_PORT}
: The remote port.{SPOT_REMOTE_NAME}
: The remote custom name, set in inventory or playbook as name
.{SPOT_REMOTE_USER}
: The remote username.{SPOT_COMMAND}
: The command name.{SPOT_TASK}
: The task name.{SPOT_ERROR}
: The error message, if any.Variables can be used in the following places: script
, copy
, sync
, delete
, wait
and env
, for example:
tasks:
name: deploy-things
commands:
- name: copy configuration
copy: {"src": "{SPOT_REMOTE_HOST}/conf.yml", "dst": "/tmp/conf.yml", "mkdir": true}
- name: sync things
sync: {"src": "testdata", "dst": "/tmp/{SPOT_TASK}/things"}
- name: some command
script: |
ls -laR /tmp/${SPOT_COMMAND}
env: { FOO: bar, BAR: "{SPOT_COMMAND}-blah" }
- name: delete things
delete: {"loc": "/tmp/things/{SPOT_REMOTE_USER}", "recur": true}
Spot supports ad-hoc commands that can be executed on the remote hosts. This is useful when all is needed is to execute a command on the remote hosts without creating a playbook file. This command is optionally passed as a first argument, i.e. spot "ls -la /tmp"
and usually accompanied by the --target=<host>
(-t <host>
) flags. Example: spot "ls -la" -t h1.example.com -t h2.example.com
.
All other overrides can be used with ad-hoc commands as well, for example --user
and --key
to specify the user and sshkey to use when connecting to the remote hosts. By default, Spot will use the current user and the default ssh key. Inventory can be passed to such commands as well, for example --inventory=inventory.yml
.
Adhoc commands always sets verbose
to true
automatically, so the user can see the output of the command.
Spot supports rolling updates, which means that the tasks will be executed on the hosts one by one, waiting for the previous host to finish before starting the next one. This is useful when you need to update a service running on multiple hosts but want to avoid downtime. To enable rolling updates, use the --concurrent=N
flag when running the spot
command. N
is the number of hosts to execute the tasks concurrently. Example: spot --concurrent=2
. In addition, the user can use a built-in wait
command to wait for a service to start before executing the next command. See the Command Types section for more details. Practically, the user will have a task with a series of commands, where the last command will wait for the service to start by running a command like curl -s --fail localhost:8080
and then the task will be executed on the next host.
Spot supports secrets, which are encrypted string values that can be used in the playbook file. This feature is useful for storing sensitive information, such as passwords or API keys. Secrets are encrypted, and their values are decrypted at runtime. Spot supports three types of secret providers: built-in, Hashicorp Vault, and AWS Secrets Manager. Other providers can be added by implementing the SecretsProvider
interface with a single GetSecrets
method.
Using secrets is simple. First, users need to define a secret provider in the command line options or environment variables. Then, users can add secrets to any command in the playbook file by setting options.secrets
, as shown in the following example:
tasks:
- name: access sensitive data
commands:
- name: read api response
script: |
curl -s -u ${user}:${password} https://api.example.com
curl https://api.example.com -H "Authorization: Bearer ${token}"
options:
secrets: [user, password, token]
In this case secrets for keys user
, password
and token
will be read from the secrets provider, decrypted at runtime, and passed to the command in the environment. Please note: if a user runs spot
with the --verbose
or --dbg
flag, the secrets will be replaced with ****
in the output. This is done to prevent secrets from being displayed or logged.
Sometimes, users may want to use the same set of secrets in multiple commands. To avoid repeating the secrets in each command, users can set secrets
at the task level, as shown in the following example:
tasks:
- name: access sensitive data
commands:
- name: read api response
script: |
curl -s -u ${user}:${password} https://api.example.com
curl https://api.example.com -H "Authorization: Bearer ${token}"
options:
secrets: [user, password, token]
Spot includes a built-in secrets provider that can be used to store secrets in SQLite, MySQL, or Postgresql databases. The provider can be configured using the following command line options or environment variables:
--secrets.provider=spot
: selects the built-in secret`s provider.--secrets.conn
or $SPOT_SECRETS_CONN
: the connection string to the database
file:///path/to/database.db
or /path/to/database.sqlite
or /path/to/database.db
, default: spot.db
user:password@tcp(host:port)/dbname
postgres://user:password@host:port/database?option1=value1&option2=value2
--secrets.key
or $SPOT_SECRETS_KEY
: the encryption key to use for decrypting secrets.If spot
provider is selected, the table spot_secrets
will be created in the database. The table has the following columns: skey
and sval
. The skey
column is the secret key, and the sval
column is the encrypted secret value. The skey
column is indexed for faster lookups. It is recommended to use application-specific prefixes for the secret keys, for example, system-name/service-name/secret-key
. This will allow to use the same database for multiple applications without conflicts.
The built-in secrets provider uses strong cryptography techniques to ensure the safety of your secrets. Below is a summary of the security methods employed:
These methods work together to provide a robust and secure way to manage secrets in Spot. By using the built-in secrets provider, users can be confident that their sensitive data is securely stored and protected from unauthorized access.
Spot supports Hashicorp Vault as a secrets provider. To use it, a user needs to set the following command line options or environment variables:
--secrets.provider=vault
: selects the Hashicorp Vault secrets provider.--secrets.vault.token
or $SPOT_SECRETS_VAULT_TOKEN
: the Vault token to use for authentication.--secrets.vault.url
or $SPOT_SECRETS_VAULT_URL
: the Vault server url.--secrets.vault.path
or $SPOT_SECRETS_VAULT_PATH
: the path to the secrets in Vault.Spot supports AWS Secrets Manager as a secrets provider. To use it, a user needs to set the following command line options or environment variables:
--secrets.provider=aws
: selects the AWS Secrets Manager secrets provider.--secrets.aws.region
or $SPOT_SECRETS_AWS_REGION
: the AWS region to use for authentication.--secrets.aws.access-key
or $SPOT_SECRETS_AWS_ACCESS_KEY
: the AWS access key to use for authentication.--secrets.aws.secret-key
or $SPOT_SECRETS_AWS_SECRET_KEY
: the AWS secret key to use for authentication.note: by default, the AWS Secrets Manager secrets provider will use the default AWS credential. This means that the provider will use the credentials from the environment variables AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
.
Spot gives the ability to use full encrypted YAML
files by ansbile-vault
--secrets.provider=ansible-vault
: selects the Ansible Vault secrets provider.
--secrets.ansible.path
or $SPOT_SECRETS_ANSIBLE_PATH
path to the ansible-vault file
--secrets.ansible.secret
or $SPOT_SECRETS_ANSIBLE_SECRET
secret string for decrypting ansible-vault file
note: encrypted values in the vault should be in the following format key[string]:value[string]
without nested lists
and maps
.
spot-secrets
Spot provides a simple way to manage secrets for builtin providers using the spot-secrets
utility. This command can be used to set, delete, get, and list secrets in the database.
spot-secrets set <key> <value>
: sets the secret value for the specified key.spot-secrets get <key>
: gets the secret value for the specified key.spot-secrets delete <key>
: deletes the secret value for the specified key.spot-secrets list
: lists all the secret keys in the database.Usage:
spot-secrets [OPTIONS] <command>
Application Options:
-k, --key= key to use for encryption/decryption [$SPOT_SECRETS_KEY]
-c, --conn= connection string to use for the secrets database (default: spot.db) [$SPOT_SECRETS_CONN]
--dbg debug mode
Help Options:
-h, --help Show this help message
Available commands:
del delete a secret
get retrieve a secret
list list secrets keys
set add a new secret
Spot is simple. It only has a few basic commands with a very limited set of options and flags. The playbook is just a list of commands to run, plus a list of remote targets to apply those commands against. Each command is made to be as intuitive and as direct as possible. Despite its simplicity, Spot is surprisingly powerful and can help get things done. This tool was built out of frustration with the complexity of similar tools. All I wanted was something that is simple, easy to use, easy to understand, and capable of handling most of the usual deployment tasks. I didn't want to have to check the documentation or resort to googling every time I used it. Spot is the result of that effort.
Spot is designed to provide a simple, efficient, and flexible solution for deployment and configuration management. It addresses the need for a tool that is easy to set up and use, while still offering powerful features for managing infrastructure.
Below are some of the reasons why you should consider using Spot:
In conclusion, Spot is a powerful and easy-to-use tool that simplifies the process of deployment and configuration management while offering the flexibility and extensibility needed to cater to various use cases.
Spot is not designed as a direct replacement for Ansible; however, in certain use cases, it can address the same challenges effectively. While both tools can be used for deployment and configuration management, there are some key differences between them:
Spot is an appealing choice for those seeking a lightweight, simple, and easy-to-use tool for deployment and configuration management, especially for smaller projects or when extensive features aren't necessary. Its single binary distribution, easy-to-comprehend structure, and minimal dependencies offer a low-maintenance solution. However, if a more comprehensive tool with a wide range of built-in modules, plugins, and integrations is needed, Ansible may be a better fit. While Ansible has advanced features and a robust ecosystem, its reliance on Python and additional libraries can sometimes be less convenient in certain environments or situations with specific constraints.
If you want to try the latest development version, you can install it directly from the master branch. There are two ways to do this:
go install github.com/umputun/spot/cmd/spot@master
and go install github.com/umputun/spot/cmd/secrets@master
. Note that this will install the latest development version of spot and secrets, which may not be stable or fully tested.git clone github.com/umputun/spot
then cd spot
and make build
. This will install the latest development version of spot and secrets to spot/.bin/spot
and spot/.bin/sport-secrets
, respectively.pls note that you need to have go 1.16+ installed on your machine.
The project is currently in active development, and breaking changes may occur until the release of version 1.0. However, we strive to minimize disruptions and will only introduce breaking changes when there is a compelling reason to do so.
Update: Version 1 has been released and is now considered stable. We do not anticipate any breaking changes for this version.
Please feel free to open a discussion, submit issues, fork the repository, and send pull requests. See CONTRIBUTING.md for more information.
This project is licensed under the MIT License. See the LICENSE file for more information.