STABILITY: PROTOTYPE

Le restique is a wrapper around the restic backup tool that allows using multiple backup profiles defined in a JSON configuration file.
You should be familiar with restic before using this. Read the restic manual first.

It's written in Python and uses only core modules so that there are no extra dependencies to install, and no compilation or build steps.
You can literally just copy this into /usr/bin/ and you're done.

:see_no_evil: I don't really know Python. So if you see some bad things in the code please feel free to file an issue or create a pull request. It will be appreciated.

Jump: Installation | Usage | Configuration | Examples | License

Installation

First, install Python 3. You probably already have it. Then install restic. Then download the restique executable and put it somewhere in your PATH. Don't forget to chmod +x path/to/restique

:penguin: This program has only been tested on Linux with Python 3.6. It might not work with older Python versions or on other platforms.

Usage

restique [-h] [--debug] [--config PATH] [--profile NAME] command [resticargs]

• -h, --help: Print the help / usage text.

• -V, --version: Print the version number.

• -d, --debug: Set the log level to debug.

• -c PATH, --config PATH: Specify the path to the configuration file. If not specified restic will check ./.restiquerc, ~/.restiquerc, ~/.config/restique/config, /etc/restique.rc in this exact order and use the first one that exists.

• -p NAME(S), --profile NAME(S): Specify the name of the backup profile to be used. If not specified, restique will use the "default_profile" provided in the config file.
  Multiple profile names can be specified as a comma-separated list (without spaces). If multiple profile names are provided they will be merged into each other in the order they are specified. This can be useful when combining partial profiles.

• command: The restic command to run. E.g. "init", "backup", "restore", ... check out the restic manual for a complete list.

• resticargs: All additional arguments will be directly passed to the restic command.

Example

# Run a backup
restique -p home backup

# Restore the latest backup to /tmp/restore-home
restique -p home restore latest --target /tmp/restore-home

Configuration

{
  "restic_path": "/path/to/restic_executable",
  "default_profile": "my_profile",
  "global": {
    // ... profile config applied to all profiles
  },
  "profiles": {
    "my_profile": {
      "json": true,
      "no_lock": true,
      "quiet": true,
      "files": [
        "/path/to/directory/or/file_to_backup",
        "/path/to/directory/or/another_file_to_backup"
      ],
      "excludes": [
        "*.go",
        "foo/**/bar"
      ],
      "keep_daily": 5,
      "keep_weekly": 4,
      "keep_monthly": 6,
      "keep_yearly": 10,
      "keep_tags": ["important"],
      "password": "correct horse battery staple",
      "repository": "/path/to/repo",
      "aws_access_key": "...",
      "aws_secret_key": "...",
      "b2_account_id": "...",
      "b2_account_key": "...",
      "azure_account_name": "...",
      "azure_account_key": "...",
      "google_project_id": "...",
      "google_application_credentials": "...",
      "hooks": {
        "pre_repo": [
          "some-shell-commands"
        ],
        "post_repo": [
          // ...
        ],
        "pre_<COMMAND>": [
          // ...
        ],
        "post_<COMMAND>": [
          // ...
        ]
      }
    },
    "my_other_profile": {
      // ...
    },
    ...
  }
}

• restic_path (string, optional, default: which(restic)): Path to the restic executable. If not specified it will look up the path in the PATH.

• default_profile (string, optional): The name of the profile that will be used when no profile name is specified through the --profile argument.
  Multiple profile names can be specified as a comma-separated list (without spaces). If multiple profile names are provided they will be merged into each other in the order they are specified. This can be useful when combining partial profiles.

• global (object, optional): A global profile that will be merged into all other profiles. If the non-global profile contains the same options the global options will be overridden by the profile. Except files and excludes will be combined.

• profiles.[NAME].initialize (boolean, optional, default: false): If true, it will automatically initialize an uninitialized repository.

• profiles.[NAME].json (boolean, optional, default: false): Calls restic with the --json argument.

• profiles.[NAME].no_lock (boolean, optional, default: false): Calls restic with the --no-lock argument.

• profiles.[NAME].quiet (boolean, optional, default: false): Calls restic with the --quiet argument.

• profile.[NAME].files (array, required): A list of paths to files and or directories that will be backed up.

• profile.[NAME].excludes (array, optional): A list of file patterns that will be excluded from the backup.
  Note that this only applies to the backup command. Exclude patterns for the restore command must be provided through the --exclude and --include arguments.

• profile.[NAME].forget_tags (array, optional): List of tags to consider when running the forget command. Equivalent of multiple --tag t arguments.

• profile.[NAME].forget_hosts (array, optional): List of hosts to consider when running the forget command. Equivalent of multiple --host h arguments.

• profile.[NAME].keep_last (number, optional): Number of most recent snapshots to keep. Equivalent of --keep-last n. Only used with forget command.

• profile.[NAME].keep_hourly (number, optional): Number of hourly snapshots to keep. Equivalent of --keep-hourly n. Only used with forget command.

• profile.[NAME].keep_daily (number, optional): Number of daily snapshots to keep. Equivalent of --keep-daily n. Only used with forget command.

• profile.[NAME].keep_weekly (number, optional): Number of weekly snapshots to keep. Equivalent of --keep-weekly n. Only used with forget command.

• profile.[NAME].keep_monthly (number, optional): Number of monthly snapshots to keep. Equivalent of --keep-monthly n. Only used with forget command.

• profile.[NAME].keep_yearly (number, optional): Number of yearly snapshots to keep. Equivalent of --keep-yearly n. Only used with forget command.

• profile.[NAME].keep_tags (array, optional): List of snapshot tags to keep. Equivalent of using multiple --keep-tag t arguments. Only used with forget command.

• profile.[NAME].keep_within (string, optional): Diration within to keep all snapshots. For example 2y5m7d3h. Equivalent of --keep-within d. Only used with forget command.

• profile.[NAME].password (string, required): The password that will be used to encrypt and sign the backup.

• profile.[NAME].repository (string, required): The repository path for the backup. Read the restic docs to see how to format the repository path for your storage backend.

• profile.[NAME].aws_access_key (string, optional): S3 access key. Used only when hosting the repository on S3.

• profile.[NAME].aws_secret_key (string, optional): S3 secret key. Used only when hosting the repository on S3.

• profile.[NAME].b2_account_id (string, optional): Backblaze B2 account ID. Used only when hosting the repository on Backblaze B2.

• profile.[NAME].b2_account_key (string, optional): Backblaze B2 access key. Used only when hosting the repository on Backblaze B2.

• profile.[NAME].azure_account_name (string, optional): Microsoft Azure account name. Used only when hosting the repository on Microsoft Azure.

• profile.[NAME].azure_account_key (string, optional): Microsoft Azure account key. Used only when hosting the repository on Microsoft Azure.

• profile.[NAME].google_project_id (string, optional): Google Cloud Storage project ID. Used only when hosting the repository on Google Cloud Storage.

• profile.[NAME].google_application_credentials (string, optional): File path to Google Cloud Storage application credentials file. Used only when hosting the repository on Google Cloud Storage.

• profile.[NAME].hooks.pre_repo (array of strings, optional): An array of shell commands that will be executed sequentially and in order before each restic command that accesses the repo. If one of the shell commands fails (i.e. exits with a non-zero exit code) all following commands including the restic command will not run and the program will exit.

• profile.[NAME].hooks.post_repo (array of strings, optional): An array of shell commands that will be executed sequentially and in order after each restic command that accesses the repo. If one of the shell commands fails (i.e. exits with a non-zero exit code) all following commands will not run and the program will exit.

• profile.[NAME].hooks.pre_[COMMAND] (array of strings, optional): An array of shell commands that will be executed sequentially and in order before each restic [COMMAND] command. If one of the shell commands fails (i.e. exits with a non-zero exit code) all following commands including the restic command will not run and the program will exit.

• profile.[NAME].hooks.post_[COMMAND] (array of strings, optional): An array of shell commands that will be executed sequentially and in order after each restic COMMAND command. If one of the shell commands fails (i.e. exits with a non-zero exit code) all following commands will not run and the program will exit.

About Hooks

Hooks can be used to do stuff before and after the repo is accessed, or before and after a specific command is executed. For example, a USB drive could be mounted in the pre_repo hook and then unmounted in the post_repo hook.

Each hook is executed in a separate shell and has access to all environment variables including the ones passed to restic. Hooks are executed in the following order:

• pre_repo
• pre_[COMMAND] (unless pre_repo failed)
• call to restic (unless pre_[COMMAND] failed)
• post_[COMMAND] (unless the restic command failed)
• post_repo (unless post_[COMMAND] failed)

So in case of restique backup it would call pre_repo, pre_backup, restic backup, post_backup, and finally post_repo.

Examples

Simple Local Backup

Config:

{
  "default_profile": "home",
  "profiles": {
    "home": {
      "repository": "/mnt/my_external_drive/backups",
      "password": "correct horse battery staple",
      "files": [
        "/home"
      ],
      "excludes": [
        "pr0n/**/*.mp4"
      ]
    }
  }
}

Initialize:

restique init

Backup:

restique backup

Integrity check:

restique check

Restore:

restique restore latest --target /tmp/home-restored

Backing Up Docker Images to Backblaze

Config:

{
  "default_profile": "dockerimages",
  "profiles": {
    "dockerimages": {
      "repository": "b2:mydockerbackups:/images",
      "b2_account_id": "1a2345b67890",
      "b2_account_key": "1234a5678b901c3d45678ef901g2h34567i89jklmn",
      "password": "correct horse battery staple"
    }
  }
}

Initialize:

restique init

Export and backup the busybox docker image:

docker save busybox | restique backup --stdin --stdin-name busybox.tar

List the contents of the latest snapshot:

restique ls latest

Output:

snapshot fb120820 of [busybox.tar] at 2017-10-01 20:30:03.222656757 +0200 CEST):
/busybox.tar

Restore the docker image:

restique restore latest --target /tmp/ --include busybox.tar

docker load < /tmp/busybox.tar

Combining Partial Profiles

Here we are going to create four partial profiles. Two of them contain information about different backups that we want to make, and the other two contain information about two different repositories where we want to store our backups. By combining one of the former profiles with one of the latter we can save the same backup on different repositories.

Config:

{
  "global": {
    "password": "correct horse battery staple"
  },
  "profiles": {
    "personal": {
      "files": [ "/home/user" ],
      "excludes": [ "/home/user/work" ]
    },
    "work": {
      "files": [ "/home/user/work" ]
    },
    "local": {
      "repository": "/mnt/my_external_drive/backups"
    },
    "remote": {
      "repository": "s3:s3.amazonaws.com/my_backups_bucket",
      "aws_access_key": "AAABBB999CCC666UU000",
      "aws_secret_key": "aaaa/BBBbBbBBbbbbBBBBBbBbBB/CCCcCcccCCCC"
    }
  }
}

Initialize S3 repository:

restique -p remote init

Initialize local repository:

restique -p local init

Backup personal files to S3:

restique -p personal,remote backup

Backup personal files locally:

restique -p personal,local backup

Backup work files to S3:

restique -p work,remote backup

Backup work files locally:

restique -p work,local backup

Restore a snapshot from the S3 repository:

restique -p remote restore e61c4ad7 --target /tmp/work-restored

Restore a snapshot from the local repository:

restique -p local restore 0eb99523 --target /tmp/personal-restored

Run Only if External Drive is Mounted

Config:

{
  "default_profile": "home",
  "profiles": {
    "home": {
      "repository": "/mnt/my_external_drive/backups",
      "password": "correct horse battery staple",
      "files": [
        "/home"
      ],
      "hooks": {
        "pre_repo": [
          "mountpoint -q '/mnt/my_external_drive/backups'"
        ]
      }
    }
  }
}

Try backup:

restique backup

Output:

ERROR: Command 'mountpoint -q '/mnt/my_external_drive/backups'' returned non-zero exit status 1.
ERROR: One or more 'pre_repo' hooks failed.

License

Copyright (c) 2017 Max Kueng

MIT License