Restic Versions Save

Fast, secure, efficient backup program

v0.16.4

1 month ago

This release works around and improves detection of a bug in the compression library used by restic. The resulting issue only happens when using restic 0.16.3 and the max compression level (the default auto and off compression levels are not affected), and when the source files being backed up have specific data in them to trigger the bug. If you use max compression, you can use restic check --read-data to make sure you're not affected.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.16.4 (2024-02-04)

The following sections list the changes in restic 0.16.4 relevant to restic users. The changes are ordered by importance.

Summary

  • Fix #4677: Downgrade zstd library to fix rare data corruption at max. compression
  • Enh #4529: Add extra verification of data integrity before upload

Details

  • Bugfix #4677: Downgrade zstd library to fix rare data corruption at max. compression

    In restic 0.16.3, backups where the compression level was set to max (using --compression max) could in rare and very specific circumstances result in data corruption due to a bug in the library used for compressing data. Restic 0.16.1 and 0.16.2 were not affected.

    Restic now uses the previous version of the library used to compress data, the same version used by restic 0.16.2. Please note that the auto compression level (which restic uses by default) was never affected, and even if you used max compression, chances of being affected by this issue are small.

    To check a repository for any corruption, run restic check --read-data. This will download and verify the whole repository and can be used at any time to completely verify the integrity of a repository. If the check command detects anomalies, follow the suggested steps.

    #4677 #4679

  • Enhancement #4529: Add extra verification of data integrity before upload

    Hardware issues, or a bug in restic or its dependencies, could previously cause corruption in the files restic created and stored in the repository. Detecting such corruption previously required explicitly running the check --read-data or check --read-data-subset commands.

    To further ensure data integrity, even in the case of hardware issues or software bugs, restic now performs additional verification of the files about to be uploaded to the repository.

    These extra checks will increase CPU usage during backups. They can therefore, if absolutely necessary, be disabled using the --no-extra-verify global option. Please note that this should be combined with more active checking using the previously mentioned check commands.

    #4529 #4681

v0.16.3

2 months ago

This release fixes a couple of bugs on Windows and in the restore command. It also works around an unlikely yet possible situation with rclone which could potentially result in data loss.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.16.3 (2024-01-14)

The following sections list the changes in restic 0.16.3 relevant to restic users. The changes are ordered by importance.

Summary

  • Fix #4560: Improve errors for irregular files on Windows
  • Fix #4574: Support backup of deduplicated files on Windows again
  • Fix #4612: Improve error handling for rclone backend
  • Fix #4624: Correct restore progress information if an error occurs
  • Fix #4626: Improve reliability of restoring large files

Details

  • Bugfix #4560: Improve errors for irregular files on Windows

    Since Go 1.21, most filesystem reparse points on Windows are considered to be irregular files. This caused restic to show an error: invalid node type "" error message for those files.

    This error message has now been improved and includes the relevant file path: error: nodeFromFileInfo path/to/file: unsupported file type "irregular". As irregular files are not required to behave like regular files, it is not possible to provide a generic way to back up those files.

    #4560 #4620 https://forum.restic.net/t/windows-backup-error-invalid-node-type/6875

  • Bugfix #4574: Support backup of deduplicated files on Windows again

    With the official release builds of restic 0.16.1 and 0.16.2, it was not possible to back up files that were deduplicated by the corresponding Windows Server feature. This also applied to restic versions built using Go 1.21.0-1.21.4.

    The Go version used to build restic has now been updated to fix this.

    #4574 #4621

  • Bugfix #4612: Improve error handling for rclone backend

    Since restic 0.16.0, if rclone encountered an error while listing files, this could in rare circumstances cause restic to assume that there are no files. Although unlikely, this situation could result in data loss if it were to happen right when the prune command is listing existing snapshots.

    Error handling has now been improved to detect and work around this case.

    #4612 #4618

  • Bugfix #4624: Correct restore progress information if an error occurs

    If an error occurred while restoring a snapshot, this could cause the restore progress bar to show incorrect information. In addition, if a data file could not be loaded completely, then errors would also be reported for some already restored files.

    Error reporting of the restore command has now been made more accurate.

    #4624 https://forum.restic.net/t/errors-restoring-with-restic-on-windows-server-s3/6943

  • Bugfix #4626: Improve reliability of restoring large files

    In some cases restic failed to restore large files that frequently contain the same file chunk. In combination with certain backends, this could result in network connection timeouts that caused incomplete restores.

    Restic now includes special handling for such file chunks to ensure reliable restores.

    #4626 https://forum.restic.net/t/errors-restoring-with-restic-on-windows-server-s3/6943

v0.16.2

4 months ago

restic 0.16.2 restores support for ARMv5 which was accidentally not included in the previous one, and also fixes missing documentation on Read the Docs.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.16.2 (2023-10-29)

The following sections list the changes in restic 0.16.2 relevant to restic users. The changes are ordered by importance.

Summary

  • Fix #4540: Restore ARMv5 support for ARM binaries
  • Fix #4545: Repair documentation build on Read the Docs

Details

  • Bugfix #4540: Restore ARMv5 support for ARM binaries

    The official release binaries for restic 0.16.1 were accidentally built to require ARMv7. The build process is now updated to restore support for ARMv5.

    Please note that restic 0.17.0 will drop support for ARMv5 and require at least ARMv6.

    #4540

  • Bugfix #4545: Repair documentation build on Read the Docs

    For restic 0.16.1, no documentation was available at https://restic.readthedocs.io/ .

    The documentation build process is now updated to work again.

    #4545

v0.16.1

5 months ago

restic 0.16.1 has been released! It contains a few usage and cosmetic improvements, but also one bugfix for a very unlikely but still possible data corruption issue when using the max compression level. Please read the changelog for more details and information on how to verify your repository if you used max compression. Please also note that the auto compression level (which is the default) was never affected.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.16.1 (2023-10-24)

The following sections list the changes in restic 0.16.1 relevant to restic users. The changes are ordered by importance.

Summary

  • Fix #4513: Make key list command honor --no-lock
  • Fix #4516: Do not try to load password on command line autocomplete
  • Fix #4523: Update zstd library to fix possible data corruption at max. compression
  • Chg #4532: Update dependencies and require Go 1.19 or newer
  • Enh #229: Show progress bar while loading the index
  • Enh #4128: Automatically set GOMAXPROCS in resource-constrained containers
  • Enh #4480: Allow setting REST password and username via environment variables
  • Enh #4511: Include inode numbers in JSON output for find and ls commands
  • Enh #4519: Add config option to set SFTP command arguments

Details

  • Bugfix #4513: Make key list command honor --no-lock

    The key list command now supports the --no-lock options. This allows determining which keys a repo can be accessed by without the need for having write access (e.g., read-only sftp access, filesystem snapshot).

    #4513 #4514

  • Bugfix #4516: Do not try to load password on command line autocomplete

    The command line autocompletion previously tried to load the repository password. This could cause the autocompletion not to work. Now, this step gets skipped.

    #4516 #4526

  • Bugfix #4523: Update zstd library to fix possible data corruption at max. compression

    In restic 0.16.0, backups where the compression level was set to max (using --compression max) could in rare and very specific circumstances result in data corruption due to a bug in the library used for compressing data.

    Restic now uses the latest version of the library used to compress data, which includes a fix for this issue. Please note that the auto compression level (which restic uses by default) was never affected, and even if you used max compression, chances of being affected by this issue were very small.

    To check a repository for any corruption, run restic check --read-data. This will download and verify the whole repository and can be used at any time to completely verify the integrity of a repository. If the check command detects anomalies, follow the suggested steps.

    To simplify any needed repository repair and minimize data loss, there is also a new and experimental repair packs command that salvages all valid data from the affected pack files (see restic help repair packs for more information).

    #4523 #4530

  • Change #4532: Update dependencies and require Go 1.19 or newer

    We have updated all dependencies. Since some libraries require newer Go standard library features, support for Go 1.18 has been dropped, which means that restic now requires at least Go 1.19 to build.

    #4532 #4533

  • Enhancement #229: Show progress bar while loading the index

    Restic did not provide any feedback while loading index files. Now, there is a progress bar that shows the index loading progress.

    #229 #4419

  • Enhancement #4128: Automatically set GOMAXPROCS in resource-constrained containers

    When running restic in a Linux container with CPU-usage limits, restic now automatically adjusts GOMAXPROCS. This helps to reduce the memory consumption on hosts with many CPU cores.

    #4128 #4485 #4531

  • Enhancement #4480: Allow setting REST password and username via environment variables

    Previously, it was only possible to specify the REST-server username and password in the repository URL, or by using the --repository-file option. This meant it was not possible to use authentication in contexts where the repository URL is stored in publicly accessible way.

    Restic now allows setting the username and password using the RESTIC_REST_USERNAME and RESTIC_REST_PASSWORD variables.

    #4480

  • Enhancement #4511: Include inode numbers in JSON output for find and ls commands

    Restic used to omit the inode numbers in the JSON messages emitted for nodes by the ls command as well as for matches by the find command. It now includes those values whenever they are available.

    #4511

  • Enhancement #4519: Add config option to set SFTP command arguments

    When using the sftp backend, scenarios where a custom identity file was needed for the SSH connection, required the full command to be specified: -o sftp.command='ssh user@host:port -i /ssh/my_private_key -s sftp'

    Now, the -o sftp.args=... option can be passed to restic to specify custom arguments for the SSH command executed by the SFTP backend. This simplifies the above example to -o sftp.args='-i /ssh/my_private_key'.

    #4241 #4519

v0.16.0

7 months ago

restic 0.16.0 has been released!

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.16.0 (2023-07-31)

The following sections list the changes in restic 0.16.0 relevant to restic users. The changes are ordered by importance.

Summary

  • Fix #2565: Support "unlimited" in forget --keep-* options
  • Fix #3311: Support non-UTF8 paths as symlink target
  • Fix #4199: Avoid lock refresh issues on slow network connections
  • Fix #4274: Improve lock refresh handling after standby
  • Fix #4319: Correctly clean up status bar output of the backup command
  • Fix #4333: generate and init no longer silently ignore unexpected arguments
  • Fix #4400: Ignore missing folders in rest backend
  • Chg #4176: Fix JSON message type of scan_finished for the backup command
  • Chg #4201: Require Go 1.20 for Solaris builds
  • Enh #426: Show progress bar during restore
  • Enh #719: Add --retry-lock option
  • Enh #1495: Sort snapshots by timestamp in restic find
  • Enh #1759: Add repair index and repair snapshots commands
  • Enh #1926: Allow certificate paths to be passed through environment variables
  • Enh #2359: Provide multi-platform Docker images
  • Enh #2468: Add support for non-global Azure clouds
  • Enh #2679: Reduce file fragmentation for local backend
  • Enh #3328: Reduce memory usage by up to 25%
  • Enh #3397: Improve accuracy of ETA displayed during backup
  • Enh #3624: Keep oldest snapshot when there are not enough snapshots
  • Enh #3698: Add support for Managed / Workload Identity to azure backend
  • Enh #3871: Support <snapshot>:<subfolder> syntax to select subfolders
  • Enh #3941: Support --group-by for backup parent selection
  • Enh #4130: Cancel current command if cache becomes unusable
  • Enh #4159: Add --human-readable option to ls and find commands
  • Enh #4188: Include restic version in snapshot metadata
  • Enh #4220: Add jq binary to Docker image
  • Enh #4226: Allow specifying region of new buckets in the gs backend
  • Enh #4375: Add support for extended attributes on symlinks

Details

  • Bugfix #2565: Support "unlimited" in forget --keep-* options

    Restic would previously forget snapshots that should have been kept when a negative value was passed to the --keep-* options. Negative values are now forbidden. To keep all snapshots, the special value unlimited is now supported. For example, --keep-monthly unlimited will keep all monthly snapshots.

    #2565 #4234

  • Bugfix #3311: Support non-UTF8 paths as symlink target

    Earlier restic versions did not correctly backup and restore symlinks that contain a non-UTF8 target. Note that this only affected systems that still use a non-Unicode encoding for filesystem paths.

    The repository format is now extended to add support for such symlinks. Please note that snapshots must have been created with at least restic version 0.16.0 for restore to correctly handle non-UTF8 symlink targets when restoring them.

    #3311 #3802

  • Bugfix #4199: Avoid lock refresh issues on slow network connections

    On network connections with a low upload speed, backups and other operations could fail with the error message Fatal: failed to refresh lock in time.

    This has now been fixed by reworking the lock refresh handling.

    #4199 #4304

  • Bugfix #4274: Improve lock refresh handling after standby

    If the restic process was stopped or the host running restic entered standby during a long running operation such as a backup, this previously resulted in the operation failing with Fatal: failed to refresh lock in time.

    This has now been fixed such that restic first checks whether it is safe to continue the current operation and only throws an error if not.

    #4274 #4374

  • Bugfix #4319: Correctly clean up status bar output of the backup command

    Due to a regression in restic 0.15.2, the status bar of the backup command could leave some output behind. This happened if filenames were printed that are wider than the current terminal width. This has now been fixed.

    #4319 #4318

  • Bugfix #4333: generate and init no longer silently ignore unexpected arguments

    #4333

  • Bugfix #4400: Ignore missing folders in rest backend

    If a repository accessed via the REST backend was missing folders, then restic would fail with an error while trying to list the data in the repository. This has been now fixed.

    #235 #4400

  • Change #4176: Fix JSON message type of scan_finished for the backup command

    Restic incorrectly set the message_type of the scan_finished message to status instead of verbose_status. This has now been corrected so that the messages report the correct type.

    #4176

  • Change #4201: Require Go 1.20 for Solaris builds

    Building restic on Solaris now requires Go 1.20, as the library used to access Azure uses the mmap syscall, which is only available on Solaris starting from Go 1.20. All other platforms however continue to build with Go 1.18.

    #4201

  • Enhancement #426: Show progress bar during restore

    The restore command now shows a progress report while restoring files.

    Example: [0:42] 5.76% 23 files 12.98 MiB, total 3456 files 23.54 GiB

    JSON output is now also supported.

    #426 #3413 #3627 #3991 #4314 https://forum.restic.net/t/progress-bar-for-restore/5210

  • Enhancement #719: Add --retry-lock option

    This option allows specifying a duration for which restic will wait if the repository is already locked.

    #719 #2214 #4107

  • Enhancement #1495: Sort snapshots by timestamp in restic find

    The find command used to print snapshots in an arbitrary order. Restic now prints snapshots sorted by timestamp.

    #1495 #4409

  • Enhancement #1759: Add repair index and repair snapshots commands

    The rebuild-index command has been renamed to repair index. The old name will still work, but is deprecated.

    When a snapshot was damaged, the only option up to now was to completely forget the snapshot, even if only some unimportant files in it were damaged and other files were still fine.

    Restic now has a repair snapshots command, which can salvage any non-damaged files and parts of files in the snapshots by removing damaged directories and missing file contents. Please note that the damaged data may still be lost and see the "Troubleshooting" section in the documentation for more details.

    #1759 #1714 #1798 #2334 #2876 https://forum.restic.net/t/corrupted-repo-how-to-repair/799https://forum.restic.net/t/recovery-options-for-damaged-repositories/1571

  • Enhancement #1926: Allow certificate paths to be passed through environment variables

    Restic will now read paths to certificates from the environment variables RESTIC_CACERT or RESTIC_TLS_CLIENT_CERT if --cacert or --tls-client-cert are not specified.

    #1926 #4384

  • Enhancement #2359: Provide multi-platform Docker images

    The official Docker images are now built for the architectures linux/386, linux/amd64, linux/arm and linux/arm64.

    As an alternative to the Docker Hub, the Docker images are also available on ghcr.io, the GitHub Container Registry.

    #2359 #4269 #4364

  • Enhancement #2468: Add support for non-global Azure clouds

    The azure backend previously only supported storages using the global domain core.windows.net. This meant that backups to other domains such as Azure China (core.chinacloudapi.cn) or Azure Germany (core.cloudapi.de) were not supported. Restic now allows overriding the global domain using the environment variable AZURE_ENDPOINT_SUFFIX.

    #2468 #4387

  • Enhancement #2679: Reduce file fragmentation for local backend

    Before this change, local backend files could become fragmented. Now restic will try to preallocate space for pack files to avoid their fragmentation.

    #2679 #3261

  • Enhancement #3328: Reduce memory usage by up to 25%

    The in-memory index has been optimized to be more garbage collection friendly. Restic now defaults to GOGC=50 to run the Go garbage collector more frequently.

    #3328 #4352 #4353

  • Enhancement #3397: Improve accuracy of ETA displayed during backup

    Restic's backup command displayed an ETA that did not adapt when the rate of progress made during the backup changed during the course of the backup.

    Restic now uses recent progress when computing the ETA. It is important to realize that the estimate may still be wrong, because restic cannot predict the future, but the hope is that the ETA will be more accurate in most cases.

    #3397 #3563

  • Enhancement #3624: Keep oldest snapshot when there are not enough snapshots

    The forget command now additionally preserves the oldest snapshot if fewer snapshots than allowed by the --keep-* parameters would otherwise be kept. This maximizes the amount of history kept within the specified limits.

    #3624 #4366 https://forum.restic.net/t/keeping-yearly-snapshots-policy-when-backup-began-during-the-year/4670/2

  • Enhancement #3698: Add support for Managed / Workload Identity to azure backend

    Restic now additionally supports authenticating to Azure using Workload Identity or Managed Identity credentials, which are automatically injected in several environments such as a managed Kubernetes cluster.

    #3698 #4029

  • Enhancement #3871: Support <snapshot>:<subfolder> syntax to select subfolders

    Commands like diff or restore always worked with the full snapshot. This did not allow comparing only a specific subfolder or only restoring that folder (restore --include subfolder filters the restored files, but still creates the directories included in subfolder).

    The commands diff, dump, ls and restore now support the <snapshot>:<subfolder> syntax, where snapshot is the ID of a snapshot (or the string latest) and subfolder is a path within the snapshot. The commands will then only work with the specified path of the snapshot. The subfolder must be a path to a folder as returned by ls. Two examples:

    restic restore -t target latest:/some/path restic diff 12345678:/some/path 90abcef:/some/path

    For debugging purposes, the cat command now supports cat tree <snapshot>:<subfolder> to return the directory metadata for the given subfolder.

    #3871 #4334

  • Enhancement #3941: Support --group-by for backup parent selection

    Previously, the backup command by default selected the parent snapshot based on the hostname and the backup targets. When the backup path list changed, the backup command was unable to determine a suitable parent snapshot and had to read all files again.

    The new --group-by option for the backup command allows filtering snapshots for the parent selection by host, paths and tags. It defaults to host,paths which selects the latest snapshot with hostname and paths matching those of the backup run. This matches the behavior of prior restic versions.

    The new --group-by option should be set to the same value as passed to forget --group-by.

    #3941 #4081

  • Enhancement #4130: Cancel current command if cache becomes unusable

    If the cache directory was removed or ran out of space while restic was running, this would previously cause further caching attempts to fail and thereby drastically slow down the command execution. Now, the currently running command is instead canceled.

    #4130 #4166

  • Enhancement #4159: Add --human-readable option to ls and find commands

    Previously, when using the -l option with the ls and find commands, the displayed size was always in bytes, without an option for a more human readable format such as MiB or GiB.

    The new --human-readable option will convert longer size values into more human friendly values with an appropriate suffix depending on the output size. For example, a size of 14680064 will be shown as 14.000 MiB.

    #4159 #4351

  • Enhancement #4188: Include restic version in snapshot metadata

    The restic version used to backup a snapshot is now included in its metadata and shown when inspecting a snapshot using restic cat snapshot <snapshotID> or restic snapshots --json.

    #4188 #4378

  • Enhancement #4220: Add jq binary to Docker image

    The Docker image now contains jq, which can be useful to process JSON data output by restic.

    #4220

  • Enhancement #4226: Allow specifying region of new buckets in the gs backend

    Previously, buckets used by the Google Cloud Storage backend would always get created in the "us" region. It is now possible to specify the region where a bucket should be created by using the -o gs.region=us option.

    #4226

  • Enhancement #4375: Add support for extended attributes on symlinks

    Restic now supports extended attributes on symlinks when backing up, restoring, or FUSE-mounting snapshots. This includes, for example, the security.selinux xattr on Linux distributions that use SELinux.

    #4375 #4379

v0.15.2

11 months ago

We're very pleased to present you restic 0.15.2! This is just (another) small bugfix release.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.15.2 (2023-04-24)

The following sections list the changes in restic 0.15.2 relevant to restic users. The changes are ordered by importance.

Summary

  • Sec #4275: Update golang.org/x/net to address CVE-2022-41723
  • Fix #2260: Sanitize filenames printed by backup during processing
  • Fix #4211: Make dump interpret --host and --path correctly
  • Fix #4239: Correct number of blocks reported in mount point
  • Fix #4253: Minimize risk of spurious filesystem loops with mount
  • Enh #4180: Add release binaries for riscv64 architecture on Linux
  • Enh #4219: Upgrade Minio to version 7.0.49

Details

  • Security #4275: Update golang.org/x/net to address CVE-2022-41723

    #4275 #4213

  • Bugfix #2260: Sanitize filenames printed by backup during processing

    The backup command would previously not sanitize the filenames it printed during processing, potentially causing newlines or terminal control characters to mangle the status output or even change the state of a terminal.

    Filenames are now checked and quoted if they contain non-printable or non-Unicode characters.

    #2260 #4191 #4192

  • Bugfix #4211: Make dump interpret --host and --path correctly

    A regression in restic 0.15.0 caused dump to confuse its --host=<host> and --path=<path> options: it looked for snapshots with paths called <host> from hosts called <path>. It now treats the options as intended.

    #4211 #4212

  • Bugfix #4239: Correct number of blocks reported in mount point

    Restic mount points reported an incorrect number of 512-byte (POSIX standard) blocks for files and links due to a rounding bug. In particular, empty files were reported as taking one block instead of zero.

    The rounding is now fixed: the number of blocks reported is the file size (or link target size) divided by 512 and rounded up to a whole number.

    #4239 #4240

  • Bugfix #4253: Minimize risk of spurious filesystem loops with mount

    When a backup contains a directory that has the same name as its parent, say a/b/b, and the GNU find command was run on this backup in a restic mount, find would refuse to traverse the lowest b directory, instead printing File system loop detected. This was due to the way the restic mount command generates inode numbers for directories in the mount point.

    The rule for generating these inode numbers was changed in 0.15.0. It has now been changed again to avoid this issue. A perfect rule does not exist, but the probability of this behavior occurring is now extremely small.

    When it does occur, the mount point is not broken, and scripts that traverse the mount point should work as long as they don't rely on inode numbers for detecting filesystem loops.

    #4253 #4255

  • Enhancement #4180: Add release binaries for riscv64 architecture on Linux

    Builds for the riscv64 architecture on Linux are now included in the release binaries.

    #4180

  • Enhancement #4219: Upgrade Minio to version 7.0.49

    The upgraded version now allows use of the ap-southeast-4 region (Melbourne).

    #4219

v0.15.1

1 year ago

We're very pleased to present you restic 0.15.1! This is just a small bugfix release, fixing some issues that arose after 0.15.0 was published.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.15.1 (2023-01-30)

The following sections list the changes in restic 0.15.1 relevant to restic users. The changes are ordered by importance.

Summary

  • Fix #3750: Remove b2_download_file_by_name: 404 warning from B2 backend
  • Fix #4147: Make prune --quiet not print progress bar
  • Fix #4163: Make self-update --output work with new filename on Windows
  • Fix #4167: Add missing ETA in backup progress bar
  • Enh #4143: Ignore empty lock files

Details

  • Bugfix #3750: Remove b2_download_file_by_name: 404 warning from B2 backend

    In some cases the B2 backend could print b2_download_file_by_name: 404: : b2.b2err warnings. These are only debug messages and can be safely ignored.

    Restic now uses an updated library for accessing B2, which removes the warning.

    #3750 #4144 #4146

  • Bugfix #4147: Make prune --quiet not print progress bar

    A regression in restic 0.15.0 caused prune --quiet to show a progress bar while deciding how to process each pack files. This has now been fixed.

    #4147 #4153

  • Bugfix #4163: Make self-update --output work with new filename on Windows

    Since restic 0.14.0 the self-update command did not work when a custom output filename was specified via the --output option. This has now been fixed.

    As a workaround, either use an older restic version to run the self-update or create an empty file with the output filename before updating e.g. using CMD:

    type nul > new-file.exe restic self-update --output new-file.exe

    #4163 https://forum.restic.net/t/self-update-windows-started-failing-after-release-of-0-15/5836

  • Bugfix #4167: Add missing ETA in backup progress bar

    A regression in restic 0.15.0 caused the ETA to be missing from the progress bar displayed by the backup command. This has now been fixed.

    #4167

  • Enhancement #4143: Ignore empty lock files

    With restic 0.15.0 the checks for stale locks became much stricter than before. In particular, empty or unreadable locks were no longer silently ignored. This made restic to complain with Load(<lock/1234567812>, 0, 0) returned error, retrying after 552.330144ms: load(<lock/1234567812>): invalid data returned and fail in the end.

    The error message is now clarified and the implementation changed to ignore empty lock files which are sometimes created as the result of a failed uploads on some backends.

    Please note that unreadable lock files still have to cleaned up manually. To do so, you can run restic unlock --remove-all which removes all existing lock files. But first make sure that no other restic process is currently using the repository.

    #4143 #4152

v0.15.0

1 year ago

We're very pleased to present you restic 0.15.0!

In this version a new rewrite command has been implemented. This allows for removing unwanted data from existing snapshots, for example if one discovers that some files were unintentionally backed up, such as files with sensitive content or files that take up a lot of space and don't need to be backed up. Please see the corresponding documentation for more information and usage examples.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.15.0 (2023-01-12)

The following sections list the changes in restic 0.15.0 relevant to restic users. The changes are ordered by importance.

Summary

  • Fix #2015: Make mount return exit code 0 after receiving Ctrl-C / SIGINT
  • Fix #2578: Make restore replace existing symlinks
  • Fix #2591: Don't read password from stdin for backup --stdin
  • Fix #3161: Delete files on Backblaze B2 more reliably
  • Fix #3336: Make SFTP backend report no space left on device
  • Fix #3567: Improve handling of interrupted syscalls in mount command
  • Fix #3897: Fix stuck copy command when -o <backend>.connections=1
  • Fix #3918: Correct prune statistics for partially compressed repositories
  • Fix #3951: Make ls return exit code 1 if snapshot cannot be loaded
  • Fix #4003: Make backup no longer hang on Solaris when seeing a FIFO file
  • Fix #4016: Support ExFAT-formatted local backends on macOS Ventura
  • Fix #4085: Make init ignore "Access Denied" errors when creating S3 buckets
  • Fix #4100: Make self-update enabled by default only in release builds
  • Fix #4103: Don't generate negative UIDs and GIDs in tar files from dump
  • Chg #2724: Include full snapshot ID in JSON output of backup
  • Chg #3929: Make unlock display message only when locks were actually removed
  • Chg #4033: Don't print skipped snapshots by default in copy command
  • Chg #4041: Update dependencies and require Go 1.18 or newer
  • Enh #14: Implement rewrite command
  • Enh #79: Restore files with long runs of zeros as sparse files
  • Enh #1078: Support restoring symbolic links on Windows
  • Enh #1734: Inform about successful retries after errors
  • Enh #1866: Improve handling of directories with duplicate entries
  • Enh #2134: Support B2 API keys restricted to hiding but not deleting files
  • Enh #2152: Make init open only one connection for the SFTP backend
  • Enh #2533: Handle cache corruption on disk and in downloads
  • Enh #2715: Stricter repository lock handling
  • Enh #2750: Make backup file read concurrency configurable
  • Enh #3029: Add support for credential_process to S3 backend
  • Enh #3096: Make mount command support macOS using macFUSE 4.x
  • Enh #3124: Support JSON output for the init command
  • Enh #3899: Optimize prune memory usage
  • Enh #3905: Improve speed of parent snapshot detection in backup command
  • Enh #3915: Add compression statistics to the stats command
  • Enh #3925: Provide command completion for PowerShell
  • Enh #3931: Allow backup file tree scanner to be disabled
  • Enh #3932: Improve handling of ErrDot errors in rclone and sftp backends
  • Enh #3943: Ignore additional/unknown files in repository
  • Enh #3955: Improve backup performance for small files

Details

  • Bugfix #2015: Make mount return exit code 0 after receiving Ctrl-C / SIGINT

    To stop the mount command, a user has to press Ctrl-C or send a SIGINT signal to restic. This used to cause restic to exit with a non-zero exit code.

    The exit code has now been changed to zero as the above is the expected way to stop the mount command and should therefore be considered successful.

    #2015 #3894

  • Bugfix #2578: Make restore replace existing symlinks

    When restoring a symlink, restic used to report an error if the target path already existed. This has now been fixed such that the potentially existing target path is first removed before the symlink is restored.

    #2578 #3780

  • Bugfix #2591: Don't read password from stdin for backup --stdin

    The backup command when used with --stdin previously tried to read first the password, then the data to be backed up from standard input. This meant it would often confuse part of the data for the password.

    From now on, it will instead exit with the message Fatal: cannot read both password and data from stdin unless the password is passed in some other way (such as --restic-password-file, RESTIC_PASSWORD, etc).

    To enter the password interactively a password command has to be used. For example on Linux, mysqldump somedatabase | restic backup --stdin --password-command='sh -c "systemd-ask-password < /dev/tty"' securely reads the password from the terminal.

    #2591 #4011

  • Bugfix #3161: Delete files on Backblaze B2 more reliably

    Restic used to only delete the latest version of files stored in B2. In most cases this worked well as there was only a single version of the file. However, due to retries while uploading it is possible for multiple file versions to be stored at B2. This could lead to various problems for files that should have been deleted but still existed.

    The implementation has now been changed to delete all versions of files, which doubles the amount of Class B transactions necessary to delete files, but assures that no file versions are left behind.

    #3161 #3885

  • Bugfix #3336: Make SFTP backend report no space left on device

    Backing up to an SFTP backend would spew repeated SSH_FX_FAILURE messages when the remote disk was full. Restic now reports "sftp: no space left on device" and exits immediately when it detects this condition.

    A fix for this issue was implemented in restic 0.12.1, but unfortunately the fix itself contained a bug that prevented it from taking effect.

    #3336 #3345 #4075

  • Bugfix #3567: Improve handling of interrupted syscalls in mount command

    Accessing restic's FUSE mount could result in "input/output" errors when using programs in which syscalls can be interrupted. This is for example the case for Go programs. This has now been fixed by improved error handling of interrupted syscalls.

    #3567 #3694 #3875

  • Bugfix #3897: Fix stuck copy command when -o <backend>.connections=1

    When running the copy command with -o <backend>.connections=1 the command would be infinitely stuck. This has now been fixed.

    #3897 #3898

  • Bugfix #3918: Correct prune statistics for partially compressed repositories

    In a partially compressed repository, one data blob can exist both in an uncompressed and a compressed version. This caused the prune statistics to become inaccurate and e.g. report a too high value for the unused size, such as "unused size after prune: 16777215.991 TiB". This has now been fixed.

    #3918 #3980

  • Bugfix #3951: Make ls return exit code 1 if snapshot cannot be loaded

    The ls command used to show a warning and return exit code 0 when failing to load a snapshot. This has now been fixed such that it instead returns exit code 1 (still showing a warning).

    #3951

  • Bugfix #4003: Make backup no longer hang on Solaris when seeing a FIFO file

    The backup command used to hang on Solaris whenever it encountered a FIFO file (named pipe), due to a bug in the handling of extended attributes. This bug has now been fixed.

    #4003 #4053

  • Bugfix #4016: Support ExFAT-formatted local backends on macOS Ventura

    ExFAT-formatted disks could not be used as local backends starting from macOS Ventura. Restic commands would fail with an "inappropriate ioctl for device" error. This has now been fixed.

    #4016 #4021

  • Bugfix #4085: Make init ignore "Access Denied" errors when creating S3 buckets

    In restic 0.9.0 through 0.13.0, the init command ignored some permission errors from S3 backends when trying to check for bucket existence, so that manually created buckets with custom permissions could be used for backups.

    This feature became broken in 0.14.0, but has now been restored again.

    #4085 #4086

  • Bugfix #4100: Make self-update enabled by default only in release builds

    The self-update command was previously included by default in all builds of restic as opposed to only in official release builds, even if the selfupdate tag was not explicitly enabled when building.

    This has now been corrected, and the self-update command is only available if restic was built with -tags selfupdate (as done for official release builds by build.go).

    #4100

  • Bugfix #4103: Don't generate negative UIDs and GIDs in tar files from dump

    When using a 32-bit build of restic, the dump command could in some cases create tar files containing negative UIDs and GIDs, which cannot be read by GNU tar. This corner case especially applies to backups from stdin on Windows.

    This is now fixed such that dump creates valid tar files in these cases too.

    #4103 #4104

  • Change #2724: Include full snapshot ID in JSON output of backup

    We have changed the JSON output of the backup command to include the full snapshot ID instead of just a shortened version, as the latter can be ambiguous in some rare cases. To derive the short ID, please truncate the full ID down to eight characters.

    #2724 #3993

  • Change #3929: Make unlock display message only when locks were actually removed

    The unlock command used to print the "successfully removed locks" message whenever it was run, regardless of lock files having being removed or not.

    This has now been changed such that it only prints the message if any lock files were actually removed. In addition, it also reports the number of removed lock files.

    #3929 #3935

  • Change #4033: Don't print skipped snapshots by default in copy command

    The copy command used to print each snapshot that was skipped because it already existed in the target repository. The amount of this output could practically bury the list of snapshots that were actually copied.

    From now on, the skipped snapshots are by default not printed at all, but this can be re-enabled by increasing the verbosity level of the command.

    #4033 #4066

  • Change #4041: Update dependencies and require Go 1.18 or newer

    Most dependencies have been updated. Since some libraries require newer language features, support for Go 1.15-1.17 has been dropped, which means that restic now requires at least Go 1.18 to build.

    #4041

  • Enhancement #14: Implement rewrite command

    Restic now has a rewrite command which allows to rewrite existing snapshots to remove unwanted files.

    #14 #2731 #4079

  • Enhancement #79: Restore files with long runs of zeros as sparse files

    When using restore --sparse, the restorer may now write files containing long runs of zeros as sparse files (also called files with holes), where the zeros are not actually written to disk.

    How much space is saved by writing sparse files depends on the operating system, file system and the distribution of zeros in the file.

    During backup restic still reads the whole file including sparse regions, but with optimized processing speed of sparse regions.

    #79 #3903 #2601 #3854 https://forum.restic.net/t/sparse-file-support/1264

  • Enhancement #1078: Support restoring symbolic links on Windows

    The restore command now supports restoring symbolic links on Windows. Because of Windows specific restrictions this is only possible when running restic with the SeCreateSymbolicLinkPrivilege privilege or as an administrator.

    #1078 #2699 #2875

  • Enhancement #1734: Inform about successful retries after errors

    When a recoverable error is encountered, restic shows a warning message saying that it's retrying, e.g.:

    Save(<data/956b9ced99>) returned error, retrying after 357.131936ms: ...

    This message can be confusing in that it never clearly states whether the retry is successful or not. This has now been fixed such that restic follows up with a message confirming a successful retry, e.g.:

    Save(<data/956b9ced99>) operation successful after 1 retries

    #1734 #2661

  • Enhancement #1866: Improve handling of directories with duplicate entries

    If for some reason a directory contains a duplicate entry, the backup command would previously fail with a node "path/to/file" already present or nodes are not ordered got "path/to/file", last "path/to/file" error.

    The error handling has been improved to only report a warning in this case. Make sure to check that the filesystem in question is not damaged if you see this!

    #1866 #3937 #3880

  • Enhancement #2134: Support B2 API keys restricted to hiding but not deleting files

    When the B2 backend does not have the necessary permissions to permanently delete files, it now automatically falls back to hiding files. This allows using restic with an application key which is not allowed to delete files. This can prevent an attacker from deleting backups with such an API key.

    To use this feature create an application key without the deleteFiles capability. It is recommended to restrict the key to just one bucket. For example using the b2 command line tool:

    b2 create-key --bucket <bucketName> <keyName> listBuckets,readFiles,writeFiles,listFiles

    Alternatively, you can use the S3 backend to access B2, as described in the documentation. In this mode, files are also only hidden instead of being deleted permanently.

    #2134 #2398

  • Enhancement #2152: Make init open only one connection for the SFTP backend

    The init command using the SFTP backend used to connect twice to the repository. This could be inconvenient if the user must enter a password, or cause init to fail if the server does not correctly close the first SFTP connection.

    This has now been fixed by reusing the first/initial SFTP connection opened.

    #2152 #3882

  • Enhancement #2533: Handle cache corruption on disk and in downloads

    In rare situations, like for example after a system crash, the data stored in the cache might be corrupted. This could cause restic to fail and required manually deleting the cache.

    Restic now automatically removes broken data from the cache, allowing it to recover from such a situation without user intervention. In addition, restic retries downloads which return corrupt data in order to also handle temporary download problems.

    #2533 #3521

  • Enhancement #2715: Stricter repository lock handling

    Previously, restic commands kept running even if they failed to refresh their locks in time. This could be a problem e.g. in case the client system running a backup entered the standby power mode while the backup was still in progress (which would prevent the client from refreshing its lock), and after a short delay another host successfully runs unlock and prune on the repository, which would remove all data added by the in-progress backup. If the backup client later continues its backup, even though its lock had expired in the meantime, this would lead to an incomplete snapshot.

    To address this, lock handling is now much stricter. Commands requiring a lock are canceled if the lock is not refreshed successfully in time. In addition, if a lock file is not readable restic will not allow starting a command. It may be necessary to remove invalid lock files manually or use unlock --remove-all. Please make sure that no other restic processes are running concurrently before doing this, however.

    #2715 #3569

  • Enhancement #2750: Make backup file read concurrency configurable

    The backup command now supports a --read-concurrency option which allows tuning restic for very fast storage like NVMe disks by controlling the number of concurrent file reads during the backup process.

    #2750

  • Enhancement #3029: Add support for credential_process to S3 backend

    Restic now uses a newer library for the S3 backend, which adds support for the credential_process option in the AWS credential configuration.

    #3029 #4034 #4025

  • Enhancement #3096: Make mount command support macOS using macFUSE 4.x

    Restic now uses a different FUSE library for mounting snapshots and making them available as a FUSE filesystem using the mount command. This adds support for macFUSE 4.x which can be used to make this work on recent macOS versions.

    #3096 #4024

  • Enhancement #3124: Support JSON output for the init command

    The init command used to ignore the --json option, but now outputs a JSON message if the repository was created successfully.

    #3124 #3132

  • Enhancement #3899: Optimize prune memory usage

    The prune command needs large amounts of memory in order to determine what to keep and what to remove. This is now optimized to use up to 30% less memory.

    #3899

  • Enhancement #3905: Improve speed of parent snapshot detection in backup command

    Backing up a large number of files using --files-from-verbatim or --files-from-raw options could require a long time to find the parent snapshot. This has been improved.

    #3905

  • Enhancement #3915: Add compression statistics to the stats command

    When executed with --mode raw-data on a repository that supports compression, the stats command now calculates and displays, for the selected repository or snapshots: the uncompressed size of the data; the compression progress (percentage of data that has been compressed); the compression ratio of the compressed data; the total space saving.

    It also takes into account both the compressed and uncompressed data if the repository is only partially compressed.

    #3915

  • Enhancement #3925: Provide command completion for PowerShell

    Restic already provided generation of completion files for bash, fish and zsh. Now powershell is supported, too.

    #3925

  • Enhancement #3931: Allow backup file tree scanner to be disabled

    The backup command walks the file tree in a separate scanner process to find the total size and file/directory count, and uses this to provide an ETA. This can slow down backups, especially of network filesystems.

    The command now has a new option --no-scan which can be used to disable this scanning in order to speed up backups when needed.

    #3931

  • Enhancement #3932: Improve handling of ErrDot errors in rclone and sftp backends

    Since Go 1.19, restic can no longer implicitly run relative executables which are found in the current directory (e.g. rclone if found in .). This is a security feature of Go to prevent against running unintended and possibly harmful executables.

    The error message for this was just "cannot run executable found relative to current directory". This has now been improved to yield a more specific error message, informing the user how to explicitly allow running the executable using the -o rclone.program and -o sftp.command extended options with ./.

    #3932 https://pkg.go.dev/os/exec#hdr-Executables_in_the_current_directoryhttps://go.dev/blog/path-security

  • Enhancement #3943: Ignore additional/unknown files in repository

    If a restic repository had additional files in it (not created by restic), commands like find and restore could become confused and fail with an multiple IDs with prefix "12345678" found error. These commands now ignore such additional files.

    #3943 https://forum.restic.net/t/which-protocol-should-i-choose-for-remote-linux-backups/5446/17

  • Enhancement #3955: Improve backup performance for small files

    When backing up small files restic was slower than it could be. In particular this affected backups using maximum compression.

    This has been fixed by reworking the internal parallelism of the backup command, making it back up small files around two times faster.

    #3955

v0.14.0

1 year ago

We're very pleased to present you restic 0.14.0! This is a great release which introduces a new repository version that adds the most requested feature: compression!

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.14.0 (2022-08-25)

The following sections list the changes in restic 0.14.0 relevant to restic users. The changes are ordered by importance.

Summary

  • Fix #2248: Support self-update on Windows
  • Fix #3428: List snapshots in backend at most once to resolve snapshot IDs
  • Fix #3432: Fix rare 'not found in repository' error for copy command
  • Fix #3685: The diff command incorrectly listed some files as added
  • Fix #3681: Fix rclone (shimmed by Scoop) and sftp not working on Windows
  • Fix #3720: Directory sync errors for repositories accessed via SMB
  • Fix #3736: The stats command miscalculated restore size for multiple snapshots
  • Fix #3861: Yield error on invalid policy to forget
  • Fix #3716: Print "wrong password" to stderr instead of stdout
  • Fix #3772: Correctly rebuild index for legacy repositories
  • Fix #3776: Limit number of key files tested while opening a repository
  • Chg #1842: Support debug log creation in release builds
  • Chg #3295: Deprecate check --check-unused and add further checks
  • Chg #3680: Update dependencies and require Go 1.15 or newer
  • Chg #3742: Replace --repo2 option used by init/copy with --from-repo
  • Enh #1153: Support pruning even when the disk is full
  • Enh #21: Add compression support
  • Enh #2162: Adaptive IO concurrency based on backend connections
  • Enh #2291: Allow pack size customization
  • Enh #2295: Allow use of SAS token to authenticate to Azure
  • Enh #2696: Improve backup speed with many small files
  • Enh #2907: Make snapshot directory structure of mount command customizable
  • Enh #3114: Optimize handling of duplicate blobs in prune
  • Enh #3465: Improve handling of temporary files on Windows
  • Enh #3709: Validate exclude patterns before backing up
  • Enh #3837: Improve SFTP repository initialization over slow links
  • Enh #2351: Use config file permissions to control file group access
  • Enh #3475: Allow limiting IO concurrency for local and SFTP backend
  • Enh #3484: Stream data in check and prune commands
  • Enh #2923: Improve speed of copy command
  • Enh #3729: Display full IDs in check warnings
  • Enh #3773: Optimize memory usage for directories with many files
  • Enh #3819: Validate include/exclude patterns before restoring

Details

  • Bugfix #2248: Support self-update on Windows

    Restic self-update would fail in situations where the operating system locks running binaries, including Windows. The new behavior works around this by renaming the running file and swapping the updated file in place.

    #2248 #3675

  • Bugfix #3428: List snapshots in backend at most once to resolve snapshot IDs

    Many commands support specifying a list of snapshot IDs which are then used to determine the snapshots to be processed by the command. To resolve snapshot IDs or latest, and check that these exist, restic previously listed all snapshots stored in the repository. Depending on the backend this could be a slow and/or expensive operation.

    Restic now lists the snapshots only once and remembers the result in order to resolve all further snapshot IDs swiftly.

    #3428 #3570 #3395

  • Bugfix #3432: Fix rare 'not found in repository' error for copy command

    In rare cases copy (and other commands) would report that LoadTree(...) returned an id [...] not found in repository error. This could be caused by a backup or copy command running concurrently. The error was only temporary; running the failed restic command a second time as a workaround did resolve the error.

    This issue has now been fixed by correcting the order in which restic reads data from the repository. It is now guaranteed that restic only loads snapshots for which all necessary data is already available.

    #3432 #3570

  • Bugfix #3685: The diff command incorrectly listed some files as added

    There was a bug in the diff command, causing it to always show files in a removed directory as added. This has now been fixed.

    #3685 #3686

  • Bugfix #3681: Fix rclone (shimmed by Scoop) and sftp not working on Windows

    In #3602 a fix was introduced to address the problem of rclone prematurely exiting when Ctrl+C is pressed on Windows. The solution was to create the subprocess with its console detached from the restic console.

    However, this solution failed when using rclone installed by Scoop or using sftp with a passphrase-protected private key. We've now fixed this by using a different approach to prevent Ctrl-C from passing down too early.

    #3681 #3692 #3696

  • Bugfix #3720: Directory sync errors for repositories accessed via SMB

    On Linux and macOS, accessing a repository via a SMB/CIFS mount resulted in restic failing to save the lock file, yielding the following errors:

    Save(<lock/071fe833f0>) returned error, retrying after 552.330144ms: sync /repo/locks: no such file or directory Save(<lock/bf789d7343>) returned error, retrying after 552.330144ms: sync /repo/locks: invalid argument

    This has now been fixed by ignoring the relevant error codes.

    #3720 #3751 #3752

  • Bugfix #3736: The stats command miscalculated restore size for multiple snapshots

    Since restic 0.10.0 the restore size calculated by the stats command for multiple snapshots was too low. The hardlink detection was accidentally applied across multiple snapshots and thus ignored many files. This has now been fixed.

    #3736 #3740

  • Bugfix #3861: Yield error on invalid policy to forget

    The forget command previously silently ignored invalid/unsupported units in the duration options, such as e.g. --keep-within-daily 2w.

    Specifying an invalid/unsupported duration unit now results in an error.

    #3861 #3862

  • Bugfix #3716: Print "wrong password" to stderr instead of stdout

    If an invalid password was entered, the error message was printed on stdout and not on stderr as intended. This has now been fixed.

    #3716 https://forum.restic.net/t/4965

  • Bugfix #3772: Correctly rebuild index for legacy repositories

    After running rebuild-index on a legacy repository containing mixed pack files (that is, pack files which store both metadata and file data), check printed warnings like pack 12345678 contained in several indexes: .... This warning was not critical, but has now nonetheless been fixed by properly handling mixed pack files while rebuilding the index.

    Running prune for such legacy repositories will also fix the warning by reorganizing the pack files which caused it.

    #3772 #3884 https://forum.restic.net/t/5044/13

  • Bugfix #3776: Limit number of key files tested while opening a repository

    Previously, restic tested the password against every key in the repository when opening a repository. The more keys there were in the repository, the slower this operation became.

    Restic now tests the password against up to 20 key files in the repository. Alternatively, you can use the --key-hint=<key ID> option to specify a specific key file to use instead.

    #3776

  • Change #1842: Support debug log creation in release builds

    Creating a debug log was only possible in debug builds which required users to manually build restic. We changed the release builds to allow creating debug logs by simply setting the environment variable DEBUG_LOG=logname.log.

    #1842 #3826

  • Change #3295: Deprecate check --check-unused and add further checks

    Since restic 0.12.0, it is expected to still have unused blobs after running prune. This made the --check-unused option of the check command rather useless and tended to confuse users. This option has been deprecated and is now ignored.

    The check command now also warns if a repository is using either the legacy S3 layout or mixed pack files with both tree and data blobs. The latter is known to cause performance problems.

    #3295 #3730

  • Change #3680: Update dependencies and require Go 1.15 or newer

    We've updated most dependencies. Since some libraries require newer language features we're dropping support for Go 1.14, which means that restic now requires at least Go 1.15 to build.

    #3680 #3883

  • Change #3742: Replace --repo2 option used by init/copy with --from-repo

    The init and copy commands can read data from another repository. However, confusingly --repo2 referred to the repository from which the init command copies parameters, but for the copy command --repo2 referred to the copy destination.

    We've introduced a new option, --from-repo, which always refers to the source repository for both commands. The old parameter names have been deprecated but still work. To create a new repository and copy all snapshots to it, the commands are now as follows:

    restic -r /srv/restic-repo-copy init --from-repo /srv/restic-repo --copy-chunker-params restic -r /srv/restic-repo-copy copy --from-repo /srv/restic-repo

    #3742 https://forum.restic.net/t/5017

  • Enhancement #1153: Support pruning even when the disk is full

    When running out of disk space it was no longer possible to add or remove data from a repository. To help with recovering from such a deadlock, the prune command now supports an --unsafe-recover-no-free-space option to recover from these situations. Make sure to read the documentation first!

    #1153 #3481

  • Enhancement #21: Add compression support

    We've added compression support to the restic repository format. To create a repository using the new format run init --repository-version 2. Please note that the repository cannot be read by restic versions prior to 0.14.0.

    You can configure whether data is compressed with the option --compression. It can be set to auto (the default, which will compress very fast), max (which will trade backup speed and CPU usage for better compression), or off (which disables compression). Each setting is only applied for the current run of restic and does not apply to future runs. The option can also be set via the environment variable RESTIC_COMPRESSION.

    To upgrade in place run migrate upgrade_repo_v2 followed by prune. See the documentation for more details. The migration checks the repository integrity and upgrades the repository format, but will not change any data. Afterwards, prune will rewrite the metadata to make use of compression.

    As an alternative you can use the copy command to migrate snapshots; First create a new repository using init --repository-version 2 --copy-chunker-params --repo2 path/to/old/repo, and then use the copy command to copy all snapshots to the new repository.

    #21 #3779 #3666 #3704 #3733

  • Enhancement #2162: Adaptive IO concurrency based on backend connections

    Many commands used hard-coded limits for the number of concurrent operations. This prevented speed improvements by increasing the number of connections used by a backend.

    These limits have now been replaced by using the configured number of backend connections instead, which can be controlled using the -o <backend-name>.connections=5 option. Commands will then automatically scale their parallelism accordingly.

    To limit the number of CPU cores used by restic, you can set the environment variable GOMAXPROCS accordingly. For example to use a single CPU core, use GOMAXPROCS=1.

    #2162 #1467 #3611

  • Enhancement #2291: Allow pack size customization

    Restic now uses a target pack size of 16 MiB by default. This can be customized using the --pack-size size option. Supported pack sizes range between 4 and 128 MiB.

    It is possible to migrate an existing repository to larger pack files using prune --repack-small. This will rewrite every pack file which is significantly smaller than the target size.

    #2291 #3731

  • Enhancement #2295: Allow use of SAS token to authenticate to Azure

    Previously restic only supported AccountKeys to authenticate to Azure storage accounts, which necessitates giving a significant amount of access.

    We added support for Azure SAS tokens which are a more fine-grained and time-limited manner of granting access. Set the AZURE_ACCOUNT_NAME and AZURE_ACCOUNT_SAS environment variables to use a SAS token for authentication. Note that if AZURE_ACCOUNT_KEY is set, it will take precedence.

    #2295 #3661

  • Enhancement #2696: Improve backup speed with many small files

    We have restructured the backup pipeline to continue reading files while all upload connections are busy. This allows the backup to already prepare the next data file such that the upload can continue as soon as a connection becomes available. This can especially improve the backup performance for high latency backends.

    The upload concurrency is now controlled using the -o <backend-name>.connections=5 option.

    #2696 #3489

  • Enhancement #2907: Make snapshot directory structure of mount command customizable

    We've added the possibility to customize the snapshot directory structure of the mount command using templates passed to the --snapshot-template option. The formatting of snapshots' timestamps is now controlled using --time-template and supports subdirectories to for example group snapshots by year. Please see restic help mount for further details.

    Characters in tag names which are not allowed in a filename are replaced by underscores _. For example a tag foo/bar will result in a directory name of foo_bar.

    #2907 #2913 #3691

  • Enhancement #3114: Optimize handling of duplicate blobs in prune

    Restic prune always used to repack all data files containing duplicate blobs. This effectively removed all duplicates during prune. However, as a consequence all these data files were repacked even if the unused repository space threshold could be reached with less work.

    This is now changed and prune works nice and fast even when there are lots of duplicate blobs.

    #3114 #3290

  • Enhancement #3465: Improve handling of temporary files on Windows

    In some cases restic failed to delete temporary files, causing the current command to fail. This has now been fixed by ensuring that Windows automatically deletes the file. In addition, temporary files are only written to disk when necessary, reducing disk writes.

    #3465 #1551 #3610

  • Enhancement #3709: Validate exclude patterns before backing up

    Exclude patterns provided via --exclude, --iexclude, --exclude-file or --iexclude-file previously weren't validated. As a consequence, invalid patterns resulted in files that were meant to be excluded being backed up.

    Restic now validates all patterns before running the backup and aborts with a fatal error if an invalid pattern is detected.

    #3709 #3734

  • Enhancement #3837: Improve SFTP repository initialization over slow links

    The init command, when used on an SFTP backend, now sends multiple mkdir commands to the backend concurrently. This reduces the waiting times when creating a repository over a very slow connection.

    #3837 #3840

  • Enhancement #2351: Use config file permissions to control file group access

    Previously files in a local/SFTP repository would always end up with very restrictive access permissions, allowing access only to the owner. This prevented a number of valid use-cases involving groups and ACLs.

    We now use the permissions of the config file in the repository to decide whether group access should be given to newly created repository files or not. We arrange for repository files to be created group readable exactly when the repository config file is group readable.

    To opt-in to group readable repositories, a simple chmod -R g+r or equivalent on the config file can be used. For repositories that should be writable by group members a tad more setup is required, see the docs.

    Posix ACLs can also be used now that the group permissions being forced to zero no longer masks the effect of ACL entries.

    #2351 #3419 https://forum.restic.net/t/1391

  • Enhancement #3475: Allow limiting IO concurrency for local and SFTP backend

    Restic did not support limiting the IO concurrency / number of connections for accessing repositories stored using the local or SFTP backends. The number of connections is now limited as for other backends, and can be configured via the -o local.connections=2 and -o sftp.connections=5 options. This ensures that restic does not overwhelm the backend with concurrent IO operations.

    #3475

  • Enhancement #3484: Stream data in check and prune commands

    The commands check --read-data and prune previously downloaded data files into temporary files which could end up being written to disk. This could cause a large amount of data being written to disk.

    The pack files are now instead streamed, which removes the need for temporary files. Please note that uploads during backup and prune still require temporary files.

    #3710 #3484 #3717

  • Enhancement #2923: Improve speed of copy command

    The copy command could require a long time to copy snapshots for non-local backends. This has been improved to provide a throughput comparable to the restore command.

    Additionally, copy now displays a progress bar.

    #2923 #3513

  • Enhancement #3729: Display full IDs in check warnings

    When running commands to inspect or repair a damaged repository, it is often necessary to supply the full IDs of objects stored in the repository.

    The output of check now includes full IDs instead of their shortened variant.

    #3729

  • Enhancement #3773: Optimize memory usage for directories with many files

    Backing up a directory with hundreds of thousands or more files caused restic to require large amounts of memory. We've now optimized the backup command such that it requires up to 30% less memory.

    #3773

  • Enhancement #3819: Validate include/exclude patterns before restoring

    Patterns provided to restore via --exclude, --iexclude, --include and --iinclude weren't validated before running the restore. Invalid patterns would result in error messages being printed repeatedly, and possibly unwanted files being restored.

    Restic now validates all patterns before running the restore, and aborts with a fatal error if an invalid pattern is detected.

    #3819

v0.13.1

1 year ago

We're very pleased to present you restic 0.13.1! This is a small bugfix release which fixes two issues that were discovered right after 0.13.0.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.13.1 (2022-04-10)

The following sections list the changes in restic 0.13.1 relevant to restic users. The changes are ordered by importance.

Summary

  • Fix #3685: Fix the diff command
  • Fix #3681: Fix rclone (shimmed by Scoop) and sftp stopped working on Windows

Details

  • Bugfix #3685: Fix the diff command

    There was a bug in the diff command, it would always show files in a removed directory as added. We've fixed that.

    #3685 #3686

  • Bugfix #3681: Fix rclone (shimmed by Scoop) and sftp stopped working on Windows

    In #3602 a fix was introduced to fix the problem that rclone prematurely exits when Ctrl+C is pressed on Windows. The solution was to create the subprocess with its console detached from the restic console. However, such solution fails when using rclone install by scoop or using sftp with a passphrase- protected private key. We've fixed that by using a different approach to prevent Ctrl-C from passing down too early.

    #3681 #3692 #3696