Virt Backup Versions Save

Fixes #51 where the clean command could fail if the json definition had an unexpected field.

Added a -c/--config option to specify a custom config path.

Cleanups and fixes in the CLI helpers.

Bug fix

• Fix setting storage type when pivoting a disk, add a compatibility layer for old pending_info.

Bug fix

• Fix resetting storage type to a qcow2 after the backup of a shutdown domain. PR #44
• Match domains by UUID, fixing issues with shutdown domains. PR #43

Thanks a lot @arkadiam for your contributions!

Bug fix

• Listing backups for specific groups now works. virt-backup list testgroup will list backups only for testgroup.
• Compares domains by using the libvirt ID instead of just object comparison.

Cleaning

• Remove the use of defusedlxml as the library is going to be deprecated. Use lxml directly instead, but remove unsafe options from the parser.
• Fix warnings with arrow.

Features

• Quiesce can now be used for external snapshots. If enabled, libvirt will try to use the guest agent to freeze and unfreeze domain's mounted file systems. For more information, look at the quiesce option in libvirt: https://libvirt.org/manpages/virsh.html#snapshot-create

  It can be enabled for an entire group or per domain. Look at the quiesce option in the example config. If the domain to backup doesn't support this option, an error message will be shown, but the backup fallbacks to a non-quiesce snapshot.

Features

• ZSTD packager can now take a threads parameter, allowing to use multithreading for the compression of a disk. Default to 0, meaning single threads. -1 can be set to use all the cores detected on the system, otherwise set an integer for the number of wanted threads.

Bugfixes

• Better handling of canceled backups. Backups can now be aborted (by a ctrl-c for example), and the cleanup will be correctly handled. Linked to #32
• ZSTD params were defined but not used.
• Fixes a bug with disks filtering in the config file. See #28 for details.

• Fixes zstd packager cleanup #29

This is a pretty big release in term of refactoring/cleanup, and will allow to add more features in the future, and make some testing way easier!

Required changes in the configuration

As packagers were introduced (see the changelog for details), modifications should be done in the configuration file. Any current configuration file should continue to work, but a warning will be print to advice the changes to be applied.

Change any usage of compress and compress_lvl to packager and packager_opts: https://github.com/aruhier/virt-backup/blob/9fac0813433cacb560a52224c695ef85e8548494/example/config.yml#L51

Changelog

Features:

• zstd backup compression. It requires the python package zstandard to be installed in order to use it.
• Add a compatibility layers for configuration, backups definition and pending info (json used to store temp data when a backup is in progress). If a new version of virt-backups brings any modification in the structure of these files, it will be handled by these layers to be able to read an old version. For the configuration, the conversion never writes the result on disk, it will only print a warning (telling also what to change). The user has to apply these changes manually to not see these warnings again.

Bug fixing:

• Fix uncleaned backups if a period is not set in the configuration (#27): before, the default value to every cleaning period in the configuration was 0, meaning that the backups would be kept indefinitely. This behavior was wanted, but is actually not intuitive. Now default to 5 kept backups for every period, if not set.

Refactoring/Cleanup:

• Add packagers: packagers all implement the same abstract class to allow storing a backup, listing files or extract a backup. Each one handle a different format: directory (just copy files somewhere), zstd or tar. It allows for the backups algorithm to not handle how the files are stored, and make it more testable.
• Compatibility layers: compatibility layers allows virt-backup to target the last possible versions of each file (configuration, definition, pending_info), and handle the conversion of an old file in a succession of layers. It removes some condition checking/branches that could easily be forgotten.