Zfsbootmenu Versions Save

This release is dedicated to the late Jürgen Buchmüller (@pullmoll), a major contributor to the Void Linux project. Although ZFSBootMenu strives to support as many Linux distributions as possible, Void Linux is the distribution of choice for the entire ZFSBootMenu team. We have benefited greatly from pullmoll's enduring commitment to Void Linux. He will be missed.

Fixes

• Snapshot duplication now carries over ZFS properties from the source (cloning has always copied properties)
• When forcing a pool read-write via MOD+W, the screen is now cleared before a password prompt
• Prompts in Arch/Ubuntu/Debian chroots are now correctly set
• Build-time checks for version-specific flags in fzf/sk/dmesg improve compatibility with older versions found on Debian and Ubuntu
• Add stty to the list of required binaries
• When generate-zbm is executed with the --debug flag, -q is no longer passed in to Dracut
• When possible, try to let Dracut determine the path to the EFI stub file. The path can be explicitly specified by setting EFI.Stub in config.yaml
• Update the list of allowed characters in a ZFS filesystem name; removing , and adding :
• Use the Dracut built-in inst_rules to install udev rules, instead of hard-coding a file path
• Correctly set a root= prefix for Gentoo systems
• Improve kernel version detection in unversioned file names
• Update documentation to clarify driver exclusions and teardown hooks

Major New features

hostid configuration assistance

A pair of new features have been developed to combat the delicate dance sometimes required to synchronize hostid in the boot environment (BE), the initramfs for the BE, and the initramfs for ZFSBootMenu. Both of these are controlled by ZFSBootMenu kernel command line options.

Set zbm.import_policy=hostid to allow run-time reconfiguration of the SPL hostid. If a pool is preferred via zbm.prefer and the pool can not be imported with a preconfigured hostid, the system will attempt to adopt the hostid of the system that last imported the pool. If a preferred pool is not set and no pools can be imported using a preconfigured hostid, the system will adopt the hostid of the first otherwise-importable pool. After adopting a detected hostid, ZFSBootMenu will subsequently attempt to import as many pools as possible.

Setting zbm.set_hostid will cause ZFSBootMenu to set the spl.spl_hostid command-line parameter for the selected boot environment to the hostid used to import its pool. The SPL kernel module will use this value as the hostid of the booted environment regardless of the contents of /etc/hostid. As a special case, if the hostid to be set is zero, ZFSBootMenu will instead set spl_hostid=00000000, which should be used by dracut-based initramfs images to write an all-zero /etc/hostid in the initramfs prior to importing the boot pool.

Minor new features

Boot environment / snapshot sorting

The main boot environment screen and snapshot list screen can now be sorted by multiple criteria. By default, these listings are sorted by name. One of zbm.sort_key=name, zbm.sort_key=creation, or zbm.sort_key=used can be set on the command line to change the default used. MOD+O can be pressed at run time to change the sort order to the next in the list.

Helper functions in the recovery shell

The internal zfsbootmenu-lib.sh library is now sourced by default in the recovery shell. This makes a number of helper functions available for general use.

Combined command line printer

Since Dracut can find command line options from multiple places, it can be difficult to determine which option was used to control bootloader behavior. The command zbmcmdline will show the command line as seen by Dracut.

Shorcut key layout improvements

Instead of flowing the helper key text to the width of the screen, shortcut key text is now arranged in one or more columns. Low resolution screens or terminals with small dimensions will use a single column to show the text. As the terminal dimensions increase, text will be laid out in up to a maximum of three left-aligned columns.

Logging system

ZFSBootMenu has an internal logging system backed by /dev/kmsg and dmesg. Logging is now enabled throughout the entire system, tracking both debug level messages as well as warnings and errors. When a warning or error condition has occurred, a yellow [!] or red [!] notification will appear in the upper left of the screen. Pressing MOD+L will access this logging system, allowing you to easily see these messages.

To aid in debugging issues, debug logging has been added in all of the internal library functions. These messages can be enabled by setting loglevel=7 on the ZFSBootMenu command line.

Binary release for x86_64

Binary releases in the form of a standalone EFI file and a kernel/initramfs pair for x86_64 will now be made available with each future tagged release. To allow for the most compatiblity with the myriad system configurations out there, a few features will be embedded in the builds:

• zbm.import_policy=hostid
• zbm.set_hostid
• xhci-teardown.sh

The EFI binary can be used as a recovery tool by naming it BOOTX86.EFI and adding it to an EF00 partition on a USB drive. It can also be used as a drop-in bootloader for your system without needing to locally build a copy. See UEFI Booting for example efibootmgr commands.

Other changes

Some command line options have now been deprecated with the inclusion of zbm.import_policy. See zfsbootmenu(7).

Testing improvements

• Ability to install Arch / Ubuntu / Debian / Void / Void-musl on ZFS, in a VM, with a single command
  • This is huge.
• Verify that ZBM builds on Arch / Ubuntu / Debian / Void / Void-musl
• Verify that ZBM built on any distro boots any other distro
• Improvments to run.sh and setup.sh testing scripts to simplify usage

Significant commits in this release

• fadfd0f - Use --props and --raw when send | recv duplicating (Andrew J. Hesford)
• 0a4b1f7 - Use signify to sign binary assets (Andrew J. Hesford)
• 98cfc69 - Show if BE is encrypted, add path to chroot prompt (#157) (Zach Dykstra)
• 848bfc9 - Do not drop to emergency shell if zbm.preferred cannot be imported (Andrew J. Hesford)
• fd94635 - Fix PS1 in arch/debian/ubuntu, set custom prompt in emergency shell (#151) (Zach Dykstra)
• dee6228 - Support older versions of fzf/sk/dmesg (Zach Dykstra)
• 6e28f42 - Non-zero return from chroot should be debug info, not an error (Andrew J. Hesford)
• 82ab9a7 - Add stty to required exectables in module-setup.sh (Andrew J. Hesford)
• f962f50 - Use configurable destinations for all paths in Makefile (Andrew J. Hesford)
• 03a38b5 - Column view for help key footers (Andrew J. Hesford)
• 951111e - Add support to discover and assume a hostid, as well as fix arguments passed to a BE (#147) (Zach Dykstra)
• 29cf15c - Logging lifecycle (#146) (Zach Dykstra)
• 629346f - Automagically source zfsbootmenu-lib in emergency shell (Zach Dykstra)
• 73dc5c2 - Move more helper functions to -lib, cleanup scripts (#144) (Zach Dykstra)
• 44ed892 - Allow BE and snapshot sorting by different criteria (#143) (Zach Dykstra)
• 9afa8b4 - comma is invalid, colon is valid (Zach Dykstra)
• a99a8de - Update README.md to explain driver exclusions and teardown hooks (RoundDuckKira)
• 5ac481f - Use inst_rules to install rule programs (Witaut Bajaryn)
• 6cf753f - Use genkernel root prefix by default on Gentoo (#139) (Witaut Bajaryn)
• a6e02c0 - Only specify --uefi-stub when EFI.Stub is configured (Andrew J. Hesford)
• 893cbe9 - Improve kernel version detection. (Andrew J. Hesford)
• 8db6d8f - README: Describe more Perl dependencies (#134) (Witaut Bajaryn)

Happy New Year! ZFSBootMenu 1.8.1 provides a few minor enhancements and bug fixes.

Fixes

• Properly handle encryption keys as raw devices rather than normal files. (#127)
• Improve detection of latest kernels when /boot contains some unversioned kernel files. (#128)
• Accept Ctrl and Ctrl-Alt as hotkey modifiers in addition to Alt to fix issues with some non-US keymaps. (#124)
• Everywhere a chroot hotkey is offered, use the same hotkey.

New features

• Add hard wraps to the hotkey menus to improve clarity; ignored for small screens.
• In the snapshot list, add an option to jump into a read-only chroot for that snapshot.
• The force_import and timeout kernel command-line options are now expected to be zbm.force_import and zbm.timeout; the old forms are deprecated but will continue to work for the forseeable future.
• New zbm.show and zbm.skip command-line options force the menu to appear or be skipped if zbm.timeout is not also set.

Significant commits in this release

• ab95346 - Expect names of the form - when finding latest kernel (Andrew J. Hesford)
• 798fce8 - Normalize tests for paths. (Andrew J. Hesford)
• d02865e - Change chroot keys, add help text (#126) (Zach Dykstra)
• 83c58b2 - Support Ctrl and Ctrl-Alt in addition to Alt as keybind modifiers (Andrew J. Hesford)
• 70c3d70 - Namespace our KCL args, organize parsing (#120) (Zach Dykstra)
• c1d6ce5 - Support chroot'ing into a snapshot (Zach Dykstra)
• 7a63beb - Support for hard wrap points in header_wrap (Andrew J. Hesford)

ZFSBootMenu 1.8.0 offers a significant list of new features, fixes and general improvements.

Fixes

• When duplicating snapshots, the process can now be interrupted with SIGINT (Ctrl-C).
• Availability of sufficient free space is confirmed before attempting snapshot duplication.
• The generate-zbm command now ensures that the target directory for boot images has sufficient space, rather than copying partial and generally broken files.
• Because ZFSBootMenu never modifies filesystem contents, ZFS filesystems are always mounted read-only, even if the pool is writable.
• Changes to the handling of encryption keys correctly handle some corner cases, such as duplicating a snapshot of a filesystem with a different encryptionroot than its parent.

New features

• Colored text and timed messages are used to bring emphasis to important messages presented by ZFSBootMenu.
• Extensive logging to the kernel ring-buffer has been enabled throughout ZFSBootMenu, with verbosity controlled by the loglevel kernel-command-line argument.
• Much of the core menu functionality has been separated into a standalone zfsbootmenu program on the initramfs, allowing the menu to be accessed over SSH using something like the dracut-crypt-ssh module.
• Boot images can be configured to include tmux, allowing the boot menu to be presented in a multi-pane, detachable view. This is primarily aimed at development/debugging efforts. See descriptions of the zbm.tmux command-line option and the zfsbootmenu_tmux dracut option in the zfsbootmenu(7) manual page.
• ZFSBootMenu can now execute arbitrary, user-supplied "setup" hooks before the menu is displayed and "teardown" hooks immediately before jumping into a selected boot environment. See descriptions of the zfsbootmenu_setup and zfsbootmenu_teardown dracut options in the zfsbootmenu(7) manual page.
• Encryption keys can now be cached by pointing the org.zfsbootmenu:keysource property of a ZFS encryptionroot to a specific filesystem. When ZFSBootMenu attempts to load a key from a file:// location, it will first attempt to load the key at that location relative to the filesystem specified by org.zfsbootmenu:keysource. If this succeeds, ZFSBootMenu will retain a copy of the key in the initramfs so that subsequent need for the key (for example, when re-importing a pool read-write to set default boot options or duplicate a snapshot) will not require re-entry of the passphrase. See descriptions of the org.zfsbootmenu:keysource ZFS property in the zfsbootmenu(7) manual page.
• The Alt+C hot key provides the means to chroot into a selected boot environment. If the pool is mounted read-write, the chroot will be writable, allowing recovery operations directly from ZFSBootMenu.

Significant commits in this release

• 8ffe139 - Add a keybind for zfs-chroot, rework script with -lib in mind (Zach Dykstra)
• 2c3e9b5 - Support configurable, opt-in caching of key files (Andrew J. Hesford)
• 45d0066 - Add zlog() logging helper, along with debug-level logging (Zach Dykstra)
• 47aa434 - Add support for richer setup and teardown hooks (Andrew J. Hesford)
• 88818a9 - Support optional "teardown" script to run before kexec (Andrew J. Hesford)
• 2045315 - Optionally launch under tmux (Zach Dykstra)
• 7f71d4f - Mount ZFS filesystems readonly (Andrew J. Hesford)
• d559b96 - Improve key handling (Andrew J. Hesford)
• ee468aa - Initial split of core menu logic from initialization (Andrew J. Hesford)
• 25baa48 - Log dracut command, add man page documentation (Zach Dykstra)
• 3f02b8f - Prevent copying of partial files when target volume is full (Andrew J. Hesford)
• 258d18a - Merge fuctionality of createInitramfs and unifiedEFI (Andrew J. Hesford)
• 4b6e9ae - Add subroutine for debug logging (Zach Dykstra)
• 82bbdc4 - Rely on local IFS override instead of global changes (Zach Dykstra)
• 402474b - Make alt-w a toggle between R/O and R/W imports (Andrew J. Hesford)
• 953c724 - Richer color handling in timed_prompt (formerly warning_prompt) (Andrew J. Hesford)
• eb39ea6 - Show countdown in warning_prompt, use to display auto-boot countdown (Zach Dykstra)
• f10e7e8 - Rough avail space validation in duplicate_snapshot (Zach Dykstra)
• bc0e117 - Move duplicate to sub shell, exit on sigint (Zach Dykstra)

This is a minor bug-fix release.

Fixes

• When ZFSBootMenu fails to import any usable pools on startup and drops to an emergency shell, the user can now manually import a pool if possible and exit the shell to attempt to continue the boot process. Previously, a reboot was required to retry the boot process.
• An oversight in the loading of encryption keys caused a harmless error message to be displayed above the password prompt. This oversight has been fixed and the error no longer appears.
• Changes made to the handling of /etc/hostid in the ZFSBootMenu dracut module in anticipation of similar changes in the upcoming OpenZFS 2.0.0 release. This caused inconsistent behavior on systems using the musl C library with current versions of ZFS on Linux, resulting in potentially unbootable systems without forcing the spl_hostid command-line parameter. Now, the ZFSBootMenu dracut module attempts to discover the installed version of ZFS and behave consistently.

Significant commits in this release

940cd4c - Fall back to legacy hostid creation for ZFS < 2.0 (Andrew J. Hesford) 6cc0076 - Fix key_wrapper calls with out CLEAR_SCREEN defined (Zach Dykstra) 65a1a33 - Loop the emergency shell when initial pool imports fail (Andrew J. Hesford)

In addition to a bug fixes, this release targets refinements that improve usability and offer contextual help within the menus.

Fixes

• ZFSBootMenu now respects the console kernel command-line option and should behave as expected over a serial console.
• Command lists at the bottom of each menu are now sensibly wrapped to the terminal width, with extra coloring to highlight key combinations.
• Rather than rely solely on the hostid(1) command to populate the default hostid in the ZFSBootMenu initramfs, the dracut module will prefer to copy the /etc/hostid from the host, which should produce more consistent behavior on musl systems.
• Boot environments are now explicitly sorted, with the default boot environment appearing at the top of the list and selected by default.

New features

• An online help system, accessible from alt-h within any menu, provides descriptions of functionality provided by ZFSBootMenu.
• The description at the top of the menu now indicates whether the selected boot environment is on a pool currently imported readonly or writable.
• New command-line arguments zbm.lines and zbm.columns allow the size of the terminal at boot time.
• When generate-zbm fails to parse the YAML configuration, more detailed messages pinpoint parsing errors.

Significant commits in this release

dbe91a1 - Fix console handling when attached to a serial line (Andrew J. Hesford) 9959d10 - Respect ZFS hostid behavior on musl (Andrew J. Hesford) f4a60e6 - Capture and print config.yaml eval failure (Zach Dykstra) 2ebad45 - Sort environments, fix preview (Zach Dykstra) cc6e27c - Control size/target of ZFSBootMenu output (Zach Dykstra) bb294b3 - Enable dynamic line wrapping for header (Zach Dykstra) 8993591 - Enable global help system (Zach Dykstra) 7879876 - Read-only helpers (Zach Dykstra)

Revert omitting rootfs-block by default from the ZFSBootMenu initramfs. rootfs-block is a hard requirement of crypt, which is used to setup LUKS beneath ZFS.

This release brings significant improvements to the pool import process. Previously, all available pools were discovered and then their health was scraped to confirm that they were in an 'ONLINE' state before importing them. This process had a few subtle shortcomings that were highlighted by the pending release of OpenZFS 2.0.0. In particular, if a zpool has been upgraded via zpool upgrade to enable OpenZFS 2.0.0 feature flags, but the ZFSBootMenu initramfs contains an older version of OpenZFS, the pool was not able to be automatically imported in ZFSBootMenu. The import process now relies on zpool import -N -a -o readonly=on to attempt to import all available and otherwise healthy pools in read-only mode. By using zpool itself to determine all of the pools that can/should be imported, ZFSBootMenu now avoids the fragile process of scraping and interpreting the human-friendly text output of zpool import.

Fixes

• Improve the reliability of returning the correct kernel command line at all times. A helper function can now return corrected arguments in cases where resume is on the command line, but the pool has been imported read-write. This function is now used when generating a preview, making the modification more transparent to user inspection.
• Omit systemd and other modules from ZFSBootMenu by default. These install hooks that interfere with the correct operation of ZFSBootMenu.
• The testing/virtualization frame work has received a lot of attention during this development cycle. This has allowed us to create a variety of pool configurations that would be otherwise difficult to accomplish with physical hardware.

New features

• When accepting user input (new filesystem name, resume protections), allow CTRL-C to be used to cancel the process.

Significant commits in this release

• 219a632 - Rewrite pool import ahead of OpenZFS 2.0.0 (Zach Dykstra, Andrew J. Hesford)
• fb866c8 - Use zfsbootmenu-input in noresume prompt (Andrew J. Hesford)
• 1bb4f41 - Allow ctrl-c to be used during user input (Zach Dykstra)
• ca8eda8 - use -F to display the file type for more filtering options (Zach Dykstra)
• 08f22e2 - Simplify handling of BE command lines, use in previews (Andrew J. Hesford)
• cd8f889 - omit rootfs-block; it will never generate a correct KCL (Zach Dykstra)
• c96165c - Omit systemd related modules (Zach Dykstra)

Fixes

• The required binaries were audited, cleaning up tools that were no longer used.
• The installation of required binaries, core scripts and other files is checked during initramfs creation. If any of these critical files can not be installed, the image is not created.
• Parameters are passed to the pool import function to control how a pool is imported. This allows for more than just readonly changes when importing.
• config.yaml defaults were adjusted to more closely match normal use cases. This works in tandem with the --enable toggle in generate-zbm to provide a better out-of-the-box experience.

New features

• When importing a pool as R/W, and a resume image is found, the kernel command line is modified to remove resume= and to subsequently append noresume.
• Support skim in place of fzf for greater platform availability. fzf is preferred when creating an initramfs.
• The health of discovered pools can now be viewed. Optionally, a checkpoint rewind can be performed if one has been set. Use caution, this action can NOT be undone.
• Global image creation can be toggled via generate-zbm --enable or generate-zbm --disable

Significant commits in this release

• cfec416 - Support automatic "noresume" when importing pools R/W (Andrew J. Hesford)
• a6a36bf - Add support for pool status and checkpoints (Zach Dykstra)
• 79bcca3 - Fix logic inversion in import_pool handling of $read_write (Andrew J. Hesford)
• 14b88b9 - generate-zbm: enable components in config.yaml (Andrew J. Hesford)
• 71cd075 - Add --enable/--disable (Zach Dykstra)
• 999e6c2 - Drop import_args variable and let import_pool build its own arguments (Andrew J. Hesford)
• b85d836 - support sk, an fzf workalike, for menu presentation (Zach Dykstra)
• 2225fea - module-setup.sh: catch installation failures, warn or fail as appropriate (#72) (Andrew J. Hesford)
• 5801e00 - Prune everything not explicitly needed (Zach Dykstra)

ZFSBootMenu 1.4.1 is a minor update, updating the provided Makefile for packagers.

ZFSBootMenu 1.4 includes significant internal changes and some user-visible functional changes in the generate-zbm script.

Fixes

• Correct an issue that required two attempts to set default boot environments.
• Internal improvements to generate-zbm to improve consistency and facilitate future development.
• Management of versioned image retention should now be more consistent with expectation. Versioned ZBM images now increment a revision number when existing images with the same version already exist, and the retention policy preserves a configurable number of revisions for the current version alongside the latest revision of each of the same number of prior versions.
• Improved error handling should make failures in generate-zbm easier to understand.

New features

• Provide man pages (generate-zbm.5, generate-zbm.8 and zfsbootmenu.7) to document the creation and use of ZFSBootMenu images.
• Move from an INI configuration format to YAML, which should provide more flexibility for future enhancements.
• Provide a --migrate command-line option to convert existing INI configurations to the new format.
• Add configuration options to change default behavior for --kernel, --kver and --prefix to make generate-zbm easier to incorporate on non-Void systems.
• Add a configuration option to change the default behavior for --version to allow customized output versioning of images.
• Support string interpolation of %current or %{current} tags in --kver and --version values.
• Add a --cmdline command-line option to override the configured CommandLine value without editing the configuration file.

Significant commits in this release

0979051 - Add documentation for generate-zbm, its config and initramfs options (Zach Dykstra, et al.) ee1d9d8 - Unmask import_args in functions calling import_pool (Zach Dykstra) 3b2b2f0 - Add explicit --migrate option to generate-zbm (Andrew J. Hesford) 3cd3a8e - Improve error handling and automatic config conversion (Andrew J. Hesford) 80e0c30 - Switch syslinux entry to heredoc, fix syslinux.cfg file copy (Zach Dykstra) 6351226 - Move to YAML configuration, improve version handling (Andrew J. Hesford, Zach Dykstra) 5fdb872 - Add configuration options for kernel, version and prefix (Andrew J. Hesford) 79295ec - Add an optional parameter to safeCopy: (Zach Dykstra) 8aa133f - Clean up control flow in generate-zbm (Andrew J. Hesford)