Apptainer Versions Save

v1.2.0 - [2023-07-18]

Changed defaults / behaviours

• Create the current working directory in a container when it doesn't exist. This restores behavior as it was before singularity 3.6.0. As a result, using --no-mount home won't have any effect when running apptainer from a home directory and will require --no-mount home,cwd to avoid mounting that directory.
• Handle current working directory paths containing symlinks both on the host and in a container but pointing to different destinations. If detected, the current working directory is not mounted when the destination directory in the container exists.
• Destination mount points are now sorted by shortest path first to ensure that a user bind doesn't override a previous bind path when set in arbitrary order on the CLI. This is also applied to image binds.
• When the kernel supports unprivileged overlayfs mounts in a user namespace, the container will be constructed by default using an overlay instead of an underlay layout for bind mounts. A new --underlay action option can be used to prefer underlay instead of overlay.
• Use fuse-overlayfs instead of the kernel overlayfs when a lower dir is a FUSE filesystem, even when the overlay layer is not writable. That always used to be done when the overlay layer was writable, but this fixes a problem seen when squashfuse (which is read-only) was used for the overlay layer.
• Fix the enable overlay = driver configuration option to always use the overlay image driver (that is, fuse-overlayfs) even when the kernel overlayfs is usable.
• Overlay is blocked on the panfs filesystem, allowing sandbox directories to be run from panfs without error.
• sessiondir maxsize in apptainer.conf now defaults to 64 MiB for new installations. This is an increase from 16 MiB in prior versions.
• The apptainer cache is now architecture aware, so the same home directory cache can be shared by machines with different architectures.
• Show standard output of yum bootstrap if log level is verbose or higher while building a container.
• Lookup and store user/group information in stage one prior to entering any namespaces, to fix an issue with winbind not correctly looking up user/group information when using user namespaces.
• A new --reproducible flag for ./mconfig will configure Apptainer so that its binaries do not contain non-reproducible paths. This disables plugin functionality.

New features / functionalities

• Support for unprivileged encryption of SIF files using gocryptfs. The gocryptfs command is included in rpm and debian packaging. This is not compatible with privileged encryption, so containers encrypted by root need to be rebuilt by an unprivileged user.
• Templating support for definition files. Users can now define variables in definition files via a matching pair of double curly brackets. Variables of the form {{ variable }} will be replaced by a value defined either by a variable=value entry in the %arguments section of the definition file or through new build options --build-arg or --build-arg-file. By default any unused variables given in --build-arg or --build-arg-file result in a fatal error but the option --warn-unused-build-args changes that to a warning rather than a fatal error.
• Add a new instance run command that will execute the runscript when an instance is initiated instead of executing the startscript.
• The sign and verify commands now support signing and verification with non-PGP key material by specifying the path to a private key via the --key flag.
• The verify command now supports verification with X.509 certificates by specifying the path to a certificate via the --certificate flag. By default, the system root certificate pool is used as trust anchors unless overridden via the --certificate-roots flag. A pool of intermediate certificates that are not trust anchors, but can be used to form a certificate chain, can also be specified via the --certificate-intermediates flag.
• Support for online verification checks of X.509 certificates using OCSP protocol via the new verify --ocsp-verify option.
• The instance stats command displays the resource usage every second. The --no-stream option disables this interactive mode and shows the point-in-time usage.
• Instances are now started in a cgroup by default, when run as root or when unified cgroups v2 with systemd as manager is configured. This allows apptainer instance stats to be supported by default when possible.
• The instance start command now accepts an optional --app <name> argument which invokes a start script within the %appstart <name> section in the definition file. The instance stop command still only requires the instance name.
• The instance name is now available inside an instance via the new APPTAINER_INSTANCE environment variable.
• Add ability to set a custom config directory via the new APPTAINER_CONFIGDIR environment variable.
• Add ability to change log level through environment variables, APPTAINER_SILENT, APPTAINER_QUIET, and APPTAINER_VERBOSE. Also add APPTAINER_NOCOLOR for the --nocolor option.
• Add discussion of using TMPDIR or APPTAINER_TMPDIR in the build help.
• The --no-mount flag now accepts the value bind-paths to disable mounting of all bind path entries in apptainer.conf.
• Support for DOCKER_HOST parsing when using docker-daemon://
• DOCKER_USERNAME and DOCKER_PASSWORD supported without APPTAINER_ prefix.
• Add new Linux capabilities CAP_PERFMON, CAP_BPF, and CAP_CHECKPOINT_RESTORE.
• Add setopt definition file header for the yum bootstrap agent. The setopt value is passed to yum / dnf using the --setopt flag. This permits setting e.g. install_weak_deps=False to bootstrap recent versions of Fedora, where systemd (a weak dependency) cannot install correctly in the container. See examples/Fedora for an example definition file.
• Warn user that a yum bootstrap of an older distro may fail if the host rpm _db_backend is not bdb.
• The remote get-login-password command allows users to retrieve a remote's token. This enables piping the secret directly into docker login while preventing it from showing up in a shell's history.
• Define EUID in %environment alongside UID.
• In --rocm mode, the whole of /dev/dri is now bound into the container when --contain is in use. This makes /dev/dri/render devices available, required for later ROCm versions.

Other changes

• Update minimum go version to 1.19.
• Upgrade squashfuse_ll to version 0.2.0, removing the need for applying patches during compilation. The new version includes a fix to prevent it from triggering 'No data available errors' on overlays of SIF files that were built on machines with SELinux enabled.
• Fix non-root instance join with unprivileged systemd-managed cgroups v2, when join is from outside a user-owned cgroup.
• Fix joining cgroup of instance started as root, with cgroups v1, non-default cgroupfs manager, and no device rules.
• Avoid UID / GID / EUID readonly var warnings with --env-file.
• Ensure consistent binding of libraries under --nv/--rocm when duplicate <library>.so[.version] files are listed by ldconfig -p.
• Ensure DOCKER_HOST is honored in non-build flows.
• Corrected apptainer.conf comment, to refer to correct file as source of default capabilities when root default capabilities = file.
• Fix memory usage calculation during apptainer compilation on RaspberryPi.
• Fix misleading error when an overlay is requested by the root user while the overlay kernel module is not loaded.
• Fix interaction between --workdir and --scratch options when the former is given a relative path.
• Remove the warning about a missing signature when building an image based on a local unsigned SIF file.
• Set real UID to zero when escalating privileges for CNI plugins, to fix issue appeared with RHEL 9.X.
• Fix seccomp filters to allow mknod/mknodat syscalls to create pipe/socket and character devices with device number 0 for fakeroot builds.
• Add 32-bit compatibility mode for 64-bit architectures in the fakeroot seccomp filter.

v1.2.0-rc.2 - [2023-07-05]

Changes since last pre-release

• Upgrade gocryptfs to version 2.4.0, removing the need for fusermount from the fuse package.
• Upgrade squashfuse_ll to version 0.2.0, removing the need for applying patches during compilation. The new version includes a fix to prevent it from triggering 'No data available errors' on overlays of SIF files that were built on machines with SELinux enabled.
• Add ability to set a custom config directory via the new APPTAINER_CONFIGDIR environment variable.
• Add ability to change log level through environment variables, APPTAINER_SILENT, APPTAINER_QUIET, and APPTAINER_VERBOSE. Also add APPTAINER_NOCOLOR for the --nocolor option.
• Add discussion of using TMPDIR or APPTAINER_TMPDIR in the build help.
• Add new option --warn-unused-build-args to output warnings rather than fatal errors for any additional variables given in --build-arg or --build-arg-file.
• Use fuse-overlayfs instead of the kernel overlayfs when a lower dir is a FUSE filesystem, even when the overlay layer is not writable. That always used to be done when the overlay layer was writable, but this fixes a problem seen when squashfuse (which is read-only) was used for the overlay layer.
• Fix the enable overlay = driver configuration option to always use the overlay image driver (that is, fuse-overlayfs) even when the kernel overlayfs is usable.
• Fix a minor regression in 1.2.0-rc.1 where starting up under unshare -r stopped mapping the user's home directory to the fake root's home directory.
• Fix interaction between --workdir and --scratch options when the former is given a relative path.
• Remove the warning about a missing signature when building an image based on a local unsigned SIF file.
• Set real UID to zero when escalating privileges for CNI plugins to fix issue appeared with RHEL 9.X.
• Fix seccomp filters to allow mknod/mknodat syscalls to create pipe/socket and character devices with device number 0 for fakeroot builds.
• Add 32-bit compatibility mode for 64-bit architectures in the fakeroot seccomp filter.

v1.2.0-rc.1 - [2023-06-07]

Changed defaults / behaviours

• Create the current working directory in a container when it doesn't exist. This restores behavior as it was before singularity 3.6.0. As a result, using --no-mount home won't have any effect when running apptainer from a home directory and will require --no-mount home,cwd to avoid mounting that directory.
• Handle current working directory paths containing symlinks both on the host and in a container but pointing to different destinations. If detected, the current working directory is not mounted when the destination directory in the container exists.
• Destination mount points are now sorted by shortest path first to ensure that a user bind doesn't override a previous bind path when set in arbitrary order on the CLI. This is also applied to image binds.
• When the kernel supports unprivileged overlay mounts in a user namespace, the container will be constructed by default using an overlay instead of an underlay layout for bind mounts. A new --underlay action option can be used to prefer underlay instead of overlay.
• sessiondir maxsize in apptainer.conf now defaults to 64 MiB for new installations. This is an increase from 16 MiB in prior versions.
• The apptainer cache is now architecture aware, so the same home directory cache can be shared by machines with different architectures.
• Overlay is blocked on the panfs filesystem, allowing sandbox directories to be run from panfs without error.
• Show standard output of yum bootstrap if log level is verbose or higher while building a container.
• Lookup and store user/group information in stage one prior to entering any namespaces, to fix an issue with winbind not correctly looking up user/group information when using user namespaces.
• A new --reproducible flag for ./mconfig will configure Apptainer so that its binaries do not contain non-reproducible paths. This disables plugin functionality.

New features / functionalities

• Support for unprivileged encryption of SIF files using gocryptfs. The gocryptfs command is included in rpm and debian packaging. This is not compatible with privileged encryption, so containers encrypted by root need to be rebuilt by an unprivileged user.
• Templating support for definition files. Users can now define variables in definition files via a matching pair of double curly brackets. Variables of the form {{ variable }} will be replaced by a value defined either by a variable=value entry in the %arguments section of the definition file or through new build options --build-arg or --build-arg-file.
• Add a new instance run command that will execute the runscript when an instance is initiated instead of executing the startscript.
• The sign and verify commands now support signing and verification with non-PGP key material by specifying the path to a private key via the --key flag.
• The verify command now supports verification with X.509 certificates by specifying the path to a certificate via the --certificate flag. By default, the system root certificate pool is used as trust anchors unless overridden via the --certificate-roots flag. A pool of intermediate certificates that are not trust anchors, but can be used to form a certificate chain, can also be specified via the --certificate-intermediates flag.
• Support for online verification checks of X.509 certificates using OCSP protocol via the new verify --ocsp-verify option.
• The instance stats command displays the resource usage every second. The --no-stream option disables this interactive mode and shows the point-in-time usage.
• Instances are now started in a cgroup by default, when run as root or when unified cgroups v2 with systemd as manager is configured. This allows apptainer instance stats to be supported by default when possible.
• The instance start command now accepts an optional --app <name> argument which invokes a start script within the %appstart <name> section in the definition file. The instance stop command still only requires the instance name.
• The instance name is now available inside an instance via the new APPTAINER_INSTANCE environment variable.
• The --no-mount flag now accepts the value bind-paths to disable mounting of all bind path entries in apptainer.conf.
• Support for DOCKER_HOST parsing when using docker-daemon://
• DOCKER_USERNAME and DOCKER_PASSWORD supported without APPTAINER_ prefix.
• Add new Linux capabilities CAP_PERFMON, CAP_BPF, and CAP_CHECKPOINT_RESTORE.
• Add setopt definition file header for the yum bootstrap agent. The setopt value is passed to yum / dnf using the --setopt flag. This permits setting e.g. install_weak_deps=False to bootstrap recent versions of Fedora, where systemd (a weak dependency) cannot install correctly in the container. See examples/Fedora for an example definition file.
• Warn user that a yum bootstrap of an older distro may fail if the host rpm _db_backend is not bdb.
• The remote get-login-password command allows users to retrieve a remote's token. This enables piping the secret directly into docker login while preventing it from showing up in a shell's history.
• Define EUID in %environment alongside UID.
• In --rocm mode, the whole of /dev/dri is now bound into the container when --contain is in use. This makes /dev/dri/render devices available, required for later ROCm versions.

Other changes

• Update minimum go version to 1.19.
• Fix non-root instance join with unprivileged systemd-managed cgroups v2, when join is from outside a user-owned cgroup.
• Fix joining cgroup of instance started as root, with cgroups v1, non-default cgroupfs manager, and no device rules.
• Avoid UID / GID / EUID readonly var warnings with --env-file.
• Ensure consistent binding of libraries under --nv/--rocm when duplicate <library>.so[.version] files are listed by ldconfig -p.
• Ensure DOCKER_HOST is honored in non-build flows.
• Corrected apptainer.conf comment, to refer to correct file as source of default capabilities when root default capabilities = file.
• Fix memory usage calculation during apptainer compilation on RaspberryPi.
• Fix misleading error when an overlay is requested by the root user while the overlay kernel module is not loaded.

v1.1.9 - [2023-06-07]

• Remove warning about unknown xino=on option from fuse-overlayfs, introduced in 1.1.8.
• Ignore extraneous warning from fuse-overlayfs about a readonly /proc.
• Fix dropped "n" characters on some platforms in definition file stored as part of SIF metadata.
• Remove duplicated group ids.
• Fix not being able to handle multiple entries in LD_PRELOAD when binding fakeroot into container during apptainer startup for --fakeroot with fakeroot command.

v1.1.8 - [2023-04-25]

Security fix

• Included a fix for CVE-2023-30549 which is a vulnerability in setuid-root installations of Apptainer and Singularity that causes an elevation in severity of an existing ext4 filesystem driver vulnerability that is unpatched in several older but still actively supported operating systems including RHEL7, Debian 10, Ubuntu 18.04 and Ubuntu 20.04. The fix adds allow setuid-mount configuration options encrypted, squashfs, and extfs, and makes the default for extfs be "no". That disables the use of extfs mounts including for overlays or binds while in the setuid-root mode, while leaving it enabled for unprivileged user namespace mode. The default for encrypted and squashfs is "yes".

Other changes

• Fix loop device 'no such device or address' spurious errors when using shared loop devices.
• Remove unwanted colors to STDERR.
• Add xino=on mount option for writable kernel overlay mount points to fix inode numbers consistency after kernel cache flush (not applicable to fuse-overlayfs).

v1.1.7 - [2023-03-28]

Changes since last release

• Allow gpu options such as --nv to be nested by always inheriting all libraries bound in to a parent container's /.singularity.d/libs.
• Map the user's home directory to the root home directory by default in the non-subuid fakeroot mode like it was in the subuid fakeroot mode, for both action commands and building containers from definition files.
• Avoid unknown option error when using a bare squashfs image with an unpatched squashfuse_ll.
• Fix GOCACHE settings for golang build on PPA build environment.
• Make the error message more helpful in another place where a remote is found to have no library client.
• Allow symlinks to the compiled prefix for suid installations. Fixes a regression introduced in 1.1.4.
• Avoid incorrect error when requesting fakeroot network.
• Build via zypper on SLE systems will use repositories of host via suseconnect-container.
• Pass computed LD_LIBRARY_PATH to wrapped unsquashfs. Fixes issues where unsquashfs on host uses libraries in non-default paths.

v1.1.6 - [2023-02-14]

Security fix

• Included a fix for CVE-2022-23538 which potentially leaked user credentials to a third-party S3 storage service when using the library:// protocol. See the link for details.

Other changes

• Restored the ability for running instances to be tracked when apptainer is installed with tools/install-unprivileged.sh. Instance tracking depends on argument 0 of the starter, which was not getting preserved.
• Fix GOCACHE environment variable settings when building debian source package on PPA build environment.
• Make PS1 environment variable changeable via %environment section on definition file that used to be only changeable via APPTAINERENV_PS1 outside of container. This makes the container's prompt customizable.
• Fix the passing of nested bind mounts when there are multiple binds separated by commas and some of them have colons separating sources and destinations.
• Added Provides: bundled(golang()) statements to the rpm packaging for each bundled golang module.
• Hide messages about SINGULARITY variables if corresponding APPTAINER variables are defined. Fixes a regression introduced in 1.1.4.
• Print a warning if extra arguments are given to a shell action, and show in the run action usage that arguments may be passed.
• Check for the existence of the runtime executable prefix, to avoid issues when running under Slurm's srun. If it doesn't exist, fall back to the compile-time prefix.
• Increase the timeout on image driver (that is, FUSE) mounts from 2 seconds to 10 seconds. Instead, print an INFO message if it takes more than 2 seconds.
• If a remote is defined both globally (i.e. system-wide) and individually, change apptainer remote commands to print an info message instead of exiting with a fatal error and to give precedence to the individual configuration.

v1.1.5 - [2023-01-10]

• Update the rpm packaging to (a) move the Obsoletes of singularity to the apptainer-suid packaging, (b) remove the Provides of singularity, (c) add a Provides and Conflicts for sif-runtime, (d) add "formerly known as Singularity" to the Summary, and (e) add a Conflicts of singularity to the apptainer package. Also update the debian and nfpm packaging with (d).
• Change rpm packaging to automatically import any modified configuration files in /etc/singularity when updating from singularity to apptainer, including importing singularity.conf using a new hidden confgen command.
• Fix the use of fakeroot, faked, and libfakeroot.so if they are not suffixed by -sysv, as is for instance the case on Gentoo Linux.
• Prevent the use of a --libexecdir or --bindir mconfig option from making apptainer think it was relocated and so preventing use of suid mode. The bug was introduced in v1.1.4.
• Add helpful error message for build --remote option.
• Add more helpful error message when no library endpoint found.
• Avoid cleanup errors on exit when mountpoints are busy by doing a lazy unmount if a regular unmount doesn't work after 10 tries.
• Make messages about using SINGULARITY variables less scary.

v1.1.4 - [2022-12-12]

• Added tools/install-unprivileged.sh to download and install apptainer binaries and all dependencies into a directory of the user's choice. Works on all currently active el, fedora, debian, and ubuntu versions except ubuntu 18.04, with all architectures supported by epel and fedora. Defaults to the latest version released in epel and fedora. Other apptainer versions can be selected but it only works with apptainer 1.1.4 and later.
• Make the binaries built in the unprivileged apptainer package relocatable. When moving the binaries to a new location, the /usr at the top of some of the paths needs to be removed. Relocation is disallowed when the starter-suid is present, for security reasons.
• Change the warning when an overlay image is not writable, introduced in v1.1.3, back into a (more informative) fatal error because it doesn't actually enter the container environment.
• Set the --net flag if --network or --network-args is set rather than silently ignoring them if --net was not set.
• Do not hang on pull from http(s) source that doesn't provide a content-length.
• Avoid hang on fakeroot cleanup under high load seen on some distributions / kernels.
• Remove obsolete pacstrap -d in Arch packer.
• Adjust warning message for deprecated environment variables usage.
• Enable the --security uid:N and --security gid:N options to work when run in non-suid mode. In non-suid mode they work with any user, not just root. Unlike with root and suid mode, however, only one gid may be set in non-suid mode.

v1.1.3 - [2022-10-25]

• Prefer the fakeroot-sysv command over the fakeroot command because the latter can be linked to either fakeroot-sysv or fakeroot-tcp, but fakeroot-sysv is much faster.
• Update the included squashfuse_ll to have -o uid=N and -o gid=N options and changed the corresponding image driver to use them when available. This makes files inside sif files appear to be owned by the user instead of by the nobody id 65534 when running in non-setuid mode.
• Fix the locating of shared libraries when running unsquashfs from a non-standard location.
• Properly clean up temporary files if unsquashfs fails.
• Fix the creation of missing bind points when using image binding with underlay.
• Change the error when an overlay image is not writable into a warning that suggests adding :ro to make it read only or using --fakeroot.
• Avoid permission denied errors during unprivileged builds without /etc/subuid-based fakeroot when /var/lib/containers/sigstore is readable only by root.
• Avoid failures with --writable-tmpfs in non-setuid mode when using fuse-overlayfs versions 1.8 or greater by adding the fuse-overlayfs noacl mount option to disable support for POSIX Access Control Lists.
• Fix the --rocm flag in combination with -c / -C by forwarding all /dri/render* devices into the container.