Cloud Hypervisor Versions Save

A Virtual Machine Monitor for modern Cloud workloads. Features include CPU, memory and device hotplug, support for running Windows and Linux guests, device offload with vhost-user and a minimal compact footprint. Written in Rust with a strong focus on security.

v39.0

1 week ago

This release has been tracked in our roadmap project as iteration v39.0. The following user visible changes have been made:

Variable Sizing of PCI Apertures for Segments

It is now possible to use --pci-segment to adjust the aperture size that devices 32-bit and 64-bit PCI device BARs will be allocated from. Previously the address space was equally distributed across all the segments which may leave insufficient space for devices that require a large 32-bit space. With this change the weighting per segment can be adjusted. (#6387)

Direct Booting with bzImages

Support for directly booting Linux from bzImages has been added.(#6200)

Support for NVIDIA GPUDirect P2P Support

The x_nv_gpudirect_clique option was added to --device to allow the configuration of device P2P support with NVIDIA GPUs. (#6235)

Guest NMI Injection Support

A new API endpoint and ch-remote option added for injecting an NMI into the guest. (#6047)

Notable Bug Fixes

  • Workaround for kernel bug affecting guest IRQ masking on AMD (#6353)
  • Correctly cleanup sigwinch_listener process (#6208)
  • Graceful shutdown of HTTP API thread (#6248, #6247)
  • Fix queue_affinity option in OpenAPI metadata (#6268)
  • Fix documentation to indicate only stream mode is supported by virtio-vsock (#6306)
  • Fix virtio-fs tag validation (#6358, #6359)
  • Add missing pvpanic device to OpenAPI metadata (#6372)
  • Fixes for nested virtualization with VFIO devices (#6110, #6298, #6297, #6319)
  • Fix for backing file for virtio-mem regions with snapshot/restore (#6337, #6338)
  • Explicitly mark FDs used for network devices as invalid across snapshot/restore (#6332, #6286)
  • Improve event-monitor events around reboot (#6277, #6274)
  • Fix potential deadlock around paused devices during live migration (#6293)
  • Fix panic when running ch-remote with no subcommand (#6230)
  • Fix hotplug of virtio devices after snapshot/restore and live migration (#6326, #6265)

Contributors

Many thanks to everyone who has contributed to our release:

v37.1

1 month ago

This is a bug fix release. The following issues have been addressed:

  • Fix several security advisories from dependencies (#6134, #6141)
  • Enable HTT flag to avoid crashing cpu topology enumeration software such as hwloc in the guest (#6146)
  • Enable nested virtualization on AMD if supported (#6106)
  • Handle non-power-of-two CPU topology properly (#6062)
  • Various bug fixes around virtio-vsock(#6080, #6091, #6095)
  • Align VFIO devices PCI BARs naturally (#6196)

v38.0

2 months ago

This release has been tracked in our roadmap project as iteration v38.0. The following user visible changes have been made:

Group Rate Limiter on Block Devices

Users now can throttle a group of block devices with the new --rate-limiter-group option. Details can be found from the I/O Throttling documentation

CPU Pinning Support for Block Device Worker Thread

Users now have the option to pin virt-queue threads for block devices to specific host cpus.

Optimized Boot Time with Parallel Memory Prefault

The boot time with prefault option enabled is optimized via parallel memory prefault.

New 'debug-console' Device

A 'debug-console' device is added to provide a user-configurable debug port for logging guest information. Details can be found from the Debug IO Ports documentation.

Improved VFIO Device Support

All non-emulated MMIO regions of VFIO devices are now mapped to the VFIO container, allowing PCIe P2P between all VFIO devices on the same VM. This is required for a wide variety of multi-GPU workloads involving GPUDirect P2P (DMA between two GPUs), GPUDirect RDMA (DMA between a GPU and an IB device).

Extended CPU Affinity Support

Users now can set the vcpu affinity to a host CPU with index larger than 255.

Notable Bug Fixes

  • Enable HTT flag to avoid crashing cpu topology enumeration software such as hwloc in the guest (#6146)
  • Fix several security advisories from dependencies (#6134, #6141)
  • Handle non-power-of-two CPU topology properly (#6062)
  • Various bug fixes around virtio-vsock(#6080, #6091, #6095)
  • Enable nested virtualization on AMD if supported (#6106)
  • Align VFIO devices PCI BARs naturally (#6196)

Contributors

Many thanks to everyone who has contributed to our release:

v28.4

3 months ago

This is a bug fix release. The following issues have been addressed:

  • Fix VFIO on platforms with non-4k page size (#5450)
  • Fix seccomp filter lists related to virtio-console, serial and pty (#5506, #5524)
  • Populate APIC ID properly (#5512)
  • Ignore and warn TAP FDs in more situations (#5522)
  • Disallow concurrent CPU resizing (#5668)
  • Use wrapping add for memory offset from instruction emulator (#5719)
  • Replace unsound static mut with once_cell (#5772)
  • Fix a deadlock when TDX is enabled (#5845)
  • Bug fix to OpenAPI specification file (#5967)
  • Error out early for live migration when TDX is enabled (#6025)

v36.1

3 months ago

This is a bug fix release. The following issues have been addressed:

  • Fix aio backend behavior for block devices when writeback cache disabled (#5930)
  • Fix PvPanic device PCI BAR alignment (#5956)
  • Bug fix to OpenAPI specification file (#5967)
  • Error out early for live migration when TDX is enabled (#6025)

v35.1

3 months ago

This is a bug fix release. The following issues have been addressed:

  • Fix a deadlock when TDX is enabled (#5845)
  • Correct default value for vCPU topology on AArch64 (#5893)
  • Fix PvPanic device PCI BAR alignment (#5956)
  • Bug fix to OpenAPI specification file (#5967)
  • Error out early for live migration when TDX is enabled (#6025)

v37.0

4 months ago

This release has been tracked in our roadmap project as iteration v37.0. The following user visible changes have been made:

Long Term Support (LTS) Release

This release is a LTS release. Point releases for bug fixes will be made for the next 18 months; live migration and live upgrade will be supported between the point releases of the LTS.

Multiple PCI segments Support for 32-bit VFIO Devices

Now VFIO devices with 32-bit memory BARs can be attached to non-zero PCI segments on the guest, allowing users to have more 32-bit devices and assign such devices to appropriate NUMA nodes for better performance.

Configurable Named TAP Devices

Named TAP devices now accepts IP configuration from users, such as IP and MAC address, as long as the named TAP device is created by Cloud Hypervisor (e.g. not existing TAP devices).

TTY Output from Both Serial Device and Virtio Console

Now legacy serial device and virtio console can be set as TTY mode as the same time. This allows users to capture early boot logs with the legacy serial device without losing performance benefits of using virtio-console, when appropriate kernel configuration is used (such as using kernel command-line console=hvc0 earlyprintk=ttyS0 on x86).

Faster VM Restoration from Snapshots

The speed of VM restoration from snapshots is improved with a better implementation of deserializing JSON files.

Notable Bug Fixes

  • Fix aio backend behavior for block devices when writeback cache disabled (#5930)
  • Fix PvPanic device PCI BAR alignment (#5956)
  • Bug fix to OpenAPI specification file (#5967)
  • Error out early for live migration when TDX is enabled (#6025)

Contributors

Many thanks to everyone who has contributed to our release:

v36.0

6 months ago

This release has been tracked in our roadmap project as iteration v36.0. The following user visible changes have been made:

Command Line Changes

We switched back to use the clap crate to create our command line, since the argh crate is barely maintained. There were several syntax changes:

  • All --option value commands now are --option=value.
  • The --disk DISK1 --disk DISK2 command now is --disk DISK1 DISK2.
  • The -v -v -vcommand now is -vvv.

Note: the released binary size increased around 0.3M due to this change.

Enabled Features Reported via API Endpoint and CLI

Now the enabled (Cargo) features of the running Cloud Hypervisor instance can be queried via API endpoint (/vmm.ping) and CLI (--version -v).

NUMA Support for PCI segments

The --numa command is augmented with a new option pci_segment=, so that users can define the relationship between PCI segments and NUMA nodes. Examples can be found from the memory documentation

CPU Topology Support on AMD Platforms

Now the CPU topology on x86_64 platforms supports multiple vendors.

Unix Socket Backend for Serial Port

The --serial command is augmented with a new option socket=, allowing users to access the serial port using a Unix socket.

AIO Backend for Block Devices

An AIO backend is added for virtio-block devices to improve block device performance when the io_uring feature is not supported by the host Operating System.

Documentation Improvements

  • New documentation for collecting coverage data
  • Various typo fixes

Notable Bug Fixes

  • Fix a deadlock when TDX is enabled (#5845)
  • Only advertise AMX feature bits to guest when the AMX cpu feature is enabled (#5834)
  • Correct default value for vCPU topology on AArch64 (#5893)

Contributors

Many thanks to everyone who has contributed to our release:

v35.0

7 months ago

This release has been tracked in our roadmap project as iteration v35.0. The following user visible changes have been made:

virtio-vsock Support for Linux Guest Kernel v6.3+

Since kernel v6.3, a vsock packet can be included in a single descriptor, instead of being split over two descriptors. Our virtio-vsock implementation now support both situations.

User Specified Serial Number for virtio-block

A new option serial is added to the --block command that allows users to specify a serial number for block devices which will be guest visible.

vCPU TSC Frequency Included in Migration State

This ensures migration works correctly between hosts that have different TSC frequencies if the guest is running with TSC as the source of timekeeping.

Notable Bug Fixes

  • Disallow concurrent CPU resizing (#5668)
  • Handle APIC EOI message for MSHV (#5681)
  • Use wrapping add for memory offset from instruction emulator (#5719)
  • Add global spell check via the 'typos' GitHub action (#5720)
  • Ensure probing reads are block size aligned (#5727)
  • Multiple bug fixes around the latency counter for block devices (#5712, #5750, #5762, #5763)
  • Replace unsound static mut with once_cell (#5772)

Contributors

Many thanks to everyone who has contributed to our release:

v34.0

8 months ago

This release has been tracked in our roadmap project as iteration v34.0. The following user visible changes have been made:

Paravirtualised Panic Device Support

A new device has been added that can communicate when the guest kernel has panicked and share those details with the VMM. This is controlled with a new --pvpanic command line option and JSON API change equivalent. (#5526)

Improvements to VM Core Dump

Requesting to dump the guest memory as core dump will now transparently pause the VM if required; returning to the original state after. (#5604)

QCOW2 Support for Backing Files

The support for QCOW2 files has been enhanced to include support for using backing files. (#5573)

Minimum Host Kernel Bump

The minimum supported host kernel is now 5.13 in order to incorporate a bug fix for KVM_FEATURE_ASYNC_PF_INT functionality. (#5626)

Notable Bug Fixes

  • The x86 emulator is only compiled in if MSHV is compiled in (the kernel carries out this job with KVM) (#5561).
  • A regression has been fixed in VFIO support for devices that use MSI rather than MSI-X (#5658).
  • When triggering a VM shutdown or reset via I/O the vCPU thread will be blocked until that asynchronous event has been received (#5645).
  • Pausing a VM is now a synchronous action with the request only completing when all vCPUs are paused (#5611).
  • Event monitor support now correctly supports concurrent access (#5633).
  • Bug fixes for the QCOW2 file support (#5573).

Contributors

Many thanks to everyone who has contributed to our release: