Vhive Versions Save

Added

• Added support for OpenYurt, an open platform that extends upstream Kubernetes to run on edge node pools. More details on how to the Knative-atop-OpenYurt mode are described here.

Changed

• Removed the utils and examples from the vHive repo, moved to vSwarm.
• Bumped Go to 1.21, Kubernetes to v1.29, Knative to v1.13, Istio to 1.20.2, MetalLB to 0.14.3, Calico to 3.27.2.
• Made the automatic patching of the knative-serving and calico manifests instead of storing the patched manifests in the repo.

Fixed

This is the release before bumping k8s to 1.29 and 1.13: https://github.com/vhive-serverless/vHive/pull/943 Such bumping would break vHive's consistency with InVitro.

Added

• Added support for NVIDIA GPU in stock-only setup, with setup script and example Knative deployment.
• Upgraded the Firecracker version. Vanilla Firecracker snapshots are supported with local snapshot storage. Remote snapshot support is added but unstable (GH-823).
• Added a new netPoolSize option to configure the amount of network devices in the Firecracker VM network pool (10 by default), which can be used to keep the network initialization off the cold start path of Firecracker VMs.

Changed

• Changed system setup script. NVIDIA helm is now one of the vHive dependencies.
• Disabled the UPF feature for Firecracker snapshots (GH-807), but it is still available in the legacy branch.
• Update quick start guide to use refactored Go version setup scripts with a unified entry for easily setting up vHive and remove some legacy bash scripts under scripts.

Fixed

• Removed the limitation on the number of functions instances that can be restored from a single Firecracker snapshot (previously it was limited to 1).

Added

• Added support for eStargz in stock-only setup, added an example of knative deployment with eStargz. Credit to @DavidBuzatu-Marian

Changed

• Bumped Knative to v1.9, Go to v1.19, Kubernetes to v1.25, Istio to 1.16.0, metallb 0.13.9, Calico to 3.25.1.
• Bumped the GitHub-hosted runner OS version to Ubuntu 20.

Fixed

• vHive CRI (end-to-end) CI for firecracker and gVisor has been rehosted and is fully operational now.

Warning: This release drops support for Kubernetes 1.23.

Upgrade of the vHive system components and the dependencies to stable versions (particularly, Knative):

• Kubernetes: 1.23.5
• Knative: 1.4
• Istio: 1.12.5
• containerd: 1.6.2
• protoc: 3.19.4
• runc: 1.1.0
• Go: 1.18

Support Ubuntu 20 host OS, discontinuing support for Ubuntu 18.

Added

• Added support for gVisor MicroVMs, as an alternative to Firecracker.
• Added vSwarm, a suite of representative serverless workloads. Currently, in a beta testing mode.
• Added Python and Go tracing modules and an example showing its usage. Moved to vSwarm.
• Added Golang and Python storage modules, supporting AWS S3 and AWS ElastiCache. Moved to vSwarm.
• Added self-hosted stock-Knative runners on KinD, see scripts/self-hosted-kind.

Changed

• Workload stdout/stderr is not directly redirected to vhive stdout/stderr anymore but is printed by vhive via logrus.WithFields(logrus.Fields{"vmID": vmID}).
• Moved the CRI non-Firecracker tests to self-hosted stock-Knative runners.

Fixed

• Minor fixes and improvements in the docs.

Added

Benchmarking and tracing and the corresponding docs

• Added 2 chained-functions microbenchmarks, synchronous and asynchronous, that use Knative Serving and Eventing, correspondingly. Tracing is fully supported for Serving function composition, partially supported for Eventing function composition.
• Added docs on vHive benchmarking methodology for arbitrary serverless deployments.
• Added docs for adding benchmarks to vHive.
• Revamped the quick-start guide for a better new-user experience.
• Added Knative Eventing Tutorial: docs and example.
• Added a Go module for tracing using Zipkin.

Improved stability and troubleshooting

• CRI test logs are now stored as GitHub artifacts.
• Added script to (re)start vHive single node cluster in a push-button.
• Added a linter for hyperlink checking in markdown files.

Changed

• Bumped Containerd to v1.5.2.
• Bumped Knative to v0.23.0.
• Bumped Go to v1.16.4.
• Frozen Kubernetes at v1.20.6-00.
• Simplified Go dependencies management by refactoring modules into packages.

Fixed

• Fixed stock Knative cluster startup.

Added

Features for performance analysis

• Zipkin support added for tracing and breaking down latencies in a distributed vHive setting (e.g., across Istio and Knative services). More info here
• [beta] Added a profiler that collects low-level microarchitectural metrics, using the Intel TopDown method. The tool aims at studying the implications of multi-tenancy, i.e., the VM number, on the tail latency and throughput.

Features for benchmarking at scale and multi-function applications

• Added cluster-local container registry support to avoid DockerHub bottleneck. Contributed by @amohoste from ETH Zurich.
• [alpha] Added Knative eventing support using In-Memory Channel and MT-Channel-broker. Integration tests and Apache Kafka support coming soon.
• Added support for MinIO object store (non-replicated, non-distributed). More info here

Other

• vHive now also supports vanilla Knative benchmarking and testing (i.e., using containers for function sandboxes). More info here.

Changed

• Bumped up the Firecracker version to v0.24 with REAP snapshots support.
• Bumped up all Knative components to version v0.21.
• MicroVMs have network access to all services deployed in a vHive/k8s cluster and the Internet by default, using an automatically detected, or a user-specified, host interface.

Fixed

• CI pulls the latest binaries from git-lfs when running tests on self-hosted runners.

Added

• Created a CI pipeline that runs CRI integration tests inside containers using kind.
• [Beta] Added a developer image that runs a vHive k8s cluster in Docker, simplifying vHive development and debugging. Currently, only single-node clusters are supported.
• Extended the developers guide on the modes of operation, performance analysis and vhive development environment inside containers.
• Added a slide deck of Dmitrii's talk at Amazon.
• Added a commit linter and a spell checker for *.md files.

Changed

• Use replace pragmas instead of Go modules.
• Bump Go version to 1.15 in CI.
• Deprecated Travis CI.

Fixed

• Fixed the vHive cluster setup issue for clusters with >2 nodes issue.