Dynamically provision Stateful Persistent Replicated Cluster-wide Fabric Volumes & Filesystems for Kubernetes that is provisioned from an optimized NVME SPDK backend data storage stack.
Release Date: 27th April, 2023
Mayastor 2.1 is the first minor release on top of the 2.0 code-base. This release introduces the ability to create thinly-provisioned volumes and has seamless upgrade support from previous 2.0 versions. This release also provides a guided migration path for users running legacy Mayastor installations (v1.0.5 and below). In addition, there are stability and observability fixes as well.
Mayastor 2.1 introduces the ability to create thin-provisioned volumes in addition to the current thickly provisioned volumes. When a thinly provisioned volume is created, space is allocated only for the metadata of the underlying replicas. As the usage increases, allocation of space happens on-demand for the new writes. The thin provisioning feature allows users to over-commit on their existing storage capacity and enables them to plan more efficiently. To track the commitment statistics, the disk pool metrics exporter now exports an additional metric named “disk_pool_committed_size“ in Bytes.
Thin provisioning is being released as an experimental feature in this release. We hope to gain valuable feedback on this from our cloud-native users. Please refer the documentation for more information.
With Mayastor 2.1 release, users get the support for upgrading of the software from version 2.0.0 or 2.0.1 to version 2.1.0. This upgrade process is made as seamless and non-disruptive as possible. The upgrade software checks and strives to preserve the availability of every volume during the upgrade process. Please note that the process is non disruptive only for volumes which have more than 1 replica. For volumes with a single replica, there will be a short duration when the volume will become unavailable while the upgrade is in-progress.
Please refer the documentation for more information.
Mayastor versions 1.0.5 and prior, are being considered as legacy versions. Due to several breaking changes in the 2.0 codebase of the software, it is not possible to support seamless upgrades from the legacy versions to the current version. Mayastor 2.1 provides a documented migration path for users to move their legacy installations to the latest version.
Please refer the documentation for more information.
Mayastor is subject to extensive unit, component and system-level testing throughout the development and release cycle. Resources for system-level (E2E) testing are currently provided by DataCore Software.
At this time, personnel and hardware resource limitations constrain testing by the maintainers to linux builds on x86. This reflects the primary use-case which the maintainers are currently targeting with the OpenEBS Mayastor project. Therefore, the use of Mayastor with other operating systems and/or architectures, if even possible, should be considered serendipitous and wholly experimental.
This release has been subject to End-to-End testing under Ubuntu 20.04.5_LTS (kernel: ubuntu-5.15.0-50-generic)
Mayastor user documentation, including a quick deployment guide, can be found here
If you are having issues during installation, configuration or upgrade, you can contact us via:
As described in the section on software testing above, the maintainers build and test Mayastor only on linux, on x86-64. The use of Mayastor in other environments is therefore not necessarily possible, at least without modification. Where possible, this is currently largely coincidental - it is not "fully" tested and therefore this should be considered an entirely experimental use-case.
The maintainers will be pleased to receive contributions in this area, with the following understanding:
Release Date: 14th March, 2023
The 2.0.1 release is a patch release, focusing on reconciliation-based fixes for the Mayastor diskpool operator and stability fixes for the core storage. This release can be independently installed.
Mayastor is subject to extensive unit, component and system-level testing throughout the development and release cycle. Resources for system-level (E2E) testing are currently provided by DataCore Software.
At this time, personnel and hardware resource limitations constrain testing by the maintainers to linux builds on x86. This reflects the primary use-case which the maintainers are currently targeting with the OpenEBS Mayastor project. Therefore, the use of Mayastor with other operating systems and/or architectures, if even possible, should be considered serendipitous and wholly experimental.
This release has been subject to End-to-End testing under Ubuntu 20.04.5_LTS (kernel: ubuntu-5.15.0-50-generic)
Mayastor user documentation, including a quick deployment guide, can be found here
Upgrading from previous versions of Mayastor is not supported at this time. To use Mayastor version 2.0.1, a fresh install is required.
If you are having issues during installation, configuration or upgrade, you can contact us via:
As described in the section on software testing above, the maintainers build and test Mayastor only on linux, on x86-64. The use of Mayastor in other environments is therefore not necessarily possible, at least without modification. Where possible, this is currently largely coincidental - it is not "fully" tested and therefore this should be considered an entirely experimental use-case.
The maintainers will be pleased to receive contributions in this area, with the following understanding:
Release Date: 8th February 2023
The 2.0.0 release is a major release, primarily focusing on enhancing the availability, stability and observability of Mayastor, the lightning fast NVMe-based block storage solution for Kubernetes stateful workloads.
A nexus is a data structure created for every Mayastor volume, within the Mayastor instance, which acts as an NVMe controller and performs IO operations for that volume. Each Mayastor volume comprises of a single nexus and one or more replicas. Prior to the 2.0.0 version, nexus had been a single point of failure. To mitigate this, a switch-over(and fail-over) logic has been added in 2.0.0. With this Nexus switch-over/fail-over logic in place, if a nexus is unavailable either as a consequence of failure scenarios associated with software errors, faulty nodes/network paths/underlying storage, or planned events like upgrades, the nexus is recreated on a node that’s most optimal and applications are reconnected to the newly created nexus instances automatically, to ensure I/O continuity.
It should be noted that the switch-over will not happen if the node housing the nexus also contains the last (or only) healthy replica of the volume and that replica is currently inaccessible.
In 2.0.0, Mayastor introduces the ability to cordon a node, effectively preventing any resources from being provisioned on the said node. Cordoning a node is an indication to the control plane that the node is going to have something done to it which is outside of its usual mode of operation, therefore the control plane should temporarily omit it from its scheduling logic.
Node cordoning is desirable when performing operations where the creation of new resources on the node would be problematic, such as during:
In 2.0.0, Mayastor introduces the ability to drain a node, effectively removing/deleting resources from the said node, such that the node ends up in an “empty“ state. Only nexus draining is supported in this release. Node draining helps lay the foundation to support non-disruptive upgrades in the future.
A metrics exporter runs as a container in every IO-engine pod and exposes pool capacity metrics in the Prometheus format.
Implemented NodeGetVolumeStats RPC service in Mayastor CSI node-plugin. These metrics are file-system statistics and are exposed by the kubelet on each node.
The 2.0.0 release provides an option in the Mayastor kubectl plug-in to collect supportability information from the cluster for better debugging purposes. The information is collected as a bundle and includes (not limited to) Mayastor component logs, command line and system outputs, Kubernetes resource specs of Mayastor resources and dump of Mayastor etcd data. For storing and retrieving historical logs, the 2.0.0. installer configures and installs a Loki stack alongside the Mayastor components.
Mayastor installation process has been simplified in 2.0.0 with a single Helm chart, an enhancement from the prior releases where there were separate Helm charts for control-plane and data-plane.
Prior to 2.0.0, it was required to configure the etcd and api-rest as services of NodePort type in addition to opening up these ports on the firewall for accessing these services from the Mayastor kubectl-plugin, from outside the cluster. This issue has now been resolved in 2.0.0 with an enhancement to the plug-in to use HTTP and TCP port-forwarding via the kube-proxy crate.
With this release, Mayastor has moved towards a more consistent and taxonomical scheme of naming for components. This scheme follows {release name}-{class}-{object} for every component. The release name is considered from the Helm chart (default is mayastor). For eg, the rest api component is now named as mayastor-api-rest, the engine mayastor has now become mayastor-io-engine and so on.
Mayastor uses etcd as a persistent store for storing configuration and state information of its resources like pool, volume, nexus, replica, etc. The etcd cluster is created as a part of Mayastor installation. Prior to 2.0.0, a Mayastor installation was designed to exclusively use the etcd cluster, expecting no interference from other users of etcd. This could lead to data inconsistency if a user-driven standalone etcd cluster is offered to more than one active Mayastor installations.
In 2.0.0, a separate namespace is allotted for every Mayastor cluster.
An enhanced API scheme with versioning v1 has been introduced in release 2.0.0 for all IO-engine RPC operations, with backward-compatible support to older v0 versioning APIs.
With 2.0.0, Mayastor completely removes the support for iSCSI transport.
NATS is no longer needed for communication between agent-core and IO-engine components. This communication is over gRPC transport.
Mayastor is subject to extensive unit, component and system-level testing throughout the development and release cycle. Resources for system-level (E2E) testing are currently provided by DataCore Software.
At this time, personnel and hardware resource limitations constrain testing by the maintainers to linux builds on x86. This reflects the primary use-case which the maintainers are currently targeting with the OpenEBS Mayastor project. Therefore, the use of Mayastor with other operating systems and/or architectures, if even possible, should be considered serendipitous and wholly experimental.
This release has been subject to End-to-End testing under Ubuntu 20.04.5_LTS (kernel: ubuntu-5.15.0-50-generic)
Mayastor user documentation, including a quick deployment guide, can be found here
Upgrading from 1.0.x is not supported at this time. To use Mayastor version 2.0.0, a fresh install is required.
If you are having issues during installation, configuration or upgrade, you can contact us via:
As described in the section on software testing above, the maintainers build and test Mayastor only on linux, on x86-64. The use of Mayastor in other environments is therefore not necessarily possible, at least without modification. Where possible, this is currently largely coincidental - it is not "fully" tested and therefore this should be considered an entirely experimental use-case.
The maintainers will be pleased to receive contributions in this area, with the following understanding:
Release Date: 13th December 2022
This patch release contains important fixes and supportability enhancements for issues identified after the release of v1.0.4 Users of prior versions are advised to upgrade to this version. New users are advised to install only this version.
Mayastor is subject to extensive unit, component and system-level testing throughout the development and release cycle. Resources for system-level (E2E) testing are currently provided by DataCore Software.
At this time, personnel and hardware resource limitations constrain testing by the maintainers to linux builds on x86. This reflects the primary use-case which the maintainers are currently targeting with the OpenEBS Mayastor project. Therefore, the use of Mayastor with other operating systems and/or architectures, if even possible, should be considered serendipitous and wholly experimental.
This release has been subject to End-to-End testing under Ubuntu 20.04.4_LTS (kernel: ubuntu-5.15.0-50-generic)
The Pool Operator is unable to provision pools directly using a file as the backing device. The operator attempts to validate any device path supplied in the pool specification as an accessible block device attached to the corresponding Mayastor node. In the case of a file store, there is no block device and the validation fails, causing provisioning of the pool to be aborted. This will be addressed in a future release
losetup) and use the device path of the loopback device in the pool specDeploying an application pod on a worker node which hosts both Mayastor and Prometheus exporter causes that node to restart.
Mayastor user documentation, including a quick deployment guide, can be found here
Upgrades from versions of Mayastor prior to v1.0.0 are not supported. Any release earlier than v1.0.4 should be removed prior to installing this version.
Upgrades are disruptive - data plane and control plane pods must be unscheduled, and hence all applications using Mayastor volumes must be stopped prior to performing the update. The upgrade process is manual. The K8s definition files corresponding to the deployment of Mayastor components should be patched to reflect the versions of images from this release (see the /deploy) folder in the appropriate repositories and/or the GitBook Quickstart instruction for details of which definition files are affected, and check the v1.0.3 versions to find the appropriate image tag values.
If you are having issues during installation, configuration or upgrade, you can contact us via:
As described in the section on software testing above, the maintainers build and test Mayastor only on linux, on x86-64. The use of Mayastor in other environments is therefore not necessarily possible, at least without modification. Where possible, this is currently largely coincidental - it is not "fully" tested and therefore this should be considered an entirely experimental use-case.
The maintainers will be pleased to receive contributions in this area, with the following understanding:
Release Date: 10th November 2022
This patch release contains important fixes and supportability enhancements for issues identified after the release of v1.0.3 Users of prior versions are advised to upgrade to this version. New users are advised to install only this version.
Mayastor is subject to extensive unit, component and system-level testing throughout the development and release cycle. Resources for system-level (E2E) testing are currently provided by DataCore Software.
At this time, personnel and hardware resource limitations constrain testing by the maintainers to linux builds on x86. This reflects the primary use-case which the maintainers are currently targeting with the OpenEBS Mayastor project. Therefore, the use of Mayastor with other operating systems and/or architectures, if even possible, should be considered serendipitous and wholly experimental.
This release has been subject to End-to-End testing under Ubuntu 20.04.4_LTS (kernel: ubuntu-5.15.0-50-generic)
The Pool Operator is unable to provision pools directly using a file as the backing device. The operator attempts to validate any device path supplied in the pool specification as an accessible block device attached to the corresponding Mayastor node. In the case of a file store, there is no block device and the validation fails, causing provisioning of the pool to be aborted. This will be addressed in a future release
losetup) and use the device path of the loopback device in the pool specDeploying an application pod on a worker node which hosts both Mayastor and Prometheus exporter causes that node to restart.
Mayastor user documentation, including a quick deployment guide, can be found here
Upgrades from versions of Mayastor prior to v1.0.0 are not supported. Any release earlier than v1.0.3 should be removed prior to installing this version.
Upgrades are disruptive - data plane and control plane pods must be unscheduled, and hence all applications using Mayastor volumes must be stopped prior to performing the update. The upgrade process is manual. The K8s definition files corresponding to the deployment of Mayastor components should be patched to reflect the versions of images from this release (see the /deploy) folder in the appropriate repositories and/or the GitBook Quickstart instruction for details of which definition files are affected, and check the v1.0.3 versions to find the appropriate image tag values.
If you are having issues during installation, configuration or upgrade, you can contact us via:
As described in the section on software testing above, the maintainers build and test Mayastor only on linux, on x86-64. The use of Mayastor in other environments is therefore not necessarily possible, at least without modification. Where possible, this is currently largely coincidental - it is not "fully" tested and therefore this should be considered an entirely experimental use-case.
The maintainers will be pleased to receive contributions in this area, with the following understanding:
Release Date: 11th October 2022
This patch release contains important fixes and supportability enhancements for issues identified after the release of V1.0.2 Users of prior versions are advised to upgrade to this version. New users are advised to install only this version.
--diagnose-stack has been added, which allows stack trace collection during such a conditionMayastor is subject to extensive unit, component and system-level testing throughout the development and release cycle. Resources for system-level (E2E) testing are currently provided by DataCore Software.
At this time, personnel and hardware resource limitations constrain testing by the maintainers to linux builds on x86. This reflects the primary use-case which the maintainers are currently targeting with the OpenEBS Mayastor project. Therefore, the use of Mayastor with other operating systems and/or architectures, if even possible, should be considered serendipitous and wholly experimental.
This release has been subject to End-to-End testing under Ubuntu 20.04.4_LTS (kernel: ubuntu-5.13.0.27-generic)
The Pool Operator is unable to provision pools directly using a file as the backing device. The operator attempts to validate any device path supplied in the pool specification as an accessible block device attached to the corresponding Mayastor node. In the case of a file store, there is no block device and the validation fails, causing provisioning of the pool to be aborted. This will be addressed in a future release
losetup) and use the device path of the loopback device in the pool specDeploying an application pod on a worker node which hosts both Mayastor and Prometheus exporter causes that node to restart.
Mayastor user documentation, including a quick deployment guide, can be found here
Upgrades from versions of Mayastor prior to v1.0.0 are not supported. Any release earlier than v1.0.3 should be removed prior to installing this version.
Upgrades are disruptive - data plane and control plane pods must be unscheduled, and hence all applications using Mayastor must be volumes stopped prior to performing the update. The upgrade process is manual. The k8s definition files corresponding to the deployment of Mayastor components should be patched to reflect the versions of images from this release (see the /deploy) folder in the appropriate repositories and/or the GitBook Quickstart instruction for details of which definition files are affected, and check the v1.0.3 versions to find the appropriate image tag values.
If you are having issues during installation, configuration or upgrade, you can contact us via:
As described in the section on software testing above, the maintainers build and test Mayastor only on linux, on x86-64. The use of Mayastor in other environments is therefore not necessarily possible, at least without modification. Where possible, this is currently largely coincidental - it is not "fully" tested and therefore this should be considered an entirely experimental use-case.
The maintainers will be pleased to receive contributions in this area, with the following understanding:
Release Date: 4th August 2022
This patch release contains important fixes for issues identified after the release of V1.0.1 Users of V1.0.1 are advised to upgrade to this version. New users are advised to install only this version.
Mayastor is subject to extensive unit, component and system-level testing throughout the development and release cycle. Resources for system-level (E2E) testing are currently provided by DataCore Software.
At this time, personnel and hardware resource limitations constrain testing by the maintainers to linux builds on x86. This reflects the primary use-case which the maintainers are currently targeting with the OpenEBS Mayastor project. Therefore, the use of Mayastor with other operating systems and/or architectures, if even possible, should be considered serendipitous and wholly experimental.
This release has been subject to End-to-End testing under Ubuntu 20.04.4_LTS (kernel: ubuntu-5.13.0.27-generic)
The Pool Operator is unable to provision pools directly using a file as the backing device. The operator attempts to validate any device path supplied in the pool specification as an accessible block device attached to the corresponding Mayastor node. In the case of a file store, there is no block device and the validation fails, causing provisioning of the pool to be aborted. This will be addressed in a future release
losetup) and use the device path of the loopback device in the pool specDeploying an application pod on a worker node which hosts both Mayastor and Prometheus exporter causes that node to restart.
Mayastor user documentation, including a quick deployment guide, can be found here
Upgrades from versions of Mayastor prior to v1.0.0 are not supported. Any release earlier than v1.0.0 should be removed prior to installing this version.
Upgrades from v1.0.1 are disruptive - data plane and control plane pods must be unscheduled, and hence all applications using Mayastor must be volumes stopped prior to performing the update. The upgrade process is manual. The k8s definition files corresponding to the deployment of Mayastor components should be patched to reflect the versions of images from this release (see the /deploy) folder in the appropriate repositories and/or the GitBook Quickstart instruction for details of which definition files are affected, and check the v1.0.1 versions to find the appropriate image tag values.
If you are having issues during installation, configuration or upgrade, you can contact us via:
As described in the section on software testing above, the maintainers build and test Mayastor only on linux on x86-64. The use of Mayastor in other environments is therefore not necessarily possible, at least without modification. Where possible, this is currently largely coincidental - it is not "fully" tested and therefore this should be considered an entirely experimental use-case.
The maintainers will be pleased to receive contributions in this area, with the following understanding:
Release Date: 22nd February 2022
This patch release contains important fixes for issues identified after the release of V1.0.0 Users of V1.0.0 are advised to upgrade to this version. New users are advised to install only V1.0.1
Mayastor is subject to extensive unit, component and system-level testing throughout the development and release cycle. Resources for system-level (E2E) testing are currently provided by DataCore Software.
At this time, personnel and hardware resource limitations constrain testing by the maintainers to linux builds on x86. This reflects the primary use-case which the maintainers are currently targeting with the OpenEBS Mayastor project. Therefore, the use of Mayastor with other operating systems and/or architectures, if even possible, should be considered serendipitous and wholly experimental.
This release has been subject to End-to-End testing under Ubuntu 20.04.3_LTS (kernel: ubuntu-5.13.0.27-generic)
The Pool Operator is unable to provision pools directly using a file as the backing device. The operator attempts to validate any device path supplied in the pool specification as an accessible block device attached to the corresponding Mayastor node. In the case of a file store, there is no block device and the validation fails, causing provisioning of the pool to be aborted. This will be addressed in a future release
losetup) and use the device path of the loopback device in the pool specDeploying an application pod on a worker node which hosts both Mayastor and Prometheus exporter causes that node to restart.
Mayastor user documentation, including a quick deployment guide, can be found here
Upgrades from versions of Mayastor prior to V1.0.0 are not supported. Any earlier release should be removed prior to installing this version.
If you are having issues during installation, configuration or upgrade, you can contact us via:
As described in the section on software testing above, the maintainers build and test Mayastor only on linux on x86-64. The use of Mayastor in other environments is therefore not necessarily possible, at least without modification. Where possible, this is currently largely coincidental - it is not "fully" tested and therefore this should be considered an entirely experimental use-case.
The maintainers will be pleased to receive contributions in this area, with the following understanding:
Release Date: 19th January 2022 Codename: New Horizons
With this release Mayastor is considered graduated from pre-release / beta status.
Complete reimplementation of the control plane, focused on modularity, future extensibility and scaling
Extensive bug fixing and stabilization
New Mayastor kubectl plugin for management and monitoring
This release has been subject to E2E system testing by MayaData / DataCore Software, running under Ubuntu 20.04.3_LTS (kernel: ubuntu-5.8.0.63-generic)
A Mayastor container may fault and restart if a disk device used by one of its associated disk pools becomes inaccessible. This is currently under investigation, with the intention of providing a fix in the next release.
Deploying an application pod on a worker node which hosts both Mayastor and Prometheus exporter causes that node to restart.
Mayastor user documentation, including a quick deployment guide, can be found here
Upgrades from previous versions of Mayastor are not supported. Any earlier release should be removed prior to installing this version. It is intended that releases forward of v1.0.0 will allow in-place upgrades.
If you are having issues during installation, configuration or upgrade, you can contact us via:
Various stability fixes