Kruise Versions Save

To install or upgrade to the old version, see installation doc.

Change log since v1.2.0

New CRD and Controller: PodProbeMarker

Kubernetes provides three Pod lifecycle management:

• Readiness Probe Used to determine whether the business container is ready to respond to user requests. If the probe fails, the Pod will be removed from Service Endpoints.
• Liveness Probe Used to determine the health status of the container. If the probe fails, the kubelet will restart the container.
• Startup Probe Used to know when a container application has started. If such a probe is configured, it disables liveness and readiness checks until it succeeds.

So the Probe capabilities provided in Kubernetes have defined specific semantics and related behaviors. In addition, there is actually a need to customize Probe semantics and related behaviors, such as:

• GameServer defines Idle Probe to determine whether the Pod currently has a game match, if not, from the perspective of cost optimization, the Pod can be scaled down.
• K8S Operator defines the main-secondary probe to determine the role of the current Pod (main or secondary). When upgrading, the secondary can be upgraded first, so as to achieve the behavior of selecting the main only once during the upgrade process, reducing the service interruption time during the upgrade process.

So we provides the ability to customize the Probe and return the result to the Pod yaml.

For more detail, please refer to its documentation and proposal.

SidecarSet

• SidecarSet support to inject pods under kube-system,kube-public namespace. (#1084, @zmberg)
• SidecarSet support to inject specific history sidecar container to Pods. (#1021, @veophi)
• SidecarSet support to inject pod annotations.(#992, @zmberg)

AdvancedCronJob

• TimeZone support for AdvancedCronJob from upstream. (#1070, @FillZpp)

WorkloadSpread

• WorkloadSpread support Native StatefulSet and Kruise Advanced StatefulSet. (#1056, @veophi)

CloneSet

• CloneSet supports to calculate scale number excluding Pods in PreparingDelete. (#1024, @FillZpp)
• Optimize CloneSet queuing when cache has just synced. (#1026, @FillZpp)

PodUnavailableBudget

• Optimize event handler performance for PodUnavailableBudget. (#1027, @FillZpp)

Advanced DaemonSet

• Allow optional filed max unavilable in ads, and set default value 1. (#1007, @ABNER-1)
• Fix DaemonSet surging with minReadySeconds. (#1014, @FillZpp)
• Optimize Advanced DaemonSet internal new pod for imitating scheduling. (#1011, @FillZpp)
• Advanced DaemonSet support pre-download image. (#1057, @ABNER-1)

Advanced StatefulSet

• Fix panic cased by statefulset pvc auto deletion. (#999, @veophi)

Others

• Optimize performance of LabelSelector conversion. (#1068, @FillZpp)
• Reduce kruise-manager memory allocation. (#1015, @FillZpp)
• Pod state from updating to Normal should all hooked. (#1022, @shiyan2016)
• Fix go get in Makefile with go 1.18. (#1036, @astraw99)
• Fix EphemeralJob spec.replicas nil panic bug. (#1016, @hellolijj)
• Fix UnitedDeployment reconcile don't return err bug. (#991, @huiwq1990)

To install or upgrade to the old version, see installation doc.

Changes since v1.1.0

New CRD and Controller: PersistentPodState

With the development of cloud native, more and more companies start to deploy stateful services (e.g., Etcd, MQ) using Kubernetes. K8S StatefulSet is a workload for managing stateful services, and it considers the deployment characteristics of stateful services in many aspects. However, StatefulSet persistent only limited pod state, such as Pod Name is ordered and unchanging, PVC persistence, and can not cover other states, e.g. Pod IP retention, priority scheduling to previously deployed Nodes.

So we provide PersistentPodState CRD to persistent other states of the Pod, such as "IP Retention".

For more detail, please refer to its documentation and proposal.

CloneSet

• Ensure at least one pod is upgraded if CloneSet has partition < 100% (Behavior Change). (#954, @veophi)
• Add expectedUpdatedReplicas field into CloneSet status. (#954 & #963, @veophi)
• Add markPodNotReady field into lifecycle hook to support marking Pod as NotReady during preparingDelete or preparingUpdate. (#979, @veophi)

StatefulSet

• Add markPodNotReady field into lifecycle hook to support marking Pod as NotReady during preparingDelete or preparingUpdate. (#979, @veophi)

PodUnavailableBudget

• Support to protect any custom workloads with scale subresource. (#982, @zmberg)
• Optimize performance in large-scale clusters by avoiding DeepCopy list. (#955, @zmberg)

Others

• Remove some commented code and simplify some. (#983, @hezhizhen)
• Sidecarset forbid updating of sidecar container name. (#937, @adairxie)
• Optimize the logic of listNamespacesForDistributor func. (#952, @hantmac)

Thanks to all our contributors! 😊

To install or upgrade to the old version, see installation doc.

Changes since v1.0.1

Project

• Bump Kubernetes dependencies to 1.22 and controller-runtime to v0.10.2. (#915, @FillZpp)
• Disable DeepCopy for some specific cache list. (#916, @FillZpp)

InPlace Update

• Support in-place update containers with launch priority, for workloads that supported in-place update, e.g., CloneSet, Advanced StatefulSet. (#909, @FillZpp)

CloneSet

• Add pod-template-hash label into Pods, which will always be the short hash. (#931, @FillZpp)
• Support pre-download image after a number of updated pods has been ready. (#904, @shiyan2016)
• Make maxUnavailable also limited to pods in new revision. (#899, @FillZpp)

SidecarSet

• Support shared volumes in init containers. (#929, @outgnaY)
• Support transferEnv in init containers. (#897, @pigletfly)
• Optimize the injection for pod webhook that checks container exists. (#927, @zmberg)
• Fix validateSidecarConflict to avoid a same sidecar container exists in multiple sidecarsets. (#884, @pigletfly)

Advanced DaemonSet

• Refactor daemonset controller and fetch upstream codebase. (#883, @FillZpp)
• Support preDelete lifecycle for both scale down and recreate update. (#923, @FillZpp)
• Fix node event handler that should compare update selector matching changed. (#920, @LastNight1997)
• Optimize dedupCurHistories func in ReconcileDaemonSet. (#912, @LastNight1997)

Advanced StatefulSet

• Support StatefulSetAutoDeletePVC feature. (#882, @veophi)

Kruise-daemon

• Support CRI-O and any other common CRI types. (#930, @diannaowa) & (#936, @FillZpp)

Other

• Add kruise_manager_is_leader metric. (#917, @hatowang)

Thanks to all our contributors! 😊

To install or upgrade to the old version, see installation doc.

Changes since v1.0.0

SidecarSet

• Fix panic when SidecarSet manages Pods with sidecar containers that have different update type. (#850, @veophi)
• Fix log error when extract container from fieldpath failed. (#860, @pigletfly)
• Optimization logic of determining whether the pod state is consistent logic. (#854, @dafu-wu)
• Replace reflect with generation in event handler. (#885, @zouyee)
• Store history revisions for sidecarset. (#715, @veophi)

Advanced StatefulSet

• Allow updating asts RevisionHistoryLimit. (#864, @shiyan2016)
• StatefulSet considers non-available pods when deleting pods. (#880, @hzyfox)

ResourceDistribution

• Improve controller updating and watching logic. (#861, @veophi)

CloneSet

• Break the loop when it finds the current revision. (#887, @shiyan2016)
• Remove duplicate register fieldindexes in cloneset controller. (#888 & #889, @shiyan2016)
• CloneSet refresh pod states before skipping update when paused (Behavior Change). (#893, @FillZpp)

PullImageJob

• Update containerd client usage and event handler for controller. (#894, @FillZpp)

Thanks to all our contributors! 😊

To install or upgrade to the old version, see installation doc.

Changes since v0.10.1

SidecarSet

• Add SourceContainerNameFrom and EnvNames in sidecarset transferenv.

Advanced StatefulSet

• Fix update expectation to be increased when a pod updated.

WorkloadSpread

• Fix bug: read conditions from nil old subset status.

Other

• Do not set timeout for webhook ready.

Thanks to all our contributors! 😊

To install or upgrade to the stable version, see installation doc.

Changes since v0.10.1

Project

• Bump CustomResourceDefinition(CRD) from v1beta1 to v1
• Bump ValidatingWebhookConfiguration/MutatingWebhookConfiguration from v1beta1 to v1
• Bump dependencies: k8s v1.18 -> v1.20, controller-runtime v0.6.5 -> v0.8.3
• Generate CRDs with original controller-tools and markers

So that Kruise can install into Kubernetes 1.22 and no longer support Kubernetes < 1.16.

New feature: in-place update with env from metadata

When update spec.template.metadata.labels/annotations in CloneSet or Advanced StatefulSet and there exists container env from the changed labels/annotations, Kruise will in-place update them to renew the env value in containers.

doc

New feature: ContainerLaunchPriority

Container Launch Priority provides a way to help users control the sequence of containers start in a Pod.

It works for Pod, no matter what kind of owner it belongs to, which means Deployment, CloneSet or any other Workloads are all supported.

doc

New feature: ResourceDistribution

For the scenario, where the namespace-scoped resources such as Secret and ConfigMap need to be distributed or synchronized to different namespaces, the native k8s currently only supports manual distribution and synchronization by users one-by-one, which is very inconvenient.

Therefore, in the face of these scenarios that require the resource distribution and continuously synchronization across namespaces, we provide a tool, namely ResourceDistribution, to do this automatically.

Currently, ResourceDistribution supports the two kind resources --- Secret & ConfigMap.

doc

CloneSet

• Add maxUnavailable field in scaleStrategy to support rate limiting of scaling up.
• Mark revision stable as currentRevision when all pods updated to it, won't wait all pods to be ready (Behavior Change).

WorkloadSpread

• Manage the pods that were created before WorkloadSpread.
• Optimize webhook update and retry during injection.

PodUnavailableBudget

• Add pod no pub-protection annotation.
• PUB controller watch workload replicas changed.

Advanced DaemonSet

• Support in-place update daemon pod.
• Support progressive annotation to control if pods creation should be limited by partition.

SidecarSet

• Fix SidecarSet filter active pods.

UnitedDeployment

• Fix pod NodeSelectorTerms length 0 when UnitedDeployment NodeSelectorTerms is nil.

NodeImage

• Add --nodeimage-creation-delay flag to delay NodeImage creation after Node ready.

Other

• Kruise-daemon watch pods using protobuf.
• Export resync seconds args.
• Fix http checker reload ca.cert.
• Fix E2E for WorkloadSpread, ImagePulling, ContainerLaunchPriority.

Thanks to all our contributors! 😊

To install or upgrade to the latest version including alpha/beta/rc, see installation doc.

Changes since v1.0.0-alpha.2

New feature: ResourceDistribution

For the scenario, where the namespace-scoped resources such as Secret and ConfigMap need to be distributed or synchronized to different namespaces, the native k8s currently only supports manual distribution and synchronization by users one-by-one, which is very inconvenient.

Therefore, in the face of these scenarios that require the resource distribution and continuously synchronization across namespaces, we provide a tool, namely ResourceDistribution, to do this automatically.

Currently, ResourceDistribution supports the two kind resources --- Secret & ConfigMap.

doc

CloneSet

• Add maxUnavailable field in scaleStrategy to support rate limiting of scaling up.
• Mark revision stable when all pods updated to it, won't wait all pods to be ready.

Advanced DaemonSet

• Support progressive annotation to control if pods creation should be limited by partition.

UnitedDeployment

• Fix pod NodeSelectorTerms length 0 when UnitedDeployment NodeSelectorTerms is nil.

Thanks to all our contributors! 😊

To install or upgrade to the latest version including alpha/beta/rc, see installation doc.

Changes since v1.0.0-alpha.1

Project

• Generate CRDs with original controller-tools and markers

WorkloadSpread

• Add discoveryGVK for WorkloadSpread

NodeImage

• Add --nodeimage-creation-delay flag to delay NodeImage creation after Node ready

Other

• Fix E2E for WorkloadSpread, ImagePulling, ContainerLaunchPriority

Thanks to all our contributors! 😊

To install or upgrade to the latest version including alpha/beta/rc, see installation doc.

Changes since v0.10.0

WorkloadSpread

• Add discoveryGVK for WorkloadSpread
• Optimize webhook injection

Kruise-daemon

• Setup generic kubeClient with Protobuf

Other

• Fix E2E for WorkloadSpread, ImagePulling

Thanks to all our contributors! 😊

To install or upgrade to the latest version including alpha/beta/rc, see installation doc.

Changes since v0.10.0

Project

• Bump CustomResourceDefinition(CRD) from v1beta1 to v1
• Bump ValidatingWebhookConfiguration/MutatingWebhookConfiguration from v1beta1 to v1
• Bump dependencies: k8s v1.18 -> v1.20, controller-runtime v0.6.5 -> v0.8.3

So that Kruise can install into Kubernetes 1.22 and no longer support Kubernetes < 1.16.

New feature: in-place update with env from metadata

When update spec.template.metadata.labels/annotations in CloneSet or Advanced StatefulSet and there exists container env from the changed labels/annotations, Kruise will in-place update them to renew the env value in containers.

doc

New feature: ContainerLaunchPriority

Container Launch Priority provides a way to help users control the sequence of containers start in a Pod.

It works for Pod, no matter what kind of owner it belongs to, which means Deployment, CloneSet or any other Workloads are all supported.

doc

WorkloadSpread

• Manage the pods that were created before WorkloadSpread.
• Optimize webhook update and retry during injection.

PodUnavailableBudget

• Add pod no pub-protection annotation.
• PUB controller watch workload replicas changed.

Advanced DaemonSet

• Support in-place update daemon pod.

SidecarSet

• Fix SidecarSet filter active pods.

Other

• Kruise-daemon watch pods using protobuf.
• Export resync seconds args.
• Fix http checker reload ca.cert.

Thanks to all our contributors! 😊