Kubespray Versions Save

Deploy a Production Ready Kubernetes Cluster

v2.25.0

3 weeks ago

Deprecation / Removal

  • Remove support for Kubernetes 1.26.x (move min version to 1.27.x) (#10817, @KubeKyrie)
  • Remove documentation for removed in-tree openstack provider (#10889, @LarssonOliver)

Feature / Major Changes

  • A check is introduced to fail the playbook if cgroups are not enabled on the node (#11165, @franznemeth)
  • Add Calico v3.27.3 and make it default (#11141, @pomland-94)
  • Add extra_vars support to vagrant setup (#10932, @VannTen)
  • Add kube-vip LeaderElection variables vip_leaseduration, vip_renewdeadline, vip_retryperiod options for kube-vip (#11021, @KubeKyrie)
  • Add new option remove_anonymous_access to prevent granting RBAC permissions to anonymous users. (#11016, @nicolas-goudry)
  • Add scheduler plugins support (scheduler_plugins_enabled enable or disable the installation scheduler plugins / scheduler_plugins_enabled_plugins describe the enabled plugins / scheduler_plugins_diabled_plugins describe the disabled plugins / scheduler_plugins_plugin_config set the custom config for enabled plugins) (#10747, @tu1h)
  • Added a config option to filter ntp interfaces (#11066, @Pavan-Gunda)
  • Adding egress IPv6 for node-local-dns queries (k8s_allowed_egress_ipv6_ips) (#10396, @raviranjanelastisys)
  • Bump docker version for kylin linux (#11203, @ErikJiang)
  • Bump docker version for openeuler linux (#11206, @ErikJiang)
  • Update almalinux-8 base image to 8.9 (#10918, @VannTen)
  • Bumping checksums and various versions (#10999, @MrFreezeex)
  • Containerd: allow to configure fallback server (#10988, @sathieu)
  • Docker upgrade from 24.0 to 26.1 (#11198, @tico88612)
  • Download hash script: auto discover versions (#10849, @VannTen)
  • Enable configuring mountOptions, reclaimPolicy and volumeBindingMode for cinder-csi StorageClasses. (#10450, @Payback159)
  • Make containerd v1.7.15 default (#11083, @Payback159)
  • Make kubernetes v1.28.6 default (#10810, @mzaian)
  • Make kubernetes v1.29.1 default Remove SecCompDefault feature gate from hardening configuration for kubernetes 1.29 (#10820, @tmurakam)
  • Make kubernetes v1.29.2 default (#10919, @mzaian)
  • Make kubernetes v1.29.3 default (#11035, @mzaian)
  • Make kubernetes v1.29.4 default (#11108, @mzaian)
  • Make kubernetes v1.29.5 default (#11196, @mzaian)
  • Metallb: added metallb_namespace variable to parameterize namespace (#11136, @oik741)
  • OpenStack Cloud Controller Manager upgrade to 1.28.2 (#11174, @tico88612)
  • Opensuse deployment is now tested in CI. (#11159, @VannTen)
  • Add selinux-ng repo in Amazon Linux to install container-selinux (#11182, @yankay)
  • Add CI Image for Ubuntu 24.04 (#11167, @yankay)
  • Allows .vagrant folder location to be configured (#10718, @kri5)
  • Prevent nodelocaldns to be OOM-killed (#11056, @sathieu)
  • Support Node Feature Discovery (#10861, @yankay)
  • Support Ubuntu 24.04 (#11132, @tico88612)
  • Support following k8s version selection pause image (#10756, @my-git9)
  • The variable old_dns_domains (list) can be used for backward compatibility when changing dns_domain (#10630, @VannTen)
  • Update external huawei cloud controller to 0.26.6 (#10824, @dabeck)
  • Update external huawei cloud controller to 0.26.8 (#11172, @dabeck)
  • Update kube-vip to v0.8.0 (#11156, @jisnardo)
  • Update metrics server to v0.7.0 (#10856, @mzaian)
  • Updated ingress controller version to 1.9.6 (#10868, @kundan2707)
  • User has a possibility to modify Service type with "ingress_nginx_service_type" property in addons. (#10925, @chrxmvtik)
  • [Terraform-openstack] Added possibility to build an octavia loadbalancer for the Kubernetes Api. (#10924, @jaszil)
  • [containerd] added distributed tracing config variables for containerd (containerd_tracing_enabled,containerd_tracing_endpoint,containerd_tracing_protocol, containerd_tracing_sampling_ratio,containerd_tracing_service_name ); it is disabled by default. (#11103, @ugur99)
  • [download] add capability to specify alternative download mirrors for files (#8474, @cristicalin)
  • [etcd] Default version to 3.5.12 for k8s 1.27 , 1.28 , 1.29 (#11036, @mzaian)
  • Minimum ansible-core version is now 2.16.4 (#10984, @VannTen)
  • Remove the archived debian apt repository when installing docker-engine (#11088, @yankay)
  • Change dependbot interval to weekly (#11189, @yankay)
  • Allow specifying CPU Manager Policy options through kubelet_cpu_manager_policy_options (#11023, @derselbst)
  • [kube-apiserver] added distributed tracing config variables for kube-apiserver (kube_apiserver_tracing,kube_apiserver_tracing_endpoint,kube_apiserver_tracing_sampling_rate_per_million); it is disabled by default. [kubelet] added distributed tracing config variables for kubelet (kubelet_tracing,kubelet_tracing_endpoint,kubelet_tracing_sampling_rate_per_million); it is disabled by default. (#10795, @ugur99)

Applications

  • [argocd] update argocd to v2.11.0 (#11193, @mzaian)
  • [helm] Upgrade to v3.14.2 (#10967, @cleman95)
  • Bump coredns version to 1.11.1 (#10719, @batazor)
  • Support CoreDNS use host network & config CoreDNS port (#10617, @liuxu623)
  • Fix secondary coredns missing var (#10821, @VannTen)
  • Revert "support CoreDNS use host network and config dns port (#10617)" (#11185, @VannTen)
  • dns_mode: coredns_dual is now tested in CI. (#10903, @VannTen)

Network

  • Adds support for cilium v1.15
    • Adds support for cilium_l2announcements to replace metallb with cilium l2 announcements, defaults to false
    • Adds support for cilium_loadbalancer_mode to switch bpf-lb-mode between snat, dsr or hybrid, default to snat (#11106, @deveshk0)
  • Adds the option to install calico 3.27.3 (#11059, @danielfrg)
  • [calico] Update default calico to v3.27.2 (#10960, @mzaian)

Container-Managers

  • crictl stop container grace period, cri_stop_containers_grace_period: 0 (#10651, @krembu)
  • Update the docker default version to 24.0 (#10873, @yankay)
  • [Containerd] Enable by default discard_unpacked_layers to save some space (see https://github.com/containerd/containerd/discussions/6295) (#10905, @VannTen)
  • [Nerdctl] Upgrade to version 1.7.4 (#10968, @cleman95)
  • [containerd] Make containerd 1.7.13 default [runc] Upgrade to v1.1.12 (#10862, @KubeKyrie)
  • [containerd] Make containerd 1.7.16 default (#11142, @mzaian)

API Change

  • Make proxy protocol in Upcloud LB configurable (#10971, @davidumea)

Design

  • Merge stop and remove systemd service task in reset/tasks/main.yml (#10902, @kimsehwan96)

Documentation

  • Add documentation for configuring nat outgoing ipv6 (#10866, @anders-elastisys)
  • Add new OpenStack Cloud for terraform (#10910, @DragomirAlin)
  • BREAKING CHANGE: This script is introduced to facilitate living documentation and its administration. This leads to a restructuring in the documentation at https://kubespray.io/#/ to simplify the automatic creation of links, as the structure in the sidebar changes. (#11128, @Payback159)
  • Change a task name Ensure kube-bench parameters are set into Ensure kubelet expected parameters are set in roles/kubernetes/preinstall/tasks/0080-system-configurations.yml for a clearer understanding of its operation (#11171, @kimsehwan96)
  • Do not disable SELinux surreptitiously (#10920, @rptaylor)
  • Doc clarification: skipping patches releases is OK (#10850, @VannTen)
  • Docs: vagrant-libvirt is tested in CI (#10847, @VannTen)
  • Explicit private/public nature of *ip vars (#10904, @VannTen)
  • Fix typo in vagrant.md (#10836, @kundan2707)
  • Fix typo mistake in roles/kubernetes/control-plane/tasks/define-first-kube-control.yml (#10835, @kimsehwan96)
  • Fixed typos in inventory/sample/group_vars/k8s_cluster (#10911, @arahmangulov)
  • Kubespray used as a collection will have the correct collection version (#10727, @VannTen)
  • Make large-deployments.md link to downloads.md (#10840, @spantaleev)
  • Removed not needed graduated feature gates. (#10448, @Smidra)
  • Update upgrades.md with serial=1 for rolling updates (#10837, @titansmc)
  • Variable cilium_ipsec_key must be base64 encoded (#10781, @ledroide)

Bug or Regression

  • Added an optional variable (cni_bin_owner) to allow the user to set a different owner for /opt/cni/bin/ and it's contents. (#10929, @Rickkwa)
  • Change the position of the containerd_extra_args parameter to enhance its universality. (#11013, @qcu266)
  • Configure crio container runtime to use kube reserved cgroup (#11028, @pedro-peter)
  • Don't overwrite changes to openstack allowed_address_pairs #10760 (#10760, @rptaylor)
  • Download cache directory permissions are no longer reset recursively (#10900, @VannTen)
  • Fix ClusterRole for Calico >=v1.26.x with Calico API Server installed (#11089, @RaSerge)
  • Fix ansible parameter ssh_args in ansible.cfg file not work (#10981, @joy717)
  • Fix boostrap for Amazon Linux (#11139, @VannTen)
  • Fix crio registries config file when using slashes in the registry path (#11030, @pedro-peter)
  • Fix file loss during download (#10779, @ErikJiang)
  • Fix kubespray-defaults: Check for boostrap-os FQCN (#11073, @KubeKyrie)
  • Fix local path provisioner image repo in sample inventory. (#11180, @tico88612)
  • Fix logical error when checking for boostrap-os (#10867, @VannTen)
  • Fix lsattr command error when kubelet has symbolic link (#11074, @KubeKyrie)
  • Fix network manage service of Debian 12 (#11058, @KubeKyrie)
  • Fix nginx controller leader election RBAC (#10913, @VannTen)
  • Fix python regex matching problem when finding docker packages (#11075, @KubeKyrie)
  • Fix waiting for MetalLB controller (#10858, @flxbwr)
  • Fix(kubernetes): taint nodes on cluster upgrade (#10705, @maxime1907)
  • Fix: config hostname as string type in kubeadmConfig rendering (#10997, @ErikJiang)
  • Fixes running recover-control-plane.yml with offline broken etcd nodes. (#10660, @yuha0)
  • Revert OCCM standard dnsPolicy to ClusterFirst to fix #10914 which was introduced with #10618 and make dnsPolicy configurable to furthermore support #10618 (#11168, @Payback159)
  • Force update helm repo if exists on host (#11043, @LuckySB)
  • Kubespray ansible version checks are now performed even when running with --limit (#10908, @VannTen)
  • None. (#11061, @bmelbourne)
  • Revert crictl version (#11042, @ErikJiang)
  • The script manage-offline-container-images.sh now supports additional environment variables, e.g. it is now possible use the script to pull images listed in a file instead of checking images in a running cluster. (#10857, @anders-elastisys)
  • Update Snapshot Controller to 7.0.2 for all supported Kubernetes versions. (#11041, @jess-sol)
  • Workaround for terraform bug related to no_floating and extra_groups (#10764, @rptaylor)
  • [etcd] fixes wrong distributed tracing flag for etcd (#11175, @ugur99)
  • Correct the POLY1305 cipher suites by adding the suffix _SHA256 (#10641, @yckaolalala)

Other (Cleanup or Flake)

  • Allow scripts/download_hash.sh to complete if new versions locations are changed of certain binaries (#10998, @MrFreezeex)
  • Ddownload: Remove deleted kubeadm config field (#10931, @VannTen)
  • Download_file playbook - Incorrect description of a task (#10875, @gianmarco-mameli)
  • EventRecordQPS (in kubelet config) now uses Kubernetes default value (50) (#10826, @VannTen)
  • Fixup galaxy.yml (#10906, @VannTen)
  • Ipaddr deprecation cleanup (using fqdn ansible.utils) (#10822, @VannTen)
  • Kubeadm images are now only downloaded where needed (#10899, @VannTen)
  • Move perma failing jobs in CI to manual run (#10886, @VannTen)
  • Remove dead link in README for debian jessie (#10827, @VannTen)
  • Remove mirwan from approvers (#10930, @VannTen)
  • Update KUBESPRAY_VERSION for v2.24.0 (#10811, @yankay)
  • Update KUBESPRAY_VERSION for v2.24.1 (#10962, @yankay)
  • Update cinder-csi from 1.22.0 to 1.29.0
  • Use the new container registry for cinder-csi (#10894, @Krast76)

Supported Components

Known issues

N/A

v2.24.1

3 months ago

Changes by Kind

Feature

  • Make kubernetes v1.28.6 default (#10810, @mzaian)

Bug or Regression

  • Add configuration to create cilium CNI plugin file when cilium>=1.14.0 (#10945, @cleman95 )
  • Fix logical error when checking for boostrap-os (#10953, @VannTen)
  • Make containerd 1.7.13 default Make runc 1.1.12 default Patch GHSA-xr7r-f8xq-vfvv (#10877, @VannTen)

Other (Cleanup or Flake)

  • Bump galaxy version before release (#10890, @VannTen)

The release intend to address GHSA-xr7r-f8xq-vfvv

v2.22.2

4 months ago

Changes by Kind

Network

  • [calico] Use calico_pool_blocksize from cluster when existing (#10516, @VannTen)

API Change

  • Make kubernetes 1.26.11 default (#10704, @VannTen)

Feature

  • Add hashes for kubernetes version 1.26.6, 1.26.7, 1.26.8 & 1.26.9 (#10444, @bozzo)
  • Don't let find search filesystem mounts in docker build run step (#10131, @tomodachi)
  • Make kubernetes 1.26.13 the default version (#10823, @VannTen)

Failing Test

  • Bump vagrant version 2.3.7 (#10789, @yankay)

Bug or Regression

  • Fix hardcoded pod infra version (#10805, @ErikJiang)
  • Make containerd 1.7.13 default Make runc 1.1.12 default Patch GHSA-xr7r-f8xq-vfvv (#10878, @VannTen)
  • [Multus] Fix loop_control template error when item is None (#10347, @nicolas-goudry)

The release intend to address GHSA-xr7r-f8xq-vfvv

v2.23.3

4 months ago

Changes by Kind

Feature

  • Update kubernetes default version to 1.27.10 (#10876, @VannTen)

Bug or Regression

  • Fix hardcoded pod infra version (#10806, @ErikJiang)
  • Make containerd 1.7.13 default Make runc 1.1.12 default Make kubernetes 1.27.10 default Patch GHSA-xr7r-f8xq-vfvv (#10876, @VannTen)

Other (Cleanup or Flake)

  • Update KUBESPRAY_VERSION in galaxy.yml and Readme for v2.23.2 (#10801, @yankay)

The release intend to address GHSA-xr7r-f8xq-vfvv

v2.24.0

4 months ago

Deprecation / Removal

  • Migrate node-role.kubernetes.io/master to node-role.kubernetes.io/control-plane (#10464, @unai-ttxu)
  • Drop support for Kubernetes 1.25.x (move min version to 1.26.x) (#10420, @yankay)
  • Drop installation notes for Debian Jessie (#10642, @jelmer)

Feature / Major Changes

  • Make kubernetes v1.28.6 default (#10810, @mzaian)
  • Add kubernetes v1.28.0, v1.28.1, v1.28.2, v1.28.3, v1.28.4, v1.28.5 hash (#10435, #10541, #10739, @mzaian ; #10390, @tmurakam ; #10624, @tmurakam)
  • Add Retry for Applying PriorityClass (#10469, @hangscer8)
  • Add option crio_criu_support_enabled to enable container forensic analysis (#10479, @tu1h)
  • Add option kubectl_alias to set bash alias of kubectl (#10552, @tu1h)
  • Add variable to configure ipvs modules (kube_proxy_ipvs_modules) (#10580, @borgiacis)
  • Check nameserver only when dns is enable (#10561, @yckaolalala)
  • Correctly handle remove_default_searchdomains when value is undefined (#10533, @yckaolalala)
  • Kube-scheduler: remove/update deprecated component component config v1beta3. (#10484, @mzaian)
  • Terraform-aws: variable driven ami selection (ami_name_pattern/ami_virtualization_type/ami_owners) (#10520, @mertcancam)
  • Terraform-openstack: Added possibility to enable dhcp flag critical on one interface (#10446, @Xartos)
  • This will introduce a new variable kube_apiserver_admission_plugins_podnodeselector_default_node_selector that can be used with kube_apiserver_admission_plugins_needs_configuration: [PodNodeSelector] defined. So allows the users to configure PodNodeSelector plugin. (#10607, @titansmc)
  • UpCloud: Terraform provider updated to v2.12.0. Server groups with strict anti-affinity (move var from anti_affinity_policy to anti_affinity) (#10474, @robinAwallace)
  • Update dockerfile to follow best practices (#10708, @maxime1907)
  • Update to ansible 2.15 and set minimum version to 2.15.5 (#10481, @MrFreezeex)
  • [etcd] Update Default etcd version to 3.5.10 for kubernetes 1.28, 1.27 and 1.26 (#10798, @VannTen)
  • [etcd] update version to 3.5.9 for k8s 1.28 , 1.27 , 1.26 (#10482, @mzaian)
  • [etcd] add 3.5.10 hashes (#10566, @mzaian)
  • [vsphere_csi] Update to 3.1.0 supports Kubernetes Version 1.28 (#10451, @mzaian)
  • [cinder_csi] Cinder-CSI now use cluster_name variable instead of the default hardcoded "kubernetes" value (#10422, @floryut)

Applications

  • [argocd] update argocd to v2.8.4 (#10568, @mzaian)
  • [helm] upgrade to 3.13.1 (#10567, @mzaian)
  • [coredns] Added option coredns_additional_error_config to allow for configuration of the coredns error plugin. (#10501, @Elias-elastisys)
  • [coredns] Support CoreDNS use host network & config CoreDNS port (#10617, @liuxu623)
  • [coredns] Support disable dns autoscaler when use CoreDNS (#10608, @liuxu623)
  • [coredns] Add pdb to coredns (#10557, @lobiyedKarim1)
  • [cert-manager] upgrade to v1.13.2 (#10616, @liuxu623)
  • [cert-manager] Upgrade to v1.12.6 (#10582, @chansuke)
  • [cert-manager] Upgrade to v1.12.5 (#10500, @chansuke)

Network

  • [cilium] Fix invalid hubble yaml if cilium_hubble_tls_generate is enabled (#10430, @toonalbers)
  • [cilium] Use correct ports in cilium metrics services if metrics are enabled. (#10519, @bakito)
  • [cilium] Adds support for deploying clusters with cilium 1.14+ (#10684, @rl0nergan)
  • [calico] Separate calico-node and calico-cni-plugin service accounts and update default calico to v3.26.1 (#10416, @mzaian)
  • [calico] Use calico_pool_blocksize from cluster when existing (#10516, @VannTen)
  • [calico] Update default calico to v3.26.3 (#10526, @mzaian)
  • [calico] Update default calico to v3.26.4 (#10669, @mzaian)
  • [kube-router] Default kube-router version updated to v2.0.0 (#10503, @bozzo)
  • [kube-router] Default kube-router version updated to v1.6.0 (#10478, @bozzo)
  • [kube-router] Add kube_router_bgp_graceful_restart optional setting for disabling graceful BGP restarts (default to true) (#10489, @rosskusler)
  • [metallb] Add option to set avoidBuggyIPs in IPAddressPools and change the default back to false (#10458, @zeeZ)
  • [metallb] Metallb --lb-class cmd arg to support multiple LoadBalancer implementations (#10550, @Seal1998)
  • [custom_cni] Add helm support for custom_cni deployment (#10529, @kukacz)
  • [kube_vip] Add kube_vip_lb_fwdmethod option for kube-vip (#10762, @tu1h)

Container-Managers

  • [containerd] Fix invalid version check in containerd jinja-template config (#10620, @khanhngobackend)
  • [containerd] Make containerd 1.7.11 default (#10671, @mzaian)
  • [containerd] Add hashes for containerd versions 1.7.6 ~ 1.7.8 default (#10439, #10525, #10589, @mzaian)
  • [containerd] Specify the runc path when we use the containerd container engine and change the bin_dir path. (#10154, @qlijin)
  • [containerd] Refactor NRI activation for containerd and CRI-O (remove crio_enable_nri and containerd_nri_disable) now only one var nri_enabled default to false (#10470, @fmuyassarov)
  • [containerd] Add Boolean option enable_cdi to enable cdi (false by default) (#10603, @krembu)
  • [containerd] Add configuration option for NRI (disable by default) in crio & containerd (using new containerd_nri_disable and crio_enable_nri) (#10454, @fmuyassarov)
  • [containerd] add config support override_path (#10776, @yankay)
  • [runc] Upgrade to v1.1.10 (#10671, @mzaian)
  • [crio] Update to v1.28.1 (#10480, @qlijin)
  • [crio] Remove crio package configuration during cleanup (#10584, @yckaolalala)
  • [crio] Update docs for crio_registry_auth (#10785, @qlijin)
  • [docker] Ability to define GPG key path for Docker APT (using new variable docker_repo_key_keyring) (#10513, @emiran-orange)
  • [kata-containers] Freshens configuration-qemu to latest template compatible with kata-containers 3.1.3. (#10466, @Alphadelta14)
  • [nerdctl] Bump nerdctl version 1.7.1 (#10685, @yankay)
  • [nerdctl] Change nerdctl version from 1.5.0 to 1.6.0 (#10475, @MaGaroo)

Documentation

  • Add link to Cilium CNI documentation (#10431, @toonalbers)
  • Update docs for calico_iptables_backend in Redhat/Centos.md (#10417, @yankay)
  • Update metallb example configs (#10485, @caruccio)
  • Updated AWS ALB ingress controller version (#10680, @kundan2707)

Bug or Regression

  • Add a variable reset_restart_network_service_name in the reset role to be able to configure the name of the service which is restarted. (#10428, @RomainMou)
  • Add dnsPolicy: ClusterFirstWithHostNet to DaemonSets with hostNetwork: true (#10618, @Payback159)
  • Check for correct conntrack module presence, regardless of kernel versions (#10662, @VannTen)
  • Fallback_ips: ignore unreachable hosts (#10601, @poblahblahblah)
  • Fix 'kube-apiserver' tag inappropriately overwriting secrets at rest encryption token (#10460, @jwitko)
  • Fix assertion for task item verify-settings (#10699, @piwinkler)
  • Fix external-lb in kubelet.conf server address and kube-proxy api-server address (#10490, @ugur99)
  • Fix forgotten update of etcd-servers list in apiserver manifest when scaling (#8253, @liupeng0518)
  • Fix metallb example yaml (#10545, @caruccio)
  • Fix reset job for cri-o container engine (#10197, @turbosnail)
  • Fix restart network task cannot be skipped (ansible boolean conversion needed) (#10512, @ErikJiang)
  • Fix: add kubelet tag in task of Fetch facts to avoid kubelet config inconsistencies (#10423, @NierYYDS)
  • Fixes the path of the certificates use in the etcdctl.sh wrapper when the deployment type is not kubeadm. (#10467, @RomainMou)
  • Hubble relay will work when cilium_cluster_name is customised. (#10614, @eugene-eeo)
  • Disable podCIDR allocation from control-plane when using calico (#10639, @VannTen)
  • Kubespray-defaults: Check for boostrap-os FQDN (#10590, @VannTen)
  • Patch for modprobe_nf_conntrack for new Linux Kernel, when using ipvs (#10625, @abhishekkr)
  • Remove always tag applied on bootstrap (#10556, @yckaolalala)
  • Set remove_default_searchdomains to false by default (#10554, @hedayat)
  • Swap is now disabled using systemd (mask of swap.target) (#10587, @VannTen)
  • Fix undefined retries variable when copying etcdctl (#10634, @ErikJiang)
  • Move control plane certs renewal "spread out" into the systemd timer (#10596, @VannTen)
  • The dhcp configuration for dns nameservers are now the same than during installation (#10548, @smutel)
  • Use correct env var name for kube-vip per service leader election (#10433, @ThisIsQasim)
  • Don't fail on 304 Not Modified for an already downloaded file (#10452, @sathieu)
  • Fix download retry when get_url has no status_code (#10613, @RomainMou)
  • Fix ntp installation on SLES and openSUSE (#10786, @goldyfruit)
  • Set the maxUnavailable of the coredns rolling update strategy to 1 (#10748, @tu1h)
  • Fix crio_version version comparison (#10780, @ledroide)
  • Fix disable swap failed in Centos/RHEL 7 (#10751, @yankay)
  • Fix image pull fail with insecure-registry (#10775, @yankay)
  • Refactor check_galaxy + fix version (#10729, @VannTen)
  • Fix Helm installation on SLES and openSUSE (#10794, @goldyfruit)
  • Fix incorrect ciliumcli binary (#10575, @tu1h)
  • Fix ntp installation on SLES and openSUSE (#10786, @goldyfruit)
  • Fix the cluster installation on cluster using etcd clients nodes (cilium / calico / ...) (#10769, @VannTen)

Other (Cleanup or Flake)

  • Cleanup a deprecation warning (ipaddr filter) (#10518, @VannTen)
  • Decouple kubespray-defaults from download (#10626, @VannTen)
  • Etcd/backup: use native ansible modules instead of shell (#10540, @VannTen)
  • Etcd: use dynamic group for certs generation check (#10610, @VannTen)
  • Factorize some identical playbooks steps into their own sub-playbooks (#10633, @VannTen)
  • Pre-upgrade tasks cleanup (#10656, @VannTen)
  • Refactor "multi" handlers to use listen (#10542, @VannTen)
  • Remove unneeded workaround for removing kubeadm DNS (#10695, @VannTen)
  • Removed DEPRECATED --logtostderr from metrics-server (#10709, @michaelkebe)
  • Update KUBESPRAY_VERSION for v2.23.1 (#10600, @yankay)
  • Update several checksum for different modules & configuration (#10606, @mzaian)
  • Use non-deprecated stdout_callback in CI (#10647, @VannTen)
  • Validate systemd unit files when generating them (#10597, @VannTen)
  • Using ctr pull instead of nerdctl to workaround https://github.com/kubernetes-sigs/kubespray/issues/10670. (#10687, @yankay)
  • Jinja syntax pre-commit validation (#10667, @VannTen)
  • Bump vagrant version 2.3.7 (#10787, @yankay)
  • Update KUBESPRAY_VERSION for v2.23.2 (#10800, @yankay)

Supported Components

Known issues

N/A

Notes

  1. Swap is now disabled using systemd instead of changing /etc/fstab. #10587
  2. download.yml path changed. #10626
  3. UpCloud: Terraform provider updated to v2.12.0. Server groups with strict anti-affinity (move var from anti_affinity_policy to anti_affinity) #10474

v2.23.2

4 months ago

Container-Managers

  • [containerd] Fix invalid version check in containerd jinja-template config (#10620, @khanhngobackend)

API Change

  • Make kubernetes 1.27.9 the default version (#10797, @VannTen)

Feature

  • Don't fail on 304 Not Modified for an already downloaded file (#10452, @sathieu)
  • Update kubernetes default version to 1.27.9
  • Update etcd version for 1.27 and 1.26 to 3.5.10 (#10797, @VannTen)

Failing Test

  • Bump vagrant version 2.3.7 (#10788, @yankay)

Bug or Regression

  • Fix calico-node in etcd mode. (#10768, @VannTen)
  • Fix download retry when get_url has no status_code (#10613, @RomainMou) (#10791, @VannTen)
  • Kube-controller-manager will no longer assign pod CIDRs to cluster nodes when using calico (with its default IPAM, calico_ipam_host_local now has a default value of false) [⚠️ NOTE users using a non-true value for calico_ipam_host_local will need to change it to true] (#10639, @VannTen)

Other (Cleanup or Flake)

  • Kubespray collection will have the correct collection version. (#10728, @VannTen)

v2.23.1

7 months ago

Network

  • [Cilium] Fix invalid hubble yaml if cilium_hubble_tls_generate is enabled (#10476, @toonalbers)

Feature

  • Add hashes for kubernetes 1.27.6 & 1.26.9 (#10443, @bozzo)
  • Make kubernetes v1.27.7 default (#10543, @mzaian)
  • [etcd] Default version to 3.5.9 for k8s 1.25 , 1.26 , 1.27 (#10483, @mzaian)
  • Add crictl 1.26.1 for Kubernetes v1.26 (#10562, @mzaian)
  • Change default cri-o versions for Kubernetes 1.25, 1.26 (#10563, @mzaian)
  • [ingress-nginx] Fix nginx controller leader election RBAC permissions (#10569, @mzaian)
  • Refactor NRI activation for containerd and CRI-O (remove crio_enable_nri and containerd_nri_disable) now only one var nri_enabled default to false (#10496, @fmuyassarov)

Bug or Regression

  • Fix get currently configured nameservers error where there are inline comments in /etc/resolv.conf (#10415, @yankay)
  • Migrate node-role.kubernetes.io/master to node-role.kubernetes.io/control-plane (#10532, @unai-ttxu)
  • [download] Don't fail on 304 Not Modified (#10559, @RomainMou)

v2.23.0

9 months ago

Deprecation / Removal

  • Ubuntu 16 and 18 are no longer tested (#10107, @MrFreezeex)
  • Drop support for ansible-core 2.11 and update tests dependencies (#10034, @MrFreezeex)
  • Drop Kubernetes 1.24 support (#10234, @MrFreezeex)

Feature / Major Changes

  • Make kubernetes v1.27.5 default (#10392, @mzaian)
  • Add kubernetes v1.27.4 (#10359, @mzaian)
  • Add Kubernetes 1.27.2 (#9976, @mzaian)
  • Add hashes for 1.27.3 1.26.6, 1.25.11 (#10220, @mzaian)
  • Add hashes for 1.27.4 1.26.7, 1.25.12 (#10300, @mzaian)
  • Add CPU Management Policies on the Node (#10309, @yankay)
  • Add Debian 12(bookworm) support (#10221, @tu1h)
  • Add download.timeout to update download timeout value (#10149, @yjqg6666)
  • Add corresponding coredns versions to all the supported kubernetes releases. (#10233, @mzaian)
  • Add growpart azure enabled (#10241, @pedro-peter)
  • Add ingressClass resource for ingress_nginx by default (#10091, @peschmae)
  • Add kubelet topology manager policy on the node (kubelet_topology_manager_scope and kubelet_topoloy_manager_policy) (#10370, @tu1h)
  • Add labels to kube-vip static pods (#10139, @liupeng0518)
  • Add node_taints to aws_inventory script (#10170, @mstoetzer)
  • Add option to set SSL_CERT_FILE for offline installation using custom CA for https proxy (#10215, @HappyFX)
  • Add terraform support for NIFCLOUD (#10227, @ystkfujii)
  • Add the huawei cloud controller as external cloud controller (#10198, @dabeck)
  • Show detected ansible version when it isn't compatible with kubespray (#10109, @jcpunk)
  • Allow to override etcd listen-metrics-urls configuration (using etcd_listen_metrics_urls variable) (#10332, @forselli-stratio)
  • Don't let find search filesystem mounts in docker build run step (#10131, @tomodachi)
  • Permit custom names for API server lb/proxy containers (#10166, @jcpunk)
  • Permit skipping helm update (#10169, @jcpunk)
  • Split defaults main file into 2 files (checksums and version) (#10121, @electrocucaracha)
  • System upgrade for Debian-family nodes is available with system_upgrade=true (#10184, @sathieu)
  • Update download_hash.sh script (#10120, @electrocucaracha)
  • Use a uniform way to get the local path of the binaries (#10211, @ErikJiang)
  • Disable fapolicyd service (#10081, @epif4nio)
  • Upgrade the load balancer ( nginx and haproxy ) image version to Nginx 1.25, Haproxy 2.8. (#10409, @yankay)
  • [etcd] Default version to 3.5.7 for kubernetes 1.27 (#10410, @mzaian)

Applications

  • [argocd] update argocd to v2.7.4 (#10226, @mzaian)
  • [argocd] update argocd to v2.8.0 (#10364, @mzaian)
  • [argocd] Add argocd_install_url option to allow changing argocd url (#10176, @liupeng0518)
  • [helm] upgrade to 3.12.1 (#10225, @mzaian)
  • [helm] upgrade to 3.12.3 (#10365, @mzaian)
  • [helm] add python dependency check for helm-apps (#10192, @palmeXx)
  • [krew] add krew_no_upgrade_check (#10175, @liupeng0518)
  • [coredns] Bump coredns version to 1.10.1 (#10199, @eminaktas)
  • [coredns] Bump nodelocaldns version to 1.22.20 (#10200, @eminaktas)
  • [cert-manager] This introduces a new variable for the cert-manager implementation that will allow one to pass in extra arguments to the cert-manager controller.(#10049, @phunyguy)
  • Update Helm (v3.12.2) / Skopeo (v1.13.0) and yq (v4.34.2) (#10295, @tu1h)
  • Upgrade many tool versions (Helm, crun, kata, youki, gvisor, skopeo, Calico, Cilium etc...) (#9798, @electrocucaracha)
  • [local_path_provisioner] Fix invalid podhelper yaml (#10237, @MrFreezeex)
  • Update metrics server to v0.6.4 (#10400, @mzaian)

Container-Managers

  • [containerd] Make containerd 1.7.5 default (#10397, @mzaian)
  • [containerd] Support containerd v1.7.2 (#10219, @Dentrax)
  • [containerd] Support containerd 1.7.3 (#10368, @mzaian)
  • [containerd] containerd config_path enable mirrors config using new variable containerd_registries_mirrors (deprecate and remove containerd_insecure_registries for containrd and nerdctl_extra_flags and insecure_registry setting for nerdctl (#10196, @yckaolalala)
  • [crio] Add crio_insecure_registries option for specifying insecure_registries of crio (#10142, @qlijin)
  • [crio] runroot now needs to be setup in storage.conf instead of crio.conf (#10372, @floryut)
  • [crio] Fix etcdctl copy operation (#10242, @ErikJiang)
  • [Kata] Set/keep owner/group root/root when unarchiving kata-containers (#10338, @rybnico)
  • [youki] Fix youki binary download url (not requiring 'v' in version) (#10337, @ErikJiang)

Network

  • [calico] Use configmap to configure calico cni config (#10177, @cyclinder)
  • [calico] Update calico v3.25.2 (#10414, @mzaian)
  • [calico] Add calico version to v3.26.0 (#10224, @mzaian)
  • [calico] Add calico version to v3.26.1 (#10235, @mzaian)
  • [calico] Clean up calicoctl_alternate_download_url and calicoctl.mirrors (#10271, @yckaolalala)
  • [cilium] Add custom rules to clusterrole for cilium operator (#10267, @jeremythuon)
  • [cilium] Upgrade to version 1.13.4 (#10269, @yulng)
  • [Cilium] Do not mount tls when 'cilium_hubble_tls_generate' is false (#10357, @charlychiu)
  • [Cilium] Update cilium to 1.13.3 (#10158, @jcpunk)
  • [flannel] Only create /var/lib/calico when needed (#10156, @jcpunk)
  • [flannel] Bump flannel version to v0.22.0 and flannel-cni-plugin version to v1.1.2. Also, changes flannel repository from flannelcni to flannel (#10205, @eminaktas)
  • [flannel] Remove unused flannel_cni_download_url (#10188, @oomichi)
  • [kube-ovn]: update version v1.11.5 (#10125, @yankay)
  • [multus] Fix loop_control template error when item is None (#10347, @nicolas-goudry)

API Change

  • Unless the pod security standard versions are changed on intentionally, as default it will be the same major version with Kubernetes version. (#10210, @ugur99)
  • Upgrade ansible to 7.0 and ansible-core to 2.14.x (#10190, @MrFreezeex) ⚠️ (See Notes 2)

Documentation

  • Add github container registry (github_image_repo) to docs/offline-environment.md (#10265, @blackliner)
  • Update doc for ansible-core 2.14 support and clarify issues running older python versions (#10261, @MrFreezeex)
  • Update links for aws_alb_ingress_controller (#10264, @kundan2707)
  • Update links in ingress-controller and kuberentes-apps (#10239, @vaibhav2107)
  • Update Calico to lowercase and fix broken calico link in README (#10232, @Xieql)
  • Document containerd command to restart nginx-proxy container when adding control plane node (#10406, @nicolas-goudry)

Failing Test

  • Increase metallb wait timeout from 30sec to 2min (#10260, @MrFreezeex)
  • Update CentOS 7 image and test fedora 37 and 38 instead of fedora 35 and 36 (#10108, @MrFreezeex)

Bug or Regression

  • Fix Dockerfile for newest directory layout (#10128, @dabeck)
  • Fix Flatcar bootstrap issues (yaml module missing and ntp issue) (#10363, @tenni-paws)
  • Fix argocd install not working using the kubespray docker image (#10371, @cortex3)
  • Fix correctly mount ssl ca directories (#9794, @maxime1907)
  • Fix etcdctl copy operation (#10230, @ErikJiang)
  • Fix gce-pd-csi driver (#10208, @ashishsinghdev)
  • Fix grep command without -w option causing prefix matched while adding one etcd member (#10291, @yangsenzk)
  • Fix hcloud-cloud-controller-manager not working in certain setups (#10297, @cortex3)
  • Fix helm (kubelet-csr-approver) installation on redhat distro (#10204, @MrFreezeex)
  • Fix kubelet-csr-approver usage with upgrade-cluster.yml and missing package with helm role (#10165, @j4m3s-s)
  • Fix nginxingress-class template (missing newline) (#10174, @richard-fairthorne)
  • Fix problem migration problem with k8s 1.27 (#10136, @batazor)
  • Fix reset_confirmation not working when inputing correct value (#10288, @somewho)
  • Fix wrong path in manage-offline-files script (#9886, @Medosopher)
  • Fix an issue where using Rocky Linux 8 as OS for Vagrant for testing purposes causing etcd to fail on start. (#10252, @nltimv)
  • Fix ansible-lint galaxy rule (#10277, @MrFreezeex)
  • Fix ansible-lint key-order error (#10314, @MrFreezeex)
  • Fix outdated tag and experimental ansible-lint rules (#10254, @MrFreezeex)
  • Fix dockerfile build error (#10127, @yankay)
  • Fix metrics-server deployment to run with kubernetes 1.26+ (#10183, @mzaian)
  • Fix undefined reset_confirmation_prompt variable in reset play (#10303, @Mishavint)
  • Fix CIS Kubernetes V1.23 Benchmark item number 4.1.9 to enhance security (Change kubelet-config.yaml and kubelet.env file permissions from 640 to 600) (#10304, @satandyh)
  • Fix parsing of RHSM proxy configuration (#10228, @tmurakam)
  • Fix var-spacing ansible rule (#10266, @MrFreezeex)
  • Fix specify owner to kube_owner in task of copy cni plugins (#10407, @NierYYDS)
  • Fix typo kubelet_topoloy_manager_policy => kubelet_topology_manager_policy (#10384, @hangscer8)
  • Fix recover_control_plane playbook (also add debian 12 with cilium as a new nightly test) (#10411, @floryut)
  • Fix nameserver inline comments in /etc/resolv.conf (#10415, @yankay)
  • Added systemd_resolved_disable_stub_listener variable to disable systemd-resolved's stub listener, defaults to true on Flatcar. (#9875, @cosandr)
  • Remove auto_attach and syspurpose in RHEL subscription Organization ID/Activation Key registration. (#10258, @yckaolalala)
  • Replace "crio_packages" with "crio_bin_files" (#10182, @yckaolalala)
  • Update MetalLB deployment, wait for resource. (#9995, @Jeroen0494)
  • Upgrade ansible to 7.0 and ansible-core to 2.14.x in Dockerfile (#10259, @yckaolalala)
  • Fix typo kubelet_topoloy_manager_policy => kubelet_topology_manager_policy (#10384, @hangscer8) ⚠️ (See Notes 1)
  • Change maximal_ansible_version to 2.15(exclusive) (#10395, @yankay)
  • Install etcdutl file by default (#10385, @liupeng0518)

Other (Cleanup or Flake)

  • [CI] Add CI VM for debian12 (#10222, @yankay)
  • [CI] Removes Ansible reinstall from build pipeline (#10032, @luksi1)
  • [CI] cleanup stale packet namespace automatically (#10245, @MrFreezeex)
  • [CI] fix tf-elastx_cleanup fail (#10133, @yankay)
  • [CI] Sanitize branch name in testing before using it in kubernetes label for packet-ci (#10315, @MrFreezeex)
  • Add an exception for youki in download_hash script (#10346, @ErikJiang)
  • Drop support for Kubernetes 1.24.x (move min version to 1.25.x) (#10126, @yankay)
  • Ensure host entries from /etc/host are absent when populate_inventory_to_hosts_file is false (#10144, @rptaylor)
  • Exclude terraform.tfstate backups in .gitignore (#10216, @rptaylor)
  • Ping is no longer reported as a changed task (#10160, @jcpunk)
  • Reading mounted volumes no longer considered a changed task (#10161, @jcpunk)
  • Resolve ansible-lint name errors (#10253, @MrFreezeex)
  • Update KUBESPRAY_VERSION for v2.22.1 (#10201, @yankay)

Supported Components

Known issues

N/A

Notes

  1. Variable kubelet_topoloy_manager_policy change to kubelet_topology_manager_policy, please update your inventory
  2. Upgrade ansible to 7.0 and ansible-core to 2.14.x
  3. ⚠️ breaking change : containerd config_path enable mirrors config using new variable containerd_registries_mirrors (#10196, @yckaolalala)

v2.22.1

1 year ago

Bug or Regression

  • Don't let find search filesystem mounts in docker build run step (#10131, @tomodachi)
  • Fix Dockerfile for newest directory layout (#10128, @dabeck)
  • Fix dockerfile build error (#10181, @yankay)
  • Fix metrics-server deployment to run with kubernetes 1.26+ (#10183, @mzaian)
  • update README for v2.22.0 (#10180, @Payback159)
  • Fix Update MetalLB deployment, wait for resource. (#9995, @Jeroen0494)

v2.22.0

1 year ago

Deprecation / Removal

  • [Cilium] Delete the probe option of cilium_kube_proxy_replacement (#9929, @XiuguangHuang)
  • [Cilium] Remove use_localhost_as_kubeapi_loadbalancer and detect wether we can use localhost apiserver loadbalancer if cilium/calico replace kube-proxy (#9718, @MrFreezeex)
  • Drop crun_bin_dir unused variable, now using only bin_dir var (#9845, @electrocucaracha)
  • Drop the canal network_plugin support because the network_plugin is unmaintained. (#10100, @oomichi)
  • Remove the support of Debian 9 (#10097, @yankay)
  • Replaces storage.googleapis.com/kubernetes-release with dl.k8s.io (#10066, @KlwntSingh)

Feature / Major Changes

  • Add Kubernetes 1.26.x (#9570, @mzaian ; #9732, @yankay; #9829, @mzaian; #9900, @mzaian)
  • Make kubernetes v1.26.5 default (#9983, @mzaian)
  • "native" snapshotter of nerdctl config is replaced by new var nerdctl_snapshotter with default "overlayfs" value (#9979, @dmitrytretyakov)
  • Support multi-arch using the same image name (#9978, @ErikJiang)
  • Add DNS configuration for cert-manager (using new variables cert_manager_dns_policy|config) (#9673, @ErikJiang)
  • Add Retry for restart kube-controller-manager (#10013, @hangscer8)
  • Add coredns_additional_configuration variable to define extra Coredns configurations (#10025, @navidnabavi)
  • Add coredns_rewrite_block to perform internal message rewriting (#10045, @maxime1907)
  • Add a new simple network_plugins custom_cni to install user provided manifests (#9819, @MrFreezeex)
  • Add back openssh-client to docker image (#9835, @maxime1907)
  • Add download retries option download_retries (#9911, @tu1h)
  • Add support to install ContainerD on any Linux Distributions using new var allow_unsupported_distribution_setup (#9827, @XDRAGON2002)
  • Add the kube-profile config to the kubeadm's kube-scheduler config. (#9993, @yankay)
  • Add vim to kubespray docker image (#9805, @XDRAGON2002)
  • Adds support for Kubelet-CSR-approver to auto-approve kubelet CSR when kubelet_rotate_server_certificates. (#9877, @j4m3s-s)
  • Add dns_cpu_limit value to support large scaled coredns deployments (#10103, @mzaian)
  • Add provider meta module_name in Equinix Metal TF configs (#10044, @vasubabu)
  • Allow to configure image garbage collection (using kubelet_image_gc_high_threshold and kubelet_image_gc_low_threshold) (#9832, @zhan9san)
  • Apply kubeadm patches during upgrade as recommended by k8s (#9781, @mvandergiesen)
  • Cinder-csi: Allow VolumeSnapshotClass' deletionPolicy to be configurable (#9736, @huangkevin404)
  • Containerd add containerd_use_config_path config field. (#9770, @lengrongfu)
  • Enable control plane load balancing for kube-vip (#9785, @ErikJiang)
  • Feat(contrib/terraform): support custom ssh port (#9836, @maxime1907)
  • Fix kube-bench 1.2.20 to enhance security (Ensure that the --audit-log-maxbackup argument is set to 10) (#9939, @yankay)
  • Fix kube-bench 1.1.19 to enhance security (Change Kubernetes Cert directory and file ownership is set to root:root) (#9937, @yankay)
  • Fix kube-bench 4.1.1 to enhance security (Change kubelet systemd init file from 644 to 600) (#9934, @yankay)
  • Fix kubernetes-app/argocd: download related things with the download role (#9786, @pli01)
  • Kube.py now supports kubeconfig (#9982, @liupeng0518)
  • MetricsServer: Add extras nodeselector, affinity, tolerations (using metrics_server_nodeselector, metrics_server_extra_affinity ,metrics_server_extra_tolerations) (#9972, @pli01)
  • Refactor Hetzner terraform (fixing flatcar configs and remove deprecated provider) (#10002, @ThisIsQasim)
  • Support for MetalLB v0.13.9 with CRD (#9120, @Jeroen0494)
  • Throw an error when specifying unsupported os in Vagrant (#9965, @THUzxj)
  • Update CoreDNS manifests (remove deprecated annotations) (#9977, @mzaian)
  • Update dns-autoscaler configuration and remove deprecated annotations (#9996, @mzaian)
  • Update metrics server to v0.6.3 (#10026, @mzaian)
  • Upgrade argocd to v2.6.3 (#9848, @panguicai008)
  • Upgrades the following Python libraries to their latest available releases (cryptography / jinja2 / jmespath / MarkupSafe/ netaddr / pbr / ruamel.yaml / ruamel.yaml.clib) (#9938, @luksi1)
  • Add IPv6 listen directive to haproxy if enable_dual_stack_networks (#9674, @yankay)
  • Add support for Ansible collections in Kubespray (⚠️ See notes !) (#9582, @luksi1)
  • Support mTLS for Hubble and upgrade backend to v0.11.0 (#9959, @jeremythuon)
  • Update nodelocaldns to 1.22.18 (#9800, @sathieu)
  • Replace disable_swap variable with kubelet_fail_swap_on (#10036, @Manuelraa)
  • Replace nodelocaldns label to k8s-app: node-local-dns (#9745, @stelucz)
  • Upgrade rancher local-path-provisioner to v0.0.23 (#9855, @panguicai008)
  • Use kube_apiserver_address variable for advertiseAddress (#9967, @liupeng0518)
  • Use string for ipv6 forward conf value (#9992, @liupeng0518)
  • Update pause image version to v3.9 (#10112, @mzaian)
  • Upgrade cni version to v1.3.0 (#10058, @cyclinder)
  • [argocd] update argocd to v2.6.7 (#9953, @mzaian)
  • [helm] support to 3.11.1 (#9849, @mzaian)
  • [helm] support to 3.11.3 (#10022, @mzaian)
  • [helm] support to 3.11.2 (#9951, @mzaian)
  • [helm] upgrade to 3.12.0 (#10085, @mzaian)
  • [UpCloud] Add server group support for vms and target port for loadbalancers (#9831, @robinAwallace)
  • [argocd] update argocd to v2.5.10 (#9753, @yanggangtony)
  • [cert-manager] Upgrade to v1.11.1 (#9964, @rtsp)
  • [flannel] update to v0.21.4 (#10027, @mzaian)
  • [nerdctl] support version 1.3.1 (#10024, @mzaian)
  • [nerdctl] update to version 1.4.0 (#10119, @mzaian)

Applications

  • [kube-vip] Support to v0.5.8 (#9734, @hangscer8)
  • [kube-vip] Support kube-vip to v0.5.11 (#9852, @panguicai008)
  • [kube-vip] Update default kube-vip to v0.5.12 (#10005, @hangscer8)
  • [vSphere-csi] Add resources section to all containers releated to Vsphere CSI driver (#9687, @JRaver)
  • [argocd] update argocd to v2.7.2 (#10086, @mzaian)

Container-Managers

  • [containerd] Add hashes for containerd version 1.6.19 (#9838, @mzaian)
  • [containerd] Add hashes for containerd version 1.6.20 (#9954, @mzaian)
  • [containerd] Add hashes for containerd version 1.7.0 (#9892, @mzaian)
  • [containerd] Add hashes for containerd versions 1.7.1, 1.6.21 (#10061, @mzaian)
  • [containerd] Support version 1.6.16 (#9727, @yanggangtony)
  • [cri-o] Bump versions to 1.26.3, 1.25.3, 1.24.5 (#9999, @dkasanic)
  • [cri-o] Fix install order -> first runc then crictl (#9780, @mvandergiesen)
  • [cri-o] Fix missed double quotes in cri-o config (#10040, @turbosnail)
  • [cri-o] Fix CRI-O amd64 v1.26.0 wrong archive checksum (#9872, @panguicai008)
  • [cri-o] cri-o restart if config change (#10057, @MrFreezeex)
  • [cri-o] Remove deprecated crio_pids_limit (default is now unlimited) (#10056, @j4m3s-s)
  • [cri-o] Fix cri-o restart if config change (#10057, @MrFreezeex)
  • [runc] Upgrade to v1.1.7 (#10039, @pomland-94)

Network

  • [Calico] Add Retry and Ignore Error for Checking calico ready (#9883, @hangscer8)
  • [Calico] Add option calico_kubeconfig_wait_timeout (#9994, @tu1h)
  • [Calico] Improve version check command (#9861, @zhan9san)
  • [Calico] Optimize the detection of calico existence (#9873, @hangscer8)
  • [Calico] Support calico version v3.25.0 (#9860, @cyclinder)
  • [Calico] upgrade default calico version to v3.25.1 (#9950, @mzaian)
  • [Calico] Add missing ipamconfigs resource in RBAC (#9755, @chaunceyjiang)
  • [Calico] Fix installation while applying CRD (#10068, @hangscer8)
  • [Calico] Add calico version to v3.24.6 (#10113, @mzaian)
  • [Cilium] Add and support v1.13.0 (#9879, @utam0k)
  • [Cilium] Fix Hubble relay configuration (#9876, @prashantchitta)
  • [Cilium] Fix the configuration of TLS for hubble (#9880, @utam0k)
  • [Cilium] Remove duplicates in the configuration of tls for hubble (#9932, @CaMoPeZzz)
  • [Cilium] Support version above 1.13.x (#9914, @wbh1)
  • [Cilium] Updates hubble certgen arguments (wrong since v0.1.7) (#9856, @XDRAGON2002)
  • [Cilium] IPAM uses "Cluster Scope" mode by default. Also add the parameters required for this mode (#9443, @dcwbq)
  • [flannel] Update image repo from flannelcni to flannel (#10041, @ErikJiang)
  • [multus] fix multus include error (#10105, @darkobas2)

API Change

  • Openstack cloud controller manager bind address is now configurable using external_openstack_cloud_controller_bind_address (#9958, @dominykasn)

Documentation

  • Add a mention for custom_cni in CNI list (#9878, @j4m3s-s)
  • ArgoCD no longer uses the pod name as initial password (#9930, @peschmae)
  • Drop remaining part for supporting ansible 2.9 and 2.10 (#9842, @oomichi)
  • Fix sidebar documentation (#9988, @lijin-union)
  • Fixup link in docs/calico.md (#9940, @kundan2707)
  • Remove stale contents for cni documention (#9778, @tu1h)
  • Reword confusing etcd download url comment when etcd_deployment=host (#9686, @tjanson)
  • Suggest to run reset.yml playbook for first-time users (#9865, @kerryeon)
  • Update docker tag to v2.21.0 in README.md (#9802, @Payback159)
  • Update link for baremetel consideration (#9944, @kundan2707)
  • Add port requirements documentation (#9969, @yankay)

Failing Test

  • Update Terraform to 1.3.7 and Vagrant to 2.3.4 (#9699, @floryut)
  • [CI] Migrate CI_BUILD_ID to CI_JOB_ID and CI_BUILD_REF to CI_COMMIT_SHA following gitlab upgrade (#10063, @floryut)

Bug or Regression

  • Add PSS labels to metallb namespace (#9713, @manzsolutions-lpr)
  • Add jmespath back to Dockerfile image (#9697, @floryut)
  • Add missing krew_download_url to offline.yml (#9788, @jianse)
  • Add proxy_env variable to apt_key cleanup task (#9766, @SamuelBECK1)
  • Add rsync in Dockerfile (#9839, @zhan9san)
  • Add ruamel.yaml back to Dockerfile image (#9707, @floryut)
  • Cleanup MetalLB install following update (#10004, @eugene-marchanka)
  • Copy contrib/ to Dockerfile (#9774, @oomichi)
  • Downgrade the version of CoreDNS to 1.8.6 for compatibility with Kubernetes versions older than 1.25. (#9846, @JiffsMaverick)
  • Explicitly disable rhsm repo when rhel_enable_repos is false (#9973, @tu1h)
  • Fix cert_manager_trusted_internal_ca manifest failing when dns policy is set (#9922, @peschmae)
  • Fix containerd_insecure_registries => move with_item to with_dict (#9729, @lengrongfu)
  • Fix allow unsupported distribution (#9904, @ErikJiang)
  • Fix cilium's hubble ui configuration (#9735, @j4m3s-s)
  • Fix comma-separated-list splitting of kubelet_enforce_node_allocatable variable (#9694, @Tristan971)
  • Fix confusing instance sizing (etcd, kube_master) in Vagrantfile (#9966, @THUzxj)
  • Fix ingress url not found issue (#9789, @JaneLiuL)
  • Fix playbook names to support import via galaxy (#10021, @dkasanic)
  • Fix restart k8s components, checking yml files instead of manifest (#9962, @liupeng0518)
  • Fix uniontech OS installation failure (#9862, @ErikJiang)
  • Fixing default cgroups for kubelet and container_manager (#9834, @MrFreezeex)
  • Localhost task (validate mirror) don't need to ask for become (#9669, @chok)
  • Remove unneeded access_ip when not wanted in terraform scripts (#9869, @maxime1907)
  • Replace semicolons by commas in networkmanager dns configuration options (#9840, @lystor)
  • Retry other masters during upgrade and not only the first one (#9768, @maxime1907)
  • Skip steps of ensuring NTP and tzdata packages in the CoreOS and Flatcar (#9742, @ErthoAers)
  • Support extended settings for the Debian os family (#9943, @ErikJiang)
  • Fix calico rbac issue (#9806, @JaneLiuL)
  • Update nodes in etc hosts after cluster scale (#9837, @zhan9san)
  • Update rhsm repo trigger if no subscriptions is found (#10001, @tu1h)
  • Bootstrap ansible requirement in the facts playbook (#10069, @MrFreezeex)
  • Clear http scheme on containerd insecure-registry tls config (#10084, @tu1h)
  • Ignore errors in check mode performing "Disable swapOnZram for Fedora" (#10077, @gorozhin)
  • [etcd] fix make-ssl-etcd.sh.j2; move pem files only if any new certs exist (#9974, @2k0ri)
  • [vSphere-csi-driver] Fixes the run of the cluster.yml playbook when vsphere_csi_namespace is set to non-default (#9946, @eugene-marchanka)

Other (Cleanup or Flake)

  • Add checksum verification for kubectl binary in dockerfile (#9963, @alekseyolg)
  • Add generic pre-commit hook to the repository (#9750, @bbaassssiiee)
  • Cleanup of external-openstack-cloud-config to be in the same order/values as the documentation and not clutter config when defaults are used. (#9899, @jadams)
  • Cleanup v1.23.x references/conditions/hashes (#9698, @floryut)
  • Dockerfile update ubuntu version to 22.04 which has newer system packages with fewer (#10033, @alekseyolg)
  • Drop support for Kubernetes 1.23.x (move min version to 1.24.x) (#9691, @floryut)
  • Fix(contrib/terraform): do not set ansible_ssh_port to 22 (#9828, @maxime1907)
  • Move multus url to k8snetworkplumbingwg repository (#9850, @panguicai008)
  • New automated method to collect binaries checksums (#9782, @electrocucaracha)
  • Reducing the number of layers and commands for docker image (#9822, @alekseyolg)
  • Remove deprecated udpIdleTimeout field in KubeProxyConfiguration (#9925, @HirazawaUi)
  • Remove invalid character in crictl tasks file (#9970, @tu1h)
  • Replace bash for loop when checking API server SANs (#9060, @rptaylor)
  • Use var etcd_deployment_type instead of etcd_kubeadm_enabled (#9823, @liupeng0518)
  • Reducing the number of layers, increasing readability, reducing the size of the image (#9821, @alekseyolg)
  • Fix arithmetic outside of jinja (#10106, @MrFreezeex)
  • Fix CI broken by flannel-cni-plugin docker hub rate limit (#10083, @yankay)
  • [CI] Add CI for containerd insecure_registries (#9797, @yankay)
  • [CI] Updated version of ara included in CI job logs collection from 1.5.7 to 1.6.1 (#9737, @dmsimard)
  • [CI] Add checksum verification of kubectl binary in pipeline image (#9971, @alekseyolg)
  • [CI] Fix CentOS Extras repo url for Oracle Linux 7 aarch64 (#9791, @bin456789)
  • [CI] Use Docker buildkit + caching for builds to speed up the CI pipeline (#10008, @luksi1)
  • [CI] Add six module into openstack-cleanup/requirements.txt (#10099, @oomichi)
  • [CI] Fix tests for files lookup path for custom-cni (#10088, @j4m3s-s)

Supported Components

Known issues

N/A

Notes

  • Support for MetalLB v0.13.9 with CRD (⚠️ This release includes user facing changes for which there is action required. The way the inventory is setup for MetalLB deployment has changed significantly. Most prominently, we have switched from underscores to a dictionary for defining resources. Please follow the documentation for restructuring your MetalLB inventory variables.
  • Replace disable_swap variable with kubelet_fail_swap_on
  • Fix playbook names to support import via galaxy (⚠️ ADD NOTE : recover-control-panel => recover_control_plane, remove-node => remove_node, upgrade-cluster => upgrade_cluster)
  • [Cilium] IPAM uses "Cluster Scope" mode by default.
  • Add support for Ansible collections in Kubespray (This would cause a change to the repository's structure, meaning downstream users would either need to change their code to point to the playbooks directory or use the ansible.builtin.import_playbook module)