Kubernetes Operations (kOps) - Production Grade k8s Installation, Upgrades and Management
This is the first beta of the 1.29 release.
Some infrastructure changes are potentially disruptive to the continued
operation of the cluster. For the most disruptive operations, particularly
those that break rolling-update of the cluster, we have started to use deferred
deletion to minimize the impact. For example, on AWS we create a second NLB
during the kops update
phase when we cannot change the NLB directly.
kops update
will report that a --prune
is needed. To minimize disruption,
we recommend you perform this after a rolling-update, for example:
kops update $MYCLUSTER --yes --admin
kops rolling-update $MYCLUSTER --yes
kops update $MYCLUSTER --yes --admin --prune # NEW!
Deferred deletion is currently used to safely introduce security groups for NLBs on AWS,
and to move to an internal load balancer for kops-controller
on GCP.
We are starting to add (experimental) support for OpenTelemetry,
in particular Tracing support. Setting OTEL_EXPORTER_OTLP_TRACES_FILE
will write a trace file which can then be read by the traceserver program.
More information and options are described in docs/opentelemetry.md.
The tracing data is not expected to be particularly useful for end-users in
this release; the (non-standard) recording approach is instead intended to
work well with our Prow end-to-end testing system so that developers can
optimize kOps.
Please note: this is not telemetry in the "phone-home" sense. The kOps project does not collect data from your machine. As an open-source project we do not even want to collect any of your data. Currently the only OpenTelemetry backend supported is writing to a filesystem (and it is opt-in). In future you will be able to configure other OpenTelemetry backends, but this data will only be sent if you enable OpenTelemetry, and only sent to where you configure.
Network Load Balancers in front of the Kubernetes API and bastion hosts now have a security group attached. These security groups are used for security group rules allowing incoming traffic to the NLBs as well as traffic between the NLBs and their target instances.
Posts event data to URL upon instance interruption action in aws-node-termination-handler with WEBHOOK_URL
.
As of Kubernetes version 1.29, credentials for private GCR/AR repositories will be handled by the out-of-tree credential provider. This is an additional binary that each instance downloads from the assets repository.
We now use a private load-balancer for in-cluster traffic on GCP, which allows us to use network tags to restrict access only to the cluster nodes.
kops toolbox dump
limits the number of nodes dumped to 500 by default. Use --max-nodes
to override.
Support for Kubernetes version 1.23 has been removed.
Support for Kubernetes version 1.24 is deprecated and will be removed in kOps 1.30.
Support for Kubernetes version 1.25 is deprecated and will be removed in kOps 1.31.
Support for AWS Classic Load Balancer for API is deprecated and should not be used for newly created clusters.
All legacy addons are deprecated in favor of managed addons, including the metrics server addon and the autoscaler addon.
kops validate cluster
improvements by @upodroid in https://github.com/kubernetes/kops/pull/16187
create cluster
command by @hakman in https://github.com/kubernetes/kops/pull/16202
k8s.io/utils/strings/slices
with golang.org/x/exp/slices
by @hakman in https://github.com/kubernetes/kops/pull/16238
domain
instead of vpc
when rendering aws_eip
by @hakman in https://github.com/kubernetes/kops/pull/16237
boskos-resource-type
flag to use different GCE projects for scale/gpu testing by @upodroid in https://github.com/kubernetes/kops/pull/16268
MACAddressPolicy=none
when using AWS VPC CNI by @moshevayner in https://github.com/kubernetes/kops/pull/16313
infinity
by @dims in https://github.com/kubernetes/kops/pull/16329
toolbox dump
by @rifelpet in https://github.com/kubernetes/kops/pull/16389
Full Changelog: https://github.com/kubernetes/kops/compare/v1.29.0-alpha.3...v1.29.0-beta.1
Full Changelog: https://github.com/kubernetes/kops/compare/v1.28.3...v1.28.4
Full Changelog: https://github.com/kubernetes/kops/compare/v1.27.2...v1.27.3
Full Changelog: https://github.com/kubernetes/kops/compare/v1.28.2...v1.28.3
Release v1.28.2
Full Changelog: https://github.com/kubernetes/kops/compare/v1.28.1...v1.28.2
Release v1.29.0-alpha.3
kops get
cmd and set node-tag ginkgo flag by @upodroid in https://github.com/kubernetes/kops/pull/16096
Full Changelog: https://github.com/kubernetes/kops/compare/v1.29.0-alpha.2...v1.29.0-alpha.3
Full Changelog: https://github.com/kubernetes/kops/compare/v1.28.0...v1.28.1
Full Changelog: https://github.com/kubernetes/kops/compare/v1.27.1...v1.27.2
--cloudonly
sound less innocuous by @danports in https://github.com/kubernetes/kops/pull/15911
SeccompDefault
kubelet config by @colinhoglund in https://github.com/kubernetes/kops/pull/15919
--node-instance-group
flag to ginkgo by @upodroid in https://github.com/kubernetes/kops/pull/16065
Full Changelog: https://github.com/kubernetes/kops/compare/v1.29.0-alpha.1...v1.29.0-alpha.2
kops get assets --copy
command no longer sets object-level public-read ACLs in the destination fileRepository.
hSupport for Kubernetes version 1.22 has been removed.
Support for Ubuntu 18.04 is has been removed.
Support for Canal, Flannel, and Kube-Router has been removed for Kubernetes 1.28 and later.
RHEL-based distros will no longer have wget
, curl
, python2
, and git
packages installed. Install them with hooks if needed.
Support for Kubernetes version 1.23 is deprecated and will be removed in kOps 1.29.
Support for Kubernetes version 1.24 is deprecated and will be removed in kOps 1.30.
Support for AWS Classic Load Balancer for API is deprecated and should not be used for newly created clusters.
All legacy addons are deprecated in favor of managed addons, including the metrics server addon and the autoscaler addon.
kube-dns-autoscaler
when upgrading to CoreDNS by @agilgur5 in https://github.com/kubernetes/kops/pull/15584
user-data
instead of startup-script
metadata key by @hakman in https://github.com/kubernetes/kops/pull/15607
--control-plane-size
as deprecated by @hakman in https://github.com/kubernetes/kops/pull/15725
ap-northeast-2d
and us-east-1e
for testing by @hakman in https://github.com/kubernetes/kops/pull/15741
--master-count
in testing for backwards compatibility by @hakman in https://github.com/kubernetes/kops/pull/15742
--endpoint/slice-updates-batch-period
by @hakman in https://github.com/kubernetes/kops/pull/15798
Full Changelog: https://github.com/kubernetes/kops/compare/v1.28.0-alpha.1...v1.28.0