Changelog

237bd359 Add docs and bump to v0.55.1 c2f6968b Fix index bug in gen logic 104cfba3 Export field 0142d9c9 Merge pull request #1526 from johnSchnake/exportJunitProcessing ea560878 Export junitResults type 692611c8 Merge pull request #1472 from NikhilSharmaWe/newFlag cdf69c82 Modify the template based on input bfce3a90 Add flag/config settings for aggregator permissions 0e7f9c90 Manually bump version to v0.55.0 for library code af1d73c6 Add more warnings for flag interactions 450be8bb Add blog for e2e-skeleton plugin 354bb678 Remove junk and fix links in docs c5269ae6 Add ci-lint options

Overview

This is a fairly big release with a number of worthwhile improvements and bug fixes that we would recommend users take advantage of, including:

• Simplified results reporting for many plugins
• Better handling of multi-container plugins
• Bug fixes related to results storage and cooperation between components
• A new command to wait on an existing Sonobuoy run
• More os/arch support
• and more

Improvements

Sonobuoy automatically reports results if your plugin completes and exits 0

One of the most frustrating types of bugs is where the plugin container runs successfully but fails to write the done file to inform the Sonobuoy worker that it has completed and where to find the results.

We decided to add a bit more intelligence to the worker to avoid many of these cases. Now, if the worker container is the only container left running and the others all exited with a status code of 0, the worker will automtaically upload the entire results directory to the aggregator on your behalf.

e4f0a0d9 Sonobuoy workers report results of completed plugins

Add environment variables and mounts to sidecar containers

Now that Sonobuoy adds a few environment variables to the containers it creates (Kuberentes version, directory locations, etc), it is reasonable that we should be adding these to the sidecars and other containers in the plugin.

9ae72e6f Add all env vars to all containers

Running Sonobuoy without any plugins

Many releases ago it was possible to run Sonobuoy without any plugins as solely a means of gathering the other debug information. This use case was dropped at some point but we have found that some users relied on it so we are adding it back in a simpler fashion.

If you want to run Sonobuoy but not run any plugins, just add SkipPlugins: true to the Sonobuoy config file (e.g. sonobuoy gen config then edit the file). This will clear out the plugin selections so that the aggregator has no plugins to load.

8c57b248 Add special config option to skip all plugins

New build artifacts: s390x and ppc64le

Kubectl supports these formats and so should we. Thanks to @barthy1 and @MatthieuSarter for this help.

06f5616e Update build setup to publish sonobuoy binary and image for s390x and ppc64le

Add a new sonobuoy wait command

Sometimes you start a sonobuoy run and then later decide you wish you had provided --wait. We added a convenience command to effectively do that. Just run sonobuoy wait to wait for the Sonobuoy run in the targeted namespace to complete.

17b70502 Add sonobuoy wait command

Rename plugins on a per-run basis without editing YAML

Now that locally Sonobuoy can install/lookup different plugins and configurations, it is much more likely that a user may want to run two different configurations of the same plugin at the same time. However, Sonobuoy requires that plugin names are unique.

To avoid having to edit the YAML each time this may occur, you can edit the plugin name for that individual run by a new bit of syntactic sugar:

$ sonobuoy run -p <old-name>@<new-name>

0434aa03 Allow plugin renaming via @ seperator in flag name

Reworking sonobuoy plugins list output

We are still working out the best approach for some of the plugin management commands. As part of that effort we've reworked the sonobuoy plugins list output to make it more succinct and to try and make it more clear how to invoke the plugin from the command line.

a01be9bd Clarify plugin list output surrounding name/file

Bug Fixes

Fix bugs involving a custom ResultsDir

A few issues have occurred lately where it became clear that the ResultsDir field of the Sonobuoy config was not being respected or passed correctly between the different bits of code.

As such, there were situations where the plugin and sonobuoy-worker disagreed about where the results should be stored or mounted from and other bugs where the CLI client and aggregator pod would disagree about where the aggregator stored the final product.

9bf47ef6 Pass ResultsDir through necessary calls

Fixed a bug causing nodeSelectors to not be properly respected

If your daemonset plugin is OS-specific, you'll need to have the nodeSelector set to target the right nodes. However, Sonobuoy goes beyond the default Kubernetes tracking and will monitor which nodes have or have not yet run the plugin. It is important that Sonobuoy properly understands the nodeSelector being used or else it will constantly expect other nodes to report results when none will occur due to the scheduling limitation.

4b072ef2 Fix plugin nodeSelector issue inc. systemd-logs

Fixed a bug causing some pod logs to be dropped

Fixed a race in sonobuoy logs that sometimes caused logs from random containers to not be gathered.

Also added a new special case to get just the logs from the Sonobuoy aggregator pod:

$ sonobuoy logs -p sonobuoy

95eb1e22 Fix logs bug causing aggregator to be dropped

Full Changelog

dc78b39a Bump version for v0.55.0 f7f9a6f6 Add flag for specifying the aggregator path to retrieve 8c57b248 Add special config option to skip all plugins d043d729 Fix bug causing some duplicate gojson tests 9ae72e6f Add all env vars to all containers e4f0a0d9 Sonobuoy workers report results of completed plugins e31cd28c Upload artifacts from integration tests 9bf47ef6 Pass ResultsDir through necessary calls bfcbf9b9 Remove deprecated fields 9109c692 Fix build image push issue c6e899d6 Fix issues related to logging and plugin cache 06f5616e Update build setup to publish sonobuoy binary and image for s390x 17b70502 Add sonobuoy wait command 4b072ef2 Fix plugin nodeSelector issue inc. systemd-logs 95eb1e22 Fix logs bug causing aggregator to be dropped 582ac8a6 Update GoReleaser config for linux-ppc64le 51f037b2 Update build script to build ppc64le binary and image f6ed8233 Bump version of systemd-logs 0434aa03 Allow plugin renaming via @ seperator in flag name a01be9bd Clarify plugin list output surrounding name/file 1ca5b4e3 Add plugin management blog

Overview

This is a big release filled with new features, bug fixes, and UX improvements for all Sonobuoy users and plugin developers.

Major Changes

• By default, sonobuoy run --wait and sonobuoy delete --wait will now output much more information. This was an experimental feature for a release and it was extremely useful. When waiting on a run, the user is effectively given timestamped changes to sonobuoy status data so you can see when plugins are progressing. When running sonobuoy delete --wait, the JSON for the namespace is output as it changes. This makes it clear what is causing namespace deletion to be slow or hang.
• Adds some basic plugin management capabilities. You can now try out sonobuoy plugin install <file or URL> and it will store the plugin in a central location so that when you run sonobuoy run -p myplugin the plugin doesn't have to reside in the present working directory. Expect a blog post on this feature as it is a very new capability and we have some ideas how to make it better.
• Sonobuoy now supports plugin results in go-json format
  • This is the format from go test --json and enables us to support the upstream e2e-framework. To write a plugin that uses this format, simply change the result-type to go-json.
• Added a securityContext by default and set the user/group/fsGroup; this prevents certain security tools from blocking typical Sonobuoy runs. This is not supported on Windows nodes so a flag was added to turn off this functionality (--security-context-mode=none) if desired.
• Adds multiple new env vars to plugin environments when run including:
  • SONOBUOY=true to indicate that it was launched by Sonobuoy
  • SONOBUOY_CONFIG_DIR to identify where configmaps will be mounted at
  • SONOBUOY_RESULTS_DIR to identify where the results should be saved
• By default, Sonobuoy will now gather logs for all pods in kube-system and will query all API resources it can, including CRDs (but not secrets!)
• Add --rerun-failed flag to sonobuoy run. This takes a filename as a parameter and will walk the given tarball for e2e results and adjust the E2E_FOCUS accordingly to target just those failed tests. This is meant to be a replacement for the older sonobuoy e2e command.

Minor Changes

• sonobuoy gen will not support all the flags that sonobuoy run does including things like --wait and --skip-preflight in order to make it easier to change back and forth between run and gen for testing
• Fixed a bug where --level was not respected on the aggregator container and added more logging at the --level=trace.
• Documented underlying issues with supporting Kubernetes for Docker Desktop
• Removed managed field data from the data that Sonobuoy queries. This is usually not relevant for users and is verbose and confusing.
• Fixed a bug which caused podLogs to only be gathered for one pod (typically the Sonobuoy aggregator)
• Adds a useful dev feature: if SONOBUOY_DEV_REPO is set, it will use that repo for the default Sonobuoy image to facilitate faster iteration
• Fixed a bug where a plugin's name would cause an invalid configmap name to be generated. A few more restrictions and validation were added to make the naming limitation more clear.
• Fixed a bug where, when a plugin failed, the Sonobuoy aggregator status would report itself as failed regardless of other plugins. This was originally by design but as we have more fine-grained reporting abilities now, it is not necessary and is potentially confusing.

Changelog

08bb70f4 Bump version and docs for v0.54.0 release b214bb79 Add go-json output support for results 0e55157c Combine gen and run flags 623d7394 Add tracing throughout code and ensure level is passed to server 57b5ecc0 Add known issue related to docker desktop af875435 Add flag/config setting for security context b633b4fe Export and tweak some of the manual results methods/values 0ffc70bf Add more auto env vars to plugins df0e77d8 Default our experimental features to true b51156c2 Increase default query data 2adb0a54 Removed managed fields from query output b84102e8 Improve various logging points on aggregator 7d59d807 Adds easy way for devs to swap their own registry fe560d01 Do not use deprecated SelfLink ca277362 Add default security context to aggregator 0f941689 Added validation to gen [plugin] for plugin name 6df070d7 Adjust the logic for processing plugin results for aggregator status 57feff6a Update contributors on site 90abde64 Fix feature gate name a189f910 Replace master with main 26b589b7 Show/maintain fewer versions of docs dc458b50 Tweaks logging and how pluginList uses cache f1060a26 Use experimental features in test 34a3fc39 Absorb e2e functionality into rerun-failed flag baffa300 Add vulnerability scanning to CI 63394b68 Fix readme for github d3bca9e7 Fix docs links for 1388 issue

Shortly after releasing v0.53.1 a bug was found that impacted certified-conformance mode. Since this is an important function of Sonobuoy, we decided to release a patch for this single change. See more details here.

Changelog

981a3ffd Point release for certified-conformance issue a6504007 Add documentation about issue 1388 3a522d4c Fix regression regarding certified-conformance mode

* NOTE *

This version of Sonobuoy was impacted by #1388 and as a result we decided to push a new point release. Users interested in certified-conformance mode should use a newer version and/or see here for more details.

Sonobuoy v0.53.1

This patch release includes a few bug and security fixes. No other new functionality was added.

Fix typo which broken custom registries for e2e plugin (373bdc2b)

It would be looking in /tmp/sonobuoy/configs/... for the file but it should have been config without an s.

Updates various references for security reasons (584a3e0c)

There were 3 separate CVEs that were reported to us as a result of different dependencies. By updating these we resolved the vulnerabilities. The CVEs were from client-go, k8s.io/api/v1/core, jwt-go, and protobuf.

Fix broken link after plugins moved (bb9776e3)

We moved all our example plugins into the separate github.com/vmware-tanzu/sonobouy-plugins repository and had failed to update the links in our documentation.

Warn if kubernetes-version and set with conflicting flags (7b2fe4d2)

If kubernetes-version and kube-conformance-image[-version] are both set then some unexpected results can occur. This mitigates the problem by providing a warning of that fact.

* NOTE *

This version of Sonobuoy was impacted by #1388 and as a result we decided to push a new point release. Users interested in certified-conformance mode should use a newer version and/or see here for more details.

Sonobuoy v0.53.0

This release includes numerous small improvements and two experimental features which we expect to become default behavior in the future: better progress reporting when --wait is used and plugin management.

Plugin Management (9539f16)

All users can take advantage of new commands to help manage your plugins. This helps keep track of plugins so that you can save them from URLs/files and invoke them more easily from any other directory. See sonobuoy plugins for commands to install, uninstall, list, and show plugins.

In the future, Sonobuoy will look for installed plugins before looking in the present working directory. If you would like to enable this behavior now, set the environment variable SONOBUOY_ALL_FEATURES=true or SONOBUOY_PLUGIN_INSTALLATION=true.

Progress Messages (eeff4bf)

Currently, when you invoke sonobuoy run --wait or sonobuoy delete --wait the default --output-mode is silent. The only alternative, until now, has been a spinner to ensure CI and end-users don't think Sonobuoy has frozen.

However, we often have more information that would be useful for a user to know about the progress made. As a result, we've added a new option --wait-output=progress which will output more valuable information.

This may become the default in the future but to enable it now you can either specify it manually with --wait-output=progress or with SONOBUOY_ALL_FEATURES=true.

Minor Changes

Sonobuoy Retrieve (8cf38a5)

Adds a new flag --filename -f to specify the exact name of the tarball once it has been downloaded. This may ease the burden when writing scripts.

Custom Log Levels (69cdfc5)

We've added a --level flag for custom log levels so you can get more/less output based on your needs.

New Fields For Plugin Definitions

Added source-url and description fields so that a plugin can be more self-documenting. This will be of great use when plugins can be installed/organized more easily.

ImagePullPolicy Change (60adfff)

If you specify the --image-pull-policy flag, it will now be applied to all plugins, not just the built in e2e and systemd-logs plugins. The reasoning is that we would like to make custom plugins and built-in plugins have as much parity as possible.

Other changes since the last release

3e8f806 Bump version 0bc9e58 pkg/client/results: fix dropped test error 0a59430 Update site docs cc3607c Tweak CLI language 9539f16 Loads plugins from installation directory if feature enabled 5d53c20 pkg/plugin/driver/utils: fix multiple imports dedbbcd Update sonobuoy.io button to edit docs and report issues f9f2b4e Adds plugin installation functionality a63033a add GcEtcdRegistry to GetDefaultImageRegistries for API server aggregator test b5dfc48 pkg/tarball: fix dropped test error a573937 Add new fields to plugin manifests 68f4b32 Blog update regarding quick mode ee7227f Blog describing Kubernetes suite 1b626d7 Update more e2e flags to use transforms and remove special logic 0ae0a6f Use transform to rewire the ssh key logic 5f6c8b5 Add configurable log levels for logrus c5063fb Add generic plugin transformations and use for kube-repo-list ac68f65 Default to assuming the conformance image can support progress d5c90a2 Rework the gen plugin commands in order to reuse more code d97f12f Wire e2e focus/skip/mode/parallel into existing env structure 9f293f2 Fix a unit test by avoiding local kubeconfig 564f622 Simplify the way mode and plugin selection is processed 7af6504 Simplify config handling 709de5b Adds extra gen tests 8e47c2a Default config.New to not provide UUID 9ffa8d7 Update github actions workflows to show history dc4be9c Add more descriptive status output c738d7c Run integration tests in parallel 576dcdb Add convenience function for updating goldenfiles 4c029aa Fix version resolution during image pulling 3889bbd Add sleep after systemd-logs again cf5bbd4 Prevent logs -f from repeating d0258d6 Add K8sOnWindows alternative to Azure

v0.52.0

This release comes on the heels of our v0.51.0 but includes an incredibly some incredibly useful improvements for custom plugins: variable image names and ConfigMap embedding.

We made these changes to better fulfill our goal of running the Kubernetes E2E tests on Windows as easily as possible. Our new windows-e2e plugins make use of both of these features.

ConfigMaps Embedded in Your Plugin (d2e9dfee)

Plugins can now specify ConfigMap data directly in the plugin specification file itself. The new config-map field is an optional, sibling field to the sonobuoy-config and spec fields. Sonobuoy will automatically generate a ConfigMap with that data and mount it as a volume on your plugin so it can consume the data. This allows you to more easily bundle and share your plugin configuration with others.

Variable-based Plugin Images (eed1cd73)

This feature allows plugins to base their plugin image on the version of Kubernetes detected in their cluster. This behavior was previously reserved for our custom e2e plugin, but has been generalized for other plugins. Simply use $SONOBUOY_K8S_VERSION in the image name to make use of it.

It is also available automatically as an environment variable (by the same name) in your plugin.

v0.51.0

Major changes

Windows support af895d76 63d15736 fcc107c5

The largest announcement of this release is that Sonobuoy will now build/publish images for Windows nodes. This means that plugins can be easily written for and run on Windows nodes. Various other changes this release were motivated by this change to work towards making the Windows experience as positive as possible. Check out our blog on more regarding Windows support.

Other Improvements

0586df2e Improve retrieve command with extraction option

sonobuoy retrieve now has an --extract flag which will download then extract the tarball in the specified directory (defaults to the present working directory).

9b9e6e8f Add support for testing against the most recent dev E2E image

This helps simplify the workflow if you find yourself actively working on the tip of Kubernetes. Test against the master branch of k8s by specifying the --kube-conformance-image-version=latest. It does use a version that is built periodically and may lag what you see in github but minutes/hours.

649cfacf E2E Image ignores metadata and provider build info

This should make things work more seamlessly when you use clusters created by cloud providers which may add their own metadata onto the build information (e.g. v0.19.3+aks.some.info). Now we'll just ignore all the metadata and assume you want to test with the upstream version provided.

d7d930c2 Extend default timeout

The amount of time Sonobuoy waits for the plugins to report is always customizable with the --timeout flag, but we extended the default timeout from 3 to 6 hours in order to avoid timeouts potentially caused by running on Windows clusters where pods take longer to startup/shutdown.

2c86eb18 Add node selector option when generating plugin spec

Adds a new flag: sonobuoy gen plugin --node-selector foo=bar to facilitate creating windows plugins more easily.

c96ae198 Adjust default nodeSelector behavior

Along with the above change, by not specifying a nodeSelector Sonobuoy will assume you want to target Linux nodes. This, in most cases, avoids an issue where you have a Windows node in a cluster and it causes your (Linux) plugin to accidently run on that node and fail.

Bug fixes

04acfcbe Avoid infinite loop when a following logs

And other misc. fixes in the documentation such as spelling, tweaks for clarity, and fixing broken links.

Sonobuoy v0.50.0

New version

This release introduce support for distroless images for Sonobuoy. The release also resetting its release version by increasing the release number to v0.50.

Read more here https://sonobuoy.io/sonobuoy-v0-50/.

Documentation update

Part of this release includes migrating the content generator, for the Sonobuoy site, to use the Hugo content server. This should make content management and generation faster and easier.

New Reliability Scanner plugin blog

The realibility scanner was announced earlier this year. It is a Sonobuoy plugin that allows cluster operators to configure a set of reliability checks to be executed against a cluster. You can find a write up here.

Changelog

d9ec0c19 v0.50 release (#1221) da8d5d33 Reliability-As-Code blogpost (#1217) d9e74851 Update to policy text in SECURITY.md (#1214) 0d4ff325 Finish contributors migration to Hugo c5c3695a Major website overhaul - moving to Hugo (#1220) 4daa503c Merge pull request #1205 from wilsonehusin/distroless 3aeb7306 Add x509ignoreCN and CGO_ENABLED=0 for integration tests cccc0b0f Update contributors section (#1208) e4b694cd Extend stale markers 13c2ffc9 Circle CI tests migration to GitHub Actions (#1204)

Sonobuoy v0.20.0

What's new

We decoupled Sonobuoy releases from Kubernetes versions and announced our roadmap towards v1.0, let us know what you think!

Changelog

f6e1914 Release 0.20 (#1198) f5437c1 Use "stale" label for stale bot (#1200) 75fb977 Merge pull request #1201 from vladimirvivien/toward1-blog 9617f99 Blog annoucing vision and roadmap fa53698 Add Docker Hub rate limit workaround (#1194) 0c072ef Dynamically determine image of plugins (#1192) df8d4eb Create codeql-analysis.yaml (#1193) 0f7c90c Merge pull request #1190 from wilsonehusin/remove-contributors f911864 Use Kramdown instead of Redcarpet 031d604 Update Jekyll and therefore Kramdown 35dc86e Update Jekyll dff8ef4 Remove contributors section 185c859 go 1.15 + unit test + lint/sync workflows (#1186) 3e0f8ed Add StaleBot (#1184) be8c386 Deterministic TagPair assertion of custom registry (#1183) 7419e1d Add golangci-lint on GitHub Actions with fixes (#1179) 883fc90 Merge pull request #1169 from vladimirvivien/inventory-plugin-doc e5b16ea Adding cluster inventory plugin doc