A modern load testing tool, using Go and JavaScript - https://k6.io
k6 v0.43.1 is a patch release containing a few bugfixes:
setup() code when vu.iterationInScenario from k6/execution was used.stdout UI.k6 v0.43.0 is here! :tada:
Notable changes in this release include:
async/await.Keep reading for the details.
#2807 Use non-zero exit codes for tests aborted by Ctrl+C or the REST API.
Aborting a test run with Ctrl+C will now exit with code 105, and stopping via the REST API will exit with code 103.
This release includes xk6-browser as an experimental module. This means you can now also use the main k6 binary for browser automation and end-to-end testing, without needing to build a custom binary with xk6.
All xk6-browser scripts that work with v0.8.0 will continue to work with the built-in module in k6 v0.43.0. To use them, change the import path from k6/x/browser to k6/experimental/browser, and set the environment variable K6_BROWSER_ENABLED to true. The requirement to specify the environment variable is temporary and may be removed in a future k6 release. It was added to minimize the risks with k6 unexpectedly launching a browser (or another process) from k6 scripts. It's also a mechanism we use in the k6 Cloud, where browser tests are currently disabled.
For details, review the script example, or the updated browser module documentation.
The module is currently under the experimental namespace, which means we reserve the decision to introduce breaking changes in the future. However, our mid-term goal is to drop the experimental label and make browser support a stable part of the k6 feature set, eventually enabling it in k6 Cloud as well.
async/await #2830In v0.35.0 we added support for asynchronous functionality in k6 scripts with the addition of Promise.
While useful, the experience wasn't very friendly. Scripts had to use the .then() API to chain Promises, instead of the await syntax available in most other JavaScript runtimes, and the async keyword wasn't supported. Some workarounds were possible, but it required a separate build pipeline to transpile the syntax into the older ES5.1+ standard supported by k6.
That is, until now! :tada: With invaluable help from @dop251, who maintains goja, the JS VM k6 uses, v0.43.0 brings native async/await to your k6 scripts. This functionality works just as you'd expect in other JS runtimes, and makes working with async APIs much more convenient. For details, review the following http.asyncRequest() example.
One caveat to note: async functions can't be passed to group() or check(). These functions are incompatible with asynchronous behavior, so you will get an error if trying to do so.
This release brings a new experimental JavaScript module that adds distributed tracing support to k6. With one call in init context, you can instrument your test script's HTTP requests. If the system you're testing is instrumented in the same way, this module brings visibility to SUT behavior for the lifetime of each request.
An example:
import tracing from 'k6/experimental/tracing';
import http from 'k6/http';
tracing.instrumentHTTP({
propagator: 'w3c',
});
export default () => {
http.get('https://httpbin.test.k6.io/get', {
headers: {
'X-Example-Header': 'instrumented/get',
},
});
};
For details and examples, refer to the tracing module documentation.
http.asyncRequest #2877The k6/http module has a new asyncRequest function that takes the same arguments as http.request(), but returns a Promise that, when used with await, will be resolved with a Response object. This gives you more control over script execution, as potentially the most time-consuming calls—making HTTP requests—will no longer block the thread of execution.
An example issuing a POST request:
import http from 'k6/http';
export default async function () {
const resPromise = http.asyncRequest(
'POST', 'https://httpbin.test.k6.io/post', { name: 'Bert' });
// Do something else here, make other requests, etc.
// Then when you're ready to use the response:
const resp = await resPromise;
console.log(resp.json().form.name); // Bert
}
This is one of the first steps towards migrating our APIs to be asynchronous, and similar changes can be expected in the future.
You can read more about asyncRequest in the documentation.
k6 version command has been enhanced to also show the version of all extensions built into the k6 binary produced by xk6. Thanks, @HarrisChu!os package.csv output now correctly shows vu and iter system tags. This fixes a regression introduced in v0.40.0. Thanks, @leonyork!k6/execution.test.abort() within a group() now correctly exits the k6 process with code 108. Thanks for reporting this, @pomeh!k6/ws when using Socket.setInterval() with values between 0 and 1.run_status value when the cloud output aborts a test.Engine, was removed, and the behavior heavily refactored. This change simplifies the code base and unblocks further improvements.k6/http module.js.Bundle was simplified.Full Changelog: https://github.com/grafana/k6/compare/v0.42.0...v0.43.0
k6 v0.42.0 is here! :tada:
This release includes:
hosts.k6/ws returns an HTTP response for failed connections instead of an undefined behavior. Thanks, @brietaylor.This release brings a new builtin Output to any Prometheus Remote Write implementations (e.g. Prometheus, Mimir). This is an experimental feature, and future releases could introduce breaking changes.
The following example uses k6 run with the new output. It uses the defaults options, such as the Remote Write server URL (http://localhost:9090/api/v1/write):
k6 run -o experimental-prometheus-rw script.js
It supports the new and convenient experimental Native Histogram feature, added in Prometheus v2.40.0. It's not enabled by default, but we expect to make it the default way to map k6 Trend metrics once the Prometheus project signals that its mature enough and when more Remote Write implementations support it. For now, if you want to use it, you need to set the environment variable K6_PROMETHEUS_RW_TREND_AS_NATIVE_HISTOGRAM to true.
You can find complete documentation with more examples, use cases, and available configurations.
The k6/experimental/websockets module that we announced in the v0.40.0 release got an update that extends its functionality.
It brings some useful features that the k6/ws module already has, like cookies, custom headers, compression and tags customization support, the syntax to define event handlers (onopen, onmessage, etc.) and ping/pong functionality.
This is still an experimental module, but with the recent changes we think it's usable for most users. So whether you're writing a new WebSocket test, or currently using the k6/ws module, we invite you to give it a try, and report any issues in the project's issue tracker.
This example customizes tags for a WebSocket connection, sets up handlers using the new on* syntax, and demonstrates the ping/pong feature.
import { WebSocket } from "k6/experimental/websockets";
import {
setTimeout,
clearTimeout,
setInterval,
clearInterval
} from "k6/experimental/timers";
const CLOSED_STATE = 3
export default function () {
const params = {
"tags": {
"my_tag": "hello"
}
};
const ws = new WebSocket('ws://localhost:10000', null, params);
ws.onopen = () => {
console.log('connected');
ws.send(Date.now().toString());
};
let intervalId = setInterval(() => {
ws.ping();
console.log("Pinging every 1 sec (setInterval test)");
}, 1000);
let timeout1id = setTimeout(function () {
console.log('2 seconds passed, closing the socket');
clearInterval(intervalId);
ws.close();
}, 2000);
ws.onclose = () => {
clearTimeout(timeout1id);
console.log('disconnected');
};
ws.onping = () => {
console.log("PING!");
};
ws.onpong = () => {
console.log("PONG!");
};
// Multiple event handlers on the same event
ws.addEventListener("pong", () => {
console.log("OTHER PONG!");
});
ws.onmessage = (m) => {
let parsed = parseInt(m.data, 10)
if (Number.isNaN(parsed)) {
console.log('Not a number received: ', m.data);
return
}
console.log(`Roundtrip time: ${Date.now() - parsed} ms`);
let timeoutId = setTimeout(function () {
if (ws.readyState == CLOSED_STATE) {
console.log("Socket closed, not sending anything");
clearTimeout(timeoutId);
return;
}
ws.send(Date.now().toString());
}, 500);
};
ws.onerror = (e) => {
if (e.error != "websocket: close sent") {
console.log('An unexpected error occurred: ', e.error);
}
};
};
The module docs has a complete reference, and some examples.
hosts option #2747Thanks to the great effort from @eugercek, the hosts option now accepts domains that contain a wildcard at the beginning.
It can be helpful for setting multiple subdomains of the same domain, so instead of setting subdomain1.k6.io': '1.2.3.4', 'subdomain2.k6.io': '1.2.3.4' it is possible to use the wildcard for setting directly *.k6.io: '1.2.3.4'.
export const options = {
hosts: {
'*.k6.io': '1.2.3.4',
},
}
setup() and teardown() only if they are defined in the script.ws_session_duration when setup throws an error.We had a few minor changes in this release:
WS.Connect() has been refactored.require and open.Full Changelog: https://github.com/grafana/k6/compare/v0.41.0...v0.42.0
k6 v0.41.0 is here! :tada: It has relatively few user-facing changes, but includes massive internal improvements that pave the way for some awesome features for the near future. Unfortunately, this work also required us to make a few minor breaking changes.
url, iter and vu system metric tagsAs we warned in the release notes for k6 v0.39.0 and v0.40.0, we've been fundamentally refactoring the metrics sub-systems of k6. We now have efficient support for time series, which required a few minor user-facing breaking changes:
http.url helper is used or the name metric tag is specified), then the url tag in the resulting http_req_* metric samples will also have the same value as the name tag. Previously, k6 did this only for the cloud output, but now it does this universally (#2703).vu and iter system tags, which are disabled by default, have been transformed into high-cardinality metrics metadata instead. It means that they will no longer be usable in thresholds, and various outputs may emit them differently or ignore them completely (#2726).While the user-facing changes from our metrics refactoring are few and relatively minor, and there are no changes to JavaScript APIs yet, we have extensively refactored our internal Go APIs (#2594, #2726, #2727). The metrics.Sample and metrics.TagSet types are now entirely different. We also have high-cardinality metadata attributes in each Sample and at the VU level (see the combined TagsAndMeta code and how it is used in the per-VU State object).
k6 convert is officially deprecated (#2714)k6 convert has been a sub-command to convert a HAR file recording of HTTP traffic into a preliminary k6 script that makes roughly the same requests. It has been long neglected and softly deprecated in favor of the newer and more feature-rich har-to-k6 standalone converter.
We have now officially deprecated k6 convert. The command still works and will continue to do so for a few more k6 versions. However, it's not visible from k6 --help and will emit a warning when used. Please see the documentation for the standalone har-to-k6 converter and open an issue (or comment on an existing one) if you have any problems with it.
maxReceiveSize and maxSendSize parameters in the gRPC's Client.connect() method. Thanks, @ariasmn!--exclude-env-vars CLI flag to k6 archive that causes it to not include the provided environment variables in the resulting archive bundle's metadata.json file.Content-Type response header. Thanks, @wingyplus!k6 version information embedded in the k6 releases.Location headers (#2474) by updating the Go version we use to compile k6 to 1.19.x. Thanks, @agilob!k6/net/grpc module (#2661). Thanks, @c47gao and @robpickerill!undefined reason.We had a few minor changes in this release:
We also have a couple of significant improvements that will help us develop exciting new features soon:
Previous to #2594, k6 didn't have an efficient way to group metric samples with the same tags. It meant that a whole class of applications for processing and aggregating metrics were nearly impossible to do or, at best, very inefficient.
At the cost of some minor breaking changes, we now have a performant internal representation to group metric samples with the same tags at the time of the action that generated them, i.e. the time of metric measurement. With this, k6 can efficiently group samples for the same action (e.g. an HTTP request to a specific URL) over time and construct time series with them.
As described in the previous section, the efficient grouping of metric samples into time series works well for relatively low-cardinality data. However, k6 needed some way to attach high-cardinality metadata as well. This is necessary for data that's unique or random, such as Trace and Span IDs in distributed tracing or user IDs in tests with huge data sets.
k6 v0.41.0 has added support for attaching high-cardinality metadata to metric samples, and the vu and iter system tags have been transformed into such metadata (see the breaking changes section above), but it is not yet accessible from user scripts. There is no JavaScript API to modify this metadata, only built-in k6 Go modules and xk6 Go extensions can make use of it, for now.
k6 v0.40.0 is here! This release includes:
Finally, the Roadmap goes over the plans for the next cycles.
metric.add in the order they are provided, we discovered that you could provide tags as a key-value pair map multiple times in the same call. This was never the intended use and was never documented. As it was undocumented, and as such functionality makes no sense alongside every other API k6 has, we decided to remove this ability.RegisterCallback result was called twice, the second call would silently break the event loop. This has never been expected behavior, and calling it twice is always a bug in the code using it. Now, calling the RegisterCallback result twice leads to a panic.tainted property of the Metric type is no longer outputted by the JSON output. That property was likely always going to have a false value as it was outputted at the beginning of the test.During the ESM changes, we found that anything defined in the main module scope was also accessible globally. This was because it was directly evaluated in the global scope. This has now been remedied and is no longer the case. This is a breaking change, but given that the whole point of modules (CommonJS or ESM) is to separate them, this is obviously rather a bug than a feature.
On that note, we've seen reports by people who have this global accessibility of the main module (intentionally or not). Still, it seems relatively rare, with only a few usages in a script. So if you need to access something globally, our suggested workaround is to set it explicitly on the global object globalThis.
k6/ws now respects the throw option #2247k6/http has used the throw option to figure out whether it should throw an exception on errors or return a response object with an error set on it (and log it).
This functionality is finally also available for k6/ws, which previously would've always thrown an exception and thus involved more scripting in handling it (#2616).
This is a minor breaking change. By default, throw is false, so it now no longer throws an exception but instead returns a Response with error property.
Thank you, @fatelei, for making this change!
As mentioned in the v0.39.0 release notes, we're happy to announce that this release brings experimental modules. The main idea behind this initiative is to get community feedback earlier, which will help us improve them faster. We encourage you to try experimental modules out and provide feedback through the community forums or GitHub issues.
This release contains three experimental modules:
k6/experimental/redis - support for interaction with Redis
k6/experimental/websockets - a new Websockets API that copies the "web" WebSocket API
k6/experimental/timers - adding setTimeout/clearTimeout and setInterval/clearInterval implementations.Important to highlight that the k6 team does not guarantee backward compatibility for these modules and may change or remove them altogether. Also, their import paths, starting with k6/experimental, will break when the modules stop being experimental. Of course, we are going to try to limit those breaking changes to a minimum and, when possible, do them in a backward compatible way for at least a version.
Here is a fairly big example using xk6-redis as an experimental module to keep track of data in Redis:
import { check } from "k6";
import http from "k6/http";
import redis from "k6/experimental/redis"; // this will be `k6/x/redis` if you are using it as extension
import { textSummary } from "https://jslib.k6.io/k6-summary/0.0.1/index.js";
export const options = {
scenarios: {
usingRedisData: {
executor: "shared-iterations",
vus: 10,
iterations: 200,
exec: "measureUsingRedisData",
},
},
};
// Instantiate a new redis client
const redisClient = new redis.Client({
addrs: __ENV.REDIS_ADDRS.split(",") || new Array("localhost:6379"), // in the form of "host:port", separated by commas
password: __ENV.REDIS_PASSWORD || "",
});
// Prepare an array of crocodile ids for later use
// in the context of the measureUsingRedisData function.
const crocodileIDs = new Array(0, 1, 2, 3, 4, 5, 6, 7, 8, 9);
export function setup() {
redisClient.sadd("crocodile_ids", ...crocodileIDs);
}
export function measureUsingRedisData() {
// Pick a random crocodile id from the dedicated redis set,
// we have filled in setup().
redisClient
.srandmember("crocodile_ids")
.then((randomID) => {
const url = `https://test-api.k6.io/public/crocodiles/${randomID}`;
const res = http.get(url);
check(res, {
"status is 200": (r) => r.status === 200,
"content-type is application/json": (r) =>
r.headers["Content-Type"] === "application/json",
});
return url;
})
.then((url) => redisClient.hincrby("k6_crocodile_fetched", url, 1));
}
export function teardown() {
redisClient.del("crocodile_ids");
}
export function handleSummary(data) {
redisClient
.hgetall("k6_crocodile_fetched")
.then((fetched) => Object.assign(data, { k6_crocodile_fetched: fetched }))
.then((data) =>
redisClient.set(`k6_report_${Date.now()}`, JSON.stringify(data))
)
.then(() => redisClient.del("k6_crocodile_fetched"));
return {
stdout: textSummary(data, { indent: " ", enableColors: true }),
};
}
This example also showcases how to write some data and clean up after yourself.
The extension does not run a Redis server. You need to separately handle running, scaling, and connecting infrastructure to Redis.
The xk6-redis repository has more examples, and the module is documented in the official k6 documentation.
This is a rewrite of the current WebSocket example at https://test-api.k6.io/.
This showcases how a single VU can run multiple WebSockets connections asynchronously and how to stop them after a period using the timeout and interval functions.
import { randomString, randomIntBetween } from "https://jslib.k6.io/k6-utils/1.1.0/index.js";
import { WebSocket } from "k6/experimental/websockets"
import { setTimeout, clearTimeout, setInterval, clearInterval } from "k6/experimental/timers"
let chatRoomName = 'publicRoom'; // choose your chat room name
let sessionDuration = randomIntBetween(5000, 60000); // user session between 5s and 1m
export default function() {
for (let i = 0; i < 4; i++) {
startWSWorker(i)
}
}
function startWSWorker(id) {
let url = `wss://test-api.k6.io/ws/crocochat/${chatRoomName}/`;
let ws = new WebSocket(url);
ws.addEventListener("open", () => {
ws.send(JSON.stringify({ 'event': 'SET_NAME', 'new_name': `Croc ${__VU}:${id}` }));
ws.addEventListener("message", (e) => {
let msg = JSON.parse(e.data);
if (msg.event === 'CHAT_MSG') {
console.log(`VU ${__VU}:${id} received: ${msg.user} says: ${msg.message}`)
}
else if (msg.event === 'ERROR') {
console.error(`VU ${__VU}:${id} received:: ${msg.message}`)
}
else {
console.log(`VU ${__VU}:${id} received unhandled message: ${msg.message}`)
}
})
let intervalId = setInterval(() => {
ws.send(JSON.stringify({ 'event': 'SAY', 'message': `I'm saying ${randomString(5)}` }));
}, randomIntBetween(2000, 8000)); // say something every 2-8seconds
let timeout1id = setTimeout(function() {
clearInterval(intervalId)
console.log(`VU ${__VU}:${id}: ${sessionDuration}ms passed, leaving the chat`);
ws.send(JSON.stringify({ 'event': 'LEAVE' }));
}, sessionDuration);
let timeout2id = setTimeout(function() {
console.log(`Closing the socket forcefully 3s after graceful LEAVE`);
ws.close();
}, sessionDuration + 3000);
ws.addEventListener("close", () => {
clearTimeout(timeout1id);
clearTimeout(timeout2id);
console.log(`VU ${__VU}:${id}: disconnected`);
})
});
}
Note that no k6 iterations finish if any WebSocket is still open or if a timeout or an interval is not cleared or triggered. This means that your script must take care of clearing all intervals and closing the WebSocket at some point. However, k6 still kills the whole process if it takes too long to stop after the maximum test duration is reached.
Current issues and future improvements for the WebSockets API can be found in its issue tracker. Currently, documentation is available through MDN, though some features are not yet supported:
onMessage and co. - only addEventListenerAs part of updating goja, k6 got native support for classes. Again, that's native, as in not by transpiling by the internal Babel.
Because this actually implements classes as described in the latest ECMAScript specification, this also means we get a ton of additional class features that were never previously supported (for example, private fields). Additionally, at least one bug #1763 was fixed as a result of this, but probably many more as well.
Due to this fairly significant change, some code could behave differently. Please report any issues, though consider that it's possible that the new behavior is just the correct one.
Other updates from goja are:
\u{01234} Unicode point syntax in regexpMany thanks to @dop251 for continuing to improve goja!
While we develop extensions internally, we often need to repeatedly create the same structure. With the addition of the event loops, it is now required to set it up as well. Even k6 team members get parts of this wrong every once in a while, so we added a small type to be used by (extension) module developers to write tests easier (#2602).
This API will likely change and evolve as we add more functionality or as we change the k6 internal API.
http.batch() now displays an error if it is not given exactly 1 argument(#1289). Thanks, @vanshaj!tainted property is no longer outputted. That property was (likely) always going to have the value false as it was outputted at the beginning of the test.rps to 0 or below leading to exceptions. Now setting it to 0 or below disables the limit. Thanks, @tbourrely. #2613
options.tags directly was not possible. This was fixed by accident by #2631. k6/execution is still the recommended way to access the final options of the test.common.Bind.//nolint comments as part of getting ready for go 1.19.lose. Thanks @spazm!This section discusses our plans for future versions. Notice that two big ticket items are here again―ESM modules and metric refactoring. They remain on the roadmap mostly for the sheer size of the work required on both, but also for some unforeseen needed changes, which actually should make them better in the long run. It also so happens that it is vacation season so the k6 team rightfully is taking some personal time away.
Native ESM support is coming. A PR to k6 is under development, and there's a branch that will become a PR to goja to add the support there. The k6 team is hopeful that this will land in the next version, v0.41.0!
It turned out that there were a lot more things to be added as functionality - dynamic import, and the tc39 group released the latest ECMAScript specification adding support of top-level await in modules. While neither k6 nor goja has support for the async/await syntax, yet, this changes significantly the internal of the change, which did require a not insignificant refactor.
Additionally, as previously mentioned, there were a bunch of changes to goja, including adding class support which also needed to be integrated.
A future breaking change is that using CommonJS style exports along import/export syntax in the same file will no longer be possible.
import http from "k6/http"
exports.default = function() {} // this will start to error out
It will still be possible to import a CommonJS module and require an ES module or use require in an ES module. Having a dependency cycle mixing and matching CommonJS and ESM is also unlikely to work properly, but might do so in particular cases.
This is really expected to have never been done by anyone as there isn't really much of a reason for this, but it is currently supported due to Babel transpiling everything to CommonJS behind the scenes.
The refactoring of metrics is underway, with a PR for more performant internal representation. Unfortunately, this took longer to get to a stable state, and given the size and the type of change, the k6 team decided to hold merging it until very near the end of the cycle. It also doesn't have any noticeable change for most users. Instead, it will be merged early in the v0.41.0 release cycle, and then more changes to use it will be made through the release cycle.
Some of those changes include supporting non-indexable tags, which also have a read PR. This change is also essential for historical reasons connected with how the name tag works. As such, it also needed to be merged to release the internal metric refactor.
Future breaking change: as part of the many changes and experiments, we found out that we can keep url as an indexable tag. Previously the plan was that it along vu and iter will become non-indexable. However, the url tag is heavily used and enabled by default. Because of that (and other internal reasons), the new change will be that url will stay indexable, but if used with name will be overwritten to have the same value as name. Arguably this is what most users would want in the case when they are using name. We plan to add a non indexable raw_url tag for those that do not. As such, we no longer will be printing a warning when url is used in thresholds.
Even so, we did make a bunch of changes to the internals of k6 that will pave the way forward (#2629, #2631).
We are also still working on incorporating the newly developed time series data model for the Prometheus remote-write output extension. We are fixing bugs and improving the extension with the goal of eventually integrating it as a core built-in k6 output module in a future k6 release.
Distributed tracing support itself needs non-indexable tag support. Once that is merged, more work in that direction will be started.
k6 v0.39.0 is here! :tada: It's a small release that includes a bunch of bugfixes and minor enhancements. Much of our focus was on some upcoming big changes. You can read about what's coming up next in the Roadmap and future plans section.
csv output with support for a new timeFormat option. The possible values are unix (default) and rfc3399. You can also configure it through the K6_CSV_TIME_FORMAT environment variable. Thanks, @rpocklin!clear() and delete() methods to the CookieJar object from the k6/http module. Thanks, @Maksimall89!any.proto was used for a type. Thanks, @Flowersea!Selection.map() from the k6/html module to the correct object types instead of a forced array of strings.Rate metric and sub-metric values were shown as NaN in the end-of-test summary if there were no measured values for them during the test run.vu.tags element from k6/execution to null or undefined.k6/execution properties outside of a VU context, e.g. trying to access execution.scenario in setup().url, error, vu or iter tags, which will become un-indexable in the future.As the lack of big changes in this release suggests, we've focused the last few months' efforts on a few areas that haven't yet been merged into the core of k6.
In this section, we'd like to inform the community about important features that we're currently working on - our short-term roadmap in a sense. We'll also use it to give notice of breaking changes we plan to make in the near future.
k6/experimental/* JS modulesOver the last several k6 releases, among a lot of other refactorings, we've added support for JavaScript event loops (#2228) in k6 VUs and added a new Go API for exposing built-in and xk6 extension modules to user scripts (announcement, docs). This has given us (and any xk6-extension authors!) the ability to better support various asynchronous streaming/messaging/etc. protocols (#882).
We've started building some of these newly possible APIs as xk6 extensions first, to be able to iterate on them more quickly and get some user feedback while we are building them. xk6-websockets, xk6-timers and xk6-redis are some of the first such APIs, but we plan to also work on support for gRPC streaming (#2020), messaging protocols (#1269), a new and better HTTP API (#2461) and many others in the future!
We want to eventually include a lot of these APIs in the k6 core as built-in modules that users can directly use, without needing to mess with xk6 or Go compilation. However, because we try to keep the built-in k6 APIs stable and backwards-compatible, we want to get more user feedback before we do that, while we are still free to iterate and make (hopefully minor) breaking changes.
So, we decided to create a new middle ground between the unstable and purely external xk6 extensions and the stable built-in k6 APIs―built-in k6/experimental/* modules! Our goal is that, starting with the next k6 v0.40.0 release, we'll start releasing some or all of these core-bound extensions as built-in k6 modules under these k6/experimental/ import paths. This will let k6 users, both OSS and Cloud, to give us feedback and help us improve the APIs before we stabilize them.
As is hopefully clear from the name, our usual guarantees of API stability won't apply to these modules while they are still experimental. We reserve the right to make breaking changes in experimental modules, up to and including completely dropping them. We don't expect big breaking changes will need to happen often, but we want to be clear they aren't impossible. Finally, when an API has been stabilized and made available under a regular import path, we'll deprecate its experimental import path. To make the transition easier, both import paths will be available simultaneously for at least one k6 version.
At the moment, k6 has support for ECMAScript modules (ESM, i.e. import, export, etc.) via automatic transpilation of scripts by the built-in Babel.js. That mostly works, but it has caused some performance and compatibility problems (#824 and #2168 among others), so we want to support ESM modules and all other ES6 features directly in k6, without the need for Babel.js (#2296). goja, the JavaScript runtime we use to evaluate k6 scripts, doesn't yet have native ESM support, so we are currently working on adding it there, to then be able to support ECMAScript modules natively in k6!
That work has been ongoing for a while and we're making progress, but it will likely not be ready in time for the next k6 v0.40.0 release. We are mentioning it here because we will probably need to make a few minor breaking changes and fixes of currently undefined behavior in k6 to support ESM modules natively.
For example, at the moment, some values like the consolidated script options were unintentionally made available globally, in all JS modules of a test, instead of just in the exported options value from the main JS module. That is not the intended or documented behavior, it's somewhere between a bug and undefined behavior, and we'll need to fix it (#2571) and other similar issues like it, starting in k6 v0.40.0. We don't expect many scripts to break because of these fixes, but we'll nevertheless announce them in the release notes of the k6 version that they happen in.
Over the last several k6 releases, we've also slowly been refactoring and improving the metrics internals in k6 (see #2071, #2330, #2426, #2463, #2433, #2442, among others). This has already brought many side benefits and minor bugfixes, and we still have a lot of work left (e.g. #1889, #572, #2430, #1831), but we've finally reached the point where we can almost start implementing some major new features effectively!
One of the upcoming next big steps is the introduction of a "time series" concept internally in k6 (#2580). We'll start to efficiently group samples (i.e. metric measurements) with the same metric and tags into a single TimeSeries, which would unlock many of possibilities that were previously too inefficient to implement.
Another upcoming metrics-related change would be the refactoring of metric tags into two distinct types (#2584) - ones that can be used for indexing (so, in the TimeSeries mentioned above, in sub-metric thresholds, in certain outputs that do aggregation based on tags, etc. ) and ones that cannot (e.g. high-cardinality metadata that's often unique for every data point).
Unfortunately, we'll need to make a few minor breaking changes. The current url, error, vu and iter system tags will be made non-indexable by default, to reflect their usually very high cardinality, so they won't be usable in thresholds. Instead of error, the error_code tag should be used there. And instead of url, users should already use the name tag to avoid issues, see the documentation about URL grouping. k6 v0.39.0 still supports thresholds with these tags, it will just print a warning if they are used.
Because of these changes, we'll finally be able to refactor and improve the Prometheus remote-write output extension. Our goal is to make it more efficient, polish its rough edges, and merge it into the k6 core as a built-in output module in the near future.
Another new feature these changes would unlock is support for distributed tracing (#2128).
Because of the upcoming support for high-cardinality unindexable metric tags, we'll be able to safely attach things like the unique trace and span IDs to every metric measurement without negatively affecting other parts of k6.
k6 v0.38.3 is a patch release containing a single fix
There was a bug where we were checking if a submetric had already been added. Unfortunately, we didn't check that this will work with the one submetric we have by default http_req_duration{expected_response:true}. After v0.38.0 defining a threshold on it would result in an error.
As this definitely shouldn't happen in that case and we don't see a particular case where that will be problematic - adding a submetric again just reuses the already added one instead.
This issue has been addressed in #2539, and k6 v0.38.3 will now lead you add a threshold on http_req_duration{expected_response:true}.
k6 v0.38.2 is a patch release containing a couple of bugfixes!
NaN (#2520)There was a bug in thresholds applied to sub-metrics set to abortOnFail: leading k6 to evaluate thresholds that would have likely aborted before they had a chance of passing (because no samples for the given metric were recorded yet). This bug would have led to such thresholds' results value to be NaN rather than a numerical value. The following script, for instance:
import { check, sleep } from 'k6';
import http from 'k6/http';
export const options = {
scenarios: {
iWillFail: {
exec: 'iWillFail',
executor: 'constant-vus',
startTime: '2s',
vus: 1,
duration: '30s',
},
},
thresholds: {
'checks{type:read}': [{ threshold: 'rate>0.9', abortOnFail: true }],
},
};
export function iWillFail() {
let res = http.get(`https://test-api.k6.io/`);
check(res, {
'read status is 200': (r) => r.status === 200,
}, { type: 'read' });
sleep(1);
}
Would result in the following:
✗ { type:read }...: NaN% ✓ 0 ✗ 0
vus...............: 0 min=0 max=0
vus_max...........: 1 min=1 max=1
This issue was introduced by recent changes to how we handle thresholds in the k6 engine and is now addressed in v0.38.2.
There was in how thresholds over sub-metrics that didn't receive any samples would be displayed under an incorrect parent metric. For instance, the following script:
import { Counter } from 'k6/metrics';
const counter1 = new Counter("one");
const counter2 = new Counter("two");
export const options = {
thresholds: {
'one{tag:xyz}': [],
},
};
export default function() {
console.log('not submitting metric1');
counter2.add(42);
}
Would have led to the following output, where the {tag:xyz} sub-metric is displayed under iterations instead of one:
data_received........: 0 B 0 B/s
data_sent............: 0 B 0 B/s
iteration_duration...: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
iterations...........: 1 499.950005/s
{ tag:xyz }........: 0 0/s
two..................: 42 20997.90021/s
When we would have expected it to produce:
one..................: 0 0/s
{ tag:xyz }........: 0 0/s
two..................: 42
This issue has been addressed in #2519, and k6 v0.38.2 now displays sub-metrics under their actual parents, even when they have received no samples.
Special thanks to @efdknittlfrank, who reported and helped us track down the issue.
k6 v0.38.1 is a patch release containing a bugfix!
There was a bug in threshold sub-metric selector parsing, which led to errors when users would use specific symbols such as {, } or : as part of their definitions. For instance, thresholds used for sub-metrics with URL Grouping like http_req_duration{name:"http://example.com/${}"} would have led to failures in v0.38.0.
The error messages for invalid metric, sub-metric and threshold definitions were also improved.
Special thanks to @efdknittlfrank, who reported and helped us track down the issue.
k6 v0.38.0 is here! 🎉
There's a new addition to the officially supported k6 JavaScript libraries: k6-jslib-aws. This library lets users interact with a selection of AWS services directly from their scripts. The library currently implements support for the S3 and the Secrets Manager services.
The AWS JS lib documentation has examples and details on how to use the library in your scripts.
The k6/execution core module now lets you access the consolidated and derived options that k6 computed before it started executing the test. You can access consolidated options through the exec.test.options property. Note that consolidated options are frozen and cannot be modified. The k6 execution module's documentation has examples and details on how to use the functionality in your scripts.
import exec from "k6/execution";
export const options = {
vus: 10,
duration: "30s",
};
export default function () {
console.log(exec.test.options.scenarios.default.vus); // 10
}
With the new consolidated script options, we've added a few helper functions to the k6-jslib-utils library. You can use them to automatically tag all the emitted metric samples by k6 with the currently running stage.
The k6 documentation has examples and details on how to use it.
This release adds the ability to dump SSL keys while making TLS connections. You then can use these keys to decrypt all traffic that k6 generates.
To accomplish this, set the SSLKEYLOGFILE environment variable to some file path and run k6.
This will populate the file with the keys.
Then you can use Wireshark to capture the traffic, decrypt it, and use that for debugging.
Here's an example that uses curl to inspect TLS traffic.
console methods now pretty print objects and arrays (2375)For convenience, all console methods such as console.log() and console.info() will now automatically JSON.stringify() objects and arrays passed to them. Thus, instead of console.log({'foo': 'bar'}) printing [object Object], it will now print {'foo': 'bar'}, which will make the experience of debugging k6 scripts easier and more intuitive.
To achieve the previous behavior, cast the Object to a String, as in console.log(String({'foo': 'bar'})).
export default function () {
console.log([1, 2, "test", ["foo", "bar"], { user: "Bob" }]);
// before: 1,2,test,foo,bar,[object Object]
// after: [1,2,"test",["foo","bar"],{"user":"Bob"}]
}
stats package were moved to the metrics package #2433For convenience and to facilitate further developments, the types and functionalities that used to live in k6's stats package have been moved to the metrics package. The stats package is, as of v0.38.0, removed in favor of the metrics package. Besides, #2442 removed the stats.New function in favor of initializing new metric via a register.NewMetric call instead.
k6 v0.37.0 already improved threshold parsing by switching its underlying implementation from JavaScript to Go. k6 v0.38.0 introduces two additional improvements:
export const options = {
// ...
thresholds: {
// Incorrect thresholds expressions:
http_req_failed: ["rave<0.01"], // e.g. "rave" is not a valid aggregation method
// Thresholds applying to a non-existing metrics:
iDoNotExist: ["p(95)<200"], // e.g. the metric 'iDoNotExist' does not exist
// Thresholds applying an aggregation method that's unsupported by the metric they apply to:
my_counter: ["p(95)<200"], // Counter metrics do not support the p() aggregation method
},
};
In addition to the --no-color CLI flag, the ANSI color escape codes emitted by k6 can now also be disabled by setting the NO_COLOR or K6_NO_COLOR environment variables, following the NO_COLOR standard.
# No color output
K6_NO_COLOR=true k6 run script.js
# No color output
NO_COLOR= k6 run script.js
You can now use passphrase-protected private keys when authenticating with TLS. Using the password property of an options' tlsAuth object, you can now indicate the passphrase to decrypt a private key. Note that this support is limited to the scope of RFC1423 and does not support PKCS8 keys, as they're not yet supported by the Golang standard library.
export const options = {
tlsAuth: [
{
domains: ["example.com"],
cert: open("mycert.pem"),
key: open("mycert-key.pem"),
password: "mycert-passphrase",
},
],
};
Thanks, @Gabrielopesantos, for the contribution.
The JSON output was optimized and now should be around 2x more performant at outputting metrics. This means that it either can export twice as many metrics, or use half the resources to do the same amount of metrics.
As a side effect, there is a slight breaking change: the tags field is no longer sorted.
We changed the behavior of how k6 treats Go panics, which may happen because of bugs in k6 or in a JavaScript k6 extension. Previously, the behavior was to catch the panic and log it as an error.
Starting with v0.38.0, whenever k6 observes a Go panic, it logs an error like before, but more importantly, it will abort the script execution and k6 will exit with a non-0 exit code. This will help extension authors to identify issues in their extensions more easily.
--quiet flag.lib/types now exposes the source of the NullHostnameTrie to simplify access to an original list of the hostnames.We built a new xk6 extension, https://github.com/grafana/xk6-websockets, with a proof of concept implementation for a new JavaScript Web Sockets API. This API uses the global event loops introduced in k6 v0.37.0 to allow a single VU to have multiple concurrent web socket connections open simultaneously, greatly reducing the resources needed for large tests. It also is a step towards supporting the official Web Sockets JS standard, potentially allowing the usage of more third-party JS libraries in the future.
Please share any feedback you have about the new extension since it's likely that we'll adopt a future version of it into the k6 core in one of the next several k6 releases.
#2484 moved out in a new dedicated Go lib/netext/grpcext package all the parts not strictly required from the js/k6/net/grpc module for binding the gRPC operations and the JavaScript runtime. It facilitates the development of extensions based on gRPC without the direct dependency on the goja runtime. Furthermore, the new Dial function accepts a grpc.DialOption variadic for customizing the dialing operation.
With this release, you can export the event loop added in v0.37.0. This lets extension developers test event-loop-dependent APIs.
There were also updates to modulestest.VU to support the new API. Head to GitHub to see it in action.
domains property empty.data property of an http.file() result as a request body.9037c2b61cbf.