Swift benchmark runner with many performance metrics and great CI support
Adds support for a new metric:.instructions
this captures the number of CPU instructions used for executing the benchmark:
This is done using perf_events
on Linux and using rusage
on macOS.
For benchmarks running in a VM, this metric might not return any data due to lack of access to CPU performance counters.
Full Changelog: https://github.com/ordo-one/package-benchmark/compare/1.22.0...1.22.1
For benchmarks that are not completely stable (in e.g. syscall/malloc count) due to use of e.g. async or networking, it is desirable to also be able to specify some leeway for benchmarks even for the absolute checks from thresholds.
This fixes https://github.com/ordo-one/package-benchmark/issues/220
By default the behaviour for absolute thresholds checks are as previously (strict, zero improvements/regressions are allowed), but now it is possible to define both absolute and relative thresholds for the benchmarks (as documented at https://swiftpackageindex.com/ordo-one/package-benchmark/1.21.3/documentation/benchmark/writingbenchmarks)
Full Changelog: https://github.com/ordo-one/package-benchmark/compare/1.21.2...1.21.3