Hyperfine Versions Save

Features

• The new --parameter-list <VAR> <VALUES> option can be used to run a parametrized benchmark on a user-specified list of values. This is similar to --parameter-scan <VAR> <MIN> <MAX>, but doesn't necessarily required numeric arguments.

  hyperfine --parameter-list compiler "gcc,clang" \
      "{compiler} -O2 main.cpp"
  

  See: #227, #234 (@JuanPotato)

• Added none as a possible choice for the --style option to run hyperfine without any output, see #193 (@knidarkness)

• Added a few new scripts for plotting various types of benchmark results (https://github.com/sharkdp/hyperfine/tree/master/scripts)

Changes

• The --prepare command is now also run during the warmup phase, see #182 (@sseemayer)

• Better estimation of the remaining benchmark time due to an update of the indicatif crate.

Other

• hyperfine is now available on NixOS, see #240 (@tuxinaut)

Features

• The --prepare <CMD> option can now be specified multiple times to run specific preparation commands for each of the benchmarked programs:

  hyperfine --prepare "make clean; git checkout master"  "make" \
            --prepare "make clean; git checkout feature" "make"
  

  See: #216, #218 (@iamsauravsharma)

• Added a new welch_ttest.py script to test whether or not the two benchmark results are the same, see #222 (@uetchy)

• The Markdown export has been improved. The relative speed is now exported with a higher precision (see #208) and includes the standard deviation (see #225).

Other

• Improved documentation for scripts folder (@matthieusb)

Features

• Added a new -D,--parameter-step-size option that can be used to control the step size for --parameter-scan benchmarks. In addition, decimal numbers are now allowed for parameter scans. For example, the following command runs sleep 0.3, sleep 0.5 and sleep 0.7:

  hyperfine --parameter-scan delay 0.3 0.7 -D 0.2 'sleep {delay}'
  

  For more details, see #184 (@piyushrungta25)

Other

• hyperfine is now in the official Alpine repositories, see #177 (@maxice8, @5paceToast)
• hyperfine is now in the official Fedora repositories, see #196 (@ignatenkobrain)
• hyperfine is now in the official Arch Linux repositories
• hyperfine can be installed on FreeBSD, see #204 (@0mp)
• Enabled LTO for slightly smaller binary sizes, see #179 (@Calinou)
• Various small improvements all over the code base, see #194 (@phimuemue)

Features

• Added a -c, --cleanup <CMD> option to execute CMD after the completion of all benchmarking runs for a given command. This is useful if the commands to be benchmarked produce artifacts that need to be cleaned up. See #91 (@RalfJung and @colinwahl)
• Add parameter values (for --parameter-scan benchmarks) to exported CSV and JSON files. See #131 (@bbannier)
• Added AsciiDoc export option, see #137 (@5paceToast)
• The relative speed is now part of the Markdown export, see #127 (@mathiasrw and @sharkdp).
• The median run time is now exported via CSV and JSON, see #171 (@hosewiejacke and @sharkdp).

Other

• Hyperfine has been updated to Rust 2018 (@AnderEnder). The minimum supported Rust version is now 1.31.

Features

• Show the number of runs in hyperfines output (@tcmal)
• Added two Python scripts to post-process exported benchmark results (see scripts/ folder)

Other

• Refined --help text for the --export-* flags (@psteinb)
• Added Snapcraft file (@popey)
• Small improvements in the progress bar "experience".

Features

• Added -S/--shell option to override the default shell, see #61 (@mqudsi and @jasonpeacock)
• Added -u/--time-unit option to change the unit of time (second or millisecond), see #80 (@jasonpeacock)
• Markdown export auto-selects time unit, see #71 (@jasonpeacock)

Feature

• Compute and print standard deviation of the speed ratio, see #83 (@Shnatsel)
• More compact output format, see #70 (@jasonpeacock)
• Added --style=color, see #70 (@jasonpeacock)
• Added options to specify the max/exact numbers of runs, see #77 (@orium)

Bugfixes

• Change Windows cmd interpreter to cmd.exe to prevent accidentally calling other programs, see #74 (@tathanhdinh)

Other

• Binary releases for Windows are now available, see #87

• Support parameters in preparation commands, see #68 (@siiptuo)
• Updated dependencies, see #69. The minimum required Rust version is now 1.24.

• Added --show-output option (@chrisduerr and @sevagh)
• Refactoring work (@stevepentland)

Features

• Support for various export-formats like CSV, JSON and Markdown - see #38, #44, #49, #42 (@stevepentland)
• Summary output that compares the different benchmarks, see #6 (@stevepentland)
• Parameterized benchmarks via -P, --parameter-scan <VAR> <MIN> <MAX>, see #19

Thanks

I'd like to say a big THANK YOU to @stevepentland for implementing new features, for reviewing pull requests and for giving very valuable feedback.