corebench - run your benchmarks against high performance computing servers with many CPU cores
Benchmark utility that's intended to exercise benchmarks and how they scale with a large number of cores.
How does your code scale and perform when running on high-core servers?
First Provider: DigitalOcean up to 48 cores currently.
Second Provider: AWS, specify your preferred instance type, us-east-1a for now
# Install corebench
go get github.com/deckarep/corebench
DigitalOcean:
Run corebench:
// Fetch instance sizes
./corebench do sizes --DO_PAT=$DO_PAT
// Run a benchmark
./corebench do bench github.com/{user}/{repo} [OPTIONS] --DO_PAT=$DO_PAT --ssh-fp=$SF
// List active instances
./corebench do list --DO_PAT=$DO_PAT
// Terminate instances created by corebench
./corebench do term --DO_PAT=$DO_PAT --all
AWS:
Run corebench:
// Run a benchmark
./corebench aws bench github.com/{user}/{repo} [OPTIONS]
//Run a benchmark on this repo with instancetype xxx and leave the resources running
./corebench aws bench github.com/deckarep/corebench --instancetype m3.medium --leave-running true
// List running instances
./corebench aws list
// Terminate/delete AWS resource stack 'corebench'
./corebench aws term
Q: Why is 48 the max amount of cores this utility supports?
A: This is DigitalOcean's beefiest box. AWS cores currently specified by instance type.
Q: What happens to the server and the code after the benchmark completes?
A: The default behavior is the server is destroyed along with the code and benchmark data. There is a setting that allows you to leave the server running if you'd like to log in and inspect the results using the --leave-running flag.
Q: Why is DigitalOcean the first provider?
A: Easy, because their droplets fire up FAST allowing a quick feedback loop during development of this project.
Q: When you will you add Google Cloud, AWS, {other-provider} next?
A: AWS support has been added. Google Cloud is next because they offer per minute billing which is great to save money. I'm hoping the community can help me build other providers along with refactoring as necessary to align the API.
Q: Why did you build this tool?
A: Because I wanted to quick way to execute remote benchmarks on cloud servers that are beefy (large number of cores).
Q: Will you eventually support other languages?
A: Meybe. :) Did I mention this code is open source?
Q: Why is your code sloppy?
A: Because I'm currently in rapid prototype mode...don't worry it will get a lot better. Also through the power of open-source...yada, yada, yada.
Q: Doesn't this cost money everytime you need to fire up a benchmark?
A: Yes, yes it does...you have been warned.
A: If you want to test-drive, you can use a weak single core instance which costs like a penny an hour.