Bencher is a suite of continuous benchmarking tools. Have you ever had a performance regression impact your users? Bencher could have prevented that from happening. Bencher allows you to detect and prevent performance regressions before they hit production.
bencher CLI simply wraps your existing benchmark harness and stores its results.For the same reasons that unit tests are run in CI to prevent feature regressions, benchmarks should be run in CI with Bencher to prevent performance regressions. Performance bugs are bugs!
Bencher consists of:
bencher CLIThough Bencher is open source, there is also a hosted version available Bencher Cloud.
The best place to start is the Bencher Quick Start tutorial.
🌐 Also available in:
Supported Benchmark Harnesses👉 For more details see the explanation of benchmark harness adapters.
Microsoft CCF
Rustls
Diesel
Hydra Database
GreptimeDB
Tailcall
Servo
Ratatui
Wire
👉 Checkout all public projects.
Install the Bencher CLI using the GitHub Action, and use it for continuous benchmarking in your project.
name: Continuous Benchmarking with Bencher
on:
push:
branches: main
jobs:
benchmark_with_bencher:
name: Benchmark with Bencher
runs-on: ubuntu-latest
env:
BENCHER_PROJECT: my-project-slug
BENCHER_API_TOKEN: ${{ secrets.BENCHER_API_TOKEN }}
steps:
- uses: actions/checkout@v4
- uses: bencherdev/bencher@main
- run: bencher run "bencher mock"
Supported Operating Systems:
👉 For more details see the explanation of how to use GitHub Actions.
Add BENCHER_API_TOKEN to you Repository secrets (ex: Repo -> Settings -> Secrets and variables -> Actions -> New repository secret). You can find your API tokens by running bencher token list my-user-slug or view them in the Bencher Console.
You can set the bencher run CLI subcommand to error if an Alert is generated with the --err flag.
bencher run --err "bencher mock"
👉 For more details see the explanation of bencher run.
You can set the bencher run CLI subcommand to comment on a PR with the --github-actions argument.
bencher run --github-actions "${{ secrets.GITHUB_TOKEN }}" "bencher mock"
👉 For more details see the explanation of bencher run.
🚨 1 ALERT: Threshold Boundary Limit exceeded!Click to view all benchmark results
There is also an optional version argument to specify an exact version of the Bencher CLI to use. Otherwise, it will default to using the latest CLI version.
- uses: bencherdev/bencher@main
with:
version: 0.5.3
Specify an exact version if using Bencher Self-Hosted. Do not specify an exact version if using Bencher Cloud as there are still occasional breaking changes.
Bencher is like CodeCov for performance metrics.
Jonathan Woollett-Light
@JonathanWoollett-Light
I think I'm in heaven. Now that I'm starting to see graphs of performance over time automatically from tests I'm running in CI. It's like this whole branch of errors can be caught and noticed sooner.
Price Clark
@gpwclark
95% of the time I don't want to think about my benchmarks. But when I need to, Bencher ensures that I have the detailed historical record waiting there for me. It's fire-and-forget.
Joe Neeman
@jneem
I've been looking for a public service like Bencher for about 10 years :)
Jamie Wilkinson
@jaqx0r
I'm happy with how quickly I was able to get Bencher configured and working.
Weston Pace
@westonpace
Bencher's main ideas and concepts are really well designed.
Free Ekanayaka
@freeekanayaka
The easiest way to contribute is to open this repo as a Dev Container in VSCode by simply clicking one of the buttons below. Everything you need will already be there! Once set up, both the UI and API should be built, running, and seeded at localhost:3000 and localhost:61016 respectively. To make any changes to the UI or API though, you will have to exit the startup process and restart the UI and API yourself.
For additional information on contributing, see the Development Getting Started guide.
🐰 All pull requests should target thedevel branch
There is also a pre-built image from CI available for each branch: ghcr.io/bencherdev/bencher-dev-container
All content that resides under any directory or feature named "plus" is licensed under the Bencher Plus License.
All other content is licensed under the Apache License, Version 2.0 or MIT License at your discretion.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in Bencher by you, as defined in the Apache-2.0 license, shall be licensed as above, without any additional terms or conditions.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4