Run the benchmarks

peryx-bench produces the tables the documentation publishes and gates a change against a regression. Every command below assumes a checked-out repository and a release build. A debug build times unoptimized code, so its numbers say nothing about peryx.

Compare peryx against the other tools

The published comparison tables come from this form, which runs every server on one machine, in one run, against the same workload:

cargo run --release -p peryx-bench                       # every ecosystem
cargo run --release -p peryx-bench -- --ecosystem pypi   # one ecosystem
cargo run --release -p peryx-bench -- --ecosystem oci

Check a change against a base commit

The A/B form builds both revisions and measures each through this same harness, so the method matches on both sides. It prints a per-metric verdict aggregated with the geometric mean and gates only the local metrics, since network variance peryx does not control dominates any row that fetches from a real upstream:

cargo run --release -p peryx-bench -- ab <base-commit>

OCI runs need Docker and a mirror

The OCI benchmarks need a running Docker daemon. Pulling from Docker Hub with nothing in front of it, an anonymous account hits the pull ceiling partway through a comparison, so export DOCKERHUB_USERNAME and a read-only access token in DOCKERHUB_TOKEN; the harness threads them into every registry and into crane.

Under --mirror the harness stands a local pull-through cache in front of Docker Hub and points every registry at it, so the run is rate-limit-free and repeatable. Without it the cold rows carry the real upstream fetch, so the harness marks them network-bound and keeps them out of the regression gate:

cargo run --release -p peryx-bench -- --ecosystem oci --mirror

Price one request, per ecosystem

The runs above time a whole client against a real network. The criterion suites price a single request served in process through the full router, with no socket and no upstream, from a warm store:

cargo bench -p peryx-ecosystem-pypi --bench operations
cargo bench -p peryx-ecosystem-oci --bench operations