Skip to content

fix(criterion): document the async API on docs.rs - #207

Merged
not-matthias merged 3 commits into
mainfrom
cod-4302-criterion-compat-docsrs-async
Nov 19, 2026
Merged

not-matthias merged 3 commits into
mainfrom
cod-4302-criterion-compat-docsrs-async

Conversation

@not-matthias

@not-matthias not-matthias commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Fixes COD-4302

Bencher::to_async was missing from the codspeed-criterion-compat docs (reported in #6). The method still exists, but it is gated behind the async feature, and docs.rs built the crate with default features only.

  • Add the same [package.metadata.docs.rs] section as upstream Criterion, so docs.rs builds with all async runtimes enabled (async_futures, async_smol, async_std, async_tokio).
  • Enable doc_cfg under docsrs, so feature-gated items get an "Available on crate feature async only" badge. The internal codspeed cfg is hidden from the badges; it only selects between the two compat_criterion modules and would otherwise mark every item as "non-codspeed only".
  • Bump the walltime fork to smol 2 and update async-std to 1.13 in the lockfile. Both previously pulled in async-io 1 → rustix 0.37, which no longer compiles on nightly (rustc_* attributes are reserved), so the async docs build would fail on docs.rs. This also fixes the fork's own docs.rs build, broken since 5.0.0. smol is not part of the public API (SmolExecutor only calls smol::block_on), so this is not a breaking change.

Verified locally:

  • RUSTDOCFLAGS="--cfg docsrs" cargo +nightly doc -p codspeed-criterion-compat -p codspeed-criterion-compat-walltime --no-deps with all async features passes (failed before the bump), and Bencher shows to_async with the async feature badge.
  • cargo test -p codspeed-criterion-compat-walltime --lib with all async features passes.

@not-matthias
not-matthias marked this pull request as ready for review November 17, 2026 17:35
@greptile-apps

greptile-apps Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Low impact] This PR appears safe to merge.

Summary

Adds docs.rs settings so codspeed-criterion-compat documents its async API.

  • The docs.rs API page now includes the async bencher method.

Reviews (1) · Last reviewed commit: "fix(criterion): document the async API o..." · Reviewed by Greptile

@codspeed

codspeed Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Merging this PR will regress 4 benchmarks

3 warnings, 1 note

⚠️ Unable to measure the execution of 59 benchmarks

These benchmarks measured no execution time, so their results are not comparable and count as unchanged. This usually happens when the compiler removes the code under test.

Prevent compiler optimizations

See affected benchmarks

⚠️ 17 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ 1 benchmark with a significant change ran on different hardware

Its base and head results were measured on machines with a different CPU, core count or memory, so the change might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

ℹ️ 57 benchmarks have no flamegraph

  • The flamegraph wasn't generated: most of the time was spent in runtime internals, which would leave nothing useful to analyze. (9 benchmarks)
  • The flamegraph wasn't generated: no execution data was recorded while the benchmark ran. Make sure the compiler didn't optimize the benchmark out. (48 benchmarks)

See affected benchmarks

⚡ 3 improved benchmarks
❌ 4 regressed benchmarks
✅ 611 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ Walltime vec_copy_with_bytes_counter 41 ns 59 ns -30.51%
❌ Walltime spawn_on_parent_runtime 21.6 µs 24.7 µs -12.52%
❌ Walltime count_set_bits[65535] 7 ns 8 ns -12.5%
❌ Walltime zeroed_buffer[1] 48.2 µs 50.2 µs -4.02%
⚡ CPU Simulation spawn_on_parent_runtime 70.1 µs 33.4 µs ×2.1
⚡ Walltime find_highest_set_bit[0] 7 ns 5 ns +40%
⚡ Walltime build_vec 237 ns 216 ns +9.72%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing cod-4302-criterion-compat-docsrs-async (2e79217) with main (0c6d629)

Open in CodSpeed

@svix-jplatte

Copy link
Copy Markdown

I would recommend you also add #![cfg_attr(docsrs, feature(doc_cfg))] to the top of lib.rs, so the feature requirements show up on docs.rs.

Comment thread crates/criterion_compat/Cargo.toml Resolved

@lvaroqui lvaroqui left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

olgtm 👍

Comment thread crates/criterion_compat/src/lib.rs Resolved
@not-matthias
not-matthias force-pushed the cod-4302-criterion-compat-docsrs-async branch from 7dd2d13 to dd256d5 Compare November 19, 2026 15:18
- `Bencher::to_async` is gated behind the `async` feature, and docs.rs built
  the crate with default features only, so the async API was missing from
  the published docs.
- Enable all async runtimes for the docs.rs build, like upstream Criterion.
- Enable `doc_cfg` under `docsrs` so rustdoc labels feature-gated items,
  e.g. `Bencher::to_async` as "Available on crate feature `async` only".
- Hide the internal `codspeed` cfg from the labels: it selects between the
  two `compat_criterion` modules, and would otherwise mark every item as
  "Available on non-codspeed only".
- docs.rs builds with nightly, and `rustix` 0.37 no longer compiles there
  (`rustc_*` attributes are reserved). Both async docs builds pulled it in:
  `smol` 1 and the locked `async-std` 1.12 depend on `async-io` 1.
- Bump the walltime fork to `smol` 2, as the compat crate already uses, and
  update `async-std` to 1.13 in the lockfile. Both now use `async-io` 2.
- `smol` is not part of the public API: `SmolExecutor` only calls
  `smol::block_on`.
@not-matthias
not-matthias force-pushed the cod-4302-criterion-compat-docsrs-async branch from dd256d5 to 2e79217 Compare November 19, 2026 15:19
@not-matthias
not-matthias merged commit 7e1dc32 into main Nov 19, 2026
49 of 50 checks passed
@not-matthias
not-matthias deleted the cod-4302-criterion-compat-docsrs-async branch November 19, 2026 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants