Skip to content

feat: report the macro runner configuration in the upload metadata - #631

Merged
adriencaccia merged 1 commit into
mainfrom
feat/report-macro-runner-config
Nov 21, 2026
Merged

adriencaccia merged 1 commit into
mainfrom
feat/report-macro-runner-config

Conversation

@GuillaumeLagrange

@GuillaumeLagrange GuillaumeLagrange commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Report the CodSpeed Macro Runner configuration in the upload metadata.

A macro runner writes a JSON object describing its configuration to /etc/codspeed/macro-runner-config.json, for example:

{
  "macroAgentVersion": "0.4.0",
  "hostImageName": "packer-ubuntu24-x86-post-20261120093416",
  "guestImageName": "packer-ubuntu24-x86-post-20261120093416",
  "runtimeConfig": { "isolation": "strict" }
}

The CLI forwards it unchanged under runner.macroRunnerConfig. A change on the machine (OS image, how the workload is sandboxed) can shift benchmark results, and CodSpeed needs this to tell such shifts apart from code changes.

The CLI does not interpret the object, so the runner can add fields without a new CLI release. Without the file, the job is not on a macro runner and the field is omitted, so other uploads and their metadata hash are unchanged. A file that cannot be read or is not a JSON object is ignored with a warning instead of failing the upload.

The hash test sets the field (cd04217a…); the backend computes the same hash for that payload.

Refs COD-3931

@GuillaumeLagrange
GuillaumeLagrange marked this pull request as ready for review November 20, 2026 15:07
@codspeed

codspeed Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

2 warnings

⚠️ Walltime results may be unreliable

Some benchmarks used the Walltime instrument outside CodSpeed Macro Runners, which can make the results inconsistent.

Learn more about Macro Runners

See affected benchmarks

⚠️ 8 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

✅ 53 untouched benchmarks
⏩ 6 skipped benchmarks1


Comparing feat/report-macro-runner-config (80072e4) with main (ffdafa6)

Open in CodSpeed

Footnotes

  1. 6 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@greptile-apps

greptile-apps Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Low impact] The PR appears safe to merge.

Summary

Adds the Macro Runner configuration to upload metadata.

  • Uploads now include the Macro Runner's configuration when available.

No actionable issues were found.

Reviews (1) · Last reviewed commit: "feat: report the macro runner configurat..." · Reviewed by Greptile

@adriencaccia adriencaccia left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Risk 3/5, human review needed on the risky files

Risk 3/5: Changes the upload metadata the API hashes; additive and omitted off macro runners. Risky files: provider.rs, interfaces.rs.

Comment thread src/run_environment/provider.rs Resolved
CodSpeed Macro Runners write a JSON object describing their configuration
to /etc/codspeed/macro-runner-config.json (agent version, image build ids,
runtime isolation). The CLI reads that file and forwards the object as is
under runner.macroRunnerConfig. Changes to a runner's configuration can
shift benchmark results, so CodSpeed needs it to tell them apart from code
changes.

The object is not interpreted here, so the runner can add fields without a
new CLI release. A missing file means the job is not on a macro runner, and
a file that cannot be read or is not a JSON object is ignored with a warning
rather than failing the upload. The field is omitted when absent, so other
uploads and their metadata hash are unchanged; the hash test now sets it, and the
upload endpoint computes the same hash for that payload.

Refs COD-3931
Co-Authored-By: Claude <noreply@anthropic.com>
@adriencaccia
adriencaccia force-pushed the feat/report-macro-runner-config branch from 7b3c6ba to 80072e4 Compare November 21, 2026 00:03
@adriencaccia

Copy link
Copy Markdown
Member

Taking this over. A missing file still stays silent, and any other read error now logs a warning; folded into the commit.

@adriencaccia
adriencaccia merged commit 2ccf52d into main Nov 21, 2026
67 checks passed
@adriencaccia
adriencaccia deleted the feat/report-macro-runner-config branch November 21, 2026 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants