Skip to content

Remove non-deterministic files from OUT_DIR to fix TreeArtifact cache misses - #3973

Merged
UebelAndre merged 2 commits into
bazelbuild:mainfrom
jsharpe:push-rnwossulpwxl
Apr 22, 2026
Merged

Remove non-deterministic files from OUT_DIR to fix TreeArtifact cache misses#3973
UebelAndre merged 2 commits into
bazelbuild:mainfrom
jsharpe:push-rnwossulpwxl

Conversation

@jsharpe

@jsharpe jsharpe commented Apr 16, 2026

Copy link
Copy Markdown
Contributor

Recursively remove files from OUT_DIR whose names appear in a known list (config.log, config.log.old, config.status, Makefile, Makefile.config, config.cache, commit_hash) or have a .d extension before Bazel captures OUT_DIR as a TreeArtifact.

These files embed sandbox-specific paths, timestamps, or volatile values that make the TreeArtifact hash non-deterministic, causing cache misses for all downstream rustc compilations on every action run.

@UebelAndre UebelAndre left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank! Can you add a test for this somewhere?

@jsharpe
jsharpe force-pushed the push-rnwossulpwxl branch 2 times, most recently from 2bced6c to 94a2144 Compare April 16, 2026 23:58
Comment thread cargo/tests/cargo_build_script/nondeterministic_out_dir/test.rs
Comment thread cargo/private/cargo_build_script_runner/bin.rs Outdated
@jsharpe
jsharpe force-pushed the push-rnwossulpwxl branch 7 times, most recently from d893275 to 5ac1e80 Compare April 22, 2026 15:16
Comment thread cargo/settings/settings.bzl Outdated
Comment thread cargo/tests/cargo_build_script/nondeterministic_out_dir/test.rs Outdated
@jsharpe
jsharpe force-pushed the push-rnwossulpwxl branch from 5dc032c to df32184 Compare April 22, 2026 15:36
jsharpe and others added 2 commits April 22, 2026 17:21
… misses

Recursively remove files from OUT_DIR whose names appear in the
`//cargo/settings:out_dir_volatile_file_basenames` string_list_flag, or have
a .d or .pc extension, before Bazel captures OUT_DIR as a TreeArtifact.

The flag value (default: config.log, config.log.old, config.status, Makefile,
Makefile.config, config.cache, commit_hash, jemalloc-config, jemalloc.sh) is
passed to the runner via the `RULES_RUST_OUT_DIR_VOLATILE_BASENAMES` env var.
The runner reads the env var at runtime; if absent, no named-file removal
occurs (extension-based removal of .d and .pc files is unconditional).

These files embed sandbox-specific paths, timestamps, or volatile values
that make the TreeArtifact hash non-deterministic, causing cache misses for
all downstream rustc compilations on every action run.

An integration test verifies both that legitimate outputs survive and that
each category of volatile file is absent from the captured TreeArtifact.
Co-authored-by: UebelAndre <github@uebelandre.com>
@jsharpe
jsharpe force-pushed the push-rnwossulpwxl branch from df32184 to 044c39f Compare April 22, 2026 16:21

@UebelAndre UebelAndre left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@UebelAndre
UebelAndre enabled auto-merge April 22, 2026 16:33
@UebelAndre
UebelAndre added this pull request to the merge queue Apr 22, 2026
Merged via the queue into bazelbuild:main with commit 6d85959 Apr 22, 2026
3 checks passed
@jsharpe
jsharpe deleted the push-rnwossulpwxl branch April 22, 2026 21:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants