Always use param files to handle large rustc commands - #4038
Merged
Conversation
UebelAndre
marked this pull request as ready for review
May 14, 2026 14:13
slackito
approved these changes
May 14, 2026
ivonnyssen
added a commit
to rusty-photon/rusty-photon
that referenced
this pull request
Jul 9, 2026
…0463 Root cause of the windows-latest bazel-build break introduced by PR #473 (rust-embed 8.11->8.12): rules_rust 0.70.0's rustc-invocation command-line construction has a bug affecting "large" commands (many --extern deps) on Windows specifically. ppba-driver's bdd and conformu_integration targets -- by far the two largest dependency graphs of any rust_test target in the repo -- deterministically failed rustc E0463 "can't find crate" for a handful of their --extern deps, 100% reproducibly (same exact crate set every run), regardless of remote-cache settings, sandbox strategy, download mode, concurrency, or Windows Defender exclusions (all tested and ruled out empirically). No other service's equivalent targets were affected, even ones with a similar dependency shape. rules_rust 0.71.0's changelog contains "Always use param files to handle large rustc commands" (bazelbuild/rules_rust#4038), landed 2026-06-25 -- after our pin (0.70.0, from 2026-04-22) and after PR #473 merged. That's an exact match for the symptom class and explains why only the repo's two largest rust_test targets tripped it. Verified locally: full `bazel build //...` + `bazel test //...` pass cleanly with 0.71.3, and both custom rules_rust patches (collect_coverage_ extra_objects, instrumented_files_include_data) still apply. This also reverts the 7 diagnostic-only commits from the windows-latest investigation (remote-cache bypass, params-file dump, sandbox bypass, forced full downloads, cut concurrency, Defender exclusion, --keep_going) -- all ruled the corresponding hypothesis out without fixing the actual bug, so none of it is needed going forward.
ivonnyssen
added a commit
to rusty-photon/rusty-photon
that referenced
this pull request
Jul 9, 2026
* chore(deps): bump rmcp from 2.1.0 to 2.2.0 Bumps [rmcp](https://github.com/modelcontextprotocol/rust-sdk) from 2.1.0 to 2.2.0. - [Release notes](https://github.com/modelcontextprotocol/rust-sdk/releases) - [Changelog](https://github.com/modelcontextprotocol/rust-sdk/blob/main/release-plz.toml) - [Commits](modelcontextprotocol/rust-sdk@rmcp-v2.1.0...rmcp-v2.2.0) --- updated-dependencies: - dependency-name: rmcp dependency-version: 2.2.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(bazel): repin MODULE.bazel.lock for dependabot/cargo/rmcp-2.2.0 * fix(deps): bump sse-stream to 0.2.4 to unbreak rmcp 2.2.0 rmcp 2.2.0 calls SseStream::from_bytes_stream, which only exists as of sse-stream 0.2.4 — the 0.2.3 resolved by the dependabot bump only has from_byte_stream (singular), so the workspace failed to compile. Also re-repins MODULE.bazel.lock so crate_universe picks up the new Cargo.lock content hash. * chore(bazel): repin MODULE.bazel.lock after merging main Merge of main (PR #473, rust-embed 8.11->8.12) into the rmcp bump branch left MODULE.bazel.lock stale; regenerate it from the merged Cargo.lock. * ci(bazel): diagnostic — bypass remote-cache reads on windows-latest build windows-latest deterministically fails E0463 "can't find crate" for several rust_test targets in services/ppba-driver (tokio, serde, ascom_alpaca, cucumber, bdd_infra), reproducing byte-identically across reruns (same remote-cache-hit count both times). The generated MODULE.bazel.lock deps for these crates are unconditional (_COMMON_CONDITION), ruling out a lockfile-content/select() bug, so this isolates whether a stale/corrupted remote-cache entry is being replayed. * ci(bazel): diagnostic — dump failing params files on windows build failure Confirmed the windows-latest E0463 failures for ppba-driver's bdd and conformu_integration targets are neither cache corruption (reproduces identically with --noremote_accept_cached, now reverted) nor simple flakiness (reproduces byte-identically across reruns). Dump the actual generated rustc params files on failure to see the real --extern args Bazel computed, instead of inferring from rustc's error output. * fix(bazel): bypass the sandbox on windows for `bazel build` too Root-caused the windows-latest bazel build break from PR #473: the plain `bazel build //...` step deterministically-but-not-repeatably (varies which target loses the race across runs) fails ppba-driver's bdd/conformu_integration rust_test binaries with rustc E0463 "can't find crate" for several --extern deps. Dumping the generated .params files showed the --extern flags were correct and complete, ruling out a lockfile/deps-graph bug -- the referenced .rlib just isn't reliably present in the sandboxed input tree at execution time on Windows. test:ci already bypasses the sandbox on windows/macOS for tests (see the comment above); this extends the same bypass to `build:windows` so it also covers the plain `bazel build` step, where these binaries are also compiled (as ordinary build outputs, tags don't affect `build`). * ci(bazel): diagnostic — force full remote-cache downloads on windows The sandbox-bypass fix (previous commit) didn't resolve the windows-latest E0463 failures -- same signature, different target (bdd this time instead of conformu_integration). Testing whether build-without-the-bytes' lazy CAS fetch (--remote_download_outputs= toplevel) is the unreliable step on Windows by forcing full downloads there. * ci(bazel): diagnostic — cut windows concurrency to --jobs=8 Neither the sandbox bypass nor forcing full remote downloads (previous 2 commits) fixed the windows-latest E0463 failures. The failure tracks dependency COUNT (only ppba-driver's two largest rust_test targets, bdd and conformu_integration, ever fail) rather than any specific crate, suggesting a low-probability per-file race that compounds across a large deps list. Testing whether cutting concurrency from --jobs=64 avoids the contention that triggers it. * ci(bazel): diagnostic — exclude workspace from Windows Defender --jobs=8 (previous commit) didn't fix the windows-latest E0463 failures either -- same signature. Every Bazel-side lever (cache accept, download mode, spawn strategy, concurrency) has now failed to change the outcome, which points away from Bazel's own scheduling and toward an OS-level cause: Windows Defender's real-time scan transiently locking a newly-written .rlib out from under a near-simultaneous rustc read. No Bazel flag can fix an OS file lock; excluding the workspace/output_base/toolchain processes from real-time scanning is the standard mitigation for this class of GitHub Actions Windows flakiness. * ci(bazel): diagnostic — --keep_going on windows to surface all failures The Defender exclusion (previous commit) didn't fix it either -- and crucially, re-examining all 7 CI runs so far shows this is NOT flaky: conformu_integration fails on exactly {bdd_infra, tokio, tracing_subscriber} and bdd fails on exactly {ascom_alpaca, bdd_infra, cucumber, serde, tokio}, 100% reproducibly, every single time. Only one of the two is ever reported per run because bazel build stops at the first failure. --keep_going surfaces both (and any other affected target, e.g. qhy-focuser's same conformu_integration/bdd shape) in one run, to see whether this is ppba-driver-specific or hits every large rust_test target with this dependency shape. * fix(bazel): bump rules_rust 0.70.0 -> 0.71.3, fixing windows-latest E0463 Root cause of the windows-latest bazel-build break introduced by PR #473 (rust-embed 8.11->8.12): rules_rust 0.70.0's rustc-invocation command-line construction has a bug affecting "large" commands (many --extern deps) on Windows specifically. ppba-driver's bdd and conformu_integration targets -- by far the two largest dependency graphs of any rust_test target in the repo -- deterministically failed rustc E0463 "can't find crate" for a handful of their --extern deps, 100% reproducibly (same exact crate set every run), regardless of remote-cache settings, sandbox strategy, download mode, concurrency, or Windows Defender exclusions (all tested and ruled out empirically). No other service's equivalent targets were affected, even ones with a similar dependency shape. rules_rust 0.71.0's changelog contains "Always use param files to handle large rustc commands" (bazelbuild/rules_rust#4038), landed 2026-06-25 -- after our pin (0.70.0, from 2026-04-22) and after PR #473 merged. That's an exact match for the symptom class and explains why only the repo's two largest rust_test targets tripped it. Verified locally: full `bazel build //...` + `bazel test //...` pass cleanly with 0.71.3, and both custom rules_rust patches (collect_coverage_ extra_objects, instrumented_files_include_data) still apply. This also reverts the 7 diagnostic-only commits from the windows-latest investigation (remote-cache bypass, params-file dump, sandbox bypass, forced full downloads, cut concurrency, Defender exclusion, --keep_going) -- all ruled the corresponding hypothesis out without fixing the actual bug, so none of it is needed going forward. * Revert "fix(bazel): bump rules_rust 0.70.0 -> 0.71.3, fixing windows-latest E0463" This reverts commit cb8e2dd. * Revert "chore(deps): bump rust-embed from 8.11.0 to 8.12.0 (#473)" This reverts commit e7c226e. * chore(bazel): repin MODULE.bazel.lock after merging the rust-embed revert Merge of revert-rust-embed-8-12-windows-fix (PR #478, reverting rust- embed to 8.11.0 to fix the windows-latest E0463 break from PR #473) into the rmcp bump branch left MODULE.bazel.lock stale; regenerate it from the merged Cargo.lock. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Igor von Nyssen (Claude Code) <igor@vonnyssen.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Using a params file helps avoid OS limitations for command line lengths. This should provide a more stable experience on all platforms.
closes #4006