Synthetic PR gate silently bypassed: same-job $[ variables['synthPr.AW_SYNTHETIC_PR'] ] resolves to empty

[jamesadevine]

Summary

The synthetic-PR gate is silently bypassed on real CI builds because the gate step reads AW_SYNTHETIC_PR from a same-job step output using a runtime expression that does not resolve. The synthPr step correctly emits the flag, but the immediately-following prGate step never sees it, so gate/bypass.ts falls into the "Not a PR build -- gate passes automatically" branch every time.

Verified against msazuresphere/4x4 build 612123 (ado-aw v0.34.2):

• Setup job, step synthPr log:

  [synth-pr] matched PR #38551 (source=refs/heads/<branch> target=refs/heads/main)
  

• Setup job, next step prGate ("Evaluate PR filters") log:

  Not a PR build -- gate passes automatically
  

• Build tagged pr-gate.passed without ever evaluating any predicate (draft check etc.).

Root cause

src/compile/filter_ir.rs:1185 emits:

AW_SYNTHETIC_PR: "$[ coalesce(variables['synthPr.AW_SYNTHETIC_PR'], '') ]"

synthPr produces the value via scripts/ado-script/src/shared/vso-logger.ts:51:

emit(`##vso[task.setvariable variable=${safeName};isOutput=true]${safeValue}`);

For an isOutput=true variable from a previous step in the same job, ADO only exposes it via macro syntax $(synthPr.AW_SYNTHETIC_PR). The runtime-expression form $[ variables['synthPr.AW_SYNTHETIC_PR'] ] looks up a regular pipeline variable whose literal name is synthPr.AW_SYNTHETIC_PR, which does not exist — so it resolves to the empty string and coalesce(...) returns ''. gate/bypass.ts then sees AW_SYNTHETIC_PR !== "true", decides the build is not synthetic, and the "Not a <bypass_label> build" bypass fires.

The doc-comment at src/compile/filter_ir.rs:1130-1140 asserts that $[ variables['synthPr.X'] ] resolves same-job step outputs. ADO does not document that, and empirically (build 612123) it does not. The Agent-job wiring at src/compile/extensions/exec_context/pr.rs:198 uses the cross-job form dependencies.Setup.outputs['synthPr.AW_SYNTHETIC_PR'], which works — but the gate step lives in the producing job itself, so that form is unavailable to it.

The same defect affects ADO_PR_ID / ADO_SOURCE_BRANCH / ADO_TARGET_BRANCH exports at src/compile/filter_ir.rs:1193 / 1196 / 1199 — they each coalesce(variables['System.PullRequest.X'], variables['synthPr.AW_SYNTHETIC_PR_X']), where the synth half is the same broken form. On a synth-promoted build these exports would silently fall through to empty even if the bypass were fixed.

Suggested fix

Switch the synth half to macro syntax in compile_gate_step_external:

step.push_str("    AW_SYNTHETIC_PR: $(synthPr.AW_SYNTHETIC_PR)\n");

For the coalesced PR-identifier exports, embed the macro inside the runtime expression literal so the real System.PullRequest.* variables still take precedence on true PR builds:

"\"$[ coalesce(variables['System.PullRequest.PullRequestId'], '$(synthPr.AW_SYNTHETIC_PR_ID)') ]\""

(ADO expands $(...) macros before evaluating $[ ... ], so coalesce receives a usable second argument when synthPr ran and the literal unexpanded $(synthPr.AW_SYNTHETIC_PR_ID) string when it did not — at which point System.PullRequest.PullRequestId is set and wins.)

Test gap

src/compile/common.rs:8456 asserts the absence of a previous buggy eq(...) form but does not assert the presence of correct same-job wiring. A regression test should assert that the gate step's env contains AW_SYNTHETIC_PR: $(synthPr.AW_SYNTHETIC_PR) (and the macro form for the three PR-identifier exports), not variables['synthPr.…'].

Impact

Any agent with on.pr and the default mode: synthetic running on Azure Repos against a CI-triggered build (no Build Validation policy installed) will pass its PR gate unconditionally — predicate checks like draft, paths, branches are never evaluated. The Agent job still runs (because synthPr succeeded and AW_SYNTHETIC_PR_SKIP is not set), so the agent fires on PRs it should have filtered out (draft PRs, etc.).

Environment

• ado-aw: v0.34.2
• ADO org: msazuresphere, project 4x4
• Repository provider: TfsGit (Azure Repos)
• Reference build: https://dev.azure.com/msazuresphere/4x4/_build/results?buildId=612123&view=logs&j=6c181f5c-fa64-5337-60e0-f5bfb300ad55