feat(compile): gate high-impact safe outputs behind manual review

.github/workflows/release.yml

@@ -77,7 +77,7 @@ jobs:
         run: |
           set -euo pipefail
           cd scripts
-          zip -r ../ado-script.zip ado-script/gate.js ado-script/import.js ado-script/exec-context-pr.js ado-script/exec-context-pr-synth.js ado-script/exec-context-manual.js ado-script/exec-context-pipeline.js ado-script/exec-context-ci-push.js ado-script/exec-context-workitem.js ado-script/exec-context-schedule.js ado-script/exec-context-pr-checks.js ado-script/exec-context-repo.js
+          zip -r ../ado-script.zip ado-script/gate.js ado-script/import.js ado-script/exec-context-pr.js ado-script/exec-context-pr-synth.js ado-script/exec-context-manual.js ado-script/exec-context-pipeline.js ado-script/exec-context-ci-push.js ado-script/exec-context-workitem.js ado-script/exec-context-schedule.js ado-script/exec-context-pr-checks.js ado-script/exec-context-repo.js ado-script/approval-summary.js
 
       - name: Upload release assets
         env:

AGENTS.md

@@ -39,6 +39,15 @@ Every compiled pipeline runs as three sequential jobs:
 3. **SafeOutputs (Stage 3)** — a non-agent executor applies approved safe outputs
    using a write-capable ADO token that the agent never sees.
 
+**Optional manual review.** When a safe output is configured with
+`require-approval` (see [`docs/safe-outputs.md`](docs/safe-outputs.md)), an
+agentless `ManualReview` job (`pool: server`, `ManualValidation@1`) is inserted
+between Detection and SafeOutputs to pause for human approval. With a mix of
+gated and non-gated outputs, Stage 3 splits into an automatic `SafeOutputs` job
+(applies non-gated outputs immediately) and a `SafeOutputs_Reviewed` job (gated
+behind `ManualReview`, publishes `safe_outputs_reviewed`). The gate is
+fail-closed and only pauses when the agent actually proposed a reviewed output.
+
 ### Architecture
 
 ```
@@ -50,7 +59,7 @@ Every compiled pipeline runs as three sequential jobs:
 │   ├── compile/          # Pipeline compilation module
 │   │   ├── mod.rs        # Module entry point and Compiler trait
 │   │   ├── common.rs     # Shared helpers across targets
-│   │   ├── agentic_pipeline.rs # Canonical Setup → Agent → Detection → SafeOutputs → Teardown shape (shared by every target); BuiltPipelineContext, build_pipeline_context, build_canonical_jobs, per-job builders, fold_agent_conditions, agent_job_variables_hoist
+│   │   ├── agentic_pipeline.rs # Canonical Setup → Agent → Detection → (ManualReview?) → SafeOutputs(+SafeOutputs_Reviewed?) → Teardown shape (shared by every target); BuiltPipelineContext, build_pipeline_context, build_canonical_jobs, per-job builders incl. build_manual_review_job + SafeOutputsVariant split, fold_agent_conditions, agent_job_variables_hoist
 │   │   ├── standalone.rs # Standalone pipeline compiler
 │   │   ├── standalone_ir.rs # Standalone target typed-IR builder
 │   │   ├── onees.rs      # 1ES Pipeline Template compiler
@@ -249,6 +258,7 @@ Every compiled pipeline runs as three sequential jobs:
 │           ├── exec-context-schedule/ # Scheduled-run context source (bundled to exec-context-schedule.js)
 │           ├── exec-context-pr-checks/ # PR validation checks context source (bundled to exec-context-pr-checks.js)
 │           ├── exec-context-repo/ # Repository identity context source (bundled to exec-context-repo.js)
+│           ├── approval-summary/ # Safe-outputs summary renderer (bundled to approval-summary.js; end-of-Agent-job summary tab)
 │           └── shared/   # Shared modules across bundles (auth, ado-client, env-facts, types.gen.ts)
 ├── tests/                # Integration tests and fixtures
 ├── docs/                 # Per-concept reference documentation (see index below)

docs/ado-script.md

@@ -3,7 +3,7 @@
 `ado-script` is the umbrella name for the TypeScript workspace at
 [`scripts/ado-script/`](../scripts/ado-script/). It produces small,
 ncc-bundled Node programs that the **compiler injects into every emitted
-pipeline** as runtime helpers. Today it produces eleven bundles:
+pipeline** as runtime helpers. Today it produces twelve bundles:
 
 - `gate.js` — trigger-filter gate evaluator (Setup job).
 - `import.js` — runtime prompt resolver described in
@@ -45,6 +45,13 @@ pipeline** as runtime helpers. Today it produces eleven bundles:
   branch, SHA, last release tag, and commits-since-tag facts under
   `aw-context/repo/` (Agent job; see
   [`execution-context.md`](execution-context.md)).
+- `approval-summary.js` — Safe-outputs summary renderer that runs at the
+  **end of the Agent job** (after proposals are collected). It reads the
+  proposed safe outputs from `safe_outputs.ndjson`, renders a sanitized
+  per-tool markdown summary (pending-approval proposals first when manual
+  review is configured), and attaches it to the build's
+  `ado-aw-safe-outputs` summary tab via `##vso[task.uploadsummary]`. See
+  [`safe-outputs.md`](safe-outputs.md).
 
 > **Internal-only.** `ado-script` is not a user-facing front-matter
 > feature. Authors never write an `ado-script:` block in their agent

docs/cli.md

@@ -36,6 +36,8 @@ Global flags (apply to all subcommands): `--verbose, -v` (enable info-level logg
   - `--ado-org-url <url>` - Azure DevOps organization URL override
   - `--ado-project <name>` - Azure DevOps project name override
   - `--dry-run` - Validate inputs but skip ADO API calls (useful for local testing and QA review)
+  - `--only <tool>` - Execute only these safe-output tools (repeatable). Used by the manual-review split for the approval-gated `SafeOutputs_Reviewed` job.
+  - `--exclude <tool>` - Skip these safe-output tools (repeatable). Used by the manual-review split so the automatic `SafeOutputs` job applies non-gated outputs while reviewed ones wait. See [`docs/safe-outputs.md`](safe-outputs.md#manual-review-require-approval).
 
 - `configure` *(deprecated; hidden in --help)* - Alias forwarding to `secrets set GITHUB_TOKEN`. Existing scripts keep working but get a stderr warning.
 

docs/safe-outputs.md

@@ -37,6 +37,123 @@ safe-outputs:
 
 Safe output configurations are passed to Stage 3 execution and used when processing safe outputs.
 
+### Manual review (`require-approval`)
+
+High-impact safe outputs can be gated behind a human approval step
+(`ManualValidation@1`) that pauses the run until a reviewer approves or rejects
+in the Azure DevOps UI. This lets agents propose more consequential actions
+(PRs, branches, queued builds, work items) safely.
+
+Set `require-approval` at the **section level** for a pipeline-wide default,
+and/or inside an **individual tool** to override the default for that tool:
+
+```yaml
+safe-outputs:
+  require-approval: true          # global default: every output below needs review
+  create-pull-request:
+    target-branch: main
+  add-pr-comment:
+    require-approval: false       # …except low-impact comments, which auto-apply
+```
+
+`require-approval` accepts either a bare boolean or an object for finer control:
+
+```yaml
+safe-outputs:
+  create-pull-request:
+    require-approval:
+      approvers: ["[MyOrg]\\release-team"]   # who may approve (empty → anyone with run permission)
+      notify-users: ["ops@example.com"]      # who is emailed (empty → no email)
+      timeout-minutes: 120                    # pending period before on-timeout fires (omit → pipeline default)
+      on-timeout: reject                      # reject (default, fail-closed) | resume
+      instructions: "Verify the proposed PR before approving."
+```
+
+Resolution per tool: the tool's own `require-approval` wins; otherwise the
+section-level `require-approval` applies; otherwise the tool is **not** gated.
+
+**Defaults (bare `require-approval: true`)** — the run pauses on a Review panel;
+**anyone with run permission** can approve or reject; **no** notification emails
+are sent; and the validation **fails closed** on timeout (`on-timeout: reject`),
+so un-approved outputs are never applied.
+
+**Timeout (`timeout-minutes` / `on-timeout`)** — `timeout-minutes` bounds the
+`ManualValidation@1` task's pending period; when it elapses the task applies
+`on-timeout` (`reject` by default, or `resume` to auto-approve). The agentless
+`ManualReview` job carries a slightly larger outer timeout as a hard bound, so a
+job-level cancellation never preempts the task's graceful `on-timeout` handling
+(in particular, `on-timeout: resume` reliably auto-approves rather than being
+cancelled). Omit `timeout-minutes` to inherit the pipeline default.
+
+**Reviewer message** — set `instructions` to control the text shown in the
+Review panel and notification emails. It is plain text and supports pipeline
+variable (`$(...)`) interpolation. When omitted, ado-aw generates a default
+message listing the reviewed safe-output type(s) awaiting approval. A run uses a
+**single** `ManualReview` gate covering every reviewed tool: the gate message
+**lists every reviewed tool** and aggregates **all** author-supplied per-tool
+`instructions` (grouped when identical), so no tool's note is dropped when
+several are gated. A single reviewed tool with its own `instructions` shows that
+message verbatim; set `instructions` on the section-level `require-approval` to
+apply one note to every tool.
+
+**Execution shape** — manual review changes the compiled pipeline:
+
+- A new agentless `ManualReview` job (`pool: server`) runs `ManualValidation@1`
+  between Detection and the safe-output execution.
+- It only pauses when Detection cleared the run (no prompt-injection / secret
+  leak) **and** the agent actually proposed a reviewed-type output (a Detection
+  step sets a `HasReviewedProposals` flag) — so the run never pauses for
+  nothing.
+- When some tools are gated and others are not, execution **splits**: an
+  automatic `SafeOutputs` job applies the non-gated outputs immediately
+  (independent of the review outcome), while a separate `SafeOutputs_Reviewed`
+  job — gated behind `ManualReview` — applies the approved outputs and publishes
+  a distinct `safe_outputs_reviewed` artifact. A rejected or timed-out review
+  fails closed: the reviewed job is skipped while the automatic outputs are
+  unaffected.
+- When **every** configured tool requires approval (no automatic tools),
+  execution is **not** split — the single `SafeOutputs` job is gated behind
+  `ManualReview` in its entirety. Note this also defers the always-enabled
+  diagnostic outputs (`noop`, `report_incomplete`, `missing-tool`,
+  `missing-data`) until after approval, since they share that one job. If you
+  want diagnostics to apply without waiting on a human, leave at least one
+  low-impact tool (e.g. `add-pr-comment`) non-gated so the automatic split job
+  is created.
+
+The Detection threat gate always runs first, so a flagged run applies nothing —
+automatic or reviewed.
+
+### Safe-outputs summary tab
+
+Every run that proposes safe outputs publishes a human-readable **build summary
+tab** titled **`ado-aw-safe-outputs`**, listing what the agent proposed. This is
+always on — it does **not** require `require-approval` — so non-elevated runs get
+the same transparency, and it is the panel a reviewer reads before approving a
+gated run.
+
+- The summary is rendered at the **end of the Agent job** (the job that produced
+  the proposals) by the `approval-summary` ado-script bundle, and attached via
+  `##vso[task.uploadsummary]`. It is **not** produced by the Detection
+  (threat-analysis) stage, whose only job is inspecting proposals for threats.
+- Each proposal is shown with per-tool key fields (e.g. PR title + target branch,
+  work-item title) plus a truncated excerpt of any long body. All content is
+  **agent-generated** and is sanitized for display (markdown/HTML escaped, code
+  fences neutralised, control characters stripped, long values truncated) so a
+  proposal cannot forge UI or break the layout.
+- When manual review is configured, the **pending-approval** proposals are listed
+  first (under a `⏳ Pending approval` heading), followed by the automatic ones.
+  With no approval configured, a single list is shown. The default review
+  message points approvers at this tab.
+- Rendering is best-effort: if it fails it is logged as a warning and never fails
+  the build or blocks the review gate.
+
+**Coexistence with your own summary tabs.** ADO derives a summary section's title
+from the uploaded file's base name and does not de-duplicate, so this feature uses
+a namespaced base name (`ado-aw-safe-outputs.md` → the `ado-aw-safe-outputs`
+section). It is additive and build-scoped: it appears as one extra section
+alongside any `task.uploadsummary` tabs your own steps publish (including under
+`target: job` / `target: stage`), and never collides with them.
+
 ### Executor authentication
 
 All write-bearing safe outputs (e.g. `create-pull-request`,

scripts/ado-script/.gitignore

@@ -11,5 +11,6 @@ exec-context-workitem.js
 exec-context-schedule.js
 exec-context-pr-checks.js
 exec-context-repo.js
+approval-summary.js
 schema
 *.tsbuildinfo

scripts/ado-script/package.json

@@ -7,8 +7,8 @@
     "node": ">=20.0.0"
   },
   "scripts": {
-    "build": "npm run codegen && npm run clean && npm run build:gate && npm run build:import && npm run build:exec-context-pr && npm run build:exec-context-pr-synth && npm run build:exec-context-manual && npm run build:exec-context-pipeline && npm run build:exec-context-ci-push && npm run build:exec-context-workitem && npm run build:exec-context-schedule && npm run build:exec-context-pr-checks && npm run build:exec-context-repo",
-    "clean": "node -e \"const fs=require('node:fs'); fs.rmSync('.ado-build',{recursive:true,force:true}); for (const n of ['gate','import','exec-context-pr','exec-context-pr-synth','exec-context-manual','exec-context-pipeline','exec-context-ci-push','exec-context-workitem','exec-context-schedule','exec-context-pr-checks','exec-context-repo']) fs.rmSync(n+'.js',{force:true});\"",
+    "build": "npm run codegen && npm run clean && npm run build:gate && npm run build:import && npm run build:exec-context-pr && npm run build:exec-context-pr-synth && npm run build:exec-context-manual && npm run build:exec-context-pipeline && npm run build:exec-context-ci-push && npm run build:exec-context-workitem && npm run build:exec-context-schedule && npm run build:exec-context-pr-checks && npm run build:exec-context-repo && npm run build:approval-summary",
+    "clean": "node -e \"const fs=require('node:fs'); fs.rmSync('.ado-build',{recursive:true,force:true}); for (const n of ['gate','import','exec-context-pr','exec-context-pr-synth','exec-context-manual','exec-context-pipeline','exec-context-ci-push','exec-context-workitem','exec-context-schedule','exec-context-pr-checks','exec-context-repo','approval-summary']) fs.rmSync(n+'.js',{force:true});\"",
     "build:gate": "ncc build src/gate/index.ts -o .ado-build/gate -m -t && node -e \"const fs=require('node:fs'); fs.copyFileSync('.ado-build/gate/index.js','gate.js'); fs.rmSync('.ado-build/gate',{recursive:true,force:true});\"",
     "build:import": "ncc build src/import/index.ts -o .ado-build/import -m -t && node -e \"const fs=require('node:fs'); fs.copyFileSync('.ado-build/import/index.js','import.js'); fs.rmSync('.ado-build/import',{recursive:true,force:true});\"",
     "build:exec-context-pr": "ncc build src/exec-context-pr/index.ts -o .ado-build/exec-context-pr -m -t && node -e \"const fs=require('node:fs'); fs.copyFileSync('.ado-build/exec-context-pr/index.js','exec-context-pr.js'); fs.rmSync('.ado-build/exec-context-pr',{recursive:true,force:true});\"",
@@ -20,10 +20,11 @@
     "build:exec-context-schedule": "ncc build src/exec-context-schedule/index.ts -o .ado-build/exec-context-schedule -m -t && node -e \"const fs=require('node:fs'); fs.copyFileSync('.ado-build/exec-context-schedule/index.js','exec-context-schedule.js'); fs.rmSync('.ado-build/exec-context-schedule',{recursive:true,force:true});\"",
     "build:exec-context-pr-checks": "ncc build src/exec-context-pr-checks/index.ts -o .ado-build/exec-context-pr-checks -m -t && node -e \"const fs=require('node:fs'); fs.copyFileSync('.ado-build/exec-context-pr-checks/index.js','exec-context-pr-checks.js'); fs.rmSync('.ado-build/exec-context-pr-checks',{recursive:true,force:true});\"",
     "build:exec-context-repo": "ncc build src/exec-context-repo/index.ts -o .ado-build/exec-context-repo -m -t && node -e \"const fs=require('node:fs'); fs.copyFileSync('.ado-build/exec-context-repo/index.js','exec-context-repo.js'); fs.rmSync('.ado-build/exec-context-repo',{recursive:true,force:true});\"",
+    "build:approval-summary": "ncc build src/approval-summary/index.ts -o .ado-build/approval-summary -m -t && node -e \"const fs=require('node:fs'); fs.copyFileSync('.ado-build/approval-summary/index.js','approval-summary.js'); fs.rmSync('.ado-build/approval-summary',{recursive:true,force:true});\"",
     "build:check": "ls -lh gate.js && wc -c gate.js",
     "codegen": "node -e \"require('node:fs').mkdirSync('schema', { recursive: true })\" && cargo run --quiet --manifest-path ../../Cargo.toml -- export-gate-schema --output schema/gate-spec.schema.json && npx json2ts schema/gate-spec.schema.json -o src/shared/types.gen.ts --bannerComment \"// AUTO-GENERATED from Rust IR via cargo run -- export-gate-schema. Do not edit; run npm run codegen.\"",
     "test": "vitest run",
-    "test:smoke": "npm run build:gate && npm run build:import && npm run build:exec-context-pr && npm run build:exec-context-pr-synth && npm run build:exec-context-manual && npm run build:exec-context-pipeline && npm run build:exec-context-ci-push && npm run build:exec-context-workitem && npm run build:exec-context-schedule && npm run build:exec-context-pr-checks && npm run build:exec-context-repo && vitest run -c vitest.config.smoke.ts",
+    "test:smoke": "npm run build:gate && npm run build:import && npm run build:exec-context-pr && npm run build:exec-context-pr-synth && npm run build:exec-context-manual && npm run build:exec-context-pipeline && npm run build:exec-context-ci-push && npm run build:exec-context-workitem && npm run build:exec-context-schedule && npm run build:exec-context-pr-checks && npm run build:exec-context-repo && npm run build:approval-summary && vitest run -c vitest.config.smoke.ts",
     "lint": "echo TODO",
     "typecheck": "tsc --noEmit"
   },