ci: add provenance guard workflow + Protect dev ruleset - #3
Merged
Conversation
- Add guard-main-provenance.yml for dev→main commit provenance checks - Create Protect dev ruleset (deletion, non-fast-forward, signed commits) - Update Protect main ruleset with guard-docs and guard-provenance status checks - Store rulesets in .github/rulesets/ for version control Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
brettdavies
added a commit
that referenced
this pull request
Apr 1, 2026
## Summary Add missing CI infrastructure for the dev→main workflow pattern. ## Changelog ### Added - Add guard-main-provenance workflow to verify commits in PRs to main came from squash-merged dev PRs - Add Protect dev branch ruleset (deletion, force-push, signed commits) - Add guard-docs and guard-provenance required status checks to Protect main ruleset ## Type of Change - [x] `ci`: CI/CD configuration changes ## Testing - [x] Workflow files match templates from github-repo-setup and rust-tool-release skills - [x] Rulesets applied via GitHub API and stored in `.github/rulesets/` for version control ## Post-Deploy Monitoring & Validation No additional operational monitoring required — CI workflow configuration only. --- > **Compound Engineered** with Claude Opus 4.6 (1M context) via ce-work Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
4 tasks
13 tasks
brettdavies
added a commit
that referenced
this pull request
Apr 22, 2026
…env-hints Pattern 2 (#26) ## Summary CLI-side implementation of v0.1.3 handoff 5 (H5). Three orthogonal features share a release vehicle because `agentnative-site`'s H6 leaderboard launch reads whatever this release emits — bundling lets the site regen wave pick up all three changes in a single pass. 1. **Audience classifier** — new read-only derivation that labels a scorecard run as `agent_optimized`, `mixed`, or `human_primary` based on `Warn` counts across 4 signal behavioral checks (`p1-non-interactive`, `p2-json-output`, `p7-quiet`, `p6-no-color-behavioral`). Returns `null` when any signal check is missing so partial signal cannot produce a misleading verdict. 2. **`--audit-profile` flag** — accepts one of `human-tui`, `file-traversal`, `posix-utility`, `diagnostic-only`. Suppresses checks that don't apply to the category (TUI apps legitimately intercept the TTY; POSIX utilities satisfy stdin-primary vacuously; diagnostic tools have no writes). Suppressed checks appear in `results[]` as `Skip` with structured evidence `"suppressed by audit_profile: <category>"` so readers see what was excluded. 3. **`p1-env-hints` Pattern 2** — extends detection beyond clap's `[env: FOO]` annotations to also recognize bash-style `$FOO` / `TOOL_FOO` references near flag definitions. Catches `ripgrep`, `aider`, and similar tools that document env bindings in free prose. Three mitigations against false positives: tool-scoped identifier shape (must be `$`-prefixed OR contain `_`), same-paragraph proximity (±4-line window or ENVIRONMENT section), shell-env blacklist. Plan + implementation log: `docs/plans/2026-04-21-v013-handoff-5-cli-audience-classifier.md` (already on `dev`). Schema-stable: `schema_version` stays `"1.1"`. `audience` and `audit_profile` move from `null` to populated values on the reserved fields, backwards-compatible with v1.1 consumers. ## Changelog ### Added - `audience` field on scorecard JSON now emits a real label (`agent_optimized` / `mixed` / `human_primary`) when all four signal behavioral checks ran, or `null` when any are missing. - `--audit-profile <category>` flag on `anc check` accepts `human-tui`, `file-traversal`, `posix-utility`, or `diagnostic-only`; the applied value echoes as the top-level `audit_profile` field on scorecard JSON. ### Changed - `p1-env-hints` now recognizes bash-style env-var references (`$FOO` / `TOOL_FOO`) near flag definitions in addition to clap `[env: FOO]` annotations. Tools like `ripgrep` and `aider` that document env bindings in free prose now Pass instead of Warn. ## Type of Change - [x] `feat`: New feature (non-breaking change which adds functionality) ## Related Issues/Stories - Plan: `docs/plans/2026-04-21-v013-handoff-5-cli-audience-classifier.md` - Compounded learning: `docs/solutions/best-practices/cli-env-var-shape-heuristic-2026-04-21.md` - Origin (combined H5): `agentnative-site/docs/plans/2026-04-20-v013-handoff-5-audience-leaderboard.md` - H4 handoff that designed Pattern 2: `docs/plans/2026-04-20-v012-handoff-4-behavioral-checks.md` - Todo 013 (pending→done): `.context/compound-engineering/todos/013-done-p3-p1-env-hints-pattern-2-bash-style-detection.md` ## Testing - [x] Unit tests added/updated - [x] Integration tests added/updated - [x] Manual testing completed - [x] All tests passing **Test Summary:** - Unit tests: 382 passing (15 new in `scorecard::audience`, 6 new in `principles::registry::tests` covering `SUPPRESSION_TABLE` + `suppresses()`, 6 new in `runner::help_probe::tests` covering Pattern 2) - Integration tests: 47 passing (6 new `test_audit_profile_*` cases covering rejection of unknown values, JSON echo, Skip evidence format, absent-flag behavior, denominator shrink, and dogfood edge case) - `cargo clippy --all-targets -- -Dwarnings`: clean - `cargo fmt --check`: clean - `cargo deny check`: advisories/bans/licenses/sources ok - `anc generate coverage-matrix --check`: exit 0 (no registry drift) - Full `scripts/hooks/pre-push`: all checks passed **Dogfood verdicts:** - `anc check . --output json` → `schema_version: "1.1"`, `audience: "agent_optimized"`, `audit_profile: null`, 27 pass / 2 warn / 4 skip / 0 fail. No regression. - `anc check . --audit-profile diagnostic-only --output json` → clean run, echoes `"audit_profile": "diagnostic-only"`, suppresses `p5-dry-run` with structured evidence, no panic. **Live-binary smoke on Pattern 2:** - `rg`: `p1-env-hints` Warn → Pass (catches `RIPGREP_CONFIG_PATH` and `RIPGREP_COLOR` in `--config` description prose). - `aider`: `p1-env-hints` Warn → Pass (catches `$FOO`-style prose mentions). - `gh`: still Warn. Named limitation — `gh`'s env docs live in `gh help environment`, not `gh --help`, which is what the behavioral check probes. Out of scope for v0.1.3. ## Files Modified **Created:** - `src/scorecard/audience.rs` — `classify()`, `SIGNAL_CHECK_IDS`, 13 unit tests including drift guards - `docs/solutions/best-practices/cli-env-var-shape-heuristic-2026-04-21.md` (in shared solutions-docs repo) **Modified:** - `src/scorecard.rs` → `src/scorecard/mod.rs` — directory-module promotion (git rename), `format_json` signature extended to accept `audience` + `audit_profile` - `src/cli.rs` — `--audit-profile` flag + `AuditProfile` ValueEnum + `From<AuditProfile> for ExceptionCategory` - `src/main.rs` — suppression wiring in check execution loop, audience computation, scorecard field threading - `src/principles/registry.rs` — `SUPPRESSION_TABLE`, `suppresses()` helper, `ExceptionCategory::as_kebab_case()`, `Diagnostic` → `DiagnosticOnly` rename (to match `"diagnostic-only"` serde output), removed the `#[allow(dead_code)]` reservation - `src/runner/help_probe.rs` — Pattern 2 implementation (`parse_env_hints_bash_style`, `extract_env_tokens`, `strip_clap_env_annotations`, `find_env_section`, `SHELL_ENV_BLACKLIST`, `PATTERN2_WINDOW`) - `tests/integration.rs` — 6 new `test_audit_profile_*` cases - `CLAUDE.md` — removed the "reserved for v0.1.3" callout on `ExceptionCategory`; updated the scorecard-fields section to document the now-consumed fields - `completions/anc.{bash,zsh,fish,elvish,powershell}` — regenerated for the new `--audit-profile` flag surface ## Key Features - **Informational audience label, not a gate.** Per CEO review Finding #3: the classifier is read-only over `results[]`. It does not mutate per-check verdicts, summary counts, or exit codes. Label mismatches on a given tool are fixed by adding a registry `audit_profile` or a new MUST — never by tweaking classifier logic. The classifier module's docstring records this doctrine verbatim. - **Suppression is categorical, not per-tool.** The four categories are fixed for v0.1.3. Adding a fifth requires a plan revision. Per-tool `audit_profile` lookups happen at the caller (site's regen script), keeping the CLI repo-agnostic — it only knows what the caller tells it. - **Pattern 2 keeps `Confidence::Medium`.** Widening detection does not raise confidence; the heuristic still has blind spots (e.g., help topics outside `--help`). Honest calibration over ambition. - **Drift guards everywhere.** `SIGNAL_CHECK_IDS` validated against the behavioral catalog; `SUPPRESSION_TABLE` validated against the full check catalog; `ExceptionCategory::as_kebab_case()` cross-checked against serde's rendering. A rename on either side of any mapping surfaces at test time. ## Breaking Changes - [x] No breaking changes Additive to the v1.1 scorecard schema — reserved `null` fields now populate with real values. Consumers already feature-detect `audience` and `audit_profile` (per site's H2 renderer); pre-v1.1 scorecards remain valid. ## Deployment Notes - [x] No special deployment steps required (for this PR into `dev`) Release-branch mechanics per `RELEASES.md` happen in a follow-up PR: branch off `origin/main`, cherry-pick these commits, bump `Cargo.toml` `0.1.2 → 0.1.3`, regenerate changelog via `generate-changelog.sh`, PR to `main`. The Homebrew-tap finalize-release dispatch will again land on the wrong repo (todo `006`) — manual recovery command is pre-staged in the plan's operational notes section. ## Checklist - [x] Code follows project conventions and style guidelines - [x] Commit messages follow Conventional Commits - [x] Self-review of code completed (including a `code-simplicity-reviewer` pass that surfaced three redundant guards — landed as the final `refactor(help_probe)` commit) - [x] Tests added/updated and passing - [x] No new warnings or errors introduced - [x] Changes are backward compatible ## Additional Context The plan doc (`docs/plans/2026-04-21-v013-handoff-5-cli-audience-classifier.md`) already landed on `dev` as a separate docs-only commit so it can serve as the citable record independent of this PR's merge. The Implementation Log section in the plan captures every deviation from the original scope (unit-bundling due to `-Dwarnings`, enum renames, suppression-table semantics, Pattern 2 heuristic tightening).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a provenance guard workflow that verifies squash-merge provenance for PRs targeting main, and add branch protection rulesets for both dev and main branches.
Type of Change
ci: CI/CD configuration changesFiles Modified
Created:
.github/workflows/guard-main-provenance.yml,.github/rulesets/protect-dev.json,.github/rulesets/protect-main.jsonChecklist