feat(p6-consistent-naming): recognize hierarchical noun-verb subcommand patterns - #73
Merged
Merged
Conversation
…nd patterns The p6-should-consistent-naming heuristic flat-classified every top-level subcommand by name alone, then Warned whenever the surface mixed verbs and nouns. That penalized the standard modern CLI shape where top-level verbs (`anc audit`, `gh pull`) coexist with noun-grouped verb hierarchies (`anc skill install`, `gh repo create`). Every major CLI uses this pattern: git, gh, kubectl, docker, npm, cargo, anc itself. The audit now probes one level deeper for each top-level non-verb subcommand and classifies it into one of three buckets: - TopLevelVerb: name is a known verb, OR name is a non-verb with no children (leaf action like `anc emit`), OR name is a non-verb whose children are all non-verbs themselves (verb-with-noun-objects shape like `anc emit coverage-matrix`). - HierarchicalNounVerb: non-verb name whose children are uniformly verbs (`anc skill install` / `skill update`). The noun groups its verbs; the action position stays predictable at the second level. - Mixed: non-verb name whose children include both verbs and non-verbs. The action position is unpredictable even within the group. This is the only case that warns. The classifier reuses the same children-probing pattern as the existing subcommand_help module (cached via the BinaryRunner), and filters clap built-ins (help, completions) out of child counts so they do not skew the all-verbs check. Also folds `emit` into COMMON_VERBS so `anc emit` is classified directly without falling through to the children-probing path (cheaper, and `emit` is a reasonable canonical verb to recognize). Self-audit score moves 99 -> 100. uv (4 noun groups with mixed children: tool, python, pip, self) and cargo (toolchain, override, self) continue to warn because their second-level mixes are genuine; the new evidence string names the offending subcommands so the operator can target the fix. Test matrix (10 cases) covers all three classifications plus the anc pattern, the git/gh pattern, clap-builtin filtering, the warn-message format, and case-insensitive helpers. Closures (`children_of`) keep the audit testable without spinning up a real BinaryRunner.
brettdavies
added a commit
that referenced
this pull request
Jun 1, 2026
…nd patterns (#73) ## Summary Teaches the `p6-should-consistent-naming` behavioral audit to recognize the standard modern CLI shape that combines top-level verbs (`anc audit`, `gh pull`) with noun-grouped verb hierarchies (`anc skill install`, `gh repo create`). The previous heuristic flat-classified every top-level subcommand by name alone and Warned whenever the surface mixed verbs and nouns, which penalized every major CLI that uses this pattern (git, gh, kubectl, docker, npm, cargo, anc itself). The audit now probes one level deeper for each top-level non-verb subcommand and classifies it into one of three buckets: - **TopLevelVerb**: name is a known verb, OR a non-verb with no children (leaf action like `anc emit`), OR a non-verb whose children are all non-verbs themselves (verb-with-noun-objects shape like `anc emit coverage-matrix`). - **HierarchicalNounVerb**: non-verb name whose children are uniformly verbs (`anc skill install` / `skill update`). The noun groups its verbs; the action position stays predictable at the second level. - **Mixed**: non-verb name whose children include both verbs and non-verbs. The action position is unpredictable even within the group. This is the only case that warns. Score impact on `anc` self-audit: **99 → 100**. `uv` and `cargo` continue to warn because their second-level mixes are genuine (`uv` has `tool`, `python`, `pip`, `self` with mixed children; `cargo` has `toolchain`, `override`, `self`). The Warn evidence now names the offending subcommands so the operator can target the fix. ## Changelog ### Changed - `p6-should-consistent-naming` heuristic now recognizes hierarchical noun-verb subcommand patterns. Top-level verbs combined with noun-grouped verbs (the standard `git`, `gh`, `kubectl`, `docker`, `npm`, `cargo`, `anc` shape) Pass; only genuine inconsistency at the second level (a non-verb subcommand whose children include both verbs and non-verbs) Warns. The audit probes one level deeper per non-verb top-level subcommand via the existing cached `BinaryRunner` infrastructure. - `emit` added to the audit's `COMMON_VERBS` list so it is classified directly as a top-level verb action. - Warn evidence now names every offending subcommand by name so the operator can target the fix. ## Type of Change - [x] `feat`: New feature (non-breaking change which adds functionality) ## Related Issues/Stories - Story: n/a - Issue: n/a - Architecture: Surfaced during the v0.5.0 self-audit pass; the prior heuristic incorrectly Warned on `anc`'s own surface (1 verb `audit`, 1 verb-with-noun-objects `emit`, 1 noun-with-verb-children `skill`). The new heuristic matches the goal of "agents can predict where the action lives" rather than insisting on a single-convention surface. - Related PRs: cherry-picked to `release/v0.5.0-audit-rename` after this PR merges. ## Testing - [x] All tests passing **Test Summary:** - `cargo test`: 800 passed, 2 ignored (was 793; +7 new red-team cases in `consistent_naming`). - `cargo build --release`: clean. - `anc emit coverage-matrix --check`: exit 0. - `cargo deny check advisories`: ok. - Score deltas across the six audit targets: | Target | Before | After | Note | |---|---|---|---| | `anc` (self) | 99 / warn | **100 / pass** | The fix. | | `jq` | 77 / skip | 77 / skip | No subcommand surface. | | `gh` | 68 / skip | 68 / skip | Probe gating unchanged. | | `uv` | 73 / warn | 73 / warn | Genuine: `tool` / `python` / `pip` / `self` have mixed children. | | `cargo` | 69 / warn | 69 / warn | Genuine: `toolchain` / `override` / `self` have mixed children. | | `bc` | 74 / skip | 74 / skip | No subcommand surface. | ## Files Modified **Modified:** - `src/audits/behavioral/consistent_naming.rs` (+273 / -41): new `Classification` enum, child-probing closure parameter, three-bucket classification logic, 10-case red-team test matrix covering anc / git / Mixed / leaf-noun / clap-builtin filtering / single-command skip / case-insensitivity / warn-message format. **Created:** - None. **Renamed:** - None. **Deleted:** - None. ## Breaking Changes - [x] No breaking changes The audit ID (`p6-consistent-naming`), the requirement ID it covers (`p6-should-consistent-naming`), the tier (SHOULD), the principle group (P6), and the layer (behavioral) are unchanged. The scorecard JSON shape is unchanged. The behavior change is one audit's Pass/Warn boundary moving to the more accurate side; consumers feature-detect status values as before. ## Deployment Notes - [x] No special deployment steps required
brettdavies
added a commit
that referenced
this pull request
Jun 1, 2026
The original v0.5.0 CHANGELOG section was generated before the hierarchical noun-verb consistent-naming fix (PR #73) was cherry-picked onto the release branch. Regenerated to include the new bullet plus a couple of others (#68, #69, #71 cherry-picks) that landed between the first generation and now. Side-effect: the section header date moves 2026-05-31 -> 2026-06-01 since git-cliff uses the regeneration date. The tag will publish today; the new date matches the actual release date.
brettdavies
added a commit
that referenced
this pull request
Jun 3, 2026
…llow-up plans Adds five plans surfaced by the post-merge adversarial review of PRs #73, #76, #77, and #79: - 2026-06-03-001 (umbrella, P1): structure-over-vocabulary audit philosophy. Spec is canonical; closed-vocabulary lookups (COMMON_VERBS, STANDARD_VERBS, DISCRIMINANT_FIELD_NAMES, SUCCESS_VALUE_STRINGS) are failed quality proxies; per-PR redesigns of p6-consistent-naming and p2-must-json-errors land under this plan. - 2026-06-03-002 (P1): PR #73 follow-up. Probe-failure now Warns instead of silent Pass, leaf non-verb requires a successful probe, drop emit from COMMON_VERBS, add 4 negative-case tests. - 2026-06-03-003 (P1): PR #76 follow-up. Scorecard transparency for .anc.toml domain_verbs (using_domain_verbs flag + domain_match_count + evidence string showing built-in vs domain ratio), trim 7 platform-specific verbs out of built-in STANDARD_VERBS, document the social-CLI .anc.toml pattern. - 2026-06-03-004 (P0): PR #77 follow-up. cfg(not(test)) is treated identically to cfg(test) in the new cfg-args parser, silently exempting production-only code from code-unwrap. Thread polarity through the recursive call site, add 8 negative-case tests, correct an inner-attribute comment. - 2026-06-03-005 (P1): PR #79 follow-up. Close three role-based-validator bypasses: numeric discriminant on info-coded values, success synonyms outside the 8-string closed set, and nested-recursion harvesting roles from a success-coded outer envelope. Add 6 adversarial tests. Plans 002, 003, and 005 are tactical stopgaps that stop active misclassification while plan 001 owns the long-term redesign. Plan 004 is a correctness bug fix and ships independently.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Teaches the
p6-should-consistent-namingbehavioral audit to recognize the standard modern CLI shape that combines top-level verbs (anc audit,gh pull) with noun-grouped verb hierarchies (anc skill install,gh repo create). The previous heuristic flat-classified every top-level subcommand by name alone and Warned whenever the surface mixed verbs and nouns, which penalized every major CLI that uses this pattern (git, gh, kubectl, docker, npm, cargo, anc itself).The audit now probes one level deeper for each top-level non-verb subcommand and classifies it into one of three
buckets:
anc emit), OR a non-verb whose children are all non-verbs themselves (verb-with-noun-objects shape likeanc emit coverage-matrix).anc skill install/skill update). The noun groups its verbs; the action position stays predictable at the second level.Score impact on
ancself-audit: 99 → 100.uvandcargocontinue to warn because their second-level mixes are genuine (uvhastool,python,pip,selfwith mixed children;cargohastoolchain,override,self). The Warn evidence now names the offending subcommands so the operator can target the fix.Changelog
Changed
p6-should-consistent-namingheuristic now recognizes hierarchical noun-verb subcommand patterns. Top-level verbs combined with noun-grouped verbs (the standardgit,gh,kubectl,docker,npm,cargo,ancshape) Pass; only genuine inconsistency at the second level (a non-verb subcommand whose children include both verbs and non-verbs) Warns. The audit probes one level deeper per non-verb top-level subcommand via the existing cachedBinaryRunnerinfrastructure.emitadded to the audit'sCOMMON_VERBSlist so it is classified directly as a top-level verb action.Type of Change
feat: New feature (non-breaking change which adds functionality)Related Issues/Stories
anc's own surface (1 verbaudit, 1 verb-with-noun-objectsemit, 1 noun-with-verb-childrenskill). The new heuristic matches the goal of "agents can predict where the action lives" rather than insisting on a single-convention surface.release/v0.5.0-audit-renameafter this PR merges.Testing
Test Summary:
cargo test: 800 passed, 2 ignored (was 793; +7 new red-team cases inconsistent_naming).cargo build --release: clean.anc emit coverage-matrix --check: exit 0.cargo deny check advisories: ok.anc(self)jqghuvtool/python/pip/selfhave mixed children.cargotoolchain/override/selfhave mixed children.bcFiles Modified
Modified:
src/audits/behavioral/consistent_naming.rs(+273 / -41): newClassificationenum, child-probing closure parameter, three-bucket classification logic, 10-case red-team test matrix covering anc / git / Mixed / leaf-noun / clap-builtin filtering / single-command skip / case-insensitivity / warn-message format.Created:
Renamed:
Deleted:
Breaking Changes
The audit ID (
p6-consistent-naming), the requirement ID it covers (p6-should-consistent-naming), the tier(SHOULD), the principle group (P6), and the layer (behavioral) are unchanged. The scorecard JSON shape is unchanged. The behavior change is one audit's Pass/Warn boundary moving to the more accurate side; consumers feature-detect status values as before.
Deployment Notes