docs: sync CLI documentation with actual command surface#1296
Merged
Conversation
- Replace `batch-eval` with `batch-evaluation` in PERMISSIONS.md for consistency - Replace `recommend` with `recommendation` in PERMISSIONS.md for consistency - Update table column alignment in PERMISSIONS.md logging section - Standardize table formatting across commands.md and evals.md documentation - Align CLI command naming conventions throughout documentation
3 tasks
tejaskash
reviewed
May 18, 2026
Per review feedback (aws#1296), the auto-prefix for bare header names without 'X-' is the legacy 'X-Amzn-Bedrock-AgentCore-Runtime-Custom-', not 'X-Custom-' as the table previously stated. The X-* pass-through half was already correct. Confirmed against normalizeHeaderName in src/cli/commands/shared/header-utils.ts and the HEADER_ALLOWLIST_PREFIX constant in src/schema/schemas/agent-env.ts.
15 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR brings the CLI documentation in line with the actual CLI surface. It does three things:
docs/commands.mdat all —import(with all 5 subcommands),add policy-engine,add policy,add runtime-endpoint, andtelemetry(withenable/disable/status).create,add agent,add evaluator,add online-eval,dev, andrun eval.docs/PERMISSIONS.md—run batch-evalandrun recommendwere used where the actual CLI commands arerun batch-evaluationandrun recommendation.After these changes, 100% of stable CLI commands are documented or mentioned in
docs/orREADME.md(preview commands are intentionally not yet documented).Files changed
docs/commands.md— most of the work (new sections + flag updates)docs/evals.md— kept in sync withcommands.mdfor theadd evaluator,run eval, andadd online-evaltablesdocs/PERMISSIONS.md— command name correctionsdocs/commands.md— new sections### import— top-level flags + 5 subcommands (runtime,memory,evaluator,online-eval,gateway) with examples and per-subcommand flag tables### add policy-engine— Cedar policy engine creation, attach modes### add policy— Cedar policy authoring (file, inline statement,--generatefrom natural language)### add runtime-endpoint— named endpoints (version aliases) for runtimes### telemetry— anonymous usage analytics:enable,disable,statusdocs/commands.md— flag updates to existing sectionscreate: clarified--name(it's the agent name, also used as project dir if--project-nameis omitted), added--project-name,--session-storage-mount-path,--with-config-bundle, addedAGUIto--protocoladd agent: addedAGUIto--protocol,--request-header-allowlist,--session-storage-mount-path,--with-config-bundleadd evaluator: refactored to support bothllm-as-a-judge(default) andcode-basedevaluator types via--type,--lambda-arn,--timeout,--kms-key-arnadd online-eval: added--endpoint <name>dev: added-b, --no-browserand--no-tracesrun eval: added--endpoint,-A, --assertion,--expected-trajectory,--expected-responseremove: addedpolicy-engine,policy, andruntime-endpointto the examples and the consolidated flag tableupdate: documented thatupdate cliis the explicit form ofupdatedocs/evals.mdSame
add evaluator/run eval/add online-evaltable updates as incommands.md, kept consistent across both.docs/PERMISSIONS.md`run batch-eval`with`run batch-evaluation`(9 occurrences) — matches the real CLI command`run recommend`with`run recommendation`(5 occurrences) — matches the real CLI commandHow I verified this
--helpoutput for every command and subcommand to extract the real flag set.agentcore <cmd> ...) across all docs and asserts every flag exists in the CLI — 0 unknown flags.docs/commands.mdor a clear mention in another doc — 0 stable commands missing.ab-test,config-bundle,archive,recommendations,promote,run batch-evaluation,run recommendation,stop batch-evaluation) were intentionally left out of this PR.Related Issue
Closes #
Documentation PR
N/A — documentation changes are included in this PR.
Type of Change
Testing
How have you tested the change?
npm run test:unitandnpm run test:integ(unit tests: 3912/3916 — the 4 failures (DeployStatus,LogPanel,SecretInput) are preexisting onmainand unrelated to this PR; verified they fail identically onupstream/mainwithout my changes)npm run typechecknpm run lint(0 errors; 29 preexisting warnings, none in files touched by this PR)src/assets/, I rannpm run test:update-snapshotsand committed the updated snapshots (no asset changes)Checklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.