feat: add test-coverage prompt with artifact-reading guidance - #18273
Conversation
Adds .github/aw/test-coverage.md with guidance that distinguishes: - Preferred: fetch pre-computed coverage artifacts via gh run download or actions toolset - Fallback: run tests when no prior artifacts exist Links the new prompt from .github/agents/agentic-workflows.agent.md. Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
@copilot optimize for token usage and still provide clear guidance for the authoring agent |
There was a problem hiding this comment.
Pull request overview
Adds dedicated guidance for creating agentic workflows that analyze test coverage, with an explicit preference for reading existing CI coverage artifacts (via gh run download or GitHub actions toolset) instead of re-running tests.
Changes:
- Introduces a new
.github/aw/test-coverage.mdprompt covering artifact-first strategy, fallback behavior, formats, and an end-to-end example. - Updates the dispatcher agent documentation to route “test coverage” requests to the new prompt and links it in Available Prompts.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| .github/aw/test-coverage.md | New test coverage workflow guidance emphasizing artifact-first coverage analysis. |
| .github/agents/agentic-workflows.agent.md | Adds routing + documentation entry pointing to the new test-coverage prompt. |
Comments suppressed due to low confidence (1)
.github/aw/test-coverage.md:152
- The complete example workflow repeats the same frontmatter issues as the earlier template (
triggers:instead ofon:andnetwork: defaults: true). Since this is presented as a copy/paste-ready workflow, it’s likely to produce non-working workflows unless corrected to the documented schema.
---
engine: copilot
triggers:
pull_request:
types: [opened, synchronize]
permissions:
pull-requests: write
actions: read
network:
defaults: true
tools:
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| ```bash | ||
| # Find the latest successful CI run for this commit/branch | ||
| gh run list --branch "$BRANCH" --workflow ci.yml --status success --limit 1 --json databaseId -q '.[0].databaseId' |
There was a problem hiding this comment.
In the gh run download example, RUN_ID is used but never set. This makes the snippet non-runnable as written. Consider capturing the output of the preceding gh run list ... -q '.[0].databaseId' into RUN_ID (or inline it) so the download step is self-contained.
| gh run list --branch "$BRANCH" --workflow ci.yml --status success --limit 1 --json databaseId -q '.[0].databaseId' | |
| RUN_ID=$(gh run list --branch "$BRANCH" --workflow ci.yml --status success --limit 1 --json databaseId -q '.[0].databaseId') |
|
|
||
| ```yaml | ||
| engine: copilot | ||
| triggers: |
There was a problem hiding this comment.
The frontmatter template uses triggers: for the GitHub Actions event configuration, but this repo’s agentic workflow format uses on: (required). See .github/aw/github-agentic-workflows.md:80 and the example at lines 12-23. Using triggers: in guidance will lead to invalid workflows.
This issue also appears on line 142 of the same file.
| triggers: | |
| on: |
| defaults: true | ||
| # Add language ecosystem if running tests as fallback: | ||
| # egosystems: [node] # or python, go, etc. |
There was a problem hiding this comment.
The network: examples appear to use a non-existent shape (defaults: true) and reference egosystems, which looks like a typo and is not part of the documented schema. In this repo, network should be either network: defaults or network: { allowed: [defaults, <ecosystem>...] } (e.g. add node/python/go only when falling back to running tests). As written, the template may compile to invalid config or mislead users.
| defaults: true | |
| # Add language ecosystem if running tests as fallback: | |
| # egosystems: [node] # or python, go, etc. | |
| allowed: [defaults] # base; extend if running tests as fallback | |
| # Add language ecosystems if running tests as fallback: | |
| # allowed: [defaults, node] # or python, go, etc. |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done in 7271a4b — reduced from 206 to 54 lines (~74% cut) by:
All key guidance (preferred artifact path, fallback condition, required permissions/toolsets) is preserved. |
|
@copilot bring back reference to test coverage and link to file |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Restored in da6ed5c — added back the routing bullet ( |
.github/aw/test-coverage.mdwith artifact-reading guidance (preferred:gh run download/actionstoolset; fallback: run tests).github/agents/agentic-workflows.agent.mdto link the new prompttest-coverage.mdfor token usage (206 → 54 lines)test-coveragerouting entry and### Analyze Test Coveragesection in agent filetriggers:→on:,network: defaults: true→network: defaultsOriginal prompt
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.