[agentic-token-optimizer] Optimize Repository Quality Improvement Agent (609 AIC/run, 53 fragmented bash calls)

[github-actions]

Target Workflow

Repository Quality Improvement Agent (repository-quality-improver.md) — highest-AIC workflow not optimized in the last 14 days. Runs daily on weekdays.

Analysis Period

7-day window ending 2026-06-15 · 1 run analyzed (run §27554699119)

Cost Profile

Metric	Value
Total AIC (1 run)	609.10
Avg AIC / run	609.10
Raw input tokens	1,711,255
Raw output tokens	31,269
Cache read tokens	1,627,442 (95.1% of input)
Cache write tokens	0
LLM turns (API calls)	35
Bash tool calls	53
Action minutes	19
Pre-computed context size	36 lines (Go metrics only)

Key Findings

The run chose Documentation as its focus area and made 53 tool calls across 35 LLM turns. The waste profile breaks down into three distinct problem areas:

Area	Calls	Turns est.	Est. AIC waste
Sequential fragmented bash exploration	36 individual bash calls (#4–39)	~25	~320
Discussion creation probing & failure-retry	13 calls (#40–52), 1 failed attempt	~8	~100
Pre-computed context too narrow	Docs analysis not in pre-agent step	~5	~60

Ranked Recommendations

1. Expand pre-agent step to cover documentation analysis 🔴

Est. savings: ~150 AIC/run

The pre-agent bash step currently collects only Go source metrics (36 lines). The agent chose "Documentation" and spent 23 consecutive bash calls (#4–26) gathering docs structure that could be computed deterministically before the agent starts.

Action: Add to the pre-agent Collect quality metrics bash step:

echo "## Docs Metrics"
echo "### Frontmatter coverage"
grep -rl "^---" docs/src/content/docs/ --include="*.md" 2>/dev/null | wc -l
grep -rL "^---" docs/src/content/docs/ --include="*.md" 2>/dev/null | wc -l

echo "### Sidebar order coverage"
grep -rl "sidebar:" docs/src/content/docs/ --include="*.md" 2>/dev/null | wc -l
grep -rL "sidebar:" docs/src/content/docs/ --include="*.md" 2>/dev/null | wc -l

echo "### TODO/WIP pages"
grep -rl "TODO\|WIP\|coming soon\|placeholder\|under construction" \
  docs/src/content/docs/ --include="*.md" 2>/dev/null | wc -l

echo "### Short pages (<10 lines)"
find docs/src/content/docs -name "*.md" | xargs wc -l 2>/dev/null \
  | awk '$1 < 10 {print}' | grep -v total | wc -l

echo "### doc.go coverage"
for dir in pkg/*/; do
  pkg=$(basename "$dir")
  [ -f "$dir/doc.go" ] && echo "HAS: $pkg" || echo "MISSING: $pkg"
done

Evidence: Bash calls #4–26 are all docs/code structure discovery that ran identically to what this snippet would produce. The pre-agent step completes in <1s and writes to /tmp/gh-aw/agent/analysis-context.md which the agent already reads on turn 1.

---

2. Add explicit safeoutputs create_discussion invocation example 🔴

Est. savings: ~100 AIC/run

The prompt says "Use the reporting MCP to create a discussion" with no concrete syntax. The agent:

1. Called safeoutputs --help (call Remove Gemini engine and all references from codebase #40)
2. Called safeoutputs create_discussion --help (call remove gemini support for now #41)
3. Failed on first creation attempt (call [Custom Engine Test] Test Pull Request - Custom Engine Safe Output #407, marked ✗ in log)
4. Wrote body to temp JSON file, re-tried via stdin pipe (calls Add workflow: githubnext/agentics/weekly-research #42–43)
5. Made 5 more inspection calls (pin claude action #44–47) to check if it worked
6. Trimmed body size (next text only for admin/maintain #48–50), created again (add ${{ github.server_url }} as allowed substitution #51), verified (Test coverage improvement august 14 metrics #52)

That's 13 calls and ~8 turns just to emit one discussion.

Action: Add to Phase 2 in the prompt:

## Phase 2: Generate Improvement Report

Use the `safeoutputs` CLI to create a discussion:

```bash
printf '%s\n' '{
  "title": "Repository Quality: [Focus Area] - [Date]",
  "body": "...",
  "category": "announcements"
}' | safeoutputs create_discussion .

Do NOT call safeoutputs --help or probe parameters first — the schema above is complete. Pass the body inline in the JSON object.

**Evidence:** The 13-call ceremony accounted for ~8 turns out of 35 total (23%). This optimization alone eliminates a common retry-loop pattern that wastes significant context.

---

#### 3. Add bash batching guidance to Phase 1 🟡
**Est. savings: ~80 AIC/run**

The prompt provides a per-category command reference table but doesn't instruct the agent to batch independent commands. This caused 8 separate calls just for Go documentation coverage (#10–17) and another 10 calls for docs sidebar/content analysis (#16–26).

**Action:** Add to the Phase 1 "Conduct Analysis" section:
```markdown
**Batching rule**: Consolidate all analysis for your chosen focus area into at most
4 bash calls, one per major topic cluster. For Documentation: one call for
frontmatter/structure, one for content quality, one for code doc-comments.
Do NOT run separate bash calls for each individual metric — combine with `&&` and
section headers.

Evidence: Calls #10–17 (8 calls) cover the same Go code documentation topic and could be a single bash block. Calls #16–26 (11 calls) explore docs sidebar configuration and could be 2 calls.

---

4. Inline sub-agent for Phase 1 data collection 🟡

Est. savings: ~150 AIC/run

The analysis phase (Phase 1) is a strong candidate for extraction into an inline sub-agent using a smaller model:

Dimension	Score	Reason
Independence	3/3	Needs only the focus area name from Phase 0
Small-model adequacy	3/3	Pure bash execution + structured result collection
Parallelism	2/2	Runs before report generation, no concurrency conflict
Size	2/2	36 bash calls, ~25 exploration turns currently
Total	10/10	Strong candidate

Action: Add an ## agent: block after Phase 0:

## agent: analysis-collector
model: github/haiku
inputs:
  focus_area: "{{ focus_area }}"
  pre_computed: "{{ analysis_context }}"

Collect analysis data for focus area `{{ focus_area }}` by running targeted bash
commands. Read pre-computed context from `/tmp/gh-aw/agent/analysis-context.md`.
Output a JSON summary of findings to `/tmp/gh-aw/agent/analysis-results.json`.
Run at most 4 bash calls. Do not write any reports.

Moving the exploratory bash work to a Haiku sub-agent (10× cheaper than Sonnet) would save ~150 AIC. The main agent then only needs to read the results JSON and write the discussion (~10 turns).

Note: Implement Recommendations 1–3 first to establish the turn baseline. Sub-agent refactor is higher risk; validate after prompt fixes.

---

Caveats

• Only 1 run was available in the 7-day window; behavior varies by chosen focus area. Documentation focus is more exploration-heavy than Security or CI/CD.
• The 95.1% cache read rate indicates prompt caching is functioning well — savings estimates target turn reduction, not cache optimization.
• The failed first discussion creation attempt (call [Custom Engine Test] Test Pull Request - Custom Engine Safe Output #407) may not be reproducible if it was caused by a transient issue, but the 10+ check/retry calls are consistently observed.
• Recommendations 1–3 are independent and low-risk. Implement together for maximum savings before attempting Rec 4.

References: §27554699119

Generated by Agentic Workflow AIC Usage Optimizer · 759.6 AIC · ⊞ 24.6K · ◷

• expires on Jun 22, 2026, 8:42 AM UTC-08:00

[github-actions]

This issue is being closed as outdated. A newer issue has been created: #39607

View newer issue

---

This action was performed automatically by the Agentic Workflow AIC Usage Optimizer workflow.