Impact Efficiency Report - 2026-06-17

[github-actions]

Executive Summary

Over the 180-day analysis window (2025-12-19 – 2026-06-17), agentic workflows consumed 43,055 AI Credits while producing 7 in-scope outcomes (PRs with linked closing issues). Three accepted outcomes — two critical bug fixes and one improvement — delivered a Total Outcome Value of 205, yielding an Impact Efficiency of 0.00476 value units/AIC. The low ratio reflects that 993 PRs were excluded for lacking linked closing issues and that the largest AIC consumers (reporting, analysis workflows) produced no measurable PR outcomes. Direct workflow attribution is unavailable this cycle due to missing attribution fields in the PR dataset.

---

Summary

Metric	Value
PRs analyzed with linked issue	merged: 3, closed: 4, open excluded: 0
PRs excluded (no linked closing issue)	merged: 497, closed: 496 — total: 993
Safe output outcomes	0
Outcomes mapped to objectives	7
Unmapped outcomes	0
Accepted outcome count	3
Total outcome value	205
AI Credits	43,055.27
Impact Efficiency	0.00476 value units / AIC

AIC source: aic-by-workflow.json from 48 daily token-audit memory snapshots (primary; no live fallback).

---

Agentic Work by Objective

All outcomes are unattributed to workflows, so per-category AIC and IE cannot be computed.

Category	Map Value	Accepted	Outcome Value	AIC	IE	Examples
bug	85	2	170	N/A	N/A	PR #37863, PR #37565
improvement	35	1	35	N/A	N/A	PR #38185

Root issues: #37835 & #37545 (bug,compiler,workflows); #38143 (improvement,quick-win).

Four rejected (closed-without-merge) PRs were mapped to improvement (obj=35, outcome value=0): #36767, #36371, #35906, #35330 — all [docs] automation PRs.

Highest AIC consumer: daily-experiment-report (3,625 AIC, 2 runs) — no accepted outcomes.

---

Which Workflows Drove That Impact

Workflow attribution for all 7 analyzed outcomes is unavailable — the PR datasets contain no workflow_run_id/workflow_name fields and workflow-logs.json covers only 10 runs from today.

Unattributed bucket: 7 outcomes, 3 accepted, Total Outcome Value = 205

Top AIC-consuming workflows (none with attributed outcome value):

Workflow	AIC	Runs
daily-experiment-report	3,625	2
Impact Efficiency Report	2,647	3
Contribution Check	2,278	5
Daily Semgrep Scan	2,209	2
Daily Firewall Logs Collector	2,144	1
Glossary Maintainer	2,039	2

⚠️ No workflow was credited with delivered outcome value this cycle — attribution data was missing, not absent agentic output.

---

Top Outcomes by Outcome Value

Outcome	Workflow	Type	Root / Objective	Obj Value	Outcome Value
PR #37863 — Fix push_to_pull_request_branch head-ref	Unattributed	Merged	#37835 bug	85	85
PR #37565 — Fix sample replay credential clobber	Unattributed	Merged	#37545 bug	85	85
PR #38185 — Add execcommandwithoutcontext linter	Unattributed	Merged	#38143 improvement	35	35
PR #36767 — [docs] Self-healing fixes 2026-06-04	Unattributed	Closed	#36275 improvement	35	0
PR #36371 — [docs] Self-healing fixes 2026-06-02	Unattributed	Closed	#34686 improvement	35	0
PR #35906 — [docs] Documentation update 2026-05-30	Unattributed	Closed	#35506 improvement	35	0
PR #35330 — [docs] Self-healing fixes 2026-05-28	Unattributed	Closed	#34686 improvement	35	0

---

Unmapped Outcomes

No unmapped in-scope outcomes. All 7 analyzed PRs had at least one mapped label through their linked closing issues. (cookie, deep-report labels are unmapped but co-occur with mapped labels.)

---

Interpretation

Accepted outcome count (3) appears low, but 993 PRs were excluded solely for lacking a Closes #N link — not for lack of agentic output.

Impact Efficiency (0.00476) is the better metric, but is artificially depressed: the denominator covers 80+ workflows over 180 days, most producing non-PR outcomes excluded from this analysis. Top AIC spenders (reporting/analysis workflows) collectively consumed ~20% of all AIC with zero attributed PR outcome value.

Key findings:

• Bug fixes deliver the most value (85/outcome): two compiler/workflow bug fixes account for 83% of total delivered value.
• Four documentation PRs were rejected — recurring pattern worth investigating in the documentation updater workflow.
• Workflow attribution gap is the single biggest blocker for useful per-workflow efficiency analysis.
• Reporting/analysis workflows dominate AIC with no PR-outcome return — this is expected, but cost visibility should drive prioritization.

---

Data Quality

Dimension	Status	Notes
PR linked-issue coverage	⚠️ Weak	99.3% of PRs excluded (993/1000). Only PRs with explicit Closes #N count.
Objective label mapping	✅ Good	All 7 in-scope outcomes mapped. cookie, deep-report unmapped but coexist with mapped labels.
Workflow attribution	❌ Missing	No workflow_run_id in PR dataset; workflow-logs.json covers only 10 runs from today.
Safe-output outcomes	✅ Complete	0 outcomes, confirmed deterministically.
AI Credits	✅ Primary	48 daily token-audit snapshots; total 43,055.27 AIC, 80 workflows.
Dataset completeness	⚠️ Capped	Both PR datasets capped at 500 records; 180-day window may contain more.

---

Human Work

Because workflow attribution is absent from the PR dataset and workflow-logs.json covers only today's 10 runs, all 500 merged PRs in the dataset appear human-authored — likely an overcount (e.g., [linter-miner] and [docs] PRs are probably agentic).

Metric	Value
Merged PRs in dataset	500 (capped)
With linked closing issue	3
Without linked closing issue	497
Mapped to objective	3
Unmapped	0

Category	Map Value	PRs	Obj Value	Examples
bug	85	2	170	#37863, #37565
improvement	35	1	35	#38185

No AIC or Impact Efficiency computed for human work.

Generated by 📊 Impact Efficiency Report · ◷