Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/adr/27137-add-ambient-context-metric.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
**Status**: Draft
**Deciders**: pelikhan, Copilot

> **Migration note:** This ADR references the legacy Effective Tokens (ET) terminology for historical context. gh-aw now uses AI Credits (AIC) as the primary cost metric.

---

## Part 1 — Narrative (Human-Friendly)
Expand Down
2 changes: 2 additions & 0 deletions docs/adr/28003-fallback-audit-metrics-without-aw-info.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
**Status**: Draft
**Deciders**: pelikhan

> **Migration note:** This ADR references the legacy Effective Tokens (ET) terminology for historical context. gh-aw now uses AI Credits (AIC) as the primary cost metric.

---

## Part 1 — Narrative (Human-Friendly)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
**Status**: Draft
**Deciders**: pelikhan, Copilot

> **Migration note:** This ADR references the legacy Effective Tokens (ET) terminology for historical context. gh-aw now uses AI Credits (AIC) as the primary cost metric.

---

## Part 1 — Narrative (Human-Friendly)
Expand Down
2 changes: 2 additions & 0 deletions docs/adr/31377-monte-carlo-projection-for-forecast-command.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
**Status**: Draft
**Deciders**: Unknown (PR authored by `app/copilot-swe-agent`; human deciders TBD)

> **Migration note:** This ADR references the legacy Effective Tokens (ET) terminology for historical context. gh-aw now uses AI Credits (AIC) as the primary cost metric.

---

## Part 1 — Narrative (Human-Friendly)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
**Status**: Draft
**Deciders**: pelikhan, Copilot

> **Migration note:** This ADR references the legacy Effective Tokens (ET) terminology for historical context. gh-aw now uses AI Credits (AIC) as the primary cost metric.

---

## Part 1 — Narrative (Human-Friendly)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
**Status**: Draft
**Deciders**: PR author (pelikhan), reviewers of PR #35338

> **Migration note:** This ADR references the legacy Effective Tokens (ET) terminology for historical context. gh-aw now uses AI Credits (AIC) as the primary cost metric.

---

## Part 1 — Narrative (Human-Friendly)
Expand Down
2 changes: 2 additions & 0 deletions docs/adr/36421-derive-effective-tokens-from-raw-usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ Recompute ET once when usage is written and persist the corrected value. Rejecte
- Reporting/output shape is unchanged; only the *source* of the ET value changed.
- The unknown-model fallback multiplier (`1.0`) and longest-prefix matching behavior are preserved.

> **Migration note:** This ADR references the legacy Effective Tokens (ET) terminology for historical context. gh-aw now uses AI Credits (AIC) as the primary cost metric.

---

*This is a DRAFT ADR generated by the [Design Decision Gate](https://github.com/github/gh-aw/actions/runs/26805931606) workflow. The PR author must review, complete, and finalize this document before the PR can merge.*
2 changes: 2 additions & 0 deletions docs/adr/36496-centralize-effective-token-limit-parsing.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ Accept short-form only at validation time and have the schema (or a pre-processi
- Documentation now defaults to short-form notation, so existing long-form examples in user workflows differ stylistically from the docs (both remain valid).
- Schema default values remain expressed as plain integers (`25000000`) even though docs prefer `25M`.

> **Migration note:** This ADR references the legacy Effective Tokens (ET) terminology for historical context. gh-aw now uses AI Credits (AIC) as the primary cost metric.

---

*This is a DRAFT ADR generated by the [Design Decision Gate](https://github.com/github/gh-aw/actions/runs/26837141982) workflow. The PR author must review, complete, and finalize this document before the PR can merge.*
3 changes: 3 additions & 0 deletions docs/src/content/docs/blog/2026-05-11-weekly-update.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ authors:
date: 2026-05-11
---

> [!NOTE]
> This post references historical **Effective Tokens (ET)** metrics. gh-aw now uses **AI Credits (AIC)** as the primary cost metric.

It was a busy week in [github/gh-aw](https://github.com/github/gh-aw)! Four releases landed between May 4 and May 7, paired with a wave of pull requests that delivered new commands, security hardening, and developer-experience polish. Here's everything that shipped.

## Releases This Week
Expand Down
3 changes: 3 additions & 0 deletions docs/src/content/docs/blog/2026-05-26-agent-of-the-day.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ metadata:
linkedPostText: "How we track Copilot agent PR performance with a daily analysis workflow"
---

> [!NOTE]
> This post references historical **Effective Tokens (ET)** metrics. gh-aw now uses **AI Credits (AIC)** as the primary cost metric.

Every morning someone at GitHub opens their laptop and wonders: how well did the coding agents do yesterday? Did they ship? Did they stall? Did they create more work than they saved? These questions used to require manual spelunking through dashboards, cross-referencing merged PRs with author names, and guessing at patterns from vibes alone.

Not anymore.
Expand Down
3 changes: 3 additions & 0 deletions docs/src/content/docs/blog/2026-05-27-agent-of-the-day.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ metadata:
linkedPostText: "Agent Performance Analyzer hits 90/100 ecosystem health"
---

> [!NOTE]
> This post references historical **Effective Tokens (ET)** metrics. gh-aw now uses **AI Credits (AIC)** as the primary cost metric.

Every day, 236 agentic workflows run inside the `gh-aw` repository. Most complete quietly. A few fail in patterns worth tracking. And once a week, one workflow reads the entire fleet, scores it, and writes up what it found. That workflow is the **Agent Performance Analyzer**, and its run on May 27, 2026 produced the clearest signal in months.

## Agent of the Day: Agent Performance Analyzer — Meta-Orchestrator
Expand Down
3 changes: 3 additions & 0 deletions docs/src/content/docs/blog/2026-05-28-agent-of-the-day.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ metadata:
linkedPostText: "Dead Code Removal Agent hits 100 automated runs"
---

> [!NOTE]
> This post references historical **Effective Tokens (ET)** metrics. gh-aw now uses **AI Credits (AIC)** as the primary cost metric.

Every codebase accumulates sediment. A helper function that made sense six months ago. A wrapper that lost its reason to exist after a refactor. Nobody deletes it on purpose — it just lingers. In Go, that lingering costs you: extra surface area to maintain, test coverage for code that does nothing new, and cognitive overhead for every engineer who reads the file.

The **Dead Code Removal Agent** is a scheduled GitHub Actions workflow that runs daily on the `gh-aw` repository. Its job is simple: find unused code, verify nothing breaks, and open a pull request. No human intervention required until review time.
Expand Down
3 changes: 3 additions & 0 deletions docs/src/content/docs/blog/2026-06-01-agent-of-the-day-2.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ metadata:
linkedPostText: "Catch architectural drift automatically with Architecture Guardian"
---

> [!NOTE]
> This post references historical **Effective Tokens (ET)** metrics. gh-aw now uses **AI Credits (AIC)** as the primary cost metric.

## 🏗️ Agent of the Day: Architecture Guardian

Architectural drift is quiet and cumulative. A file grows past 600 lines. A function absorbs one more responsibility. An import cycle sneaks in between two packages that "just need to share a little logic." None of it trips a CI gate, no test turns red, and six months later a new engineer opens that directory and wonders how it got this bad. The Architecture Guardian workflow exists precisely to interrupt that pattern before it becomes load-bearing.
Expand Down
3 changes: 3 additions & 0 deletions docs/src/content/docs/blog/2026-06-01-weekly-update.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ authors:
date: 2026-06-01
---

> [!NOTE]
> This post references historical **Effective Tokens (ET)** metrics. gh-aw now uses **AI Credits (AIC)** as the primary cost metric.

It's been a busy week in [github/gh-aw](https://github.com/github/gh-aw)! Five releases landed between May 28 and May 31, capped off by [v0.77.4](https://github.com/github/gh-aw/releases/tag/v0.77.4) — one of the biggest releases in recent memory. Here's everything that shipped.

## Release: v0.77.4
Expand Down
14 changes: 9 additions & 5 deletions docs/src/content/docs/reference/audit.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@ sidebar:

The `gh aw audit` commands download workflow run artifacts and logs, analyze MCP tool usage and network behavior, and produce structured reports suited for security reviews, debugging, and feeding to AI agents.

> [!NOTE]
> AI Credits (AIC) are the primary spend metric in gh-aw. Legacy Effective Tokens (ET) fields remain available for backward compatibility in report output.

## `gh aw audit <run-id-or-url> [<run-id-or-url>...]`

Audit one or more workflow runs. When a single run is provided, a detailed Markdown report is generated. When two or more runs are provided, the first is used as the base (reference) run and the remaining runs are compared against it, producing a diff report.
Expand Down Expand Up @@ -82,7 +85,7 @@ gh aw audit 12345 12346 --repo owner/repo # Specify repository
The Metrics section includes an `ambient_context` object when available. Ambient context captures the first LLM inference footprint for the run:
- `ambient_context.input_tokens` — input tokens for the first invocation
- `ambient_context.cached_tokens` — cache-read tokens reused by the first invocation
- `ambient_context.effective_tokens` — `input_tokens + cached_tokens`
- `ambient_context.effective_tokens` — legacy ET field (`input_tokens + cached_tokens`) retained for compatibility

**Diff output** includes:
- New and removed network domains
Expand All @@ -91,8 +94,8 @@ The Metrics section includes an `ambient_context` object when available. Ambient
- Anomaly flags (new denied domains, previously-denied domains now allowed)
- MCP tool invocation changes (new/removed tools, call count and error count diffs)
- Run metrics comparison (token usage, duration, turns)
- Token usage and spend breakdown: input tokens, output tokens, cache read/write tokens, effective tokens, AIC, total API requests, and cache efficiency per run
- Tokens per turn: effective tokens divided by turn count for each run, with the change between runs
- Token usage and spend breakdown: input tokens, output tokens, cache read/write tokens, AIC, legacy effective tokens (ET), total API requests, and cache efficiency per run
- Tokens per turn: legacy ET divided by turn count for each run, with the change between runs
- AIC reporting: AI Credits are shown alongside token metrics for spend tracking
- Tool call breakdown: per-tool call counts (new, removed, and changed tools) with max input/output sizes
- Bash command breakdown: aggregated call counts and max input/output sizes for each distinct bash command invoked
Expand Down Expand Up @@ -122,7 +125,7 @@ This feature is built into the `gh aw logs` command via the `--format` flag.

The report output includes an executive summary, domain inventory, metrics trends, MCP server health, and per-run breakdown. It detects cross-run anomalies such as domain access spikes, elevated MCP error rates, and connection rate changes.

For each run in detailed logs JSON output, an `ambient_context` object is included when token usage data is available. It reflects only the first LLM invocation in the run (`input_tokens`, `cached_tokens`, `effective_tokens`).
For each run in detailed logs JSON output, an `ambient_context` object is included when token usage data is available. It reflects only the first LLM invocation in the run (`input_tokens`, `cached_tokens`, and legacy `effective_tokens`).

**`--stdin` mode:** Pass `--stdin` to supply an explicit list of run IDs or URLs instead of letting the command discover runs from the GitHub API. Date, count, and workflow-name filters are ignored; `--engine`, `--firewall`, `--safe-output`, and other content filters still apply. Blank lines and `#`-prefixed lines are ignored. Bare numeric IDs require `--repo owner/repo`.

Expand All @@ -146,7 +149,8 @@ gh aw logs --format markdown --repo owner/repo --count 10

- [Cost Management](/gh-aw/reference/cost-management/) — Track AIC-first spend and token usage
- [Artifacts](/gh-aw/reference/artifacts/) — Artifact names, directory structures, and token usage file locations (`token-usage.jsonl` in `firewall-audit-logs`)
- [Effective Tokens Specification](/gh-aw/specs/effective-tokens-specification/) — Legacy effective token computation details
- [AI Credits Specification](/gh-aw/specs/ai-credits-specification/) — Primary AIC computation details
- [Effective Tokens Specification](/gh-aw/specs/effective-tokens-specification/) — Legacy ET computation details
- [Network](/gh-aw/reference/network/) — Firewall and domain allow/deny configuration
- [MCP Gateway](/gh-aw/reference/mcp-gateway/) — MCP server health and debugging
- [CLI Commands](/gh-aw/setup/cli/) — Full CLI reference
Expand Down
2 changes: 1 addition & 1 deletion docs/src/content/docs/reference/footers.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ safe-outputs:
title-prefix: "[bot] "
```

The `messages.footer` template supports variables like `{workflow_name}`, `{agentic_workflow_url}`, `{run_url}`, `{triggering_number}`, `{effective_tokens_suffix}`, and more. `{agentic_workflow_url}` links directly to the agentic workflow file view for the run (equivalent to `{run_url}/agentic_workflow`), while `{run_url}` links to the plain Actions run page. `{effective_tokens_suffix}` is a pre-formatted, always-safe suffix (e.g. `" · sonnet46 1.2K"` or `""`) that you can place directly before `{history_link}`. When the run's engine model is known, the suffix is prefixed with a deterministic compact model identifier (`sonnet46`, `gpt55`, `opus47`, `haiku45`, `gem25`, …), and direct short aliases like `opus`, `sonnet`, and `haiku` are preserved. See [Custom Messages](/gh-aw/reference/safe-outputs/#custom-messages-messages) for complete documentation on message templates and available variables.
The `messages.footer` template supports variables like `{workflow_name}`, `{agentic_workflow_url}`, `{run_url}`, `{triggering_number}`, `{ai_credits_suffix}`, and more. `{agentic_workflow_url}` links directly to the agentic workflow file view for the run (equivalent to `{run_url}/agentic_workflow`), while `{run_url}` links to the plain Actions run page. `{ai_credits_suffix}` is a pre-formatted, always-safe suffix (e.g. `" · sonnet46 12.4 AIC"` or `""`) that you can place directly before `{history_link}`. When the run's engine model is known, the suffix is prefixed with a deterministic compact model identifier (`sonnet46`, `gpt55`, `opus47`, `haiku45`, `gem25`, …), and direct short aliases like `opus`, `sonnet`, and `haiku` are preserved. Legacy templates that still use `{effective_tokens_suffix}` continue to work for backward compatibility. See [Custom Messages](/gh-aw/reference/safe-outputs/#custom-messages-messages) for complete documentation on message templates and available variables.

## Related Documentation

Expand Down
31 changes: 21 additions & 10 deletions docs/src/content/docs/reference/frontmatter-full.md
Original file line number Diff line number Diff line change
Expand Up @@ -2226,7 +2226,7 @@ engine:
# (optional)
api-target: "example-value"

# Custom model token weights for effective token computation. Overrides or extends
# Custom model token weights for legacy Effective Tokens (ET) computation. Overrides or extends
# the built-in model multipliers from model_multipliers.json. Useful for custom
# models or adjusted cost ratios.
# (optional)
Expand Down Expand Up @@ -2529,19 +2529,30 @@ engine:
# the default engine when engine.id is not specified.
model: "example-value"

# Explicit ET budget control for firewall cost enforcement. Omit this field to
# leave ET budget protection unset. Set to a negative value to disable budget
# enforcement and token steering.
# Explicit AI Credits budget control for firewall cost enforcement. Omit this
# field to leave AIC budget protection unset. Set to -1 to disable budget
# enforcement.
Comment on lines +2532 to +2534
# (optional)
# Accepted formats:

# Format 1: Maximum effective-token (ET) budget for AWF API proxy enforcement. Use
# a negative value to disable budget enforcement and token steering.
max-effective-tokens: 1
# Format 1: Maximum AI Credits (AIC) budget for AWF API proxy enforcement.
max-ai-credits: 1

# Format 2: Maximum effective-token (ET) budget as a numeric string or GitHub
# Actions expression.
max-effective-tokens: "example-value"
# Format 2: Maximum AI Credits (AIC) budget as a numeric string with optional
# K/M suffix or GitHub Actions expression.
max-ai-credits: "example-value"

# Explicit daily AI Credits guardrail per workflow across the previous 24-hour
# window. Omit to disable, or set to -1 to explicitly disable.
Comment on lines +2545 to +2546
# (optional)
# Accepted formats:

# Format 1: Maximum daily AI Credits (AIC) budget.
max-daily-ai-credits: 1

# Format 2: Maximum daily AI Credits (AIC) budget as a numeric string with
# optional K/M suffix or GitHub Actions expression.
max-daily-ai-credits: "example-value"

# DEPRECATED: Legacy alias for AWF invocation cap (`apiProxy.maxRuns`).
# Use `max-turns` instead. Defaults to 500 when omitted.
Expand Down
12 changes: 6 additions & 6 deletions docs/src/content/docs/reference/model-tables.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: Model Aliases & Multipliers
description: Reference tables for the built-in model alias map and per-model Effective Token multipliers used by GitHub Agentic Workflows.
description: Reference tables for the built-in model alias map and legacy per-model Effective Token multipliers used by GitHub Agentic Workflows.
sidebar:
order: 297
---

This page lists the built-in model aliases and the per-model Effective Token (ET) multipliers used by GitHub Agentic Workflows.
This page lists the built-in model aliases and the legacy per-model Effective Token (ET) multipliers used by GitHub Agentic Workflows. AI Credits (AIC) is the primary cost metric.

> [!CAUTION]
> The multiplier values shown on this page are **approximations**. They are used solely for the purpose of normalizing token usage across models into a single comparable metric (Effective Tokens) and do **not** represent precise cost ratios. Values may be inaccurate for specific model versions and may become out of date as providers update their offerings. Do not use these numbers for billing or financial calculations.
> The multiplier values shown on this page are **approximations** for legacy ET normalization only. gh-aw uses AI Credits (AIC) as the primary billing metric. Do not use ET multipliers for billing or financial calculations.

## Model Aliases

Expand Down Expand Up @@ -66,7 +66,7 @@ Meta-aliases reference other aliases by name. They are resolved recursively unti
| `mini` | `haiku` → `gpt-5-mini` → `gpt-5-nano` → `gemini-flash-lite` |
| `large` | `sonnet` → `gpt-5-pro` → `gpt-5` → `gemini-pro` |
| `agent` | `sonnet-6x` → `gpt-5.4` → `gpt-5.3` → `gemini-pro` → `any` |
| `small-agent` | `haiku` → `gpt-5-mini` → `gemini-flash` → `any` |
| `small-agent` | `haiku` → `gpt-5-mini` → `gemini-flash` |
| `copilot` | `agent` |
| `claude` | `agent` |
| `codex` | `agent` |
Expand All @@ -75,9 +75,9 @@ Meta-aliases reference other aliases by name. They are resolved recursively unti

## Model Multipliers

Effective Token multipliers scale the weighted token total for each model relative to the reference model (`claude-sonnet-4.5`, multiplier = 1.0). A multiplier of 5.0 means that a run on that model counts as five times as many Effective Tokens as the same run on the reference model.
Legacy Effective Token multipliers scale the weighted token total for each model relative to the reference model (`claude-sonnet-4.5`, multiplier = 1.0). A multiplier of 5.0 means that a run on that model counts as five times as many Effective Tokens as the same run on the reference model.

See the [Effective Tokens Specification](/gh-aw/specs/effective-tokens-specification/) for the full formula.
See the [AI Credits Specification](/gh-aw/specs/ai-credits-specification/) for primary billing details and the [Effective Tokens Specification](/gh-aw/specs/effective-tokens-specification/) for legacy ET formulas.

### Token Class Weights

Expand Down
Loading