[WIP] Refactor extract functions in token-parsers file by Copilot · Pull Request #5310 · github/gh-aw-firewall

Copilot · 2026-06-19T22:29:51Z

Run baseline tests/build for affected area
Refactor extractCacheReadTokens with helper extraction
Refactor extractUsageFromJson into a small orchestrator
Refactor extractUsageFromSseLine to reduce complexity
Add/update focused parser tests
Run targeted tests and finalize

Copilot

Pull request overview

Refactors token-usage extraction logic in the API proxy parser to reduce complexity and improve maintainability, while extending cache-read token parsing to handle deeper/nested token-entry structures.

Changes:

Refactored cache-read token extraction by introducing recursive helpers to sum token_type: "cache_read" across nested details arrays.
Refactored JSON/SSE usage parsing into small “builder” helpers plus a shared Copilot-breakdown merge step.
Added a focused test covering nested cache_read token entries in OpenAI Responses usage.*_tokens_details.details.

Show a summary per file

File	Description
containers/api-proxy/token-parsers.js	Extracted helper functions for cache-read summing and usage object construction/merging to simplify `extractUsageFromJson`/`extractUsageFromSseLine`.
containers/api-proxy/token-tracker.parsing.test.js	Added coverage for nested `cache_read` token entries to validate the new recursive extraction behavior.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 2/2 changed files
Comments generated: 0

github-actions · 2026-06-19T23:22:55Z

✅ Coverage Check Passed

Overall Coverage

Metric	Base	PR	Delta
Lines	97.62%	97.66%	📈 +0.04%
Statements	97.56%	97.60%	📈 +0.04%
Functions	98.85%	98.85%	➡️ +0.00%
Branches	93.24%	93.27%	📈 +0.03%

📁 Per-file Coverage Changes (1 files)

File	Lines (Before → After)	Statements (Before → After)
`src/workdir-setup.ts`	92.7% → 94.5% (+1.82%)	92.7% → 94.5% (+1.82%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

lpcox · 2026-06-19T23:50:31Z

@copilot address the review feedback and fix the pr title and summary

lpcox · 2026-06-20T02:02:30Z

@copilot address the review feedback and fix the pr title and summary

github-actions · 2026-06-20T02:25:49Z

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

github-actions · 2026-06-20T02:25:49Z

✅ Smoke Gemini completed. All facets verified. 💎

Testing safeoutputs

github-actions · 2026-06-20T02:25:49Z

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

github-actions · 2026-06-20T02:25:50Z

❌ Security Guard failed. Please review the logs for details.

github-actions · 2026-06-20T02:25:50Z

✅ Build Test Suite completed successfully!

github-actions · 2026-06-20T02:25:50Z

✅ Contribution Check completed successfully!

github-actions · 2026-06-20T02:25:51Z

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

github-actions · 2026-06-20T02:25:51Z

✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

github-actions · 2026-06-20T02:25:51Z

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

github-actions · 2026-06-20T02:25:52Z

🔑 Smoke Copilot PAT PAT auth validated. All systems operational. ✅

github-actions · 2026-06-20T02:25:52Z

❌ Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed...

github-actions · 2026-06-20T02:25:52Z

🔌 Smoke Services — All services reachable! ✅

github-actions · 2026-06-20T02:25:54Z

✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓

github-actions · 2026-06-20T02:26:03Z

❌ Smoke Claude failed

github-actions · 2026-06-20T02:28:44Z

Smoke Test: Copilot BYOK (Direct Mode) ✅

✅ GitHub MCP connectivity (tested)
✅ GitHub.com HTTP 200
✅ BYOK inference (api-proxy → api.githubcopilot.com)
✅ Running in direct BYOK mode

Status: PASS

cc/ @lpcox @Copilot

🔑 BYOK report filed by Smoke Copilot BYOK

github-actions · 2026-06-20T02:28:59Z

🔬 Smoke Test Results

PR: [WIP] Refactor extract functions in token-parsers file
Author: @Copilot | Assignees: @lpcox @Copilot

Test	Result
GitHub MCP connectivity	✅
GitHub.com HTTP connectivity	✅
File write/read	✅

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot

github-actions · 2026-06-20T02:29:16Z

🔬 Smoke Test: Copilot PAT Auth — Results

Test	Status
GitHub MCP connectivity	✅
GitHub.com HTTP connectivity	❌ template vars not expanded
File write/read	❌ template vars not expanded

Overall: FAIL — Pre-step outputs (SMOKE_HTTP_CODE, SMOKE_FILE_PATH, SMOKE_FILE_CONTENT) were not passed to the agent (workflow template substitution failed).

Auth mode: PAT (COPILOT_GITHUB_TOKEN)
PR: "[WIP] Refactor extract functions in token-parsers file" — @Copilot (author), @lpcox (assignee)

🔑 PAT report filed by Smoke Copilot PAT

github-actions · 2026-06-20T02:29:44Z

Gemini Engine Smoke Test Results

GitHub MCP Testing: ❌ (Tools missing)
GitHub.com Connectivity: ❌ (Status 000, Exit 35)
File Writing Testing: ✅
Bash Tool Testing: ✅

Overall Status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini

github-actions · 2026-06-20T02:30:09Z

[WIP] Refactor extract functions in token-parsers file
✅ PR reads
✅ Browser title
✅ File write
✅ Build
✅ Discussion lookup
Overall: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

github-actions · 2026-06-20T02:30:17Z

🔬 Smoke Test: API Proxy OpenTelemetry Tracing

Scenario	Result	Notes
1. Module Loading	✅ Pass	`otel.js` loads cleanly; exports: `startRequestSpan`, `setTokenAttributes`, `setBudgetAttributes`, `endSpan`, `endSpanError`, `shutdown`, `isEnabled`
2. Test Suite	✅ Pass	`otel.test.js`: 39/39 tests passed (1 suite)
3. Env Var Forwarding	✅ Pass	`OTEL_*` vars forwarded via `observability-environment.ts`; `GH_AW_OTLP_ENDPOINTS`, `OTEL_EXPORTER_OTLP_ENDPOINT`, `GITHUB_AW_OTEL_TRACE_ID/PARENT_SPAN_ID` forwarded via `api-proxy-service-config.ts`
4. Token Tracker Integration	✅ Pass	`onUsage` callback present in `token-tracker-http.js` (line 324); `otel.js` `setTokenAttributes` sets `gen_ai.usage.*` semantic conventions
5. OTEL Diagnostics	✅ Pass (graceful degradation)	No OTEL endpoint configured in this run; module degrades gracefully (file exporter fallback to `/var/log/api-proxy/otel.jsonl`)

All scenarios pass. Parent context propagation reads GITHUB_AW_OTEL_TRACE_ID / GITHUB_AW_OTEL_PARENT_SPAN_ID env vars and injects them as W3C trace context on each span.

📡 OTel tracing validated by Smoke OTel Tracing

github-actions · 2026-06-20T02:30:46Z

Chroot Version Comparison Results

Runtime	Host Version	Chroot Version	Match?
Python	`Python 3.12.13`	`Python 3.12.3`	❌
Node.js	`v24.16.0`	`v22.22.3`	❌
Go	`go1.22.12`	`go1.22.12`	✅

Overall: ❌ Not all tests passed — Python and Node.js versions differ between host and chroot environments.

Tested by Smoke Chroot

github-actions · 2026-06-20T02:31:16Z

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	1/1 passed	✅ PASS
Go	env	✅	1/1 passed	✅ PASS
Go	uuid	✅	1/1 passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	passed	✅ PASS
Node.js	execa	✅	passed	✅ PASS
Node.js	p-limit	✅	passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #5310 · 41.1 AIC · ⊞ 7.7K · ◷

github-actions · 2026-06-20T02:32:09Z

Smoke Test: GitHub Actions Services Connectivity

Check	Result
Redis `PING`	❌ timeout (no response on port 6379)
PostgreSQL `pg_isready`	❌ no response on port 5432
PostgreSQL `SELECT 1`	❌ no response (pg_isready failed)

Overall: FAIL

host.docker.internal resolves to 172.17.0.1 but TCP connections to both ports time out. No local listeners on 6379/5432 either — the GitHub Actions service containers do not appear to be running in this environment.

🔌 Service connectivity validated by Smoke Services

github-actions · 2026-06-20T02:33:39Z

@Copilot

GitHub MCP connectivity: ✅
GitHub.com HTTP response: ✅
Agent file I/O: ✅
Direct BYOK mode via api-proxy (Azure OpenAI, o4-mini-aw): ✅

PASS: Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry)

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

Initial plan

bcca911

Copilot AI assigned Copilot and lpcox Jun 19, 2026

Copilot started work on behalf of lpcox June 19, 2026 22:52 View session

refactor: decompose token parser helpers

0684eab

Copilot AI requested a review from lpcox June 19, 2026 23:03

lpcox marked this pull request as ready for review June 19, 2026 23:21

Copilot AI review requested due to automatic review settings June 19, 2026 23:21

Copilot started reviewing on behalf of lpcox June 19, 2026 23:21 View session

Copilot AI reviewed Jun 19, 2026

View reviewed changes

Copilot started work on behalf of lpcox June 19, 2026 23:58 View session

Copilot started work on behalf of lpcox June 20, 2026 02:10 View session

Copilot finished work on behalf of lpcox June 20, 2026 02:16

Copilot started work on behalf of lpcox June 20, 2026 02:16 View session

lpcox added the ready-for-aw label Jun 20, 2026

lpcox had a problem deploying to aoai-model June 20, 2026 02:25 — with GitHub Actions Failure

lpcox temporarily deployed to aoai-model June 20, 2026 02:28 — with GitHub Actions Inactive

github-actions Bot added the smoke-copilot-byok label Jun 20, 2026

github-actions Bot added the smoke-copilot label Jun 20, 2026

lpcox temporarily deployed to aoai-model June 20, 2026 02:29 — with GitHub Actions Inactive

github-actions Bot mentioned this pull request Jun 20, 2026

[aw] Contribution Check is missing required data #5289

Closed

github-actions Bot added the smoke-codex label Jun 20, 2026

github-actions Bot added the build-test label Jun 20, 2026

github-actions Bot added the smoke-copilot-byok-aoai-apikey label Jun 20, 2026

Copilot finished work on behalf of lpcox June 20, 2026 03:15

lpcox merged commit 29f8e2a into main Jun 20, 2026
87 of 93 checks passed

lpcox deleted the copilot/refactor-token-parsers branch June 20, 2026 16:03

lpcox mentioned this pull request Jun 20, 2026

perf(security-guard): prioritize security-relevant files in PR diff #5329

Merged

Conversation

Copilot AI commented Jun 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

github-actions Bot commented Jun 19, 2026

✅ Coverage Check Passed

Overall Coverage

Uh oh!

lpcox commented Jun 19, 2026

Uh oh!

lpcox commented Jun 20, 2026

Uh oh!

github-actions Bot commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 20, 2026

Smoke Test: Copilot BYOK (Direct Mode) ✅

Uh oh!

github-actions Bot commented Jun 20, 2026

🔬 Smoke Test Results

Uh oh!

github-actions Bot commented Jun 20, 2026

🔬 Smoke Test: Copilot PAT Auth — Results

Uh oh!

github-actions Bot commented Jun 20, 2026

Gemini Engine Smoke Test Results

Uh oh!

github-actions Bot commented Jun 20, 2026

Uh oh!

github-actions Bot commented Jun 20, 2026

🔬 Smoke Test: API Proxy OpenTelemetry Tracing

Uh oh!

github-actions Bot commented Jun 20, 2026

Chroot Version Comparison Results

Uh oh!

github-actions Bot commented Jun 20, 2026

Copilot AI commented Jun 19, 2026 •

edited

Loading

github-actions Bot commented Jun 20, 2026 •

edited

Loading

github-actions Bot commented Jun 20, 2026 •

edited

Loading

github-actions Bot commented Jun 20, 2026 •

edited

Loading

github-actions Bot commented Jun 20, 2026 •

edited

Loading

github-actions Bot commented Jun 20, 2026 •

edited

Loading

github-actions Bot commented Jun 20, 2026 •

edited

Loading

github-actions Bot commented Jun 20, 2026 •

edited

Loading

github-actions Bot commented Jun 20, 2026 •

edited

Loading

github-actions Bot commented Jun 20, 2026 •

edited

Loading

github-actions Bot commented Jun 20, 2026 •

edited

Loading

github-actions Bot commented Jun 20, 2026 •

edited

Loading

github-actions Bot commented Jun 20, 2026 •

edited

Loading

github-actions Bot commented Jun 20, 2026 •

edited

Loading

github-actions Bot commented Jun 20, 2026 •

edited

Loading