Skip to content

feat(#531): max_tokens two-step packing for codedb_context#610

Merged
justrach merged 1 commit into
release/0.2.5825from
feat/issue-531-context-budget
Jun 11, 2026
Merged

feat(#531): max_tokens two-step packing for codedb_context#610
justrach merged 1 commit into
release/0.2.5825from
feat/issue-531-context-budget

Conversation

@justrach

Copy link
Copy Markdown
Owner

Pick 5 of the #531 roadmap (token-budget-aware context packing) — step 2 of the two-step finish.

What

codedb_context gains max_tokens (~4 chars/token estimate, min 256). The handler now works in two steps:

  1. Render: every section — reader.md, head, symbol definitions (a rich variant with inlined bodies plus a lean def-lines-only fallback), callers, calls, file list, snippets — renders into its own arena buffer.
  2. Pack: sections are admitted by value order (head always → files → symbols rich-falling-back-to-lean → reader.md → callers → calls → snippets) until the byte budget runs out, then emitted in document order. Every omitted section leaves a one-line [max_tokens: omitted … (~N tokens)] marker so the agent knows what to raise the budget for or fetch directly.

So "give me the best ~2k tokens for this task" keeps the cheap high-value head (file ranking, symbol definition locations) and sheds the fat tail (snippet blocks, inlined bodies) — instead of truncating mid-section.

Compatibility

Without max_tokens every section is admitted and emitted in the original order — output is unchanged, and the pre-existing context tests (issue-570 fallback, etc.) pass untouched. The pure-render refactor also tightened the shared-lock scope to just the snippet-extraction block.

Tests

  • issue-531: codedb_context max_tokens packs sections by value under the budget — fat symbol body + five mention files; asserts: unbudgeted output unchanged (no markers) and larger; budgeted output ≤ budget×4 + marker slack; head and ## Most-relevant files survive while ## Top sites is dropped.

Suite: 814/814.

🤖 Generated with Claude Code

Pick 5 of the #531 roadmap: every truncation in the composer was by
lines/bytes, never tokens, and an oversized response could not be
bounded by the caller.

handleContext now renders in two steps. Step 1 writes each section —
reader.md, head, symbol definitions (a rich variant with inlined bodies
and a lean def-lines-only fallback), callers, calls, file list,
snippets — into its own arena buffer. Step 2 admits sections by VALUE
order (head always, then files, symbols rich-or-lean, reader.md,
callers, calls, snippets) under the byte budget (max_tokens x 4, the
~4-chars-per-token estimate) and emits the admitted ones in DOCUMENT
order; every omitted section leaves a one-line [max_tokens: omitted ...]
marker with its approximate cost. Without max_tokens every section is
admitted, so existing output is unchanged — the pre-existing context
tests pass untouched.

Suite: 814/814.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Benchmark Regression Report

Thresholds: 10.00% and 50,000 ns absolute delta

NOISE means the percentage threshold was exceeded, but the absolute delta was too small to fail CI.

Tool Base (ns) Head (ns) Delta Abs Delta (ns) Status
codedb_bundle 115940 114577 -1.18% -1363 OK
codedb_changes 11409 11126 -2.48% -283 OK
codedb_context 2008097 2008144 +0.00% +47 OK
codedb_deps 325 362 +11.38% +37 NOISE
codedb_edit 42238 38080 -9.84% -4158 OK
codedb_find 11231 11028 -1.81% -203 OK
codedb_hot 26658 31763 +19.15% +5105 NOISE
codedb_outline 38289 46889 +22.46% +8600 NOISE
codedb_read 17913 17709 -1.14% -204 OK
codedb_search 30135 29282 -2.83% -853 OK
codedb_snapshot 78621 76220 -3.05% -2401 OK
codedb_status 9805 9624 -1.85% -181 OK
codedb_symbol 50724 53942 +6.34% +3218 OK
codedb_tree 47230 46970 -0.55% -260 OK
codedb_word 13332 12606 -5.45% -726 OK

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant