Skip to content

feat: generation rotation for /full/current ref#781

Merged
pfleidi merged 22 commits intomainfrom
feat/checkpoints-v2-transcript-rotation
Mar 26, 2026
Merged

feat: generation rotation for /full/current ref#781
pfleidi merged 22 commits intomainfrom
feat/checkpoints-v2-transcript-rotation

Conversation

@pfleidi
Copy link
Copy Markdown
Contributor

@pfleidi pfleidi commented Mar 26, 2026

Summary

Implements generational rotation for the v2 /full/current ref (task A6 from the checkpoints v2 design spec). When /full/current accumulates >= 100 checkpoints, it is archived as a numbered ref and a fresh orphan is created.

  • generation.json at tree root tracks checkpoint IDs and timestamps, updated on every WriteCommitted (skipped for UpdateCommitted since that's stop-time finalization, not a new checkpoint)
  • 2-phase rotation: archive current ref as refs/entire/checkpoints/v2/full/0000000000001, create fresh orphan /full/current with seed generation.json
  • Non-fatal rotation — if rotation fails, the checkpoint write already succeeded; next writer retries
  • Concurrent rotation guard — re-reads /full/current before archiving to skip if another instance already rotated
  • Dropped deprecated TranscriptLinesAtStart from v2 metadata writes (v2 is a clean break)

New files

  • checkpoint/v2_generation.goGenerationMetadata type, read/write helpers, rotation, archived generation listing
  • checkpoint/v2_generation_test.go — 17 unit tests

Modified files

  • paths/paths.goV2FullRefPrefix, GenerationFileName constants
  • checkpoint/v2_store.goMaxCheckpointsPerGeneration field for testability
  • checkpoint/v2_committed.go — generation tracking + rotation check wired into write path
  • checkpoint/v2_store_test.go — 2 integration tests for rotation triggering

Test plan

  • Unit tests cover: read/write round-trips, accumulation, dedup, tree preservation, archive listing, sequential numbering, rotation mechanics, concurrent rotation guard, threshold triggering, below-threshold no-op
  • mise run test:ci passes (unit + integration + E2E canary)
  • mise run lint clean

🤖 Generated with Claude Code


Note

Medium Risk
Adds automatic archival/rotation of the v2 refs/entire/checkpoints/v2/full/current ref based on checkpoint count, which changes git ref/tree update behavior and could impact transcript retention if bugs exist. Risk is mitigated by extensive new unit/integration tests and making rotation non-fatal to the write path.

Overview
Adds generation tracking + rotation for v2 raw transcript storage under /full/current: each WriteCommitted now updates a root generation.json (checkpoint IDs + oldest/newest timestamps) and, once the checkpoint count hits a threshold (default 100), archives the current ref as a numbered refs/entire/checkpoints/v2/full/<NNN...> and resets /full/current to a fresh orphan commit.

Introduces new generation/rotation helpers on V2GitStore (including archived generation discovery and sequential numbering), a test-only MaxCheckpointsPerGeneration override, and new path constants (V2FullRefPrefix, GenerationFileName). Also drops writing the deprecated TranscriptLinesAtStart field in v2 committed metadata and adds comprehensive tests covering metadata updates, deduping, rotation mechanics, and threshold behavior.

Written by Cursor Bugbot for commit 3dd257d. Configure here.

pfleidi added 13 commits March 25, 2026 11:01
Add V2FullRefPrefix and GenerationFileName constants to the paths
package, and introduce GenerationMetadata struct for tracking /full/*
generation state (checkpoint count, checkpoint IDs, timestamps).

Entire-Checkpoint: c23e9d9220b4
Implement readGeneration, readGenerationFromRef, writeGeneration, and
addGenerationToRootTree on V2GitStore. These helpers read/write
GenerationMetadata (generation.json) from git trees, supporting the
generation rotation lifecycle.

Entire-Checkpoint: b90857dc67c4
…rent write

Add updateGenerationForWrite helper that maintains checkpoint IDs and
timestamps in generation.json. Wire it into writeCommittedFullTranscript
so every WriteCommitted updates the generation metadata at the tree root.
UpdateCommitted (stop-time finalization) does not modify generation.json
since it replaces an existing transcript rather than adding a new
checkpoint.

Changes:
- cmd/entire/cli/checkpoint/v2_generation.go — added updateGenerationForWrite
- cmd/entire/cli/checkpoint/v2_committed.go — wired generation tracking into writeCommittedFullTranscript
- cmd/entire/cli/checkpoint/v2_generation_test.go — 5 new tests

Entire-Checkpoint: 0be8f3fb287a
Scan refs under refs/entire/checkpoints/v2/full/ (excluding current) to
enumerate archived generations. nextGenerationNumber finds the highest
existing archive number and returns max+1 for the next rotation.

Changes:
- cmd/entire/cli/checkpoint/v2_generation.go — added listArchivedGenerations, nextGenerationNumber
- cmd/entire/cli/checkpoint/v2_generation_test.go — 5 new tests + createArchivedRef helper

Entire-Checkpoint: c68e85deaf3a
…fresh orphan

Three-phase rotation: (1) finalize generation.json on /full/current with
the archive sequence number, (2) create archived ref pointing to the
finalized commit, (3) create fresh orphan /full/current with seed
generation.json (generation: 0, empty checkpoints).

Changes:
- cmd/entire/cli/checkpoint/v2_generation.go — added rotateGeneration
- cmd/entire/cli/checkpoint/v2_generation_test.go — 2 new tests + populateFullCurrent helper

Entire-Checkpoint: f32242003084
After each successful write to /full/current, check if the checkpoint
count has reached the threshold. If so, trigger rotation (non-fatal on
failure). Add MaxCheckpointsPerGeneration field to V2GitStore for
testability (defaults to 100).

Changes:
- cmd/entire/cli/checkpoint/v2_store.go — added MaxCheckpointsPerGeneration field, maxCheckpoints() helper
- cmd/entire/cli/checkpoint/v2_committed.go — added rotation check after updateRef
- cmd/entire/cli/checkpoint/v2_store_test.go — 2 new tests (triggers at threshold, no rotation below)

Entire-Checkpoint: 02f58fcff274
strings.TrimLeft on an all-zero string returns "", causing Atoi to fail.
strconv.ParseInt handles zero-padded decimal strings natively.

Entire-Checkpoint: dbb1cf5bbaef
Eliminates stringly-typed checkpoint IDs in generation.json. The typed
ID is already used everywhere else and marshals identically to JSON.

Entire-Checkpoint: 83c2919167b8
v2 is a clean break — no need to carry the backward-compat field that
v1 still needs for older CLI versions.

Entire-Checkpoint: cca08210a63b
Re-read /full/current before archiving and skip if checkpoint count is
already below threshold. Prevents archiving an empty generation if
another CLI instance already rotated.

Entire-Checkpoint: f2f088150117
@pfleidi pfleidi requested a review from a team as a code owner March 26, 2026 00:20
Copilot AI review requested due to automatic review settings March 26, 2026 00:20
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds generational rotation to the v2 /full/current checkpoint ref so large transcript histories are periodically archived into numbered refs, keeping /full/current bounded while preserving access to older generations.

Changes:

  • Introduces generation.json metadata at the root of each /full/* generation and updates it on each WriteCommitted.
  • Implements rotation: archive /full/current to a numbered ref and reset /full/current to a fresh orphan commit when the checkpoint threshold is reached.
  • Adds unit/integration tests validating generation metadata behavior, archived generation discovery, numbering, and rotation triggering.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
cmd/entire/cli/paths/paths.go Adds constants for /full/* ref prefix and generation.json filename.
cmd/entire/cli/checkpoint/v2_store.go Adds a test override for the max checkpoints-per-generation threshold.
cmd/entire/cli/checkpoint/v2_committed.go Wires generation metadata updates and post-write rotation checks into /full/current writes.
cmd/entire/cli/checkpoint/v2_generation.go Implements generation metadata read/write, archived generation listing, numbering, and rotation logic.
cmd/entire/cli/checkpoint/v2_generation_test.go Adds unit tests covering generation read/write, accumulation/dedup, listing, and rotation behavior.
cmd/entire/cli/checkpoint/v2_store_test.go Adds tests asserting rotation triggers at threshold and not below it.

pfleidi added 6 commits March 25, 2026 18:08
Previously all errors from tree.File() were treated as "missing file".
Now only ErrFileNotFound/ErrEntryNotFound are ignored; real failures
(tree corruption, entry conflicts) are surfaced.

Entire-Checkpoint: be7d8ec41404
Re-read /full/current after archiving and before resetting to orphan.
If another writer advanced the ref, abort the reset to prevent making
their writes unreachable. The archive ref is harmless and the next
writer will trigger rotation again.

Entire-Checkpoint: e118558f00d3
Check if the target archived ref exists before creating it. If another
instance already rotated and created the same archive number, skip
instead of overwriting.

Entire-Checkpoint: 4faaf3537e42
Move timestamp updates inside the dedup guard so they reflect checkpoint
creation times, not last-write times for duplicate multi-session writes.

Entire-Checkpoint: 1bc2f6399ff9
Only include refs whose suffix matches the expected 13-digit format.
Prevents unexpected refs under the prefix from breaking
nextGenerationNumber parsing.

Entire-Checkpoint: 9922d253ea7f
Iterate all archived generation names and take the max, skipping any
that fail to parse instead of failing the entire rotation.

Entire-Checkpoint: 58a487f7498f
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Testing override only — tests are in the same package so they can set
the unexported field directly. Avoids suggesting it's a supported
runtime tuning knob.

Entire-Checkpoint: d9436b8f2630
Soph
Soph previously approved these changes Mar 26, 2026
pfleidi added 2 commits March 26, 2026 11:03
Address review comments: clarify that a generation is a batch of
transcripts under a single ref, and that timestamps exist for the
future cleanup tool (RFD-009).

Entire-Checkpoint: 948bcc63eb93
Add offset parameter to populateFullCurrent so sequential rotation
tests generate unique IDs per cycle, avoiding accidental collisions.

Entire-Checkpoint: 3288358d689d
@pfleidi pfleidi merged commit 26e5a5a into main Mar 26, 2026
3 checks passed
@pfleidi pfleidi deleted the feat/checkpoints-v2-transcript-rotation branch March 26, 2026 18:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants