feat(cache): add in flight deduping

[MasterPtato]

Description

Please include a summary of the changes and the related issue. Please also include relevant motivation and context.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes.

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[railway-app]

🚅 Deployed to the rivet-pr-4459 environment in rivet-frontend

Service	Status	Web	Updated (UTC)
frontend-cloud	😴 Sleeping (View Logs)	Web	Mar 24, 2026 at 7:53 am
frontend-inspector	😴 Sleeping (View Logs)	Web	Mar 24, 2026 at 1:03 am
website	❌ Build Failed (View Logs)	Web	Mar 19, 2026 at 8:53 pm
mcp-hub	✅ Success (View Logs)	Web	Mar 19, 2026 at 8:52 pm
ladle	❌ Build Failed (View Logs)	Web	Mar 19, 2026 at 8:51 pm

[MasterPtato]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat(pb): rewrite serverless handlers #4463
• feat(ups): implement queue subs #4486
• feat(cache): add in flight deduping #4459 👈 (View in Graphite)
• fix(cache): clean up lib #4457
• fix(config): allow configuring topo dcs via map, fix pg ssl mode config #4456
• fix(api): dont 500 on cross dc response #4449
• chore: update byte units on frontend #4444
• fix: runner alloc idx logic, api auth for actor get #4443
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[pkg-pr-new]

More templates

• actor-actions
• actor-actions-vercel
• ai-agent
• ai-agent-vercel
• example-ai-and-user-generated-actors-freestyle
• chat-room
• chat-room-vercel
• example-cloudflare-workers
• example-cloudflare-workers-hono
• collaborative-document
• collaborative-document-vercel
• cross-actor-actions
• cross-actor-actions-vercel
• example-cursors
• example-cursors-raw-websocket
• example-cursors-raw-websocket-vercel
• example-cursors-vercel
• custom-serverless
• example-elysia
• example-experimental-durable-streams-ai-agent
• example-experimental-durable-streams-ai-agent-vercel
• geo-distributed-database
• geo-distributed-database-vercel
• hello-world
• hello-world-vercel
• example-hono
• hono-react
• hono-react-vercel
• example-kitchen-sink
• example-kitchen-sink-vercel
• multi-region-vercel
• multiplayer-game
• example-multiplayer-game-patterns
• example-multiplayer-game-patterns-vercel
• multiplayer-game-vercel
• native-websockets
• native-websockets-vercel
• next-js
• per-tenant-database
• per-tenant-database-vercel
• example-raw-fetch-handler
• example-raw-fetch-handler-vercel
• example-raw-websocket-handler
• example-raw-websocket-handler-proxy
• example-raw-websocket-handler-proxy-vercel
• example-raw-websocket-handler-vercel
• example-react
• example-react-vercel
• sandbox
• sandbox-coding-agent
• sandbox-coding-agent-vercel
• sandbox-vercel
• scheduling
• scheduling-vercel
• example-sqlite-drizzle
• example-sqlite-raw
• state
• state-vercel
• example-stream
• example-stream-vercel
• example-trpc

@rivetkit/cloudflare-workers

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/cloudflare-workers@4459

@rivetkit/framework-base

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/framework-base@4459

@rivetkit/next-js

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/next-js@4459

@rivetkit/react

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/react@4459

rivetkit

pnpm add https://pkg.pr.new/rivet-dev/rivet/rivetkit@4459

@rivetkit/sql-loader

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/sql-loader@4459

@rivetkit/sqlite-vfs

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/sqlite-vfs@4459

@rivetkit/traces

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/traces@4459

@rivetkit/workflow-engine

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/workflow-engine@4459

@rivetkit/virtual-websocket

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/virtual-websocket@4459

@rivetkit/engine-runner

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/engine-runner@4459

@rivetkit/engine-runner-protocol

pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/engine-runner-protocol@4459

commit: 6570bf2

[claude]

Code Review: feat(cache): add in flight deduping. Good feature addition. Two correctness issues: (1) Critical - In-flight entries are leaked when the leased getter returns an error. tokio::try_join! propagates via ? before the broadcast send and remove_async cleanup, permanently poisoning the in_flight map for those keys. Fix: restructure so cleanup always runs before the ? propagation. (2) Bug - Broadcast is only sent inside if !entries_values.is_empty(). If getter returns no values, broadcast is never sent and waiters block for the full 5s IN_FLIGHT_TIMEOUT. Fix: send broadcast unconditionally before lease cleanup. Minor: timeout_falls_back_to_getter waits 5s per CI run - make IN_FLIGHT_TIMEOUT configurable for tests. Minor: HashMap deduplication in GetterCtx::new is implicit - add a comment. Nit: broadcast::channel capacity 1 is sufficient since one message per lease. Positive: GetterCtx HashMap refactor is clean (O(1) lookups), CacheKey: Eq+Hash bound is right, &[RawCacheKey] API is better, test split into fetch.rs/in_flight.rs/ttl.rs is much cleaner than the old monolithic integration.rs, and the dedup tests give solid happy-path coverage.

[claude]

line1\nline2\nline3