Skip to content

perf(server): serving-throughput defaults: parallel decode, batched prefill, prompt cache#714

Merged
inureyes merged 3 commits into
mainfrom
perf/628-serving-throughput-defaults
Jul 9, 2026
Merged

perf(server): serving-throughput defaults: parallel decode, batched prefill, prompt cache#714
inureyes merged 3 commits into
mainfrom
perf/628-serving-throughput-defaults

docs: sync turbo-kv-cache budget default with the auto guard

c0f5486
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar