Context
PR #745 (issue #734) made the B=1 MTP burst tick-cooperative: one speculative round per scheduler tick, one slice in flight at a time, and any other speculative-eligible request falls back to classic decode while a slice is parked (scheduler.rs, single-slice admission guard).
The pr-security-checker review of #745 rated this MEDIUM (QoS, explicitly non-blocking): unlike the legacy run-to-completion burst, which held the worker for one tick-equivalent and then freed the speculative path for the next request, a slice holds the single speculative slot for its whole generation, which spans many ticks and is longer in wall-clock due to the strict round/classic alternation. A client keeping one long speculative stream open (or issuing back-to-back long speculative requests) denies speculative acceleration to every other client for as long as it chooses.
Impact is bounded: classic decode still progresses for everyone (the point of #734), total speculative capacity is not below legacy (also one request at a time), and the hold is bounded by the offender's max_tokens. It is a fairness degradation, not a hard DoS.
What's needed
Either or both:
- (a) Cap a single slice's slot hold with a rounds or wall-ms budget, after which the job yields the speculative slot back and continues classically or re-queues for a new slice grant, round-robining the slot across waiting speculative requests.
- (b) Priority-aware slice admission so a higher-priority speculative request preempts (at a round boundary) a lower-priority parked slice.
Acceptance Criteria
Relations
Context
PR #745 (issue #734) made the B=1 MTP burst tick-cooperative: one speculative round per scheduler tick, one slice in flight at a time, and any other speculative-eligible request falls back to classic decode while a slice is parked (scheduler.rs, single-slice admission guard).
The pr-security-checker review of #745 rated this MEDIUM (QoS, explicitly non-blocking): unlike the legacy run-to-completion burst, which held the worker for one tick-equivalent and then freed the speculative path for the next request, a slice holds the single speculative slot for its whole generation, which spans many ticks and is longer in wall-clock due to the strict round/classic alternation. A client keeping one long speculative stream open (or issuing back-to-back long speculative requests) denies speculative acceleration to every other client for as long as it chooses.
Impact is bounded: classic decode still progresses for everyone (the point of #734), total speculative capacity is not below legacy (also one request at a time), and the hold is bounded by the offender's max_tokens. It is a fairness degradation, not a hard DoS.
What's needed
Either or both:
Acceptance Criteria
Relations