Skip to content

tonic: avoid excess decode buffer reservation#2735

Open
mingley wants to merge 2 commits into
grpc:masterfrom
mingley:mingley/optimize-decode-buffer-reserve
Open

tonic: avoid excess decode buffer reservation#2735
mingley wants to merge 2 commits into
grpc:masterfrom
mingley:mingley/optimize-decode-buffer-reserve

Conversation

@mingley

@mingley mingley commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Avoid redundant decode-buffer growth when the existing capacity can already
    hold the announced gRPC message.
  • Add exact-content coverage across fragmented, coalesced, and custom-buffer
    decode paths.
  • Add representative unary and streaming decode benchmarks.

Motivation

BytesMut::reserve treats its argument as additional capacity. Calling
self.buf.reserve(len) after consuming the gRPC header can therefore target
self.buf.len() + len bytes even when some or all of the message is already
buffered.

For complete or coalesced messages, this can trigger an unnecessary allocation
and copy of the buffered payload. It can also retain substantially more memory
than the message requires.

Solution

Check whether the current BytesMut capacity is smaller than the announced
message length. Only in that case, reserve the difference between the message
length and the bytes already buffered.

The subtraction is safe because BytesMut::len() cannot exceed
BytesMut::capacity(), and it only executes when capacity is below the message
length. The decode state machine, size limits, compression handling, and wire
behavior are unchanged.

Benchmark Results

Measured on an Apple M4 Pro running macOS 26.5.1 with rustc 1.97.0. Baseline
and optimized benchmark binaries were compiled from the same source path.
Each result is the median of 10 interleaved A/B process runs using the release
benchmark profile. Negative changes are faster.

Workload Baseline Optimized Change
1 KiB unary, single chunk 183.5 ns 179.5 ns -2.2%
10 KiB unary, single chunk 726.0 ns 477.5 ns -34.2%
64 KiB unary, 16 KiB chunks 3,492.5 ns 3,503.0 ns +0.3%
1 MiB unary, 16 KiB chunks 32,130.5 ns 32,211.5 ns +0.3%
100 x 1 KiB streaming, 16 KiB chunks 8,404.0 ns 6,654.5 ns -20.8%
100 x 1 KiB streaming, 100-byte chunks 23,041.5 ns 22,373.0 ns -2.9%
1 MiB unary, single coalesced chunk 39,914.5 ns 26,315.5 ns -34.1%

The large-message 16 KiB chunk cases are unchanged within measurement noise.
The improvement is concentrated where a body chunk already contains a complete
message or multiple messages, which is where the previous reservation could
force an avoidable reallocation.

Testing Done

  • Local code review completed
  • Added exact-content unit coverage for fragmented, partial, coalesced, and
    custom-buffer decoding
  • cargo fmt --all -- --check
  • cargo clippy --workspace --all-features --all-targets
  • cargo test --workspace --all-features --no-fail-fast
  • Checked tonic with no default features and representative gzip, deflate,
    zstd, server, and channel feature configurations
  • Ran the interleaved decode benchmark matrix described above

mingley added 2 commits July 14, 2026 15:52
Reserve only when the current decode buffer capacity cannot hold the announced message. This avoids reallocating and copying data that is already fully buffered while retaining the existing fast path for fragmented messages.\n\nAdd representative unary and streaming decode benchmarks covering default HTTP/2 frame sizing, fragmentation, and coalesced data.
Exercise fragmented, coalesced, and custom-buffer decoding with distinct message payloads. Rename the representative benchmarks around body chunk sizes so they remain transport agnostic.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant