diff --git a/.agents/benchmark-record.md b/.agents/benchmark-record.md index 0c1354a76..109228b12 100644 --- a/.agents/benchmark-record.md +++ b/.agents/benchmark-record.md @@ -11639,3 +11639,79 @@ hazard-A handling (double-buffer `exec_state_` so freeing the prior StepInputs doesn't race the in-flight forward) and hazard-C (block-table device buffer). Not a "small scoped drain" — it is the device-resident-sampled-tokens architecture. Evidence: `dgx:~/work/q35-regrid/evidence/raw/35/ours/c16-r{1,2,3}-{abdrain,abevent}.json`. + +## Qwen3.5-4B revalidation across the 217-commit rebase and GCC 15 repair (2026-08-03) + +`CLAIM-CPU-GCC15-WERROR-LOCAL`. Measured, not assumed. `main` advanced from +`f3ecbe70d` to `265e3bf98` (217 upstream commits) and the GCC 15 `-Werror` +repair landed on top, so the 4B lever was rerun against the pinned oracle on +the identical workload. + +| Axis | ours | vLLM at pin | ratio | 2026-07-29 | Disposition | +|---|---:|---:|---:|---:|---| +| Total throughput (tok/s) | 6611.397 | 6624.585 | 0.9980x | 0.9972x | FAIL | +| Output throughput (tok/s) | 731.070 | 732.528 | 0.9980x | 0.9972x | FAIL | +| Requests/s | 5.710 | 5.723 | 0.9978x | 0.9971x | FAIL | +| Mean TTFT (ms) | 728.243 | 948.961 | 0.7674x | 0.7701x | PASS | +| Mean TPOT (ms) | 38.160 | 33.942 | 1.1243x | 1.1247x | FAIL | +| Peak PSS (GiB) | 2.147 | 7.621 | 0.2816x | | PASS | +| Peak VRAM (MiB) | 12850.7 | 12832.0 | 1.0015x | | FAIL | + +Every axis reproduced the 2026-07-29 series inside run-to-run noise. Our +generated tokens were bit-identical, 128/128 within each repetition and +128/128 against the earlier series for every arm. Agreement with vLLM remained +89/128, so the residual stayed the known near-tie sampling difference. The +0.0008x ratio movement came from the denominator: ours moved +0.017% while the +oracle ran 0.06% slower than its previous series. This was a null result, not +an improvement. + +The open gaps remained total/output throughput, requests/s, TPOT, and peak +VRAM. Peak host memory remained the decisive win. Oracle: +`.venv-vllm-pin`, vLLM `0.23.1rc1.dev1511+g555967922`, built from source at the +parity pin rather than the faster 0.24.0 release. A clean CUDA build passed +899/899. Three interleaved repetitions per arm ran under one `flock /tmp/gpu`; +all 18 legs observed 0% GPU utilization before starting. No 4B result implies +support or speed for the 27B/35B gates. + +Evidence and full reproduction: +[Qwen3.5-4B GCC 15 fix revalidation](../docs/bench-evidence/qwen35-4b-gcc15fix-20260803.md). + +## PR #28 sanitizer CI disk and leak repair (2026-08-04) + +`HARDEN-DETECTOR-LANES`, infrastructure, performance NOT APPLICABLE. GitHub run +`30819266647`, job `91704728276`, did not report a sanitizer finding. The +hosted runner reached 99 MiB free and `ld` failed with `No space left on +device`, because every sanitizer test force-linked another complete +instrumented static engine. + +Sanitizer tests now share one internal instrumented image and emit `-g1`. The +ASan+UBSan build footprint fell from 93 GiB to 5.6 GiB; the TSan tree occupies +1.9 GiB. CI uses `VT_POOL_BYPASS=1`, so intentionally cached scratch blocks are +really freed during detector runs. The complete leak survey then found and +fixed a genuine minja ownership cycle: a `MacroNode` callable strongly captured +the context that owned the callable. Two Nix-only test environment assumptions +were also removed. + +Binding GCC 15.2 local evidence: ASan+UBSan **331/331 PASS** with +`detect_leaks=1`, `strict_string_checks=1`, and `VT_POOL_BYPASS=1`; TSan +**331/331 PASS**; the affected plain `-Werror` tests **3/3 PASS**. No model, +kernel, runtime default, GPU path, or timing path changed, so no performance +number is applicable. Exact commands and binary sizes are retained in +`.agents/state.md` under "PR #28 sanitizer CI disk and leak repair". + +## PR #28 post-upstream-merge re-gate (2026-08-05) + +Merged `upstream/main` at `35542ad3a` into the PR branch and repeated the full +GCC 15.2 detector lanes. ASan+UBSan with `detect_leaks=1`, +`strict_string_checks=1`, and `VT_POOL_BYPASS=1` passed **333/333**; TSan with +`VT_POOL_BYPASS=1` passed **333/333**. The merged ASan+UBSan tree is 5.7 GiB +and the TSan tree remains 1.9 GiB. + +The first merged ASan survey exposed a stale exact error-message expectation: +the architecture list in `test_model_loader_gguf` omitted upstream's newly +registered `KimiLinearForCausalLM`. Updating that expectation restored the +full lane, and the focused plain GCC tests (`test_chat_template`, +`test_none_hash_determinism`, `test_serve_low_tools`, and +`test_model_loader_gguf`) passed **4/4**. This integration repair changes no +model, kernel, runtime default, GPU path, or timing path, so performance remains +NOT APPLICABLE. diff --git a/.agents/parity-ledger.md b/.agents/parity-ledger.md index 1156d1926..606340574 100644 --- a/.agents/parity-ledger.md +++ b/.agents/parity-ledger.md @@ -886,3 +886,4 @@ Columns: | 2026-07-31 (`SERVE-C-ABI` W0 contract spike; `CLAIM-SERVE-C-ABI-SPIKE`; CPU-only records/docs) | Accepted `.agents/specs/c-api-library.md` for the already-shipped original C packaging layer: complete scope, vLLM semantic chain/deviation, ABI v10/19-symbol baseline, ownership/error/version/dispatch rules, exact code/test anchors, gates, dependencies, risks, and W1-W5 follow-ons. Also fixes the verified stale public `VLLM_ABI_VERSION 9` labels in README/USAGE to the source-of-truth v10 and adds the missing v10 usage-table entry. No production/test/CMake source changed. | Pinned vLLM `555967922` has no C ABI; behavior beneath the adapter remains owned by its vLLM-derived engine rows. The flat ABI is the recorded llama.cpp-style packaging deviation and may translate, never reimplement, policy. | **CPU/records gate only; benchmark NOT APPLICABLE.** Focused C11/C++/dlopen/export gate passed 3/3 after explicitly building `vllm_shared`; five record checkers pass. `check-agent-record` reports the base tree's same six missing closing-commit objects (`444ea9d7`, `7a3f04b2`, `164453a2`), none in this row/diff. Row stays `ANCHOR-BACKFILL` because all-symbol dlsym coverage (chat symbols currently omitted), historical-layout compatibility, allocation-failure no-throw proof, lifetime sanitizer stress, and a standalone real-model C consumer remain W1-W5. | | 2026-07-31 (`CLAIM-CPU-GCC12-WERROR-PORTABILITY`; maintenance, rows `QUANT-GGUF-KEEPQ-LOADER` + `KV-OFFLOAD`; lifecycle unchanged) | Removes two GCC 12 production-library `-Werror` blockers without suppressions: the GGUF prefault keeps the same one-byte-per-page volatile XOR but uses simple assignment, and the KV filesystem tier builds the identical `...tmp` suffix with append operations inside its thread-local initializer. No API, algorithm, default, CUDA, fixture, or golden change. | Behavior remains grounded in the accepted loader and KV-persistence leaf specs: llama.cpp mmap prefault intent and vLLM `tiering/fs/io.py` unique temporary-file publication. This is compiler portability, not a parity-surface change. | RED: GCC 12 failed first at `qwen3_5_gguf_weights.cpp:49` (`-Wvolatile`), then at `fs_io.cpp:66` (`-Wrestrict`). GREEN: production `vllm` and focused test targets build clean; focused CTest 2/2 (`test_gguf_keep_quant`, `test_kv_offload_fs`). Full all-target build is PARTIAL at 42% on unrelated test-only GCC 12 `-Wrestrict` diagnostics in `test_deepseek_v2_paged_engine.cpp` and `test_glm4_moe_lite_paged_engine.cpp`; no full-CTest claim. Benchmark NOT APPLICABLE. | | 2026-08-01 (`SERVE-CLI-CHAT` W0 contract spike; `CLAIM-SERVE-CLI-CHAT-SPIKE`; CPU-only records/spec) | Accepts `.agents/specs/cli-chat-complete.md`, corrects the inventory from “no direct commands” to the actual pinned `chat`/`complete` surface, and decomposes a dual-mode port: exact remote OpenAI HTTP/SSE commands plus preservation of the existing in-process invocation as a compatibility alias. No production, test, CMake, model, kernel, fixture, or generated file changes. | Pinned vLLM `5559679229`: command registration `vllm/entrypoints/cli/main.py:17-37,73-98`; model/auth resolution and stream shaping `vllm/entrypoints/cli/openai.py:30-100`; chat `:155-234`; complete `:237-312`. The local compatibility baseline is `examples/cli/main.cpp:1-207`. | CPU record/doc gates only; benchmark `NOT APPLICABLE`, `benchmark_binding=false`. Implementation remains absent and the row moves `INVENTORIED` -> `SPIKE`. W1-W5 name parse, transport, complete, chat, and packaging gates, including fake-server request/SSE transcript parity, Release `-Werror`, ASan+UBSan, and TSan. | +| 2026-08-04 (`HARDEN-DETECTOR-LANES` PR #28 CI repair; `CLAIM-HARDEN-SANITIZER-DISK`; Ordino task `t-e19dc73f`; CPU-only, lifecycle unchanged; closing commit: this checkpoint) | Repairs the hosted ASan+UBSan build's filesystem exhaustion without weakening detector coverage. Sanitizer tests share one internal fully instrumented engine image instead of force-linking another static copy into every executable, and `-g1` retains file/line traces without full type/local-variable DWARF. CI enables the existing `VT_POOL_BYPASS=1` exact-allocation/real-free detector mode. The newly reachable leak survey removes a real minja `MacroNode` ownership cycle by weakly capturing the context that owns the callable. It also closes the two Nix-only suite portability gaps: resolve `true` and the active Python executable instead of assuming `/usr/bin`, and remove inherited `PYTHONHASHSEED` only from the unconfigured control suite. Default build linkage and packaged C ABI exports remain unchanged. | No vLLM behavioral counterpart: this is local build/test infrastructure plus a vendored minja lifetime repair. Anchors: `CMakeLists.txt`, `tests/CMakeLists.txt`, `.github/workflows/ci.yml`, `third_party/minja/minja.hpp`, `tests/tools/test_gdn_packed_component.py`, and `tests/tools/test_online_gate_client.py`. Remote root-cause evidence: GitHub run `30819266647`, job `91704728276`, 99 MiB free then `ld: No space left on device`. | **PASS.** GCC 15.2.0 ASan+UBSan full suite **331/331** under leak detection and pool bypass; TSan full suite **331/331** under pool bypass; affected plain GCC 15 `-Werror` suites **3/3**. ASan+UBSan tree **93 GiB -> 5.6 GiB** (about 94% smaller); TSan tree **1.9 GiB**. `benchmark_binding=false`, performance **NOT APPLICABLE**. Hosted PR confirmation is the next external gate; `continue-on-error` stays until that confirmation. | diff --git a/.agents/specs/hardening-adoption-2026-07-27.md b/.agents/specs/hardening-adoption-2026-07-27.md index e049fb0a5..fe598aaa2 100644 --- a/.agents/specs/hardening-adoption-2026-07-27.md +++ b/.agents/specs/hardening-adoption-2026-07-27.md @@ -56,9 +56,10 @@ separate jobs. Design points, all from the guide: project code, and the plain build — the one that DOES enforce `-Werror` — stays clean. Letting a false positive in a system header stop the runtime detectors would defeat the point. -- Landed `continue-on-error: true` because the lanes have never run: their FIRST - run is a survey, and a pre-existing finding must not block unrelated work. The - closing step of this row is triaging that first run and REMOVING the flag. +- `continue-on-error: true` remains until the repaired PR validates the two + lanes on the hosted runner. The first complete local survey is now triaged: + both full suites pass, and removing the flag after remote confirmation is the + closing step of this row. **The lane paid for itself before it was even committed.** Its first real build failed on `-Werror=unused-function`: `AsyncDeviceMirrorEnvDefault()`, added by the @@ -71,11 +72,16 @@ compiles the other `cfg` branches. **Verified end to end**, not merely configured: `address,undefined` and `thread` both configure; the two guards fire (a lane value outside the allowlist, and a -lane with the CUDA backend on, are both `FATAL_ERROR`); the ASan+UBSan lane -builds `test_input_batch`, `test_combine_tokens` and `test_arena` clean and all -three PASS under `ASAN_OPTIONS=detect_leaks=1 UBSAN_OPTIONS=print_stacktrace=1` -(25/25 + 183 asserts, 7/7 + 14, 4/4 + 18), which includes the new W4 op-log -cases. The plain CPU build was re-verified clean and green in the same pass. +lane with the CUDA backend on, are both `FATAL_ERROR`). On GCC 15.2.0 the full +ASan+UBSan suite passes 331/331 under +`ASAN_OPTIONS=detect_leaks=1:strict_string_checks=1`, +`UBSAN_OPTIONS=print_stacktrace=1`, and `VT_POOL_BYPASS=1`; the full TSan suite +also passes 331/331 with the bypass. The detector-specific shared test image +reduces the ASan+UBSan build from 93 GiB to 5.6 GiB, while keeping file/line +traces with `-g1`; TSan is 1.9 GiB. The survey also removed the minja macro +context ownership cycle that leak detection exposed. The plain GCC 15 `-Werror` +build and the affected `test_serve_low_tools`, `test_none_hash_determinism`, and +`test_chat_template` cases pass 3/3. ### 2. Allocator-blind-spot bypass (§4) @@ -121,8 +127,8 @@ reproduce here, which is worth recording so a later reader does not re-audit: ## Follow-up rows -1. Triage the first `sanitize-cpu` run and remove `continue-on-error` (closes - this row). +1. Confirm both repaired `sanitize-cpu` jobs on the hosted PR runner, then + remove `continue-on-error` (closes this row). 2. Validate the three residual `atoi` sites. 3. A concurrency matrix per §7 (independent concurrent engines, callback re-entry, forced worker failure). diff --git a/.agents/state.md b/.agents/state.md index 9aa1eea77..59a7a792b 100644 --- a/.agents/state.md +++ b/.agents/state.md @@ -34537,6 +34537,50 @@ This is build portability only and changes no runtime, model lifecycle, correctness result, or benchmark disposition. Darwin consumer CI remains the binding AppleClang verification. +## 2026-08-04 - PR #28 sanitizer CI disk and leak repair + +Ordino task `t-e19dc73f`, PR #28, branch +`fix/minja-gcc15-werror-20260803`. GitHub run `30819266647`, job +`91704728276`, failed during the ASan+UBSan **Build** at about 60%: the hosted +runner reported 99 MiB free and `ld` returned `No space left on device` while +linking `test_mtp_speculator` and `test_rejection_sampler`. There was no +sanitizer runtime finding in that run. Each test force-linked the full static +instrumented engine, duplicating hundreds of MiB about 330 times. + +The sanitizer configuration now emits `-g1` and links helper-generated tests to +one internal, unversioned `libvllm_sanitize_test_shared.so` that retains the +whole static archive and its registrars. The packaged `libvllm.so` and default +static test linkage are unchanged. The ASan+UBSan build fell from 93 GiB to +5.6 GiB (shared image 289,836,680 bytes; `test_version` 3,025,144 bytes), and +the TSan build is 1.9 GiB. CI runs the existing `VT_POOL_BYPASS=1` detector mode +so the intentionally retained scratch pool uses real frees. The now-reachable +leak survey found one genuine minja ownership cycle: `MacroNode` stored a +callable in a context while strongly capturing that same context. A weak +capture preserves invocation lifetime through the caller's parent chain and +releases the graph after rendering. + +Two previously recorded Nix host-portability failures are also closed. The +packed-component test resolves `true` from `PATH`, its clean-environment import +uses the current absolute Python executable, and CTest removes an inherited +`PYTHONHASHSEED` only for the suite whose control arm explicitly requires no +seed configuration. + +Binding local evidence on GCC 15.2.0: + +- ASan+UBSan full build, then `UBSAN_OPTIONS=print_stacktrace=1 + ASAN_OPTIONS=detect_leaks=1:strict_string_checks=1 VT_POOL_BYPASS=1 ctest + --test-dir build-nix-sanitize-g1 --output-on-failure`: **331/331 PASS**. +- TSan full build, then the same CTest command against + `build-nix-sanitize-thread-shared`: **331/331 PASS**. +- Plain GCC 15 `-Werror` affected-target build and focused CTest + (`test_serve_low_tools`, `test_none_hash_determinism`, + `test_chat_template`): **3/3 PASS**. + +No model, kernel, runtime default, CUDA path, correctness lifecycle, or +benchmark headline changed. Performance is NOT APPLICABLE. Next external gate: +push the checkpoint through the Ordino action-request path and confirm both +hosted sanitizer jobs before removing their first-survey `continue-on-error`. + ## 2026-08-03 - MLX system-header dependency boundary LocalAI Darwin consumer run `30783379823` showed that translation-unit warning diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index bcaad189e..0700f1502 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -291,4 +291,8 @@ jobs: env: UBSAN_OPTIONS: print_stacktrace=1 ASAN_OPTIONS: detect_leaks=1:strict_string_checks=1 + # The production DevicePool deliberately retains scratch blocks. Its + # detector lane uses exact allocations and real frees so ASan can + # distinguish that cache from a leak and can see use-after-free. + VT_POOL_BYPASS: "1" run: ctest --test-dir build-sanitize --output-on-failure diff --git a/CMakeLists.txt b/CMakeLists.txt index 363ce7924..73a426e63 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -142,9 +142,12 @@ endif() # # -fno-sanitize-recover=all makes a finding ABORT instead of print-and-continue, # so ctest FAILS the lane rather than burying the report inside passing output. -# Frame pointers plus -g keep the traces attributable; the lane keeps the -# project's -ffp-contract=off numerics so a bit-identity test still means what it -# means in the production build. +# Frame pointers plus -g1 keep file/line traces attributable without emitting +# full type/local-variable DWARF. Every test statically links the instrumented +# library, so full -g duplicates hundreds of MiB into each executable; the +# ASan+UBSan GitHub runner exhausted its filesystem before the suite finished +# linking. The lane keeps the project's -ffp-contract=off numerics so a +# bit-identity test still means what it means in the production build. set(VLLM_CPP_SANITIZE "OFF" CACHE STRING "Host sanitizer lane: OFF | address | undefined | address,undefined | thread") set_property(CACHE VLLM_CPP_SANITIZE PROPERTY STRINGS @@ -166,7 +169,7 @@ if(NOT VLLM_CPP_SANITIZE STREQUAL "OFF") $<$:-fsanitize=${VLLM_CPP_SANITIZE}> $<$:-fno-sanitize-recover=all> $<$:-fno-omit-frame-pointer> - $<$:-g>) + $<$:-g1>) # -Werror itself is dropped for this lane in cmake/CompilerWarnings.cmake, where # the per-target warning flags are set; see the rationale there. add_link_options(-fsanitize=${VLLM_CPP_SANITIZE}) @@ -784,6 +787,21 @@ target_include_directories(vllm PRIVATE src) # src/vllm/entrypoints/chat_template.cpp. target_include_directories(vllm SYSTEM PUBLIC third_party) vllm_cpp_set_warnings(vllm) +# SYSTEM covers the FRONT-END diagnostics from those vendored headers, but not +# the ones the OPTIMIZER raises after inlining: those are attributed to the TU +# being compiled, not to the header, so -isystem cannot reach them. GCC 15 +# inlines minja::Parser::parseValueExpression (third_party/minja/minja.hpp), +# loses track of the allocation base of a std::vector it is +# destroying, and reports -Wfree-nonheap-object against correct vendored code. +# Demote just that diagnostic, just here: the warning stays visible, -Werror +# stays in force for every other diagnostic and every other TU, and the +# vendored source stays unmodified. Ordered after vllm_cpp_set_warnings so the +# target-level -Werror cannot promote it again (the lesson of the MLX header +# exception below). cmake/CompilerWarnings.cmake documents the same GCC +# false-positive class for the sanitizer lanes. +set_source_files_properties(src/vllm/entrypoints/chat_template.cpp PROPERTIES + COMPILE_OPTIONS + "$<$:-Wno-error=free-nonheap-object>") # --- Vendored BLAKE3 (LMCache MODE-1 lm:// token hash, KV-EXTERNAL-CACHE W1) -- # The official BLAKE3 C implementation (third_party/blake3, upstream tag 1.5.5 # / commit 81f772a; dual CC0-1.0 / Apache-2.0), the same algorithm the `blake3` @@ -1417,6 +1435,26 @@ if(UNIX AND NOT APPLE) "LINKER:--version-script,${CMAKE_CURRENT_SOURCE_DIR}/cmake/vllm_export.map") endif() +# Sanitizer-only test linkage. Every normal test force-links the complete static +# `vllm` archive so its registrars are retained. With ASan+UBSan that makes each +# executable hundreds of MiB and duplicates the same instrumented engine across +# the whole suite; GitHub's hosted runner ran out of disk while linking. Keep one +# unversioned internal shared image of the fully instrumented archive instead. +# This is deliberately separate from packaged `vllm_shared`: that library's +# version script must continue exporting only the stable C ABI, while the tests +# need the C++ internals. Production/default builds never create this target. +if(NOT VLLM_CPP_SANITIZE STREQUAL "OFF") + add_library(vllm_sanitize_test_shared SHARED "${_vllm_shared_stub}") + add_library(vllm::sanitize_test_shared ALIAS vllm_sanitize_test_shared) + target_link_libraries(vllm_sanitize_test_shared PRIVATE vllm) + target_include_directories(vllm_sanitize_test_shared INTERFACE + "$") + target_include_directories(vllm_sanitize_test_shared SYSTEM INTERFACE + "$") + target_compile_definitions(vllm_sanitize_test_shared INTERFACE + "$") +endif() + # make install => a consumable libvllm.so + libvllm.a + vllm.h (llama.cpp-style). install(TARGETS vllm vllm_shared ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR} diff --git a/docs/BENCHMARKS.md b/docs/BENCHMARKS.md index 74616eb73..76015b6e1 100644 --- a/docs/BENCHMARKS.md +++ b/docs/BENCHMARKS.md @@ -38,7 +38,7 @@ The binding comparison. vLLM runs its **production graphed config**, never | Qwen3.6-27B | NVFP4 | 0.25.0 | **115/124** | Effective parity-or-better, two-grid totality | | Qwen3.6-35B-A3B | NVFP4 `modelopt_mixed` | 0.25.0 | 2/18 | fresh 3-rep grid 2026-08-05 @`1ea26427`: tput 0.93-1.03x (c4 wins), TTFT 0.93-0.98x; c1 closed 0.82→0.98; c16 0.93x. c16 drain-sync lever A/B'd (below): blocking-event LOST -1.9%, drain kept; cost = serialization | | DeepSeek-V2-Lite | bf16 MLA | 0.25.0 | 4/25 | Attributed miss, row stays `ACTIVE` | -| Qwen3.5-4B | bf16 direct-load | 0.24.0 | 5/8 | Throughput 0.98x, TTFT and memory win | +| Qwen3.5-4B | bf16 direct-load | 0.26.0.dev0 | 2/7 | 0.998x throughput; TTFT/PSS win. TPOT 1.124x and VRAM open ([evidence](bench-evidence/qwen35-4b-gcc15fix-20260803.md)) | ### Qwen3.6-27B by concurrency @@ -258,6 +258,12 @@ compaction, CI concurrency, anchor backfill, the operator/helper protocol W0-W5 with role discipline now enforcing, and the upstream/device inventory) touched no engine code and moved no number: NOT APPLICABLE, nothing to reproduce. +The PR #28 sanitizer repair is also NOT APPLICABLE to performance: both full +333-test CPU detector lanes pass after merging upstream `main`, while the +ASan+UBSan build footprint falls from 93 GiB to 5.7 GiB and TSan occupies +1.9 GiB. Reproduce with the sanitizer +CTest commands recorded in `.agents/state.md`. + **Vocabulary.** *Token-exact* means our output ids equal the reference's, byte for byte. *Near-tie* means the reference's own greedy decode is not deterministic at this precision, so the gate is distributional: our output must fall inside the diff --git a/docs/STATUS.md b/docs/STATUS.md index afbc54efd..9f050b981 100644 --- a/docs/STATUS.md +++ b/docs/STATUS.md @@ -1246,12 +1246,12 @@ protocol is ACCEPTED; W0-W5 LANDED, enforcement opt-in ([spec](../.agents/specs/operator-helper-protocol.md)). No engine code, no kernel, no numbers changed. -Measured against an oracle built from source at the ACTUAL parity pin, the gap -is 0.9970x total throughput, not the 0.9819x published against the older -pip-installed 0.24.0 release: that release is 1.25% faster than the pin, so -the old denominator was understating us. TPOT (+12.4%) is the one real gap. -Re-validated 2026-07-29 after rebasing onto 139 upstream commits: 0.9972x / -TPOT 1.1247x, every axis inside noise and token-identical to the prior series. +Qwen3.5-4B direct-load remains speed-pending against the oracle built at the +actual parity pin. The 2026-08-03 revalidation across another 217 upstream +commits plus the GCC 15 repair measured 0.9980x total throughput and 1.1243x +TPOT, with output bit-identical to the prior series (128/128 per arm). The +0.0008x ratio movement was denominator noise. The older 0.9819x result used the +1.25%-faster pip 0.24.0 release and is not the binding comparison. One open lead is on record from the same profiling pass: cuBLASLt resolves Ampere-class GEMM kernels on this Blackwell device. It is unmeasured and may be @@ -1265,33 +1265,27 @@ Benchmark provenance is gated as well as measured: both the comparison and the same-binary A/B harness refuse to start a leg on a GPU that is not idle, after a cooldown rather than before it. -Alongside the default build, `-DVLLM_CPP_SANITIZE=address,undefined` and -`-DVLLM_CPP_SANITIZE=thread` build the CPU tier under the dynamic detectors, and -CI runs both as separate jobs. Verified end to end: the ASan+UBSan lane builds -and passes `test_input_batch`, `test_combine_tokens` and `test_arena` with leak -detection on. The lanes keep the warnings but drop `-Werror`, because sanitizer -instrumentation makes GCC's range and initialization analyses fire inside -libstdc++ on correct code; the plain build is the one that enforces `-Werror`. The lane refuses to configure with the CUDA -backend on, because a host sanitizer runtime does not instrument nvcc device -translation units and reports false positives against the CUDA driver; the CUDA -tier's equivalent is `compute-sanitizer`, and `VT_POOL_BYPASS=1` makes the device -scratch pool hand out exact-size, really-freed allocations so that tool can see -tensor boundaries and use-after-free the caching pool otherwise hides. - -CI concurrency is split by GATE SCOPE, not applied workflow-wide. The two -DIFF-scoped gates, `documentation-checkpoint` and `commit-protocol-tag`, -evaluate `github.event.before..github.sha`, so each owns a commit range that no -later run re-covers (the next run's `before` is this run's `sha`). They -deliberately carry no concurrency group and are never cancelled. The six -TREE-scoped jobs (`agent-record`, `cuda-arch-features`, `device-leakage`, -`build-test-cpu`, and both `sanitize-cpu` lanes) validate HEAD, so only the -newest push to a ref is meaningful; each carries a per-job group keyed on -`github.ref` with `cancel-in-progress`. Net effect: a superseded push to `main` -keeps its 2 per-commit gates and drops the other 6 jobs. The `sanitize-cpu` -group includes `matrix.lane`, without which the two mutually exclusive -sanitizer legs of the SAME run would share a group and cancel each other. At -the workflow level the group is keyed on the SHA for a push, so two pushes to -`main` never share a group; only pull-request pushes are deduplicated there. +The ASan+UBSan and TSan CPU lanes both pass **333/333** on GCC 15.2 after +merging upstream `main`; ASan leak detection stays enabled. PR #28's hosted +failure was filesystem exhaustion, not +a sanitizer finding: sharing one internal instrumented image and using `-g1` +cut the ASan+UBSan tree from 93 GiB to 5.7 GiB; TSan occupies 1.9 GiB. +`VT_POOL_BYPASS=1` makes intentional scratch retention visible as real frees, +and the complete survey fixed one genuine minja context/callable ownership +cycle. Host sanitizers remain CPU-only; CUDA uses `compute-sanitizer`. + +The plain GCC 15.2 `-Werror` build exposed two optimizer false positives. The +project-owned `std::vector` assignment now avoids the packed word-copy +path without suppression; the vendored minja diagnostic is demoted only for +`chat_template.cpp` under GNU. Every other diagnostic and unit remains fatal. +Sanitizer lanes alone omit `-Werror` because instrumentation triggers additional +libstdc++ range-analysis false positives. + +CI concurrency follows gate scope. The diff-scoped documentation and commit +protocol jobs never cancel because no later run rechecks their commit range. +Tree-scoped jobs cancel superseded PR runs per ref; sanitizer groups also key on +the matrix lane so ASan+UBSan and TSan cannot cancel each other. Push workflows +key on SHA, while pull-request workflows deduplicate the same updated ref. Known failing on discrete sm_120 (RTX 5070 Ti), pre-existing and not introduced by the lanes: `test_cuda_ops` "CUDA matmul (cuBLASLt) matches CPU on odd sizes" diff --git a/docs/bench-evidence/qwen35-4b-gcc15fix-20260803.md b/docs/bench-evidence/qwen35-4b-gcc15fix-20260803.md new file mode 100644 index 000000000..9d46c43e6 --- /dev/null +++ b/docs/bench-evidence/qwen35-4b-gcc15fix-20260803.md @@ -0,0 +1,103 @@ +# Qwen3.5-4B re-validated after the 217-commit rebase and the GCC 15 build fix, 2026-08-03 + +Immutable evidence index for the re-measurement of the 4B lever after `main` was +advanced from `f3ecbe70d` to `265e3bf98` (217 upstream commits) and the GCC 15 +`-Werror` build repair landed on top as `4e43aa5e`. + +## Headline + +**Nothing moved.** 217 upstream commits and one build repair later, every axis +reproduces the 2026-07-29 series inside run-to-run noise, and our generated +tokens are **bit-identical** to that series. The ratio reads **0.9980x** total +throughput (was 0.9972x), TTFT **PASS**, TPOT still the one failing axis at +**+12.4%**. + +The published ratio moved by 0.0008x, and the honest attribution is the +DENOMINATOR, not us: our own total throughput moved +0.017% (6610.270 to +6611.397, ratio-to-historical 1.00017x) while the oracle came in 0.06% slower +than its previous series (6628.651 to 6624.585). Both are inside noise. This is +a null result, and it is the useful kind: the rebase did not regress the lever, +and the build fix did not touch this workload. + +## What was measured + +- Ours: `4e43aa5e` on `fix/minja-gcc15-werror-20260803`, clean CUDA rebuild + 899/899, 0 errors, 0 warnings. +- Oracle: `.venv-vllm-pin`, vLLM `0.23.1rc1.dev1511+g555967922`, built from + source at the parity pin. Pin re-checked before the run and unchanged + (`AGENTS.md` still specifies `555967922`), so the same denominator remains + correct. This is NOT the `.venv-vllm` 0.24.0 release, which is 1.25% faster + than the pin and would understate us. +- Build: RelWithDebInfo, `VLLM_CPP_CUDA_ARCHITECTURES=120a`, FlashAttn ON, + Triton AOT ON with regen, vendored arch `sm_120`. +- Three performance repetitions per arm, interleaved, one `flock /tmp/gpu` held + across the whole 18-leg series. +- **All 18 legs recorded 0% GPU utilization** and 539 MiB resident (the X + server) before starting. The series is uncontended and therefore binding. + +## Binding result + +| Axis | ours | vLLM @ pin | ratio | 2026-07-29 ratio | Disposition | +|---|---:|---:|---:|---:|---| +| Total throughput (tok/s) | 6611.397 | 6624.585 | 0.9980x | 0.9972x | FAIL | +| Output throughput (tok/s) | 731.070 | 732.528 | 0.9980x | 0.9972x | FAIL | +| Requests/s | 5.710 | 5.723 | 0.9978x | 0.9971x | FAIL | +| Mean TTFT (ms) | 728.243 | 948.961 | 0.7674x | 0.7701x | PASS | +| Mean TPOT (ms) | 38.160 | 33.942 | 1.1243x | 1.1247x | FAIL | +| Mean ITL (ms) | 38.160 | 33.942 | 1.1243x | | FAIL | +| Peak PSS (GiB) | 2.147 | 7.621 | 0.2816x | | PASS | +| Peak VRAM (MiB) | 12850.7 | 12832.0 | 1.0015x | | FAIL | + +Per-axis spread across the three repetitions is tight: total throughput +6608.16 to 6614.46, mean TPOT 38.15 to 38.17, mean TTFT 726.85 to 730.61. + +The acceptance rule is match or beat on EVERY axis, so this remains an OPEN GAP, +not a pass. Throughput sits 0.2% below the oracle, TPOT 12.4% above, and peak +VRAM 18.7 MiB above. Peak host memory is the one decisive win (2.147 GiB against +7.621 GiB, and stable PSS 0.760 GiB against 4.442 GiB). + +## Output stability + +- Ours is bit-stable within the run: 128/128 identical generated tokens across + all three repetitions, both arms. +- Ours is bit-identical to the 2026-07-29 series: 128/128 for `cpp-on`, + `cpp-off` and the `vllm` arm alike. **The 217-commit advance plus the build + fix changed no output on this workload.** +- Ours versus vLLM agreement is 89/128, UNCHANGED from the 2026-07-29 series. + The residual divergence is the known near-tie sampling difference, not new + drift. + +## Direct-load arm + +The `cpp-off` (no direct loader) arm reproduces as well: 6494.79 tok/s against +its own 2026-07-29 6493.39 (1.00022x). Direct loading remains worth 1.8% total +throughput and drops peak PSS from 8.574 to 2.147 GiB. + +## Interpretation + +TPOT remains owned by `ENG-ASYNC-SCHED`, and the W4 finding stands: closing it +needs the ASYNC engine loop, which `vllm-bench` does not drive. Nothing in this +change set touches it. The build fix altered a CMake warning property and one +`std::vector` assignment in a tool parser that this dense 4B decode +workload never executes, so a null result was the prediction and the measurement +confirms it rather than assuming it. + +## Reproduction + +```sh +nix develop .#cuda --command bash -c 'REQUIRE_TRITON_AOT=1 \ + CPP_BENCH="$PWD/build-nix-cuda-transplant-triton/examples/vllm-bench" \ + CMAKE_CACHE="$PWD/build-nix-cuda-transplant-triton/CMakeCache.txt" \ + VLLM_PYTHON="$PWD/.venv-vllm-pin/bin/python" \ + VLLM_CUDA_HOME="$PWD/.venv-vllm-pin/lib/python3.12/site-packages/nvidia/cu13" \ + flock /tmp/gpu tools/bench/run_qwen35_4b_compare.sh /tmp/qwen35-gcc15fix-4e43aa5ee' + +python3 tools/bench/summarize_qwen35_4b_compare.py \ + --root /tmp/qwen35-gcc15fix-4e43aa5ee \ + --historical-root /tmp/qwen35-postrebase-13906f189 \ + --output /tmp/qwen35-gcc15fix-4e43aa5ee/aggregate.json +``` + +Evidence root `/tmp/qwen35-gcc15fix-4e43aa5ee` (306 files), aggregate +`aggregate.json`. No 4B result implies support or speed for the 27B/35B gate +checkpoints, which remain hardware-unavailable on this host and `PENDING`. diff --git a/src/vllm/entrypoints/openai/tool_parsers/hunyuan_a13b.cpp b/src/vllm/entrypoints/openai/tool_parsers/hunyuan_a13b.cpp index 22e62cc3f..7bced431d 100644 --- a/src/vllm/entrypoints/openai/tool_parsers/hunyuan_a13b.cpp +++ b/src/vllm/entrypoints/openai/tool_parsers/hunyuan_a13b.cpp @@ -200,7 +200,12 @@ std::optional HunyuanA13BToolParser::handle_test_compatibility( d.function.name = function_name; DeltaMessage msg; msg.tool_calls = std::vector{std::move(d)}; - hy_current_tools_sent = {true}; + // assign() rather than `= {true}`: the initializer_list assignment routes + // through vector::_M_insert_range -> _M_copy_aligned, where GCC 15 + // mis-analyses the bit-packing word copy and reports a false + // -Warray-bounds ("forming offset 8 is out of the bounds [0, 8]") against + // a correct one-element assignment. Same semantics, no suppression. + hy_current_tools_sent.assign(1, true); hy_current_tool_id = 0; hy_current_tool_index = 0; if (hy_sent_tools.empty()) { diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 0cba56464..241548ea3 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -17,7 +17,11 @@ set_tests_properties(test_serve_low_tools PROPERTIES function(vllm_cpp_add_test name) add_executable(${name} ${ARGN}) - target_link_libraries(${name} PRIVATE vllm::vllm vllm_test_main) + if(TARGET vllm::sanitize_test_shared) + target_link_libraries(${name} PRIVATE vllm::sanitize_test_shared vllm_test_main) + else() + target_link_libraries(${name} PRIVATE vllm::vllm vllm_test_main) + endif() vllm_cpp_set_warnings(${name}) add_test(NAME ${name} COMMAND ${name}) endfunction() @@ -223,6 +227,11 @@ vllm_cpp_add_test(test_single_type_kv_cache_manager vllm/v1/test_single_type_kv_ vllm_cpp_add_test(test_kv_cache_coordinator vllm/v1/test_kv_cache_coordinator.cpp) vllm_cpp_add_test(test_kv_cache_manager vllm/v1/test_kv_cache_manager.cpp) vllm_cpp_add_test(test_none_hash_determinism vllm/v1/test_none_hash_determinism.cpp) +# This suite explicitly exercises configured and unconfigured hash seeding in +# child processes. Insulate its unconfigured control arm from a developer shell +# that happens to export Python's seed (the Nix dev shell exports 0). +set_tests_properties(test_none_hash_determinism PROPERTIES + ENVIRONMENT_MODIFICATION "PYTHONHASHSEED=unset:") vllm_cpp_add_test(test_prefix_cache_stats vllm/v1/test_prefix_cache_stats.cpp) vllm_cpp_add_test(test_prometheus_metrics vllm/v1/test_prometheus_metrics.cpp) vllm_cpp_add_test(test_kv_offload_cpu vllm/v1/test_kv_offload_cpu.cpp) diff --git a/tests/tools/test_gdn_packed_component.py b/tests/tools/test_gdn_packed_component.py index c7d235aef..ca3155401 100644 --- a/tests/tools/test_gdn_packed_component.py +++ b/tests/tools/test_gdn_packed_component.py @@ -1078,8 +1078,11 @@ def _write_complete_evidence(root: pathlib.Path, *, packed_better: bool = True) class GdnPackedComponentTest(unittest.TestCase): @classmethod def setUpClass(cls) -> None: + true_binary = shutil.which("true") + if true_binary is None: + raise unittest.SkipTest("the platform has no 'true' executable") cls.cuda_compiler_patch = mock.patch.object( - gdn_component, "DGX_CUDA_COMPILER", pathlib.Path("/usr/bin/true") + gdn_component, "DGX_CUDA_COMPILER", pathlib.Path(true_binary) ) cls.cuda_compiler_patch.start() diff --git a/tests/tools/test_online_gate_client.py b/tests/tools/test_online_gate_client.py index f1716af66..5b8ebadc4 100644 --- a/tests/tools/test_online_gate_client.py +++ b/tests/tools/test_online_gate_client.py @@ -1462,7 +1462,8 @@ def test_trace_driver_sets_h1d_plan_environment_before_manifest(self) -> None: "-c", definitions + '"${benchmark_clean_env[@]}" "${h1d_plan_env[@]}" ' - "python3 -c 'import tools.bench.online_gate'", + + shlex.quote(sys.executable) + + " -c 'import tools.bench.online_gate'", ], env=polluted, check=True, diff --git a/tests/vllm/test_model_loader_gguf.cpp b/tests/vllm/test_model_loader_gguf.cpp index acd50e050..24978efc1 100644 --- a/tests/vllm/test_model_loader_gguf.cpp +++ b/tests/vllm/test_model_loader_gguf.cpp @@ -80,7 +80,8 @@ TEST_CASE("FromModelDir rejects an unknown dense architecture before loading") { "'Gemma4ForConditionalGeneration', 'GemmaForCausalLM', " "'Glm4ForCausalLM', 'Glm4MoeLiteForCausalLM', 'GraniteForCausalLM', " "'InternLM2ForCausalLM', 'InternLM3ForCausalLM', " - "'KimiK3ForConditionalGeneration', 'LagunaForCausalLM', " + "'KimiK3ForConditionalGeneration', 'KimiLinearForCausalLM', " + "'LagunaForCausalLM', " "'LlamaForCausalLM', " "'MiniCPM3ForCausalLM', 'MiniCPMForCausalLM', 'MistralForCausalLM', " "'OPTForCausalLM', 'Olmo2ForCausalLM', 'Olmo3ForCausalLM', " diff --git a/third_party/minja/minja.hpp b/third_party/minja/minja.hpp index f414cca9c..3dc121419 100644 --- a/third_party/minja/minja.hpp +++ b/third_party/minja/minja.hpp @@ -1063,7 +1063,15 @@ class MacroNode : public TemplateNode { void do_render(std::ostringstream &, const std::shared_ptr & macro_context) const override { if (!name) throw std::runtime_error("MacroNode.name is null"); if (!body) throw std::runtime_error("MacroNode.body is null"); - auto callable = Value::callable([this, macro_context](const std::shared_ptr & call_context, ArgumentsValue & args) { + // The callable is stored in macro_context itself. Capturing that context + // strongly creates a permanent cycle (Context -> Value -> callable -> + // Context) after rendering any template with a macro. Callers keep the + // context alive through call_context's parent chain, so a weak capture + // preserves the lifetime needed during invocation without leaking it. + std::weak_ptr weak_macro_context = macro_context; + auto callable = Value::callable([this, weak_macro_context](const std::shared_ptr & call_context, ArgumentsValue & args) { + auto macro_context = weak_macro_context.lock(); + if (!macro_context) throw std::runtime_error("Macro context has expired"); auto execution_context = Context::make(Value::object(), macro_context); if (call_context->contains("caller")) {