fix(mesh): add inference-only embedded peer surface - #1
Open
ccarpene-blk wants to merge 516 commits into
Open
Conversation
* ci: make PR rust cache restores non-fatal * ci: clean up workflow lint warnings * ci: gate Windows Node release addon builds * ci: remove Docker builds from PRs * Update PR CI routing and cache warm keys * Document PR CI timing heuristics * Stop CI-only changes from fanning out * Document CI-only routing contract * Trigger & test React build timing
* updates to ROADMAP.md * Update ROADMAP.md * Update ROADMAP.md
Fix Skippy prefix-cache reuse for shared-prefix agent prompts Skippy split-prefill recording now stores the same bounded shared-prefix candidates that lookup already probes. This lets repeated agent/tool prompts with a stable long prefix and changing tail reuse the resident prefix cache instead of falling back to only exact-prompt hits. The change keeps exact prompt records intact, preserves the existing bounded candidate policy, and adds regressions proving that same-prefix/different-tail prompts share the recorded grid page while distinct exact prompts remain distinct. Validation * Validation tier: Tier 2 - narrow Skippy prefix-cache record/lookup repair for shared-prefix agent prompts. * git fetch --no-tags origin main:refs/remotes/origin/main: PASS, origin/main at 9b51d01. * git rebase origin/main: PASS. * git diff --check origin/main...HEAD: PASS, no output * git diff --check: PASS, no output * git diff --cached --check: PASS, no output * cargo fmt --all -- --check: PASS * cargo test -p skippy-cache --lib: PASS, 31 passed * LLAMA_STAGE_BUILD_DIR=<prepared llama stage build> cargo test -p skippy-server prefix --lib: PASS, 4 passed * LLAMA_STAGE_BUILD_DIR=<prepared llama stage build> cargo test -p skippy-server --lib: PASS, 98 passed * LLAMA_STAGE_BUILD_DIR=<prepared llama stage build> cargo check -p mesh-llm: PASS * LLAMA_STAGE_BUILD_DIR=<prepared llama stage build> cargo-clippy clippy -p skippy-server --all-targets -- -D warnings: PASS * Ledger: not applicable - not required for selected validation tier/change family. * Version: not applicable - cache behavior repair only; no release/version sync required. * Not run: live shared-prefix agent smoke - no local model/runtime endpoint was available; targeted identity/server tests cover the record/lookup path. Rollback * git revert HEAD
Fix Blacksmith CI discrepancies
Fix metadata-only package verification cache
Fix dispatched Swift release manifest flow
Validation * Validation tier: Tier 4 - release publish verification tooling and crates.io partial-publish recovery for issue Mesh-LLM#691. * git fetch --no-tags origin main:refs/remotes/origin/main: PASS, origin/main at 5c805ea. * git diff --check: PASS, no output * git diff --cached --check: PASS, no output * bash -n scripts/publish-crates.sh: PASS * scripts/publish-crates.sh --allow-dirty: PASS, rejected because --allow-dirty requires --dry-run * python3 -m unittest scripts.tests.test_publish_crates: PASS, 7 passed * python3 -m unittest discover -s scripts/tests: PASS, 27 passed * cargo run -p xtask -- repo-consistency release-targets: PASS * CARGO_TARGET_DIR=$(mktemp -d /tmp/mesh-llm-publish-target.XXXXXX) scripts/publish-crates.sh --dry-run --allow-dirty: PASS; dry-run verified publishable crates through model-artifact and deferred downstream crates whose 0.66.0 registry dependencies are not published yet. * Ledger: not applicable - not required for selected validation tier/change family. * Version: not applicable - release publish tooling/docs only; no release version sync required. * Not run: actionlint - not installed locally and workflow file was not changed. * Not run: real cargo publish / just release - intentionally avoided for PR validation. Rollback * git revert HEAD
Summary Adds a repeatable KV/tool-loop stability certification harness for direct-model agent/tool-call pressure runs, including plan output, manifest evidence, transcript handling, native-log checkpoint scanning, docs, tests, and a repo-local agent skill. Validation * Validation tier: Tier 4 - verification tooling and testing documentation for KV/tool-loop stability certification; no runtime, protocol, workflow, or CI gate change. * git fetch --no-tags origin main:refs/remotes/origin/main: PASS, origin/main at 12aa611. * git rebase origin/main: PASS, rebased cleanly after Mesh-LLM#695/Mesh-LLM#696. * git diff --check origin/main...HEAD: PASS, no output * git diff --check: PASS, no output * git diff --cached --check: PASS, no output * python3 -m unittest scripts.tests.test_qa_kv_tool_loop_stability: PASS, 16 passed * python3 -m unittest discover -s scripts/tests: PASS, 43 passed * python3 -m py_compile scripts/qa-kv-tool-loop-stability.py scripts/tests/test_qa_kv_tool_loop_stability.py: PASS * mkdir -p target/kv-tool-loop-stability && python3 scripts/qa-kv-tool-loop-stability.py --models auto,mesh --attempts 1 --pressure-turns 2 --timeout 30 --min-cached-tokens 128 --suffix-prefill-limit 64 --output-dir target/kv-tool-loop-stability/review-smoke --print-plan > /tmp/kv-tool-loop-plan.json && python3 -m json.tool /tmp/kv-tool-loop-plan.json >/dev/null: PASS * Remote PR checks after rebase: PASS, changes + summary green; non-applicable build/test matrices skipped by path filters. * Review/merge state: APPROVED, MERGEABLE, CLEAN. * Ledger: not applicable - not required for selected validation tier/change family. * Version: not applicable - verification tooling/docs only; no release/version sync required. * Not run: cargo check/tests - not required for selected validation tier; no Rust/runtime code changed. * Not run: live direct-model Skippy certification run - no local loaded direct-model endpoint was available; deterministic unit and plan-smoke coverage proves the harness behavior. Rollback * Revert this PR.
…#697) * skippy-runtime: filter DEBUG-level lines from native log callback llama.cpp's llama_log_internal_v invokes the registered log callback unconditionally with the level argument — the level filter is expected to live inside the callback. mesh-llm's native log callback (write_native_log) was ignoring _level and writing every line to ~/.mesh-llm/runtime/<pid>/logs/skippy-native.log, which meant LLAMA_LOG_DEBUG output ended up in the file regardless of GGML_LLAMA_LOG_LEVEL. Observed impact: on studio today, ~5400 'Grammar still awaiting trigger after token …' DEBUG lines for 18 actual tool-call grammar triggers — roughly 300× verbosity per completion. The lines are emitted per-token during MiniMax-M2.5's reasoning phase before the <minimax:tool_call> regex fires. Fix: drop ggml_log_level=DEBUG (1) unless GGML_LLAMA_LOG_LEVEL=4 (the existing opt-in that enable_verbose_native_logs sets). Keep CONT (continuation) lines regardless so multi-line INFO/WARN messages don't get truncated. Tests added in skippy-runtime::tests: - native_log_filter_drops_debug_by_default - native_log_filter_keeps_debug_when_verbose - native_log_filter_keeps_continuation_lines Env-var tests run serially via per-test mutex to avoid racing the rest of the file's GGML_LLAMA_LOG_LEVEL-touching tests. * mesh: harden relay-only heartbeat threshold against path flap Three related changes to the heartbeat failure-detection path: 1) Bump relay-only failure threshold from 3 to 5. Observed today on the public mesh: a Sydney<->Sydney peer (mini -> studio) that should hole-punch a direct LAN path but doesn't (mini's macOS VPN system extension blocks the UDP hole-punch) is forced onto a relay-only path via services.iroh.computer. Steady-state relay RTT is ~200ms but transient renegotiation spikes it to 10s+. With 60s heartbeats and a 3-miss threshold, mini declared studio dead twice in a 25 minute window — and each declaration knocked MiniMax-M2.5 out of the MoA reducer's eligible-peer set, causing 'HTTP 502: Reducer failed (tried 3)' for ~90s while the fallback to weaker peers also failed. 5 misses = 5 min grace covers the typical iroh relay path-renegotiation window. Direct paths stay at 2. 2) Fix is_relay_only detection so it survives mid-failure. The original check was , which returns (not Relay) when no path is currently selected — exactly what happens during a heartbeat failure when the connection is between path selections. That meant the relay-only-grace policy *never fired during the failures it was designed to protect against*: every flap landed on the strict direct-threshold path. New is_relay_only_connection inspects every advertised path; if none is IP (i.e. only relay paths are known), treat as relay-only. Also fixes the no-connection case: previously a peer with no live Connection object got the strict direct threshold, which is backwards — no connection at all is the most failure-prone state. Now defaults to lenient. 3) Cosmetic: '💚 Heartbeat: <peer> recovered (was N/2)' was hardcoded '/2', misleading for relay-only peers whose actual threshold was 3 (now 5). Read the real threshold from the policy and display it. Tests (mesh::tests::): - relay_only_peers_get_extra_heartbeat_grace (renamed, threshold updated 3 -> 5) - direct_peers_use_strict_heartbeat_threshold (new) - is_relay_only_path_set_classifies_correctly (new, exercises empty/all-relay/mixed/all-direct path sets) is_relay_only_connection is implemented as a thin wrapper over is_relay_only_path_set(IntoIterator<Item=bool>) so the classification logic is unit-testable without constructing a real iroh Connection. * clippy: drop redundant closure in heartbeat relay-only check * skippy: make CONT pass-through explicit in native log filter The original ordering made the CONT check dead code: the level-!=-DEBUG guard above it already returned false for CONT (level 5), so the explicit CONT branch could never execute. The comment claimed CONT was being protected by that branch but the protection was incidental. Reorder so CONT is checked first and is the real explicit pass-through, matching the documented intent. * heartbeat: default no-Connection to strict, not lenient PR review (P2, James): when state.connections.get(peer_id) returns None the old code in this PR defaulted is_relay_only to true, which extended the 5-min relay grace to *any* peer without a live Connection object — including previously-direct peers that simply got cleanly disconnected (QUIC idle-expired, peer departed). That kept stale model routes alive for up to 5 min instead of 2 for direct paths. The lenient threshold exists to absorb mid-flap path renegotiation, which only happens while iroh still owns the Connection. Once Connection is gone, the peer should be judged by the strict (direct) threshold so the eligible-peer set converges promptly. Introduce classify_relay_only_for_policy(Option<bool>) so the no-Connection default is testable in isolation. Live Connection still gets is_relay_only_connection's path-set verdict (unchanged); only the no-Connection fallback changes from lenient → strict. Test: classify_relay_only_defaults_to_strict_when_no_connection covers all three Option states. * Revert skippy-runtime DEBUG log filter — defer to a focused PR Pulled at James's P3 feedback. The native log filter as shipped only honors GGML_LLAMA_LOG_LEVEL=4 for DEBUG and ignores 0/1/2/3, so values like silent / error / warn are silently ineffective. Fixing it properly means parsing the env as a full threshold, deciding whether to cache (Copilot's perf concern) and how CONT should behave when its base line was filtered (James's part-b). That is a meaningful design discussion in its own right and doesn't belong tacked onto the heartbeat PR. Reverts to byte-for-byte parity with main on crates/skippy-runtime/src/lib.rs so Mesh-LLM#697 is unambiguously 'relay-only heartbeat grace + path-set classification fix', nothing else. The log filter will land separately once the threshold + CONT semantics are settled. heartbeat fixes (relay-only grace bump + is_relay_only_connection path-set classifier + classify_relay_only_for_policy strict-on-no-conn default + recovery-message threshold) are unchanged.
* MoA: paint winner content as multiple SSE deltas Today MoA emits the full winning text in a single SSE delta, so chat UIs jump from spinner to wall-of-text once the arbiter commits (~3s). This splits the buffered winner into ~25 word-boundary chunks with a short inter-chunk delay so the response paints progressively over ~500ms. Tool-call deltas remain atomic — harness parsers (Goose, OpenCode) still see a single well-formed tool_call object. Tunable via MESH_MOA_STREAM_CHUNKS (default 25) and MESH_MOA_STREAM_CHUNK_DELAY_MS (default 20). Verified on M4 against a real mesh: main: 1 delta, 0ms paint spread, TTFB 3212ms this branch: 22 deltas, 457ms paint spread, TTFB 3184ms Refs Mesh-LLM#618. * MoA: drip progress into reasoning_content while arbiter waits Streaming MoA requests previously sat silent for ~3s until the arbiter committed. Now the gateway sends response headers immediately and emits short progress lines ("Routing through mesh…", "Querying peer models…", "Comparing responses…") into delta.reasoning_content once per second. Goose and other OpenAI-shape clients route reasoning_content to the thinking pane, so the lines appear as visible activity without polluting the final answer. Clients that ignore the field (pi, openai SDK without reasoning support) parse the chunks normally and only see the final content. Responses-API requests drip into output_text.delta instead, since the Responses spec has no dedicated reasoning channel — the lines render inline but are short enough that the real answer effectively replaces them within a few hundred ms. Trade-off: HTTP headers must precede the body, so this path loses the post-hoc x-moa-* observability headers (workers, elapsed, etc.). Verified locally on M4 against a real mesh: - Wire: 3 reasoning_content drips over the 3s wait, then 22-chunk content paint (from the previous commit on this branch) - Goose chat-completions: tool calls still atomic, content arrives - Pi chat-completions: streams correctly, ignores reasoning_content Refs Mesh-LLM#618. * MoA progress: drip into reasoning_text.delta on Responses path The mesh-llm web UI (and any Responses-API consumer that follows the OpenAI spec) routes response.reasoning_text.delta to a separate thinking pane, but appends response.output_text.delta to the visible answer. Emitting progress lines as output_text.delta polluted the final answer — the 'Routing through mesh…' / 'Querying peer models…' / 'Comparing responses…' lines stuck to the top of the assistant's reply instead of staying in the thinking section. Switch to response.reasoning_text.delta for the Responses adapter so progress shows in the thinking pane and collapses cleanly when the real answer arrives. Chat-completions path (goose etc.) continues to use delta.reasoning_content, which is the analogous channel there. Verified on M4 against a real mesh via /v1/responses streaming probe: - 3× response.reasoning_text.delta at 1s/2s/3s (progress) - 22× response.output_text.delta from 3.0s-3.5s (real answer) And confirmed in the mesh-llm web console: progress shows in thinking pane, real answer in the bubble, no concatenation. Refs Mesh-LLM#618. * fmt: nest matches!() under streaming guard * MoA progress: cycle tail lines, decompose for clippy Two small follow-ups on the streaming MoA work after live-testing against a slow public peer: 1. Progress lines now cycle through a tail set after the initial three. Previously a slow MoA call (10-20s, common on public mesh when a peer is loaded) repeated 'Still working…' over and over. Now the opening sequence (Routing / Querying / Comparing) fires once each, then the tail (Waiting on a slow peer / Still gathering / Hold on…) cycles deterministically. 2. Split run_moa_turn_with_progress into three named helpers (send_progress_headers, drip_progress_until_moa_completes, write_progress_body) so each piece stays under the clippy cognitive-complexity threshold. Extracted the duplicated SSE-header-write block in send_moa_as_*sse_inner to a shared write_sse_response_headers helper. Tests updated to cover the cycle behaviour and renamed to match. * MoA streaming: drop env-var overrides, constants only Copilot review on PR Mesh-LLM#685 flagged that the MESH_MOA_STREAM_CHUNKS and MESH_MOA_STREAM_CHUNK_DELAY_MS env vars contradicted the PR description ("no env vars") and that the env-mutating tests were racy under parallel test runs. - Replace DEFAULT_MOA_STREAM_CHUNKS + getter with const MOA_STREAM_CHUNKS = 25 - Replace DEFAULT_MOA_STREAM_CHUNK_DELAY_MS + getter with const MOA_STREAM_CHUNK_DELAY = 20ms - Rename MOA_STREAM_MIN_CHARS → MOA_STREAM_MIN_BYTES (the check is .len(), not chars().count(); fixes a doc-vs-code mismatch Copilot flagged separately) - Delete env-var mutation in chat_sse / responses_sse delta tests; accept the ~500ms test runtime each (real-world chunk delay × N) - Rename chunk_helper_empty_input_returns_empty_slice → chunk_helper_empty_input_returns_single_empty_chunk (the helper returns vec![""], not an empty vec; better failure message) No runtime behaviour change — the defaults were already the values the env vars defaulted to, and no production code path relied on the overrides. * MoA progress: id-consistency + Responses ordering fixes Two correctness bugs flagged by Copilot review on PR Mesh-LLM#685: 1. Progress chunks hard-coded id=chatcmpl-mesh, but the body writer used the real MoA id (chatcmpl-moa-<hex>). Clients that correlate stream chunks by id (most chunk-aggregating SDKs) saw the progress and the content as belonging to different completions. 2. Responses-API streams emitted reasoning_text.delta events BEFORE response.created (the progress drip fired during MoA wait; response.created came from the body writer once MoA finished). The Responses-API contract is strict: created must precede all delta events, otherwise strict clients reject the stream. The body writer was also emitting a second response.created (now two in one stream — also a protocol violation). Changes: - Generate a single chatcmpl-moa-<hex-nanos> id up front in run_moa_turn_with_progress (same shape as MoA's own short_id). - Thread it through write_progress_event, write_failure_as_sse_tail. - After MoA returns, mutate response_body.id so the body writer reuses the same id (no new helper to plumb the id into the body writer). - On the Responses path, emit response.created up front right after headers via a new write_progress_response_created helper. - send_moa_as_responses_sse_inner: when header_already_sent is true, skip its own response.created (the progress path already sent it). - item_id derived from completion_id via item_id_from_completion_id helper, mirroring short_id_from_response shape so progress and final events share item_id within one Responses stream. - Failure-tail also reuses the completion_id and adds 'id' on the Responses error envelope. Tests added: - progress_event_chat_uses_reasoning_content_field: now asserts id matches the supplied completion_id. - progress_event_responses_uses_reasoning_text_delta: now asserts item_id derives correctly from completion_id. - failure_tail_emits_error_then_done_for_chat_adapter: now asserts the failure chunk carries the completion_id. - responses_progress_path_emits_exactly_one_response_created: new, end-to-end test that runs headers → created → progress delta → body writer (header_already_sent=true) and asserts exactly one response.created event, in the right ordering position. - item_id_derives_short_suffix_from_completion_id: new unit test for the helper. Verified live on M4 against the public mesh: - Wire probe: 1 response.created, 3 reasoning_text.delta, 30 output_text.delta. All 34 events share id chatcmpl-moa-<X> and item_id msg_moa_<X>. - Goose tool-call (write a file): tool argument parsed correctly, file produced, content matches. * fmt: collapse send_responses_created_for_progress signature * clippy: collapse nested inter-chunk delay ifs in MoA streaming * moa progress: sequence_number on Responses deltas + Skip ticker behavior Two correctness fixes for the streaming progress path. 1. Responses-API progress events (response.reasoning_text.delta) now carry sequence_number, starting at 1 to follow response.created at 0. Strict Responses clients (OpenAI SDK, Vercel AI SDK) use the field for ordering and dedup within a single response stream; omitting it was inconsistent with the helpers in openai_frontend::responses and with the body writer's events. 2. The 1s progress ticker now uses MissedTickBehavior::Skip. The default (Burst) would fire the ticker N times back-to-back if a progress write stalled (slow client, transient backpressure), dumping a burst of progress lines once writes resumed. Skip drops the missed ticks so cadence stays at one line per real interval. Tests: - progress_event_responses_uses_reasoning_text_delta now asserts sequence_number == 1. - New progress_event_responses_sequence_number_increments verifies two consecutive writes produce monotonically increasing sequence_number (1, 2). * moa progress: stream-wide monotonic sequence + cancel on client disconnect Two fixes for the Responses-API progress streaming path (PR review, P1 and P2 from James). P1 — stream-wide monotonic sequence_number + stable created_at: Previously the body writer (send_moa_as_responses_sse_inner) called helpers that hard-code sequence_number=0 and recomputed created_at locally, so a slow MoA response produced a wire stream like created(seq=0, created_at=T1) reasoning.delta(seq=1), reasoning.delta(seq=2), … output_text.delta(seq=0) ← reset, collides with created output_text.done(seq=0) completed(seq=0, created_at=T2) ← different timestamp, same id Strict Responses-API clients (OpenAI SDK, Vercel AI SDK) use sequence_number for ordering/dedup and created_at to correlate the response object across events; both invariants were broken. Thread the running sequence_number and the original created_at from the progress phase into the body writer via a new ProgressContinuation struct, and switch to the *_with_sequence helper variants. Non-progress callers pass None and the existing behaviour (created_at = now, sequence_number starts at 0) is preserved. P2 — cancel MoA on client disconnect: When a progress write fails (almost always: client closed the TCP connection), the old code logged 'falling back' and then *awaited* the MoA future to completion, burning peer compute and reducer budget for a dead request — up to MoA's ~60s timeout. drip_progress_until_moa_completes now returns Result<(TurnResult, i32), ClientGone>. On Err the caller bails out and the pinned MoA future is dropped, cancelling worker dispatch and any reducer call at the next .await point. Refactor split out drip_progress_against_future as the generic core (takes any Future<Output = TurnResult>) so tests can substitute a hand-rolled never-yielding future to verify the drop-on-disconnect behaviour without running a real MoA gateway. run_moa_turn_with_progress also gained two small helpers (send_progress_response_created_if_responses, drip_progress_phase) to keep clippy's cognitive_complexity check satisfied. New tests: - responses_progress_path_emits_monotonic_sequence_and_stable_created_at parses the full wire trace and asserts sequence_number is strictly monotonic with no duplicates AND created_at matches between response.created and response.completed. - progress_drops_moa_future_when_client_disconnects uses a DropTrackingPendingFuture + tokio::test(start_paused) to verify the MoA future is dropped (not awaited) when the client socket closes mid-progress. * moa_gateway: extract progress streaming into its own module PR review (P3, James): moa_gateway.rs grew to 3,020 lines as the streaming progress UX accreted around the gateway entry, well over the repo's 2,000-line file limit. Split the progress-streaming surface into a dedicated progress submodule so the gateway file can stay focused on routing, scoring, and worker dispatch. The progress module owns: - The streaming entry (run_moa_turn_with_progress) and the ticker drip loop (drip_progress_phase, drip_progress_until_moa_completes, drip_progress_against_future). - All progress-specific wire writers (write_progress_response_created, send_progress_headers, write_progress_event, write_progress_body, write_failure_as_sse_tail). - The Responses-API continuation type (ProgressContinuation, ClientGone) and the small helpers tied to the progress path (item_id_from_completion_id, overwrite_response_id, progress_line, short_hex_nanos). - All progress-related tests. The gateway module keeps the body writers (send_moa_as_*_sse_inner) and shared helpers (is_moa_failure_body, write_sse_response_headers, chunking, MoA decision logic). Helpers needed by progress are re-scoped to pub(in crate::network::openai::moa_gateway) so the visibility boundary matches the module tree. mod.rs: 1,953 lines. progress.rs: 1,108 lines. Both now under the 2,000-line limit. No behaviour change \u2014 all 53 existing tests pass without modification (just relocated into the progress test module where they live next to the code they exercise).
* Add optional SDK console asset server * extract shared mesh config crate * Include console server crate in client Docker build * Address config crate review feedback
* feat(cli): --relay-auth URL=TOKEN for gated iroh-relays Adds a per-relay bearer token to the iroh relay map so mesh-llm can register with a gated iroh-relay (one running AccessConfig::Restricted) while public relays in the same map continue to register without auth. The token is sent as 'Authorization: Bearer <TOKEN>' on the WebSocket upgrade to the matching --relay URL, via iroh::RelayConfig::with_auth_token. Repeatable. Splits on the first '=' only so tokens may contain '=' (base64 padding, JWTs, etc.). Why: enables embedders (e.g. Sprout) to launch mesh-llm against an operator-hosted iroh-relay that authenticates members via a bearer scheme (NIP-98, JWT, opaque API key, ...) without forking mesh-llm. Admission policy stays at the relay; mesh-llm just carries the token. Threaded through Node::start -> bind_mesh_endpoint and maybe_start_control_listener -> configure_control_relay so both the data-plane and owner-control endpoints honour per-relay tokens. Tests: parser handles trailing '=' and rejects malformed input; relay-map builder leaves untokened relays unauthenticated and attaches tokens only to matching URLs. * fix(clippy): group relay urls/auths into RelayConfig to keep Node::start under arg limit * test(relay-auth): in-process gated-relay e2e + scanner regression for --relay-auth before serve/client PR feedback fixes for Mesh-LLM#641: - ndizazzo: add --relay-auth to normalize_runtime_surface_args' value-taking flag list so 'mesh-llm --relay-auth URL=TOKEN serve/client …' no longer stops scanning at the token. Adds two regression tests covering both surfaces and a base64-padded NIP-98-style token. - Copilot: drop the broken intra-doc link [`RelayMap`] (RelayMap isn't in scope at the call site) for [`iroh::RelayMap`]. Plus a real defence for the feature itself: spin up an in-process iroh-relay with AccessConfig::Restricted, build an iroh::Endpoint from relay_map_from_urls' output, and assert: 1. Matching token → endpoint.online() resolves. 2. Wrong token → home_relay_status reports 'not authorized' and online() never resolves. 3. Missing token → online() never resolves. 4. Mixed map (gated + public) authenticates only the gated relay and still comes online. This is the missing end-to-end check: if iroh changes how with_auth_token is sent on the WebSocket upgrade, or if a future refactor drops relay_auths from the call chain, these tests fail. * fix(relay-auth): redact token portion from parser error messages Copilot review surfaced a real leak: parse_relay_auth_pair includes the full URL=TOKEN input in error strings via {s:?}. If a user mistypes the flag, the bearer token ends up in terminal output, logs, and bug reports. Redaction rules: - Missing '=' separator: redact whole input (we cannot tell URL from token). - Empty URL ('=token'): redact (the value after '=' is the secret). - Empty token ('URL='): URL is safe to name; no token to leak. New test parser_errors_never_leak_token_portion pins the property: inject a known token string, drive each error path, assert the string never appears in the error message. * test(mesh): bind empty relay-auth map locally instead of &HashMap::new() Copilot review pass flagged 7 sites where the test threads `&std::collections::HashMap::new()` straight into `maybe_start_control_listener(...).await`. Compiles fine today because the callee only borrows the reference synchronously before any internal `.await`, but the pattern is fragile: if the signature ever shifts the borrow across an await point, every test breaks at once. Centralise on an `empty_relay_auths()` helper bound to a local before each call. Cheap future-proofing, removes 7 lookalikes from review chatter on future PRs touching this file, no behaviour change. All 119 mesh tests still pass. * style: rustfmt for Rust 2024 edition import ordering cargo fmt under edition 2024 sorts uppercase types alongside lowercase modules, which reorders the gated_relay_e2e_tests imports. Pure formatting, no logic change. Also adds dist/native-sdk*/ and dist/llama-stage-static/ to .gitignore so locally-packaged release artifacts don't leak into commits.
* Fix split serving for HF layer packages * Trim split docs quickstart * Add two-node split smoke CI * Watch both nodes in split smoke * Harden layer package stage preparation (Mesh-LLM#707) * Harden layer package stage preparation * Reduce stage control handler complexity * Fix split runtime partial-load test mock --------- Co-authored-by: James Dumay <jameswdumay@gmail.com>
* Use resumable hf-hub fork * Use combined hf-hub branch
* Move CLI integrations into bundled plugins * Support OpenCode JSONC config files * Extract blackboard plugin crate * Stage bundled plugin binaries in CI smokes * Stop shipping blackboard with mesh-llm * Make PR Linux tests tolerate sccache cache outages * Drop unnecessary workflow and build-script changes
Complete Skippy shared-prefix record candidates (Mesh-LLM#694) Fixes the Skippy prefix-cache record site for issue Mesh-LLM#683 so shared-prefix agent prompts can produce reusable cache candidates instead of repeatedly missing on the common “same long prefix, different tail” workload. Validation * Validation tier: Tier 2 - narrow Skippy prefix-cache record-site repair for issue Mesh-LLM#683, refreshed on current main. * git fetch --no-tags origin main:refs/remotes/origin/main: PASS, origin/main at 6669b41. * git rebase origin/main: PASS * git diff --check origin/main...HEAD: PASS, no output * git diff --check: PASS, no output * git diff --cached --check: PASS, no output * cargo fmt --all -- --check: PASS * cargo test -p skippy-cache --lib: PASS, 31 passed * cargo test -p skippy-server prefix --lib: PASS, 7 passed * cargo test -p skippy-server resident_activation --lib: PASS, 4 passed * cargo test -p skippy-server --lib: PASS, 104 passed * cargo check -p mesh-llm: PASS * cargo-clippy clippy -p skippy-server --all-targets -- -D warnings: PASS * Ledger: not applicable - not required for selected validation tier/change family. * Version: not applicable - cache behavior repair only; no release/version sync required. * Not run: live shared-prefix agent smoke - no local model/runtime endpoint was available; deterministic candidate-planning and cache unit tests cover the changed paths. Rollback * git revert HEAD
Refresh Skippy package preflight diagnostics (Mesh-LLM#703) Validation * Validation tier: Tier 2 - narrow skippy-model-package CLI diagnostics refresh for issue Mesh-LLM#630 package-preflight support after rebasing onto current main. * git fetch --no-tags origin main:refs/remotes/origin/main: PASS, origin/main at 4f52c43. * git fetch --no-tags fork codex/skippy-split-package-preflight:refs/remotes/fork/codex/skippy-split-package-preflight: PASS. * git rebase origin/main: PASS. * git diff --check origin/main...HEAD: PASS, no output. * git diff --check: PASS, no output. * git diff --cached --check: PASS, no output. * cargo fmt --all -- --check: PASS. * cargo test -p skippy-model-package preflight: PASS, 6 passed, 10 filtered out. * cargo test -p skippy-model-package --bin skippy-model-package: PASS, 16 passed. * cargo check -p skippy-model-package --bin skippy-model-package: PASS. * cargo-clippy clippy -p skippy-model-package --all-targets -- -D warnings: PASS. * Remote CI on final head 506fe47: PASS. * Ledger: not applicable - not required for selected validation tier/change family. * Version: not applicable - package diagnostics/tooling only; no release/version sync required. * Not run: live HF package/direct split startup smoke - not required for this package preflight PR; deterministic manifest/artifact/stage diagnostics are covered by targeted unit tests. Rollback * git revert HEAD
…s active (Mesh-LLM#952) * fix: gate passive inference streams behind admission when a trust policy is active stream_allowed_before_admission exempted STREAM_TUNNEL_HTTP (0x04) and STREAM_ROUTE_REQUEST (0x05) from the quarantine gate unconditionally, so a caller holding a leaked invite token could route inference through a node even after the trust gate rejected it at gossip (e.g. UntrustedOwner under TrustPolicy::Allowlist). The token was a bearer credential for inference regardless of the node's admission policy. The exemption now applies only under TrustPolicy::Off (the default open-mesh posture — no behavior change there). With PreferOwned/RequireOwned/Allowlist, only STREAM_GOSSIP bypasses the gate: if a node enforces who may join, the same enforcement covers who may consume. Found by an embedded-SDK admission harness (allowlist serve node + trusted client + stranger client reusing the trusted invite token): the stranger was rejected at gossip but still completed a chat completion via 0x04. New regression test: passive_streams_are_gated_when_trust_policy_active. * fix: keep prefer-owned passive access advisory --------- Co-authored-by: James Dumay <jameswdumay@gmail.com>
* fix(tui): indicate clipped join tokens * test(tui): cover empty join token slices
* Improve plugin documentation * Address plugin documentation review feedback
A node with a verified owner identity started a second iroh endpoint (the owner-control listener, ALPN mesh-llm-control/1) using the SAME secret key as the main mesh endpoint, so both endpoints presented the SAME iroh endpoint id. The control protocol requires this: it validates the dialed target_node_id against the main endpoint id (verify_control_plane_target_node), so the control endpoint must present that id. But an iroh relay keeps only ONE active connection per endpoint id. The control listener enabled relay and bound AFTER the main mesh endpoint, so it evicted the main endpoint's relay registration (relay: 'Another endpoint connected with the same endpoint id. No more messages will be received.'). The main mesh endpoint then went dark on the relay: all relay-delivered mesh traffic (gossip, joins, inference routing) stopped, so any peer that could not reach the node directly never completed its join, never synced the model catalog, and every inference returned 429/503. This is the consume-side 'connects but never joins' failure. Fix: bind the owner-control listener with RelayMode::Disabled so it never contends for the shared id's relay slot, leaving the main mesh endpoint as the sole relay registrant. The main mesh endpoint is unchanged and keeps relay/NAT traversal. Also clear_ip_transports() on the control endpoint so an explicit IPv4 bind does not leave iroh's implicit [::]:0 socket (which can trip MultipathNotNegotiated with relay disabled), matching the main endpoint's LAN-only handling. Trade-off: owner-control is now reachable over its direct/advertised address only; relay-assisted remote owner-control across NAT is not supported while the control and mesh endpoints share one id. Giving control its own relay presence would require a distinct id, which the control target validation forbids. Regression test: control_plane_listener_token_carries_no_relay_urls asserts the control token advertises zero relay URLs while keeping the main endpoint id. Empirically: with owner identity, a cross-machine consumer never joined; with this fix (proven via owner-suppressed A/B) it joins over the relay and inference returns HTTP 200.
Signed-off-by: ccarpene-blk <ccarpene@block.xyz>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
zizmor found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
There was a problem hiding this comment.
Semgrep OSS found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
Signed-off-by: ccarpene-blk <ccarpene@block.xyz>
Signed-off-by: ccarpene-blk <ccarpene@block.xyz>
ccarpene-blk
changed the title
Signed-off-by: ccarpene-blk <ccarpene@block.xyz>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
peer_inference_onlyembedded mode that keeps mesh routing and OpenAI inference streams while rejecting raw tunnels, plugins, subprotocols, and Skippy stage ALPN from peersContext
Buzz currently pins MeshLLM at
f455d493a2ae82baf2a326e2d0fda351433b4b30, where the inbound QUIC HTTP tunnel forwards arbitrary HTTP requests to the local:9337proxy. That means a trusted mesh peer can reach control routes such as/mesh/loadand/mesh/drop, not just inference. A deeper review also found that admitted peers can reach non-HTTP mesh capabilities such as Skippy stage control and plugin/subprotocol streams.This patch makes the intended contract explicit for embedded consumers:
peer_inference_only(true)limits remote peers to mesh maintenance/routing plus the OpenAI inference tunnel, and the tunnel itself allows only model discovery and inference routes. Restricted embedded runtimes also stop appending ambient plugins discovered from the process-wide installed plugin store. Local loopback APIs and the default MeshLLM CLI behavior remain unchanged.block/mesh-llmis currently behind the upstream revision Buzz consumes, so this PR includes the upstream catch-up needed to reach the affected tunnel implementation plus the narrow fix on top.Verification
cargo fmt --all --checkRUSTFLAGS="-C link-arg=-fuse-ld=ld" cargo test -p mesh-llm-host-runtime --no-default-features --features dynamic-native-runtime restricted_host_mode_does_not_import_ambient_installed_plugins --libRUSTFLAGS="-C link-arg=-fuse-ld=ld" cargo test -p mesh-llm-host-runtime --no-default-features --features dynamic-native-runtime network::tunnel --libRUSTFLAGS="-C link-arg=-fuse-ld=ld" cargo test -p mesh-llm-host-runtime --no-default-features --features dynamic-native-runtime inference_only_peer_surface_keeps_routing_but_blocks_extended_capabilities --libRUSTFLAGS="-C link-arg=-fuse-ld=ld" cargo test -p mesh-llm-host-runtime --no-default-features --features dynamic-native-runtime inference_only_peer_surface_rejects_stage_control_after_admission --libRUSTFLAGS="-C link-arg=-fuse-ld=ld" cargo check -p mesh-llm-host-runtime --no-default-features --features dynamic-native-runtimeRUSTFLAGS="-C link-arg=-fuse-ld=ld" cargo clippy -p mesh-llm-host-runtime --no-default-features --features dynamic-native-runtime --all-targets -- -D warningsRUSTFLAGS="-C link-arg=-fuse-ld=ld" MESH_LLM_SKIP_UI=1 cargo build -p mesh-llm --bin mesh-llmMESH_TWO_NODE_MAX_WAIT=240 scripts/ci-two-node-client-serving-smoke.sh target/debug/mesh-llm target/debug /tmp/SmolLM2-135M-Instruct-Q8_0.gguf/v1/modelspassedNotes
mesh-llm-host-runtimelib test suite was not used as a signal locally because many unrelated tests require socket capabilities that are blocked in the default sandbox; the focused socket tests and real two-node smoke were run outside the sandbox instead.zizmorandSemgrep OSScheck failures are annotations on imported upstream.github/workflows/*and composite action files from the Block fork catch-up, not on the runtime files changed by this boundary patch. The Block mirror needs a sync/base-strategy decision before this can merge cleanly..peer_inference_only(true)opt-in after this commit is available from the Block fork; it should not pin to a personal fork or PR ref.