Skip to content

ci: prove the relay-driven mesh lifecycle — discover, join, infer, deny — with real nodes - #3862

Open
michaelneale wants to merge 2 commits into
mainfrom
micn/mesh-relay-lifecycle-ci
Open

ci: prove the relay-driven mesh lifecycle — discover, join, infer, deny — with real nodes#3862
michaelneale wants to merge 2 commits into
mainfrom
micn/mesh-relay-lifecycle-ci

Conversation

@michaelneale

@michaelneale michaelneale commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Summary

CI now proves the full Buzz shared-compute join story end to end: a member can discover another member's served model through the Buzz relay alone and run inference over the mesh, while a non-member gets nothing — no relay visibility and no mesh admission, even holding a leaked endpoint address.

This is deliberately different from mesh-llm's own CI smokes (which bootstrap two nodes with a hand-carried invite token / mdns): here the relay is the control plane, exactly like the desktop app:

  1. Membership — identities A and B are added via buzz-admin (kind:13534 NIP-43 roster); C is not.
  2. Advertise — each member publishes a client-signed kind:30003 discovery note carrying its MeshLLM owner binding and (for the serve node) serveTargets[].endpointAddr, covered by an endpoint-binding signature — the exact payload shape the desktop coordinator publishes.
  3. Trust — the serve node derives its admission allowlist from the relay (statuses ∩ roster, owner_ids_from_events semantics) and starts with TrustPolicy::Allowlist.
  4. Join — the client verifies owner + endpoint bindings and membership, then dials the relay-discovered endpoint (the desktop join-watcher's dial_endpoint_addr step). No out-of-band token.
  5. Infer — a chat completion against the client's local OpenAI endpoint routes over QUIC to the serve node's model (CPU, SmolLM2-135M, ~105MB).
  6. Deny — the stranger's REQ returns zero mesh statuses from the membership-gated relay, and dialing the leaked endpoint directly is refused mesh admission (verified: 503, no route).

What's in the PR

  • crates/buzz-relay/examples/mesh_relay_lifecycle_smoke.rs — the harness. One process per node (mesh-llm keeps process-global state under ~/.mesh-llm), orchestrator + serve/client/stranger roles, byte-identical binding payloads to desktop/src-tauri/src/mesh_llm/identity.rs (called out with keep-in-sync comments).
  • scripts/ci-mesh-lifecycle-smoke.sh — provisions a membership-gated relay (throwaway owner + signing identities via buzz-admin generate-key), runs the harness, cleans up. Fails fast if :3000 is already occupied (a stale open relay would mask gating).
  • scripts/start-relay-for-tests.sh — gains opt-in NIP-43 membership env passthrough (BUZZ_REQUIRE_RELAY_MEMBERSHIP + RELAY_OWNER_PUBKEY + BUZZ_RELAY_PRIVATE_KEY). Default behavior unchanged.
  • .github/workflows/mesh-lifecycle.yml — separate, path-filtered, non-required workflow (mesh paths + this harness + dispatch). Caches the mesh native runtime + HF model keyed on the mesh pin.

Relationship to mesh-llm's CI

Follows the shape mesh-llm's own CI proved stable (tiny CPU model, one runner, multiple real mesh-llm processes over real QUIC — cf. their ci-two-node-client-serving-smoke.sh), but swaps the token bootstrap for the relay-driven lifecycle, which is the part only Buzz can test.

Validation

Rebased onto main after #3467 (mesh v0.74) landed — the harness rides the repo's [dev-dependencies] pin and passed the full run against v0.74.0 locally (macOS, --profile ci):

PASS 1/5: relay-derived allowlist resolved
PASS 2/5: serve member ready with model: jc-builds/SmolLM2-135M-Instruct-Q4_K_M-GGUF:Q4_K_M
PASS 3/5: client member discovered + joined via relay
PASS 4/5: inference routed over the mesh: "PONG"
PASS 5/5: stranger denied by relay and by mesh admission
PASS: full relay-driven mesh lifecycle verified

cargo fmt --all --check and cargo clippy -p buzz-relay --all-targets -- -D warnings pass.

Notes / open questions

  • Draft until the Mesh Lifecycle workflow has a green run on Actions (Linux runner; local validation was macOS. The native runtime install path is the same SDK-owned dynamic path the desktop uses, and mesh-llm's own CI runs the same model CPU-only on ubuntu-24.04).
  • The harness follows the repo's mesh pin automatically, so it doubles as a canary for future mesh upgrades (it already caught the v0.73.1 → v0.74.0 bump during development).
  • The stranger "deny" accepts either shape mesh-llm exhibits: no model visibility at all, or gossip visibility with inference refused (admission gates routing, not gossip).

permissions:
contents: read
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
@michaelneale
michaelneale force-pushed the micn/mesh-relay-lifecycle-ci branch from 152f526 to c8239e7 Compare July 31, 2026 06:21
@Chessing234

Copy link
Copy Markdown
Contributor

cool ci. if the smoke is flaky on shared runners, maybe pin a longer join timeout in the workflow rather than retrying the whole job

…inference, denial

Run the full Buzz shared-compute join story in CI with three real mesh-llm
node processes on one runner, using the Buzz relay as the control plane
(no hand-carried invite tokens):

1. membership  — A and B added via buzz-admin (kind:13534 roster); C is not
2. advertise   — members publish client-signed kind:30003 discovery notes
                 with owner + endpoint binding signatures (the exact payload
                 the desktop coordinator publishes)
3. trust       — the serve node derives its admission allowlist from the
                 relay (statuses ∩ roster) and starts TrustPolicy::Allowlist
4. join        — the client verifies bindings + membership and dials the
                 relay-discovered endpoint
5. infer       — chat completion routes client → QUIC → serve node (CPU,
                 SmolLM2-135M)
6. deny        — the stranger gets zero statuses from the membership-gated
                 relay AND is refused mesh admission with the leaked endpoint

Mirrors the shape mesh-llm's own CI uses for its two-node smokes (tiny CPU
model, one runner, real QUIC mesh), swapping mdns/token bootstrap for the
relay-driven lifecycle. start-relay-for-tests.sh gains opt-in NIP-43
membership gating env passthrough.

Signed-off-by: Michael Neale <michael.neale@gmail.com>
@michaelneale
michaelneale force-pushed the micn/mesh-relay-lifecycle-ci branch from c8239e7 to 8c56830 Compare August 1, 2026 02:14
- Stranger relay denial now requires the relay's own NIP-42 membership
  rejection ('not a relay member'); successful auth or any other connect
  failure fails the run instead of passing as denial.
- Differential admission proof: after the stranger's attempt, the trusted
  client must still route inference (VERIFY_AGAIN over stdin), so a dead
  serve node can no longer masquerade as an admission denial. Stranger
  child exit status is also checked.
- Allowlist gate now requires the exact expected {A, B} owner-id set
  (MESH_EXPECTED_OWNERS carries ids, not a count).
- Hard deadlines everywhere: child stdout is pumped through a reader
  thread (recv_timeout), wait_child kills on timeout; outer dial retries
  reduced to 3 (mesh-llm already retries internally per attempt).
- Workflow: pin ubuntu-24.04, trigger paths + cache key include
  Cargo.lock (mesh pin bumps roll the runtime cache), add buzz-admin/
  test-client/ws-client paths, upload the harness log artifact too.
- Scope note in the module docs: this is an independent protocol harness
  mirroring the desktop payloads byte-for-byte, not a link against
  desktop discovery code.

Signed-off-by: Michael Neale <michael.neale@gmail.com>
@michaelneale
michaelneale marked this pull request as ready for review August 1, 2026 04:16
@michaelneale
michaelneale requested a review from a team as a code owner August 1, 2026 04:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants