Upgrade llama-cpp-sys-2 when Gemma 4 12B peg-gemma4 parser fixes land upstream

[malibio]

Overview

Gemma 4 12B tool calls work correctly via the Ollama backend but fail on the GGUF/llama.cpp path. The root cause is that `llama-cpp-sys-2 0.1.146` (released 2026-04-30) predates Gemma 4 12B (released 2026-06-03) and the bundled llama.cpp has known bugs in the `peg-gemma4` parser for 12B-scale tool call payloads.

Root Cause

The `peg-gemma4` streaming parser (llama.cpp PR #21326) enters an infinite repetition loop when the model generates large tool call JSON (issue #21375 upstream). For `create_schema` with full field definitions, 12B generates ~500+ tokens of JSON; the parser re-parses on every token and never exits the tool call block. EOS (`<turn|>`) fires mid-JSON, delivering truncated `{}` args to the tool executor.

Verified in the #1329 A/B trial on M4 Pro / 48GB:

• E4B GGUF: ✅ works (generates minimal 15-token args, closes cleanly)
• 12B GGUF (Unsloth): ❌ peg-gemma4 loop, EOS at token 544 with `brace_depth=3`
• 12B via Ollama: ✅ works (Ollama has its own template handling, uses non-streaming API)

Fix

When a new `llama-cpp-sys-2` release bundles llama.cpp with the upstream peg-gemma4 fixes (PRs #21375, #21697), bump the version constraint in `packages/nlp-engine/Cargo.toml` and re-run the `aichat-matrix.ts` scenario matrix against `gemma-4-12b-unsloth-q4km` to verify.

This is a passive wait — no code change needed on our side until the crate is updated.

Acceptance Criteria

• `llama-cpp-sys-2` releases a version bundling llama.cpp with peg-gemma4 loop fix
• Bump version in `packages/nlp-engine/Cargo.toml`
• Re-run `aichat-matrix.ts 12b-gguf` — `create_schema` executes with full field definitions without truncation
• E4B GGUF behavior unchanged

Technical Specifications

Reference Files

• Cargo constraint: `packages/nlp-engine/Cargo.toml` — `llama-cpp-2 = { version = ">=0.1.146" }`
• Trial harness: `scripts/aichat-matrix.ts` — re-run against `gemma-4-12b-unsloth-q4km`
• Upstream issues: Eval bug: Infinite repetition loop in llama-server with peg-gemma4 parser during tool calls ggml-org/llama.cpp#21375 (peg-gemma4 loop), common : enable reasoning budget sampler for gemma4 ggml-org/llama.cpp#21697 (reasoning budget fix)

Watch For

• New `llama-cpp-sys-2` release on crates.io: https://crates.io/crates/llama-cpp-sys-2
• The bundled llama.cpp version must be newer than the June 3 12B release date

Related Issues

• A/B test Gemma 4 12B (Q4_K_M) vs E4B for the local ai-chat agent #1329 (A/B trial that surfaced this)
• Fix Gemma 4 12B tool-call routing: chat template mismatch causes Python-prose output instead of structured tool calls #1347 (chat_format diagnostics — already merged)
• Model-capability guardrails + larger-model prerequisites (anti-fabrication, KV-cache quant, per-model n_ctx, Gemma-12B tool parsing) #1332 (per-model n_ctx, KV cache quant — already merged)