Local-first codebase indexer with semantic search and chunk exports for agent consumption
Transform large codebases into searchable, intelligently-chunked datasets with real neural-network embeddings running entirely on-device. llmx ships as a native MCP server and CLI: no server-side indexing, no code upload, and no data leaving your machine.
Proof: 7,147 files indexed in 31 MB -> 1,625 tokens retrieved (99.98% savings) | 180+ tests | MCP server
Neural Semantic Search - Burn-powered mdbr-leaf-irembeddings running on-device (CPU by default, optional GPU acceleration)
Hybrid Search - Combines BM25 + vector search with RRF (Reciprocal Rank Fusion) for best results
Smart Chunking - Deterministic chunking by file type (functions, headings, JSON keys)
Semantic Exports - Hierarchical outline format with function names and heading breadcrumbs
Privacy-First - Your code stays local; the embedding model ships with the binary, so indexing makes zero network calls
Agent-Ready - MCP tools and exports designed for selective retrieval, not bulk ingestion
LLMs have limited context windows. Loading an entire codebase is:
- Token-expensive - Wastes context on irrelevant code
- Slow - Reading hundreds of files takes time
- Inefficient - Agents can't filter until after reading everything
llmx builds a searchable index with semantic chunk exports that enable agents to:
- Scan the manifest (
llm.md) to understand structure - Search for relevant concepts using BM25, vectors, or symbol lookup
- Retrieve only the specific chunks needed
- Navigate via function names, heading hierarchies, and the call/import graph
llmx exports token-efficient manifests (llm.md + manifest.llm.tsv) with semantic labels for intelligent chunk selection:
### src/auth.js (js, 47 lines)
- c0001 (1-15) `loginUser()`
- c0002 (17-30) `validateToken()`
- c0003 (32-47) `logout()`
### docs/api-reference.md (md, 234 lines)
- c0004 (1-45) API Reference
- c0005 (46-102) API Reference > Authentication
- c0006 (103-156) API Reference > Rate Limiting > Quotas
Agents can scan headings, function names, and file types to select relevant chunks--without opening any files.
cargo install --locked llmx-mcpbrew install johnzfitch/llmx/llmxyay -S llmx-bin # or paru, pakku, etc.All methods install both llmx (CLI) and llmx-mcp (MCP server).
MCP Server Setup (Claude Code, Cursor, etc.)
Create a .mcp.json in your project root for MCP discovery:
echo '{"mcpServers":{"llmx":{"command":"llmx-mcp"}}}' > .mcp.jsonThen restart your MCP client. The first session auto-starts a shared backend on localhost:19100 so multiple sessions share one index in memory instead of each loading its own copy.
| Environment Variable | Effect |
|---|---|
LLMX_PORT |
Override backend port (default 19100) |
LLMX_NO_AUTOSTART=1 |
Disable auto-start, run standalone per-session |
LLMX_STORAGE_DIR |
Override index storage location |
The server provides:
llmx_status- Index readiness, file counts, and background task progress
llmx_search- Semantic / keyword / hybrid search with token-budgeted inline content
llmx_lookup- Exact or prefix symbol resolution by name
llmx_refs- Graph traversal — callers, callees, imports, type references
llmx_explore/llmx_symbols/llmx_get_chunk/llmx_index/llmx_manage- Structure browsing, symbol tables, full-chunk fetch, and index lifecycle
Build from Source
git clone https://github.com/johnzfitch/llmx.git
cd llmx
# Default build: GPU-capable embeddings (wgpu/Metal/Vulkan) with CPU fallback
cargo build --release -p llmx-mcp --bin llmx --bin llmx-mcp
# CPU-only build: lighter and faster to compile, embeddings run on CPU
cargo build --release -p llmx-mcp --bin llmx --bin llmx-mcp \
--no-default-features --features treesitter,mcp,mcp-http,cli,ndarray-backendBinaries output to target/release/llmx and target/release/llmx-mcp. The embedding
model is committed under ingestor-core/models/, so builds are fully offline -- no model
download step.
# Index a codebase
llmx index ./my-project
# Search with token budget
llmx search "authentication login" --limit 10 --max-tokens 4000
# Explore structure
llmx explore files
llmx explore symbols --path src/
# Export for agents
llmx export --format zip -o ./export.zipOnce registered (see MCP Server Setup above), an agent calls llmx_status to check
index readiness, then llmx_search / llmx_lookup / llmx_refs to retrieve only the
chunks it needs. Indexing runs as a background job and the server stays responsive,
returning the best available results as the index warms up.
Give the agent an export bundle (compact, recommended):
llmx-export/
├── llm.md # Compact pointer manifest (recommended)
├── manifest.llm.tsv # Token-efficient chunk table for LLMs
└── chunks/
├── c0001.md # Chunk body (minimal header + content)
└── ...
Agent workflow:
- Read
llm.mdfor the compact workflow and artifact pointers - Scan
manifest.llm.tsvto identify relevant files/chunks by label - Open only the matching
chunks/<ref>.mdfiles
- Search from the CLI or via your MCP-enabled editor
- Export for offline analysis
- Share the exported
*.llmx-<id8>.zipbundle with team members (no server needed)
flowchart LR
A[Codebase] --> B[Chunker]
B --> C[Index + BM25 + Embeddings]
C --> D[llm.md manifest]
D --> E[Agent queries]
E --> F[Relevant chunks only]
llmx chunks files deterministically by type:
| File Type | Chunking Method |
|---|---|
| JavaScript/TypeScript | Function/class declarations (via tree-sitter or fallback) |
| Rust / Python / Go / Java / C / C++ / C# | Symbol-aware via tree-sitter |
| Markdown | Heading boundaries with ancestry preserved |
| JSON | Top-level keys or array ranges (max 50 elements) |
| HTML | Heading tags, scripts/styles stripped |
| Text | Paragraph boundaries |
| Images | Indexed by path, bytes included in export |
Hybrid search combining two approaches:
- BM25 (Keyword Search) - TF/IDF with document-length normalization; fast lexical matching.
- Neural Semantic Search -
mdbr-leaf-irvia Burn (768dimensions, INT8Q8Squantized), running on-device. Understands meaning, not just keywords. - RRF Fusion - Reciprocal Rank Fusion combines both rankings for results better than either method alone.
Embeddings run natively: CPU by default, or GPU-accelerated (wgpu over Metal/Vulkan/DX12)
when built with the wgpu-backend feature.
| Format | Contents | Use Case |
|---|---|---|
| llm.md | Semantic manifest with outline | Quick scanning, agent navigation |
| manifest.json | Optimized columnar format | Machine parsing, tooling |
| index.json | Full index + inverted index | Offline search, backup |
| export.zip | All above + chunk files + images | Complete portable package |
Tested on the Apple Human Interface Guidelines archive (1980-2009):
| Metric | Value |
|---|---|
| Files | 7,147 |
| Chunks | 21,369 |
| Raw size | 31 MB (~7.8M tokens) |
| Access Method | Tokens | Savings |
|---|---|---|
| Read all files | ~7,800,000 | -- |
Scan manifest (llm.md) |
~208,000 | 97% |
| Targeted search (3 queries) | ~1,625 | 99.98% |
The agent found relevant content spanning 4 decades using 0.02% of the total corpus tokens.
- Language: Rust
- Architecture: Native MCP server + CLI, with an auto-started local REST backend so multiple sessions share one in-memory index
- ML Framework: Burn (Rust-native)
- Embedding Model:
mdbr-leaf-ir(768-dim output; nativef32andq8artifacts committed underingestor-core/models/) - Embedding Backends:
ndarray(CPU) orwgpu(GPU via Metal/Vulkan/DX12) - Storage: On-disk index store (default
~/.local/share/llmx/indexes, configurable viaLLMX_STORAGE_DIR) - Search: Hybrid (BM25 + neural embeddings) with RRF fusion
- Chunking: Deterministic, content-hash based IDs
- Integrity: The model id/SHA-256 is derived at build time; indexes record the model they were built with and reject mismatches (re-index after a model change)
llmx/
├── ingestor-core/ # Rust crate: chunking, indexing, search, RRF, MCP + CLI
│ ├── src/
│ │ ├── index.rs # Indexing + hybrid search
│ │ ├── chunk/ # Per-language chunkers (tree-sitter)
│ │ ├── embeddings*.rs # Native Burn embeddings
│ │ ├── mcp/ # MCP server tools
│ │ └── bin/
│ │ ├── mcp_server.rs # llmx-mcp (MCP server + REST backend)
│ │ └── llmx.rs # llmx (CLI)
│ ├── models/ # Committed mdbr-leaf-ir artifacts (f32 + q8 + tokenizer)
│ └── build.rs # Verifies committed model + emits model id/sha256
├── docs/ # Specifications and usage guides
└── pkg/ # Packaging (Homebrew, AUR)
# Core library tests
cargo test --package ingestor-core
# CLI integration tests
cargo test --features cli
# MCP protocol tests
cargo test --features mcp180+ tests covering token savings, CLI commands, MCP protocol, edge cases, and all 30+ file types.
- Zero network calls during indexing - Your code never leaves your machine, and the embedding model ships with the binary (no runtime download)
- No external dependencies for core indexing functionality
- Content treated as untrusted - prompt-injection-resistant handling
- Deterministic output - Same input = same index every time
- Model integrity verification - the build derives a SHA-256 for the committed model; indexes are tagged with their model and refuse to serve semantic results on a mismatch
Export Format Details
Header:
# llm.md (pointer manifest)
Index ID: <sha256>
Files: 42 Chunks: 187
Chunk files live under `chunks/` and are named `{ref}.md`.
Prefer search to find refs, then open only the referenced chunk files.File sections:
### src/utils.ts (js, 89 lines)
- abc123def (1-20) `parseDate()`
- ghi456jkl (22-45) `formatCurrency()`Chunk files (chunks/<ref>.md):
---
ref: abc123def
id: <full-sha256>
slug: parseDate
path: src/utils.ts
kind: java_script
lines: [1, 20]
token_estimate: 145
heading_path: []
symbol: parseDate
---
export function parseDate(input) {
// ... function body
}MIT License - See LICENSE file for details
Contributions welcome! Please:
- Read the specs under docs/ for architecture
- Check existing issues before opening new ones
- Run tests before submitting PRs
- Follow the existing code style
Made for agents, by humans 