feat: Lanham style training + ingestion pipeline extensions#47
Open
ProfessorVR wants to merge 4 commits into
Open
feat: Lanham style training + ingestion pipeline extensions#47ProfessorVR wants to merge 4 commits into
ProfessorVR wants to merge 4 commits into
Conversation
…nary POS-free L2 port of the god-agent TypeScript analyzer (lanham-shared.ts). Axes split into focused submodules (lexical, noun_verb, parataxis, prosody, opacity, tacit) under a shared foundation; every public function is golden- tested against the TS reference. Includes the archon-style-train binary and output-style renderer.
…traction Token-aware chunking, structural table detection/gating, and marker-based locator extraction for document ingestion. Unit-tested with a Python parity fixture (marker_selftest.json).
Register archon-lanham in the workspace and add the `style train` subcommand that trains an output-style from sample prose via archon_lanham::train_to_output_style. Extracts StyleAction into its own cli_args module.
… kb ingest Source-verified, phased plan (PR1 MVP behind flag -> PR2 Marker routing -> PR3 flip default + S-1 parity gate) for routing the token-aware chunker into archon-docs: the synthesized-Block + ChunkOut->PageChunk adapter, the inert HARD_MAX oversized-split caveat, the injectable ChunkerKind switch, and the chunk_id/re-embed migration blast radius. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three additive commits landing together — both features are offline-only (no LLM, no network for training or chunking):
feat(lanham)archon-lanhamcrate — POS-free Lanham prose analyzer +archon-style-trainbinaryfeat(cli)archon style traincommand wired into the CLIfeat(ingest-ext)archon-ingest-extcrate — token-aware chunker, Marker walker, table gate, layout locatorFeature 1 —
archon style trainTrains an Archon output-style from sample prose by running a Lanham rhetorical analyzer over the text, writing a
.mdprofile to~/.archon/output-styles/. The analyzer is a faithful Rust port of the god-agentlanham-shared.ts, measuring eight rhetorical dimensions without a POS tagger: noun/verb ratio, parataxis/hypotaxis, periodic/running architecture, voice, register markedness (Latinate/Germanic), opacity, tacit patterns (anaphora, chiasmus, antithesis, isocolon, alliteration, polyptoton), and sentence/tone stats.Golden-tested byte-for-byte against the TypeScript reference (axes, foundation tokenization, lexical analysis, renderer parity).
Feature 2 — Ingestion extensions (
archon-ingest-ext)Four Rust-native ingestion components, each a port of the god-agent Python reference and golden-tested against the Python output:
[TABLE]chunks; never silently drops content_BEKKER_REThis crate is a standalone, additive workspace member — it is not yet wired into the live
archon kb ingestpipeline; the Python-reference golden gate (S-1) remains for wire-in time.Verification
Built and tested locally on
stable(rustc 1.96):cargo test -p archon-lanham -p archon-ingest-ext— 35 passed, 0 failedcargo build --bin archon— full binary compiles cleanarchon style trainsmoke-tested end-to-end (stdin → rendered output-style)cargo fmt -p archon-lanham -p archon-ingest-ext --check— cleancargo clippy— 3 minor advisory lints only (CI clippy is advisory, non-blocking)unsafeblocksBreaking changes
None. Both features are purely additive; the existing
archon kb ingestCLI surface and output schema are unchanged.🤖 Generated with Claude Code