diff --git a/.devflow/docs/reviews/fix-esc-injection-176/2026-07-25_1625/resolution-summary.md b/.devflow/docs/reviews/fix-esc-injection-176/2026-07-25_1625/resolution-summary.md new file mode 100644 index 00000000..a947d569 --- /dev/null +++ b/.devflow/docs/reviews/fix-esc-injection-176/2026-07-25_1625/resolution-summary.md @@ -0,0 +1,491 @@ +# Resolution Summary + +**Branch**: fix/esc-injection-176 -> main +**Date**: 2026-07-25 +**Review**: .devflow/docs/reviews/fix-esc-injection-176/2026-07-25_1625 +**Command**: /resolve + +## Decisions Citations + +- applies PF-014 — resolve-B2a (regression-1 cluster: sanitize inputs, never the rendered frame), resolve-B2b, resolve-B1b +- avoids PF-013 — resolve-B1a (testing-3, testing-7), resolve-B4 (python-2), resolve-B5 (consistency-1), resolve-B7 (testing-1, testing-9, testing-12), resolve-B3 (display_sanitized tests) +- avoids PF-004 — resolve-B2a (formatter.rs parallel NamedSource path), resolve-B4 (python-5 typed path), resolve-B2b (5 hand-rolled sites unified) +- applies PF-007 — resolve-B4 (typed/wire parity), resolve-B5 (native-vs-WASM differential) +- avoids PF-008 — resolve-B7 (testing-12 fail-closed gates) + +## Statistics +| Metric | Value | +|--------|-------| +| Total Issues | 110 | +| Fixed | 94 | +| False Positive | 0 | +| By Design | 0 | +| Deferred | 11 | +| Blocked | 0 | +| Escalated | 5 | + +_(Note: `Deferred` = `## Fix Separately` count + `## Deferred to Tech Debt` count combined — the two sections are distinct by scope, but the Statistics row aggregates both for the convergence parser.)_ + +## Verification +| Command | Result | +|---------|--------| +| cargo fmt --all -- --check | PASS | +| cargo clippy --workspace --all-targets -- -D warnings | PASS | +| cargo nextest run --workspace (1925 tests) | PASS | +| cargo test --doc --workspace (36 doc tests) | PASS | +| maturin develop + pytest crates/mds-python/tests (223 tests) | PASS | +| npm test -w @mdscript/mds-napi (98 tests) | PASS | +| npm test -w @mdscript/mds (263 tests) | PASS | + +Regression tests added: 27 + +Final gate: PASS + +## Fixed Issues +| Issue | File:Line | Commit | +|-------|-----------|--------| +| performance-1: sanitize_control_chars always allocates (String::with_capacity(s.len())) and copies char-by-char even when the | crates/mds-core/src/lint/diagnostic.rs:416 | ed24492 | +| rust-7: sanitize_control_chars returns String unconditionally where Cow<'_, str> is the idiomatic signature—a &str -> | crates/mds-core/src/lint/diagnostic.rs:418 | ed24492 | +| reliability-3: String::with_capacity(s.len()) is guaranteed insufficient for inputs containing control characters: every esca | crates/mds-core/src/lint/diagnostic.rs:417 | ed24492 | +| performance-7: fmt::write(&mut out, format_args!("\\u{:04X}", ch as u32)) invokes the full width/zero-pad formatting machiner | crates/mds-core/src/lint/diagnostic.rs:424 | ed24492 | +| security-3: to_canonical_json() sanitizes message and help but emits the per-file group key verbatim: "file": file | crates/mds-core/src/lint/diagnostic.rs:341 | ed24492 | +| reliability-5: file is the one user-controlled text channel left unsanitized on every JSON and typed surface | crates/mds-core/src/lint/diagnostic.rs:292 | ed24492 | +| compliance-2: The PR claim 'every serialization boundary' overstates coverage: in to_canonical_json(), message and help are | crates/mds-core/src/lint/diagnostic.rs:344 | ed24492 | +| security-8: Compile warnings are printed and serialized unsanitized at main.rs:278, :287, :340, build.rs:694, :1148, and C | crates/mds-cli/src/main.rs:278 | ed24492 | +| architecture-8: Unsanitized warning emission via emit_warnings (lib.rs:507-511), CompileResult::to_canonical_json warnings (li | crates/mds-core/src/lib.rs:507 | ed24492 | +| rust-8: let _ = fmt::write(&mut out, format_args!("\\u{:04X}", ch as u32)) discards a Result and uses fmt::write + for | crates/mds-core/src/lint/diagnostic.rs:426 | ed24492 | +| complexity-8: let _ = fmt::write(&mut out, format_args!(...)) is an unusual form that reads as though something subtle is ha | crates/mds-core/src/lint/diagnostic.rs:424 | ed24492 | +| rust-9: Public sanitize_control_chars lacks #[must_use] and a doctest, breaking mds-core's own API convention | crates/mds-core/src/lint/diagnostic.rs:418 | ed24492 | +| testing-3: The diff adds help sanitization at two new boundaries: diagnostic.rs:328 and mds-python/src/lib.rs:784 | crates/mds-core/src/lint/diagnostic.rs:328 | ed24492 | +| testing-7: Idempotency is claimed in three places (diagnostic.rs:414, output.rs:494, and the KB) and asserted nowhere | crates/mds-core/src/lint/diagnostic.rs:414 | ed24492 | +| regression-1: Whole-frame sanitization escapes miette's own ANSI colour codes, so interactive mds lint and mds watch output | crates/mds-cli/src/lint.rs:289 | 5d35da0 | +| security-1: Whole-frame sanitization mangles miette's own ANSI styling: interactive mds lint / mds watch output is now lit | crates/mds-cli/src/lint.rs:289 | 5d35da0 | +| security-2: Caret/underline is misaligned whenever the source line contains a control char | crates/mds-cli/src/output.rs:495 | 5d35da0 | +| reliability-1: Post-render sanitization desynchronizes miette's caret/underline from the source line it points at | crates/mds-cli/src/output.rs:496 | 5d35da0 | +| reliability-2: render_result_human iterates result.diagnostics (capped at MAX_DIAGNOSTICS=1000) | crates/mds-cli/src/lint.rs:283 | 5d35da0 | +| architecture-3: The field-level pre-sanitize in render_diag_human builds a full LintDiagnostic clone to sanitize message and h | crates/mds-cli/src/lint.rs:266 | 5d35da0 | +| complexity-4: The comment at lint.rs:266-268 claims the double-pass 'guards against any future path that might bypass the ou | crates/mds-cli/src/lint.rs:266 | 5d35da0 | +| documentation-8: The comment at lint.rs:266-268 justifies the field-level sanitize as guarding 'against any future path that mi | crates/mds-cli/src/lint.rs:266 | 5d35da0 | +| rust-2: The comment at lint.rs:265-267 claims the inner field-level pass 'guards against any future path that might by | crates/mds-cli/src/lint.rs:265 | 5d35da0 | +| reliability-4: The field-level pass builds a complete LintDiagnostic clone including span.clone(), file.clone(), fix_removals | crates/mds-cli/src/lint.rs:266 | 5d35da0 | +| performance-3: The redundant field-level sanitize pass in render_diag_human is fully subsumed by the whole-report pass in epr | crates/mds-cli/src/lint.rs:272 | 5d35da0 | +| rust-1: render_diag_human builds a render-only sanitized local with a full-field struct literal that includes fix_remo | crates/mds-cli/src/lint.rs:269 | 5d35da0 | +| performance-6: The sanitized LintDiagnostic clone at lint.rs:276-277 clones fix_removals and fix_edits (Vec, each w | crates/mds-cli/src/lint.rs:276 | 5d35da0 | +| testing-4: The named_source = None branch of render_diag_human is unreachable | crates/mds-cli/src/lint.rs:288 | 5d35da0 | +| documentation-6: output.rs:509-511 claims 'miette box-drawing and carets therefore survive intact' and lint.rs:284-287 claims ' | crates/mds-cli/src/output.rs:509 | 5d35da0 | +| testing-2: All five new e2e vectors force NO_COLOR=1 deliberately, so the suite has zero coverage of the default interact | crates/mds-cli/src/lint.rs:283 | 5d35da0 | +| regression-3: All five new ESC tests force NO_COLOR=1 deliberately, making the suite pin only the non-default configuration | crates/mds-cli/tests/cli_lint.rs:343 | 5d35da0 | +| testing-10: T-10 depends on ambient colour configuration: format!("{report:?}") uses miette's global handler | crates/mds-cli/src/output.rs:812 | 5d35da0 | +| reliability-8: T-10's assertion set (no 0x1B, contains \u001B, contains Hello) cannot distinguish a correct frame from a care | crates/mds-cli/src/output.rs:812 | 5d35da0 | +| testing-5: Two defence-in-depth layers added in this diff cannot be distinguished from their own absence by any test: (1) | crates/mds-cli/src/lint.rs:271 | 5d35da0 | +| rust-5: After this PR the crate ships two text accessors with opposite safety properties: e.serialize().message is san | crates/mds-core/src/error.rs:826 | 5d35da0 | +| architecture-2: MdsError derives Display via thiserror and is publicly re-exported; a downstream Rust consumer writing the nat | crates/mds-core/src/error.rs:826 | 5d35da0 | +| consistency-7: error.rs uses crate::lint::sanitize_control_chars(...) inline twice at lines 826 and 828; the symbol is re-exp | crates/mds-core/src/error.rs:826 | 5d35da0 | +| rust-12: error.rs uses crate::lint::sanitize_control_chars(...) inline at lines 826 and 828 while crate::sanitize_contr | crates/mds-core/src/error.rs:826 | 5d35da0 | +| compliance-3: err.detail bypasses the new MdsError::serialize() choke-point: it is set by the catch_unwind wrappers outside | crates/mds-napi/src/lib.rs:310 | 5d35da0 | +| python-1: Change #2 breaks typed-vs-wire parity: message/help are sanitized when populating the typed pyclass at lines 7 | crates/mds-python/src/lib.rs:780 | dfb21a5 | +| architecture-4: The Python belt-and-suspenders guard re-sanitizes json_str(d, "message") where d is already a serde_json::Valu | crates/mds-python/src/lib.rs:776 | dfb21a5 | +| python-4: The code comment at lib.rs:775-779 claims the wrap 'ensures as_json()/to_dict() parity for any future code pat | crates/mds-python/src/lib.rs:775 | dfb21a5 | +| performance-4: message: mds::sanitize_control_chars(&json_str(d, "message")) causes two wasted allocations per diagnostic fie | crates/mds-python/src/lib.rs:780 | dfb21a5 | +| rust-4: message: mds::sanitize_control_chars(&json_str(d, "message")) double-allocates: json_str already returns an ow | crates/mds-python/src/lib.rs:779 | dfb21a5 | +| python-5: LintFileReport.file carries raw C0/DEL/C1 bytes—json_str(file_val, "file") is populated unsanitized in the sam | crates/mds-python/src/lib.rs:752 | dfb21a5 | +| python-2: E12 cannot fail if the pyclass sanitize wrap at lib.rs:780/784 is deleted—it validates core B4, not the Python | crates/mds-python/tests/test_errors.py:232 | dfb21a5 | +| python-6: The (b) comment block inside E12's first loop sits inside for diag in all_diags: but the (b) assertion actuall | crates/mds-python/tests/test_errors.py:274 | dfb21a5 | +| python-7: Only ESC (U+001B) is exercised on the Python surface | crates/mds-python/tests/test_errors.py:200 | dfb21a5 | +| python-3: E12 docstring states the vector uses 'a module whose NAME contains a raw ESC byte', but the resulting file key | crates/mds-python/tests/test_errors.py:272 | dfb21a5 | +| consistency-1: Three lint-path binding tests accept either escape casing via || alternatives: napi T-13b, Python E12, and WAS | crates/mds-napi/__test__/index.spec.mjs:1272 | 555095f | +| consistency-2: The vector-label scheme has three conflicting accountings and T-11 does not exist | crates/mds-napi/__test__/index.spec.mjs:1193 | 555095f | +| testing-11: Binding surfaces test ESC only (U+001B); DEL/C1 stop at core + CLI | crates/mds-napi/__test__/index.spec.mjs:1200 | 555095f | +| testing-6: Six per-surface goldens exist but zero differential assertions | crates/mds-napi/__test__/index.spec.mjs:1275 | 555095f | +| consistency-5: The control-char assertion predicate is spelled four ways with naming inconsistency: napi uses assertNoControl | crates/mds-napi/__test__/index.spec.mjs:1203 | 555095f | +| complexity-3: The control-byte assertion predicate (C0-excl-\t\n / DEL / C1) appears six times across the PR | crates/mds-wasm/tests/web.rs:790 | 555095f | +| consistency-6: The feature-KB anchor note states E11/E12 live in test_errors.py/test_lint.py, but crates/mds-python/tests/tes | .devflow/features/mds-lint/KNOWLEDGE.md | 555095f | +| architecture-6: Five sibling sites (lint.rs:95, lint.rs:1408, main.rs:232, main.rs:348, build.rs:1548) still hand-roll eprintl | crates/mds-cli/src/lint.rs:95 | 9b1e3c3 | +| security-7: mds build / mds check already mangle miette colours via pre-existing sanitize_control_chars(&format!("{e:?}")) | crates/mds-cli/src/main.rs:232 | 9b1e3c3 | +| security-5: mds lint status lines print attacker-controlled filenames raw at lint.rs:792, :820, :843, :866, :1163, :1300, | crates/mds-cli/src/lint.rs:792 | 9b1e3c3 | +| architecture-10: ESC-bearing filenames are a distinct unaddressed vector: eprintln!("error writing {}: {e}", file.display()) at | crates/mds-cli/src/lint.rs:1136 | 9b1e3c3 | +| security-6: Same raw-filename pattern across fmt.rs (lines 61, 178, 181, 189, 194, 272, 318) and build.rs (lines 564, 1028 | crates/mds-cli/src/fmt.rs:61 | 9b1e3c3 | +| regression-4: mds check, mds build, and mds fmt already mangle colour via pre-existing eprint_error and inline-sanitize site | crates/mds-cli/src/fmt.rs:232 | 9b1e3c3 | +| documentation-12: eprint_error's doc states handlers 'MUST use this helper' and that centralizing the render 'means the sanitize | crates/mds-cli/src/output.rs:501 | 9b1e3c3 | +| consistency-4: eprint_error call style forks within this PR: watch.rs:42 adds eprint_error to its use crate::output::{...} bl | crates/mds-cli/src/watch.rs:42 | 9b1e3c3 | +| rust-3: render_error_sanitized is pub(crate) but its only non-test caller is eprint_error at output.rs:513—same module | crates/mds-cli/src/output.rs:495 | 9b1e3c3 | +| reliability-7: render_error_sanitized doc claims 'The render+sanitize pass is idempotent: calling it a second time on already | crates/mds-cli/src/output.rs:492 | 9b1e3c3 | +| documentation-14: render_diag_human's summary line still reads 'applying sanitize_control_chars at the boundary', written for th | crates/mds-cli/src/lint.rs:256 | 9b1e3c3 | +| documentation-4: 'ALL serialization and render boundaries' is factually false on two counts: (1) CompileResult::to_canonical_js | crates/mds-core/src/lint/diagnostic.rs:7 | 9ef6576 | +| architecture-1: The rewritten module doc asserts sanitization is applied at 'ALL serialization and render boundaries' and enum | crates/mds-core/src/lint/diagnostic.rs:7 | 9ef6576 | +| consistency-3: Two boundary lists in the same file (module header lines 7-14 and sanitize_control_chars fn doc lines 405-407, | crates/mds-core/src/lint/diagnostic.rs:7 | 9ef6576 | +| documentation-7: The module header (lines 7-17) and the sanitize_control_chars fn doc (lines 405-408), both rewritten in this P | crates/mds-core/src/lint/diagnostic.rs:405 | 9ef6576 | +| documentation-3: Line 135 reads '**Sanitization**: apply sanitize_control_chars at the CLI render boundary **only**.' This is f | crates/mds-core/src/lint/diagnostic.rs:135 | 9ef6576 | +| rust-6: LintDiagnostic struct doc at line 135 reads '**Sanitization**: apply sanitize_control_chars at the CLI render | crates/mds-core/src/lint/diagnostic.rs:135 | 9ef6576 | +| architecture-7: Line 135 still reads '**Sanitization**: apply sanitize_control_chars at the CLI render boundary only.' The mod | crates/mds-core/src/lint/diagnostic.rs:135 | 9ef6576 | +| documentation-2: The LintDiagnostic doc comment reads: 'Implements std::error::Error + miette::Diagnostic so it can be rendered | crates/mds-core/src/lint/diagnostic.rs:123 | 9ef6576 | +| complexity-5: LintDiagnostic struct doc at line 123 still reads 'so it can be rendered by miette at the CLI boundary: eprint | crates/mds-core/src/lint/diagnostic.rs:123 | 9ef6576 | +| documentation-16: The field doc at line 143 reads 'Raw — do not sanitize in the constructor' which is accurate and should stay, | crates/mds-core/src/lint/diagnostic.rs:143 | 9ef6576 | +| documentation-11: sanitize_control_chars is now a load-bearing public API of the crates.io-published mds-core, re-exported at th | crates/mds-core/src/lint/diagnostic.rs:401 | 9ef6576 | +| testing-1: T-9 is fully vacuous for three independent reasons: (1) the ESC never reaches any output field—serde_yaml_ng r | crates/mds-cli/tests/cli_lint.rs:1795 | a2a9cb0 | +| documentation-15: T-9's redesign rationale (that serde_yaml_ng rejects raw ESC in YAML frontmatter keys so T-9 became a wire-lev | crates/mds-cli/tests/cli_lint.rs:1784 | a2a9cb0 | +| testing-12: T-9's Gate 3 fails open: if let Some(files) = json["files"].as_array() silently skips the whole check when the | crates/mds-cli/tests/cli_lint.rs:1841 | a2a9cb0 | +| complexity-1: T-9's Gate 2 iterates stdout_str.bytes() and tests (0x80..=0x9F).contains(&byte) for C1 | crates/mds-cli/tests/cli_lint.rs:1827 | a2a9cb0 | +| complexity-2: Four new tests (lines 1614, 1663, 1700, 1757) hand-roll command construction that lint_path and lint_stdin hel | crates/mds-cli/tests/cli_lint.rs:1614 | a2a9cb0 | +| complexity-9: let stderr = out.stderr.clone() clones the buffer needlessly in four new tests (lines 1622, 1671, 1717, 1767) | crates/mds-cli/tests/cli_lint.rs:1622 | a2a9cb0 | +| testing-9: Pre-existing ESC test lint_esc_byte_in_syntax_error_is_sanitized_on_stderr (unchanged by this PR) calls lint_p | crates/mds-cli/tests/cli_lint.rs:1563 | a2a9cb0 | +| testing-8: A3 (11 watch.rs call sites) ships with no test coverage: crates/mds-cli/tests/cli_watch.rs is untouched and no | crates/mds-cli/src/watch.rs:790 | a2a9cb0 | +| documentation-1: CHANGELOG.md is untouched on this branch: grep for 'sanitiz', 'control char', 'CWE-150', '#176', and 'injectio | CHANGELOG.md | ae0fd23 | +| compliance-1: git diff main...HEAD touches 15 files and zero of them is CHANGELOG.md | CHANGELOG.md | ae0fd23 | +| regression-2: The PR body states a Behavioral change noting err.message, err.help, and lint diagnostic messages now carry \u | CHANGELOG.md | ae0fd23 | +| documentation-5: spec.md §7.5 is the normative wire-format documentation for mds lint --format json | spec.md:971 | ae0fd23 | +| architecture-5: to_canonical_json() now mutates the value domain of message/help while the envelope version stays 1 | crates/mds-core/src/lint/diagnostic.rs:327 | ae0fd23 | +| documentation-9: The PR body states '@mdscript/bundler-utils (normalizeError) .. | (none) | ae0fd23 | +| regression-5: The PR body names the bundler entry point normalizeError; the actual export is formatMdsError (packages/bundle | (none) | ae0fd23 | +| documentation-10: The PR body claims '15 test vectors added across all surfaces' and maps T-11..T-15 to four binding surfaces (f | (none) | ae0fd23 | + +## False Positives +| Issue | File:Line | Reasoning | +|-------|-----------|-----------| + +(none — every spot-checked reviewer claim held) + +## By Design +| Issue | File:Line | Rationale (ADR/doc) | +|-------|-----------|---------------------| + +(none) + +## Fix Separately +| Issue | File:Line | Reason | Tracked | +|-------|-----------|--------|---------| +| performance-2: render_diag_human clones the entire source file for every single diagnostic via src.to_str | crates/mds-cli/src/lint.rs:285 | src.to_string() per diagnostic identical to main; Arc refactor is its own ticket | #255 | +| performance-5: accumulate_result_json deep-clones every diagnostic Value via json_files.extend(arr.iter() | crates/mds-cli/src/lint.rs:1416 | accumulate_result_json deep clone, pre-existing; sibling to #173 | #173 (comment) | +| complexity-6: run_watch_file is ~311 lines, fully pre-existing | crates/mds-cli/src/watch.rs:886 | run_watch_file ~311 lines, fully pre-existing; PR changed one line | #256 | +| complexity-7: eprint_error + error-settle pattern is repeated in two identical pairs: watch.rs:869/:877 | crates/mds-cli/src/watch.rs:869 | watch.rs duplicate pairs; restructuring cost exceeds benefit in this PR | #257 | +| reliability-6: eprintln! panics with 'failed printing to stderr' if the write fails (e.g | crates/mds-cli/src/output.rs:513 | eprintln! EPIPE panic — every site was already eprintln! on main | #258 | +| rust-10: Public LintDiagnostic is not #[non_exhaustive] while its sibling public Message is | crates/mds-core/src/lint/diagnostic.rs | #[non_exhaustive] on LintDiagnostic — PRE-TAG deadline: free at zero users, breaking after v0.4.0 | #259 | +| rust-11: format!("{report:?}") materializes the entire rendered frame, then sanitize_control_chars | crates/mds-cli/src/output.rs:495 | double materialization; likely moot after PF-014 redesign — re-check then close or fix | #260 | +| architecture-9: render_error_sanitized name describes the transformation but not that it is the boundary; | crates/mds-cli/src/output.rs:495 | rename render_error_sanitized — revisit now that PF-014 reshaped it | #261 | +| architecture-11: sanitize_control_chars lives in the lint module but is now a core cross-cutting concern: e | crates/mds-core/src/lint/diagnostic.rs:416 | move sanitizer to mds-core/src/sanitize.rs; orthogonal to closing #176 | #262 | +| documentation-13: spec.md documents the lint JSON wire format (§7.5) but has no comparable normative section | spec.md | normative spec section for serialized error shape; pre-existing gap | #263 | +| python-8: E11 uses try/except/else + pytest.fail rather than pytest.raises, which is less idiomatic | crates/mds-python/tests/test_errors.py:246 | pytest.raises idiom conditioned on file-wide style migration | #264 | + +## Deferred to Tech Debt +| Issue | File:Line | Risk Factor | +|-------|-----------|-------------| + +(none — all deferrals are scoped FIX_SEPARATE tickets) + +## Escalations +| Issue | File:Line | Security Concern | Decision (2026-07-25) | +|-------|-----------|-----------------|----------------------| +| security-4: The sanitizer character class is C0 (minus \n/\t), DEL, and C1 | crates/mds-core/src/lint/diagnostic.rs:416 | Sanitizer omits bidi/Trojan Source U+202E (CVE-2021-42574) + U+2028/29 — widening the escape class is a wire-format decision | **WIDEN** (implemented 2026-07-26): extend escape class to bidi/Trojan-Source characters (U+200E/200F, U+202A–202E, U+2066–2069) and JS-hazard separators/BOM (U+2028/2029, U+FEFF), escaped as the existing uppercase `\uXXXX` literal form. Rationale: CVE-2021-42574 class; these pass through C0/DEL/C1 filtering untouched. | +| security-9: \n pass-through permits diagnostic-line forging in unwrapped consumers: a YAML key 'a\nerr | crates/mds-core/src/lint/diagnostic.rs:419 | The \n carve-out enables CWE-117 log-record forging in unwrapped consumers — keep or close is an owner decision | **ESCAPE `\n` ON WIRE ONLY** (implemented 2026-07-26): wire/API/JSON surfaces escape newline as `\n` literal to close CWE-117 log-record forging; the CLI human render keeps real newlines so multi-line diagnostics still render. One shared escape map behind a mode flag — deliberately NOT two tables. | +| security-10: Sanitization is not injective: source text containing the 6 literal chars \u001B is indist | crates/mds-core/src/lint/diagnostic.rs:416 | Sanitizer is non-injective (no backslash escaping); fixing churns the wire format on all 5 surfaces pre-tag | **ACCEPT + DOCUMENT** (joint decision with reliability-9, implemented 2026-07-26): escaping stays one-way, lossy and non-injective. A literal 6-character `\u001B` string in template source and a real ESC byte are indistinguishable after sanitization. The contract now explicitly forbids consumers un-escaping `\uXXXX` back to bytes; round-tripping is a permanent non-goal. No backslash escaping — it would churn the wire format across five surfaces for no security gain. Documented normatively in `spec.md` §7.5 and in `sanitize_control_chars` rustdoc. | +| reliability-9: sanitize_control_chars is not injective: sanitize_control_chars("\\u001B") and sanitize_co | crates/mds-core/src/lint/diagnostic.rs:416 | Duplicate of security-10 — decide once | **ACCEPT + DOCUMENT** (same decision as security-10 — decided as one, implemented 2026-07-26). See security-10 row for full rationale. | +| security-11: --diff / --check output echoes raw source lines to the terminal at lint.rs:1431 and fmt.rs | crates/mds-cli/src/lint.rs:1431 | --diff/--check echo raw source bytes to stdout by design; explicit accept-or-fix decision requested | **TTY-GATED NEUTRALIZE** (implemented 2026-07-26): `--fix --diff` preview output neutralizes control bytes when stdout is a terminal, and stays byte-faithful when piped so diffs remain usable by `patch`/tooling. NO_COLOR does not affect it — this is safety, not styling. Note precisely: neutralization applies to the `--diff` preview text; `--check` alone emits only `Would fix:`/`Would reformat:` status lines, which are unconditionally sanitized via `safe_path` and are not TTY-gated. | + +> **Decisions recorded 2026-07-25; implementation landed 2026-07-26.** Originally escalated as escape-map / wire-contract decisions (bidi coverage, \n carve-out, non-injectivity, --diff raw echo). All five decided as a batch in the last free wire-format window: zero users, pre-v0.4.0-tag, so wire-format churn costs nothing now and would be breaking later. + +**Implementing commits (branch `fix/esc-injection-176`):** `d5c7975` (core: widened class + wire mode), `3c383ad` (cross-surface tests), `7f26e3f` (spec §7.5 + CHANGELOG), `5fa54f4` (TTY-gated preview), `4f591c7` (diff-renderer consolidation), `ad2a673` (WASM parity test), `61120e7` (strip raw control bytes from comments), `ba5dde0` (P0: SanitizedReport at the stderr choke-point), `f2e2874` (boundary-table closure), `cb6d860` (eprint_warning). + +**Mid-flight expansion (2026-07-26, owner-approved):** After the five escalation decisions were implemented, a Scrutinizer pass demonstrated a raw-ESC-to-stderr path in `MdsError` message text on the CLI human error path (found by the Scrutinizer), plus the CLI-authored `miette!()` error family and warning prints. Fixing these was approved mid-flight as an addition to the original five escalations — not one of the five — and committed to the same branch. + +**Alignment-review finding (round 1, closed):** A subsequent alignment review found the boundary-closure claim still incomplete: a bare `eprintln!` for unknown `mds.json` rule names at `crates/mds-cli/src/lint.rs:197`, and raw `MdsError` Display interpolated into `fix rejected:` reasons. Both were fixed (`e145e41`, `46fb326`) and the claims narrowed (`e1d1d73`, `35195d4`). + +--- + +### Decision 3 RE-RATIFIED as a per-field rule (2026-07-26) + +A **second** alignment review falsified the narrowed warning-path claim again — a third distinct unescaped print (`output.rs`'s own walker depth-limit warning) plus the discovery that the round-1 `lint.rs` fix used HUMAN mode, so a newline in the rule name still forged standalone status lines. Two rounds of "fix the enumerated sites" had each been correct and each been superseded. + +The owner therefore **re-ratified Decision 3** in a stronger, per-field form, **superseding the earlier "wire mode at exactly four boundaries" enumeration**: + +> **Untrusted identifiers and filenames are WIRE-escaped on every surface, human output included. Prose (diagnostic message / help bodies) stays HUMAN so multi-line frames keep rendering.** + +Rationale: a filename or a config key is never legitimately multi-line, so preserving `\n` in one only enables status-line forgery (CWE-117); a diagnostic body legitimately *is* multi-line. This makes each remaining site decidable **by rule** rather than by re-deriving a list of boundaries. Recorded normatively in `spec.md` §7.5 and in the `crates/mds-core/src/lint/diagnostic.rs` module doc. + +### Systemic guard approved and landed (2026-07-26) + +The owner also approved a **systemic guard** as the deliverable that ends the whack-a-mole: `crates/mds-cli/tests/print_discipline.rs` fails CI if any print macro under `crates/mds-cli/src/**` interpolates a value that is not passed through one of the escape helpers, and applies the same rule to `format!` invocations nested inside `eprint_warning` calls. Exceptions live in an explicit allowlist with a written justification per entry; a companion test fails if an allowlist entry stops matching. + +Consequences recorded here because they change earlier decisions: + +- **`watch.rs` is no longer carved out.** Previously documented as a pre-existing gap outside #176's diff, its lifecycle status lines are now routed through `safe_path` / `safe_inline` / `eprint_warning`. Allowlisting them would have been a deliberate hole in the guard. +- **`eprint_warning` alone is explicitly not sufficient**, and is no longer documented as if it were. The boundary table now records its row as *prose HUMAN, interpolated identifiers/paths WIRE*. +- **Known residual, deliberately not claimed closed:** CLI `miette::miette!(…)` message construction. Those reports are HUMAN-escaped at `eprint_error` before miette renders them, so no raw control byte reaches stderr, but a `\n` in an interpolated path survives inside the rendered (indented, box-drawn) frame. + +**Round-2 implementing commits:** see the branch log for `fix/esc-injection-176` after `35195d4`. + +### Round 3 — the guard hardened, the normative claim narrowed (2026-07-26) + +A **third** adversarial alignment review attacked the guard itself rather than the +prints, and falsified the spec's central claim. Both are closed here; this is intended +as the final code round. + +**B1 (critical) — hoisting `format!` into a `let` defeated the guard.** `collect_sites` +looked for `format!` only where it appeared lexically inside the `eprint_warning(...)` +parens, so `let msg = format!("... {name}"); eprint_warning(&msg);` — completely +idiomatic — reintroduced M2 verbatim and invisibly. **B2 — `eprint_warning()` was unchecked**, with five live instances (`build.rs:711`, `build.rs:1166`, +`main.rs:279`, `main.rs:288`, `main.rs:341`). + +Fixed together. The helper's argument is now classified in its own right: a string +literal, a whole-expression sanitizer call, or a `format!` whose interpolations are each +accepted. A bare local is traced **one hop** through its `let` binding in the same file +and judged by the same rule. Anything unresolved is **reported, not trusted** — the +guard fails closed. Disposition of the five bare-`w` sites: **allowlisted with written +justification** in a new `ALLOWED_UNTRACED_HELPER_ARGS` — **two entries** (`build.rs`/`w` +and `main.rs`/`w`) covering all five sites, since the list is keyed by `(file, +expression)` — kept separate from the general +allowlist so the exemption applies *only* in the helper-argument position. The +justification states plainly what the reviewer observed: their safety rests on mds-core +producer discipline (`resolver.rs`, `evaluator.rs` WIRE-escape at construction), which +this lexical guard cannot verify across a crate boundary. Nothing mechanical holds it; +that is now written down rather than implied. + +**B3 — `is_sanitizer_call` accepted postfix continuations** (`safe_path(p) + &evil`, +`safe_path(p).replace("a", &evil)`), contradicting the guard's own self-test, which +asserted the property but tested only the prefix direction. The call must now be the +whole expression; the self-test covers both directions and asserts strictly more than +before. **B4 — `write!` / `writeln!` to a stream handle was unscanned** (latent: zero +instances in the crate). Now scanned when the first argument names stdout/stderr, +including through a `let` binding. + +**B5 / B6 accepted as limits, not chased**, and stated in the guard's own rustdoc under +"Accepted limits": sanitizers are matched by the last path segment (an alias, or a local +`fn safe_path`, defeats it); allowlist entries are **anti-rot, not anti-reuse** (keyed by +`(file, expression)`, so a future variable reusing an exempted name in the same file +inherits the exemption). Added alongside them: the trace is one hop within one file, and +stream detection is by name. The rustdoc now says outright that this is a lexical +scanner whose bar is *accidental* reintroduction, and that closing the remaining gaps +would need a rustc lint or a `syn`-based HIR analysis. + +Each of B1-B4 was proven by **injecting the bypass into real source**, confirming the +guard fails naming the exact site, then reverting. + +### Normative claim narrowed — option (b), with the mds-core residual named + +spec 7.5 asserted filenames, paths and causes are "**WIRE everywhere**". Falsified: +mds-core `MdsError` message bodies interpolate exactly those values and stay HUMAN on +terminal surfaces (`fs.rs:478` `cannot read {normalized}: {e}`, `:487` invalid-UTF-8, +`parser_helpers.rs:853` `invalid import alias: '{alias}'`). The declared residual had +also been scoped to *CLI `miette!()` construction* only, understating it — the same +defect exists at a second construction site of equal severity. + +**Chose (b) — narrow the claim — over (a) — make it true.** The blast radius of (a) is +not small: 110+ `MdsError::*(format!(...))` construction sites across `parser_helpers.rs`, +`evaluator.rs`, `resolver.rs`, `builtins.rs`, `fs.rs` and `lib.rs`, changing the public +`MdsError` message text seen by all three binding layers (a further breaking wire change +beyond what is already declared) and churning goldens in four suites. Decisively: fixing +only the two sites the reviewer named would leave the claim false at ~100 others — the +exact overclaim that reopened this issue twice already. + +The rule is therefore stated per FIELD, precisely: a path in a `file` **field** (CLI +status line, `[file:line:col]` header, JSON `file` key) is WIRE on every surface; a path +or identifier interpolated into a message **body** is prose and follows the message row. +The residual is named in all four places — a new "Residual: paths and identifiers inside +a message body" section in spec 7.5, the boundary table in +`crates/mds-core/src/lint/diagnostic.rs`, a "Declared residual" paragraph in the +CHANGELOG, and this ledger. The reviewer's characterization is preserved: frame content +is indented and box-prefixed, and the prefix survives `strip()`, so it cannot masquerade +as a bare status line — the surface is genuinely weaker, only its scope was understated. + +**Documentation overclaims corrected:** "All serialization and diagnostic-render +boundaries are now hardened" (closed-set-by-enumeration, in the file that retires +enumeration) is now an audit list; "Human-render output is unchanged" is scoped to +diagnostic prose; "sanitizes renderer inputs byte-length-preservingly" is scoped to +source text (`message` / `help` are escaped to `\uXXXX` literals, not length-preserved); +"all five surfaces" corrected to four; two bidi rustdoc lists that omitted U+061C (11 of +12) completed; and the escape class, previously defined as "C0 except `\n` / `\t`" in +`diagnostic.rs` against the spec's "`\n` in class, `\t` sole exemption", now reads the +same in both — `\n` is in the class, and HUMAN/WIRE is the mode choice, not a class +difference. + +**Round-3 implementing commits:** `0e79385` (guard hardening), `c973331` (normative +claim + overclaims). + +--- + +## Round 4 (2026-07-26) — adversarial pass #4, and the final code round + +A fourth adversarial pass found one **reproduced** coverage gap and a set of sentences +that claimed more than the code delivered. Every prior round of #176 died on that same +failure mode, so this round's rule was: **narrow the wording to what is demonstrably +true; add no new absolute claim anywhere.** + +### C1 — source-map sidecar carries raw control bytes (REPRODUCED) + +`mds build --source-map` on a file named `evil[31m.mds` writes a sidecar whose +decoded `file` / `sources` hold a real newline and a real ESC, while the CLI status lines +of the same run are correctly escaped: + +``` +# both lines describe the SAME run, on a file whose name holds a real LF and a real ESC +stderr : Compiled to .../ev\u000Ail\u001B[31m.md <- escaped +sidecar : {"version":3,"file":"ev\nil\u001b[31m.md", ...} +python : json.load(sidecar)['file'] -> real newline: True, real ESC: True +``` + +That falsified the shared per-field sentence at seven sites (`spec.md` x3, +`diagnostic.rs`, `CHANGELOG.md`, this ledger, `KNOWLEDGE.md`). + +**Decision — carve source maps out; do NOT escape them.** Source Map v3 `file` / +`sources` are *functional path references*: devtools, bundlers and IDEs resolve them +against the filesystem. WIRE-escaping one to a `\uXXXX` literal would make the map point +at a path that does not exist — breaking source-map resolution and dependency tracking in +order to defend against a pathological filename. It is the same product-versus-display +distinction that already keeps the `compiled` stdout allowlist entry unescaped: escaping +the artefact corrupts the artefact. + +Implemented as an explicit, **named third carve-out** stated at every site where the +per-field rule appears — `spec.md` §7.5 (new "Carve-out: functional path references" +subsection, plus the `file` invariant row, the governing blockquote, the supersession +paragraph and a new row in the mode table), `crates/mds-core/src/lint/diagnostic.rs` +("three categories remain outside the table"), `CHANGELOG.md` ("Declared carve-out"), +`.devflow/features/mds-lint/KNOWLEDGE.md`, and this section. Scope covers source-map +`file` / `sources` / `sourcesContent` **and** `CompileResult.dependencies`, in the sidecar +and in the `sourceMap` embedded in `CompileResult::to_canonical_json()`. The contract is +stated one-way and normatively: **consumers MUST treat these paths as untrusted**; JSON +string encoding is not escaping, since a decoded `"\n"` is a real newline again. Rustdoc +now says so on `SourceMap` and on `CompileResult::dependencies` / `to_canonical_json`, +where a consumer will actually read it. + +The per-field sentence itself was **narrowed** everywhere from "on every surface" to "on +the diagnostic surfaces — the `"version": 1` JSON wire, CLI status and warning lines, +`[file:line:col]` frame headers". `CHANGELOG.md`'s "every wire boundary" became "the +diagnostic wire boundaries"; "escaped … on every surface" became "on every surface that +renders one"; "display-hazardous, on every surface" became "on every surface that +escapes". + +**Input-boundary rejection — assessed, not implemented.** Rejecting control characters in +filenames at the input boundary (the tree walk and the `@import` path parser) would be a +strictly stronger defence than output escaping: it fails closed once, at one place, +instead of requiring every output site to remember. It would also remove the need for the +carve-out to be a hazard at all, since the paths reaching a source map could then not +carry the bytes. It is **out of scope here** — it changes which inputs compile at all +(a breaking behavioural change), needs its own AC on `/`-and-NUL-only POSIX semantics +versus a stricter allowlist, and belongs in a separate issue. Recorded as an assessment, +not a plan. + +### C2 — two FALSE statements inside the guard's own security justification + +`print_discipline.rs` claimed the mds-core producer precondition was "upheld by … mds-core's +own tests." **No such test existed**; `source_map_vfs.rs:967` only asserts +`w.contains("segment cap")`. It also named an `evaluator.rs` producer of "rejected-value +text" that does not exist — both evaluator sites interpolate only `inc.alias`. + +Rather than only striking the clauses, the precondition was **made true where it can be**. +`mds-core` has exactly three warning producers that interpolate a runtime value: + +| Producer | Hostile input reachable? | Status now | +|---|---|---| +| `resolver.rs:1069` imported-module filename | **Yes** — a module key is a filesystem path | **Tested**: new `crates/mds-cli/tests/producer_discipline.rs` | +| `evaluator.rs:1148` `@include` alias | No — parser requires `[A-Za-z_][A-Za-z0-9_]*` | Upheld by review, stated as such | +| `evaluator.rs:1305` `@include` alias | No — same gate | Upheld by review, stated as such | + +The test lives in `mds-cli`, not `mds-core`, deliberately: it asserts the precondition on +the exact value `build.rs` / `main.rs` hand to `eprint_warning` (`for w in +&result.warnings`), which is what the `ALLOWED_UNTRACED_HELPER_ARGS` entries depend on — +and it keeps `mds-core` free of executable change, so the binding suites do not need to +re-run. Vector: an entry module importing a module named `big[31m.mds` whose +evaluation exceeds `MAX_SOURCEMAP_SEGMENTS`. PF-013 evidence: positive (both the `\u001B` and `\u202E` literals present), negative (no raw ESC, no raw U+202E, no raw `\n`), +non-vacuity (the segment-cap warning must exist, else the haystack is empty), and +**guard-removal proven** — deleting the `sanitize_control_chars_wire(…)` wrapper at +`resolver.rs` makes the test fail on the raw ESC. + +A behavioural test of the two alias sites would assert on an input the parser rejects — +vacuous, the PF-013 failure mode — so none is written and the asymmetry is stated in the +guard's module doc instead of papered over. + +### C3 — undocumented guard bypass: non-`let` binder name collision + +`print_discipline.rs` affirmatively claimed a loop variable or function parameter is +"reported, not assumed safe". It was not: `let` bindings are matched **file-wide**, so a +`for` variable, parameter or closure param was resolved against unrelated `let`s of the +same name and accepted when all of them were safe. Reproduced on real source — `lint.rs` +has three `let label = safe_path(…)` bindings, so + +```rust +fn atk_v12(rules: &[String]) { for label in rules { eprint_warning(label); } } +``` + +produced **zero sites**. + +**Fixed, not merely documented.** A new `collect_non_let_binders` collects every `for` +variable, function parameter and closure parameter in a file; each **poisons** its own +name, so `classify_helper_arg` reports such an argument instead of resolving it. +Over-collection is the safe direction and is chosen deliberately. Proven by injecting the +exact construct above into real `crates/mds-cli/src/lint.rs`: + +``` +print-discipline violation: 1 interpolation(s) reach a terminal stream unescaped. + lint.rs:1547: eprint_warning(untraced) interpolates unsanitized `label` +``` + +`lint.rs` was restored with `git checkout --` immediately after. The fix introduced **zero +false positives** on real source: `cli_print_sites_sanitize_every_interpolated_value` and +`every_allowlist_entry_is_live` both still pass unchanged, and no allowlist entry was +added. A new non-vacuity assertion (>= 50 non-`let` binders found) keeps the poison set +from silently emptying, and a new self-test +(`the_guard_refuses_to_resolve_a_non_let_binder`) covers the `for` / parameter / closure +shapes, asserts the collector finds each shape it models, and asserts a bitwise `|` / +`||` operand is **not** read as a closure parameter. The residual — `if let` / `while let` +/ `match`-arm binders are not modelled — is now **limit 5** in "Accepted limits", stated +as the narrowed remnant of the wider hole rather than as a closed one. + +### C4 — doc inconsistencies + +- `spec.md` claimed both "C0 … except `\t`" and "`\t` is the only character **in the + class** that is preserved". Unified on the framing the other two documents already use: + `\t` is the **sole exemption** from the C0 range and is never escaped, in either mode. +- `crates/mds-core/src/error.rs`'s `display_sanitized` rustdoc still used the retired "C0 + except `\t` and `\n`" framing that `diagnostic.rs` says docs must not use — the + round-3 unification missed this third site. It now states the class once and names + HUMAN mode as the reason `\n` survives, with the retired framing called out explicitly. +- `crates/mds-python/src/lib.rs` and `crates/mds-python/tests/test_errors.py` said "the + other four surfaces", implying five. There are four in total (CLI, napi, WASM, Python), + so from Python's own surface it is **three**, now named. +- `ALLOWED_UNTRACED_HELPER_ARGS` is **two entries covering five sites** (keyed by `(file, + expression)`), not five entries. Corrected in this ledger and in the entry's own + justification. + +### Residuals after round 4 — the complete list + +Four adversarial alignment passes have now run against this branch. What remains is +enumerated rather than implied: + +1. **Message-body residual** — a path / identifier / cause interpolated into an `MdsError` + or `miette!()` message body is prose, so it stays HUMAN on terminal surfaces and a `\n` + in it survives inside the rendered frame. Weaker than a status line (frame content is + indented and `│`-prefixed, and the prefix survives `strip()`). ~110 construction sites; + a separate change. +2. **Source-map / `dependencies` carve-out** — paths verbatim, by decision (C1). The + one-way consumer contract is the mitigation. +3. **Guard limits 1–5** — name-matched sanitizers, anti-rot-not-anti-reuse allowlists, the + one-hop single-file trace, name-based stream detection, and unmodelled `if let` / + `while let` / `match`-arm binders. The guard's bar is *accidental* reintroduction. +4. **Two `evaluator.rs` producers upheld by review only** — untestable today because the + parser closes the vector (C2). +5. **Escaping is one-way and non-injective** — a decided, documented non-goal, not a gap. + +### Round-4 verification + +| Command | Result | +|---------|--------| +| `cargo fmt --all --check` | PASS | +| `cargo clippy --workspace --all-targets -- -D warnings` | PASS (zero warnings) | +| `cargo nextest run --workspace` | PASS — **1980** tests across 28 binaries (baseline 1978, +2 new) | +| `cargo test --doc --workspace` | PASS — **37** doctests (unchanged) | +| `mds-cli::security` | 22 tests (unchanged) | +| M1 / M2 forgery re-verification | PASS — 0 forged bare lines, 0 raw control bytes | +| Binding suites (pytest / napi / universal JS / wasm-pack) | **not run — not required**: the round's diff under `crates/{mds-core,mds-wasm,mds-napi,mds-python}` contains zero non-`///`/`//!` lines apart from one Python test *docstring*, so no executable code changed | + +Raw-control-byte audit of every changed file: **0** (the two ESC bytes the tool +layer decoded into `producer_discipline.rs`, and two pre-existing ones in +`KNOWLEDGE.md`, were replaced with the six-character literal text). + +**Round-4 implementing commits:** `0ce1966` (guard fail-closed + `producer_discipline.rs`), +`731c3b3` (source-map carve-out + claim narrowing + doc unification). + +## Blocked +| Issue | File:Line | Blocker | +|-------|-----------|---------| + +(none) diff --git a/.devflow/features/index.md b/.devflow/features/index.md new file mode 100644 index 00000000..33a096c9 --- /dev/null +++ b/.devflow/features/index.md @@ -0,0 +1,5 @@ +# Feature Knowledge Index + +- **mds-fmt** — crates/mds-core/src, crates/mds-cli/src — Use when modifying the mds fmt formatter engine (crates/mds-core/src/formatter.rs), the mds fmt CLI subcommand (crates/mds-cli/src/fmt.rs), any change to mds-core's output model (clean_output, evaluate_nodes, @message/@define body evaluation, the lexer's fence recognition) that could silently break the formatter's compile-equivalence guarantee, or changes to the shared directory walker (output.rs). Keywords: mds fmt, format_str, format_str_with, format_str_named, FormatterInvariant, clean_output, compile-equivalence, idempotent, assert_equivalent, structural_equivalent, strip_trailing_insignificant_text, in_raw_content, raw_content_spans, protected_spans, R1 R2 R3 R4, safety gate, token lossiness, @message body, @define body, @block body, FmtConfig, FmtFlags, interior-verbatim contract, try_scan_fence_at, FenceMatch, deep_merge_yaml, RESERVED_MERGE_KEYS, is_default_excluded_dir, is_within_default_excluded_dir, walker exclusions, node_modules, hidden dirs, effective_parent, bare filename, atomic_write_file. +- **mds-lint** — crates/mds-core/src/lint, crates/mds-cli/src, crates/mds-wasm/src, crates/mds-napi/src, crates/mds-python/src, packages/mds/src — Use when adding or modifying lint rules, extending the --fix pipeline, changing the JSON wire format, wiring lint into a binding layer, debugging unexpected exit codes and reverify gate refusals, or working on the ESC/bidi/newline injection defences. Keywords: mds lint, LintDiagnostic, fix_removals, fix_edits, TextEdit, FixLineSpan, diag_to_edits, LintResult, LintConfig, to_canonical_json, fix tier, reverify gate, FixOutcome, PartiallyFixed, apply_fixes_incremental, preview_fixes, PreviewOutcome, set_diag_display_path, AnalysisContext, ElseifBranch, end_offset, DefineFact, assertKnownKeys, CheckOptions, unreachable-branch, unused-variable, duplicate-import, empty-block, legacy-interpolation, is_output_neutral, all_output_neutral, Tier A Tier B Tier C, structural-standalone, compile-clean, is_standalone, sanitize_control_chars, sanitize_control_chars_wire, named_source_for_render, neutralize_source_for_render, SanitizedReport, SanitizedNode, MAX_AUX_DEPTH, EscapeMode, HUMAN WIRE, eprint_warning, safe_path, safe_inline, safe_file_display, preview_text_for, print_discipline, reverify_failure_reason, LintDirCtx, config_cache, dedup_contained_or_identical, EXIT 0 1 2 3, render_error_sanitized, eprint_error, display_sanitized, MdsError::display_sanitized, ESC-injection, CWE-150, CWE-117, bidi, Trojan-Source, CVE-2021-42574, U+061C, U+202E, U+FEFF, U+2028, U+2029, PF-014, PF-005, construction-time sanitization, per-field rule, Cow, #176. +- **source-map-security** — crates/mds-core/src, crates/mds-cli/src, packages/mds/src — Use when working with Source Map v3 generation, sources[] path relativization, the relativize_source choke-point, FileSystem::source_root(), CompileOptions.source_map_base, cross-surface source-map parity tests, or the Windows verbatim UNC path fix. Keywords: source map, sources[], relativize_source, source_map_base, source_root, path containment, basename fallback, PF-005, ADR-005, SEC-3, Windows verbatim UNC, path_to_unified, compute_source_map_base, apply_source_map_file_label, CF-SM2, V-SM1, differential test, two-level anchoring, map-relative, root-relative. diff --git a/.devflow/features/mds-lint/KNOWLEDGE.md b/.devflow/features/mds-lint/KNOWLEDGE.md new file mode 100644 index 00000000..2c7bc283 --- /dev/null +++ b/.devflow/features/mds-lint/KNOWLEDGE.md @@ -0,0 +1,577 @@ +--- +feature: mds-lint +name: mds lint — Static Analysis Engine and Tiered --fix +description: "Use when adding or modifying lint rules, extending the --fix pipeline, changing the JSON wire format, wiring lint into a binding layer, debugging unexpected exit codes and reverify gate refusals, or working on the ESC/bidi/newline injection defences. Keywords: mds lint, LintDiagnostic, fix_removals, fix_edits, TextEdit, FixLineSpan, diag_to_edits, LintResult, LintConfig, to_canonical_json, fix tier, reverify gate, FixOutcome, PartiallyFixed, apply_fixes_incremental, preview_fixes, PreviewOutcome, set_diag_display_path, AnalysisContext, ElseifBranch, end_offset, DefineFact, assertKnownKeys, CheckOptions, unreachable-branch, unused-variable, duplicate-import, empty-block, legacy-interpolation, is_output_neutral, all_output_neutral, Tier A Tier B Tier C, structural-standalone, compile-clean, is_standalone, sanitize_control_chars, sanitize_control_chars_wire, named_source_for_render, neutralize_source_for_render, SanitizedReport, SanitizedNode, MAX_AUX_DEPTH, EscapeMode, HUMAN WIRE, eprint_warning, safe_path, safe_inline, safe_file_display, preview_text_for, print_discipline, reverify_failure_reason, LintDirCtx, config_cache, dedup_contained_or_identical, EXIT 0 1 2 3, render_error_sanitized, eprint_error, display_sanitized, MdsError::display_sanitized, ESC-injection, CWE-150, CWE-117, bidi, Trojan-Source, CVE-2021-42574, U+061C, U+202E, U+FEFF, U+2028, U+2029, PF-014, PF-005, construction-time sanitization, per-field rule, Cow, #176." +category: domain-knowledge +directories: + - crates/mds-core/src/lint + - crates/mds-cli/src + - crates/mds-wasm/src + - crates/mds-napi/src + - crates/mds-python/src + - packages/mds/src +created: 2026-07-11 +updated: 2026-07-26 +--- + +# mds lint — Static Analysis Engine and Tiered --fix + +## Overview + +`mds lint` is a phase-3.5 static analysis pass: it runs AFTER `mds check` (resolve+validate) confirms the template compiles, then applies 10 rules over the raw AST and token stream. The engine lives entirely in `crates/mds-core/src/lint/` and is exposed unchanged via all five surfaces: CLI, WASM, napi, Python, and the `packages/mds` universal wrapper. + +The feature has two key design invariants that touch every layer: (1) `LintResult::to_canonical_json()` is the ONE serializer for all surfaces — byte-parity across surfaces is enforced by goldens; (2) `tier.rs` is the single source of truth for which rules are auto-fixable — both `diagnostic.rs` and `fix.rs` import from it to break what would otherwise be a circular dependency. + +Fix edits take two forms: **line-removal** (`fix_removals: Option>`) for rules that delete whole lines (e.g. `duplicate-import`), and **in-place replacement** (`fix_edits: Option>`) for rules that need to replace text without changing line structure (e.g. `legacy-interpolation` rewrites `{x}` → `{{x}}`). Both paths converge in `diag_to_edits` and produce `ByteEdit`s that `apply_plan_unchecked` applies right-to-left via `replace_range`. + +## Business Context + +Issue #61 closes the v0.4.0 gate. Users are prompt-library authors (dead `@define` functions accumulate silently), CI gatekeepers (PR template already mandates "no new linter warnings"), and future LSP integrations (stable JSON schema). `mds check` answers "will it compile?"; `mds lint` answers "should it compile this way?". + +## Core Business Rules + +### Rule Catalog + +Ten rules across three severities. Defaults are built-in and overridable per-rule in `mds.json`. + +| Rule | Default Severity | Fix Tier | Notes | +|------|-----------------|----------|-------| +| `duplicate-import` | Error | A | Lexical path normalization via `normalize_import_path` (interior segment-collapse) | +| `duplicate-export` | Error | A | Spans from D2 `offset` on `ExportDirective` variants | +| `unreachable-branch` | Error | A | Literal↔literal Eq/NotEq + duplicate structural @elseif only; variable compares never flagged | +| `empty-block` | Warn | A | Empty OR whitespace-only-Text bodies; NEVER @block (intentional placeholder); fires on @if/@elseif/@else/@for/@define/@message | +| `legacy-interpolation` | Warn | A | TOKEN-based (Token::Text + @message directives, structurally skips fences); detects old `{x}` single-brace syntax + `\{`/`\}` remnants + `@message {expr}:`; `${...}` skipped; fix via `fix_edits` (atomic TextEdit replacement, not FixLineSpan) | +| `unused-import` | Warn | B | Merge imports always "used"; selective flagged per-name; re-export (Named/ReExport) exempts; `fix_removals: None` (report-only in practice) | +| `unused-function` | Warn | B | Only fires when `has_explicit_exports`; self-recursion treated as used; `fix_removals = Some([whole-@define block])` | +| `unused-variable` | Warn | C | Frontmatter keys with no body reference; Tier C = report-only | +| `redundant-else` | Warn | C | @else body structurally identical to @if then-body via structural_eq | +| `shadow-variable` | Info | C | Default OFF — must be enabled in mds.json; never affects exit code | + +### The Tier Model + +Tier classification is the central safety contract for `--fix`. `tier.rs` is a dedicated leaf module that both `fix.rs` and `diagnostic.rs` import — it was extracted to break a potential circular dependency. It also hosts the `first_occurrence` helper (shared by `duplicate_import.rs` and `duplicate_export.rs`) as it is the one lint leaf module with no rule-specific imports. + +- **Tier A**: Auto-fixable via `fix_removals` (line-removal) or `fix_edits` (text-replacement) + gated by the reverify pipeline. Currently: duplicate-import, duplicate-export, unreachable-branch, empty-block, legacy-interpolation. +- **Tier B**: Fixable only when the file is "structural-standalone" (no `@import` or `@extends`). Currently: `unused-function` (has `fix_removals = Some([whole-@define block])`, applied when standalone); `unused-import` has `fix_removals: None` — partial-name removal from an import list is structurally ambiguous and unsafe, so the rule is Tier B but never emits an edit (report-only in practice). A file that triggers `unused-import` is, by definition, not structural-standalone. +- **Tier C**: Never fixed — report-only. Currently: unused-variable, redundant-else, shadow-variable. + +**Terminology (spec §7.5)**: +- **Structural-standalone**: a file with no `@import`, `@extends`, or use as a partial target. Gates Tier B `--fix`. +- **Compile-clean**: a file that compiles without any runtime `--vars`. Gates the output-equality reverify for Tier B fixes: removing an unused import or function must produce byte-identical compiled output. + +```rust +// tier.rs — the single source of truth. DO NOT re-inline this table in fix.rs or diagnostic.rs. +pub fn rule_tier(rule: &str) -> FixTier { + match rule { + "duplicate-import" | "duplicate-export" | "unreachable-branch" + | "empty-block" | "legacy-interpolation" => FixTier::A, + "unused-import" | "unused-function" => FixTier::B, + _ => FixTier::C, // all others, including unknown rules + } +} + +// is_output_neutral: true for all rules EXCEPT legacy-interpolation. +// All Tier A/B rules are output-neutral (fix preserves compiled output) except +// legacy-interpolation, which migrates {x} (plain text) to {{x}} (interpolation), +// intentionally changing the compiled output. The CLI uses this to skip the +// output byte-equality reverify sub-check when the plan contains that rule. +pub fn is_output_neutral(rule: &str) -> bool { + rule != "legacy-interpolation" +} +``` + +The `fixable` flag in the canonical JSON output is computed as `(fix_removals.is_some() || fix_edits.is_some()) && tier::is_fixable(rule, is_standalone)` inside `to_canonical_json()` — it is NOT stored on `LintDiagnostic` directly. + +### Rule Semantics — Key Edge Cases + +**legacy-interpolation**: Token-based scan over `Token::Text` (skips fences/frontmatter/directives automatically) and `Token::Directive` (for `@message {expr}:` patterns). Detects: `\{`/`\}` remnants (fix = delete backslash), `${...}` (skip, never flagged), `{expr}` (fix = atomic TextEdit replacing entire `{expr}` with `{{expr}}`), `@message {expr}:` dynamic role (same fix). Edits are always single atomic TextEdits — never split open/close. Suppressible via `mds.json`. + +**Suppression configs**: `examples/edge-cases/mds.json`, `examples/stress-test/edge/mds.json`, and `crates/mds-cli/tests/fixtures/mds.json` all set `"legacy-interpolation": "off"` — these directories intentionally contain literal-brace teaching content. + +**unreachable-branch**: The rule only fires on literal↔literal comparisons (`@if "x" == "x":`). Always-true conditions only flag IF there are LATER branches to be unreachable. Each duplicate `@elseif` gets at most one finding — "duplicate" OR "always-true/false", never both. Diagnostic spans use `branch.offset`. All messages end with trailing periods. Per-case `fix_removals`: A/B/C/E/G have `Some(spans)`, D/F have `None` (when @elseif branches make safe removal ambiguous). + +**empty-block**: "Empty" = `body.is_empty()` OR all nodes are whitespace-only `Text`. The `@block` directive is deliberately excluded (intentional "inherit parent default" pattern). All messages end with trailing periods. Per-case `fix_removals`: whole-block removal (`to_inclusive=true`) for ①@for ②@define ③bare-@if; `None` for ④@if-with-branches (unsafe partial removal); exclusive removal (`to_inclusive=false`) for ⑤@else ⑥terminal-@elseif; `None` for ⑦/⑧ non-terminal-@elseif or @else-follows; `@message` always `None`. + +**unused-import (merge)**: Merge imports (`@import "path"`) are always treated as used — they inject all exports plus `prompt` into scope. Conservative; false negatives are acceptable here. + +**unused-variable**: Uses are tracked across all recursive bodies: `Expr::Var`, `Arg::Var`, `Arg::MemberAccess`, condition operands, `@for` iterables, call arguments. Code-fence content is `Text` (interpolation suppressed) — correct by construction. Reserved skip-set: `{imports, type, extends, prompt}`. + +**unused-* suppression**: When `ctx.is_partial_or_extends == true` (file starts with `_` OR has `@extends`), the unused-variable, unused-import, and unused-function rules are entirely suppressed. shadow-variable is NOT suppressed — it is already default-off. + +**wildcard re-export exemption**: The `@export * from "path"` directive does NOT exempt an `@import` from being flagged as unused. Only `Named`/`ReExport` exports suppress their corresponding import. + +### Configuration + +`mds.json` `lint.rules` section: +```json +{ "lint": { "rules": { "unused-variable": "off", "shadow-variable": "warn" } } } +``` + +**Unknown rule NAMEs** → warn-and-ignore at CLI (forward compat). +**Unknown severity VALUES** → hard parse error → exit 2 (closed enum, no sensible fallback). + +`LintConfig` lives in `mds-core` (not mds-cli). The CLI `LintCliConfig` from `build.rs` converts to it via `into_core_config()`. + +## Technical Implementation Patterns + +### Engine Pipeline (per-file) + +```rust +// lint_source() in crates/mds-core/src/lint/mod.rs — the engine entry point. +// Called by the public mds::lint(), mds::lint_str_with(), mds::lint_virtual() API. +pub(crate) fn lint_source(source, filename, config) -> Result { + // Step 1: re-parse entry independently (mirrors scan_imports pattern) + let tokens = lexer::tokenize(source, filename)?; + let module = parser::parse_with_ctx(&tokens, filename, source)?; + + // Step 2: facts walk — one traversal building AnalysisContext. + let ctx = collect_facts(&module, is_partial || is_extends, source)?; + + // Step 3: standalone detection for Tier B eligibility + let is_standalone = !ctx.is_partial_or_extends && ctx.imports.is_empty(); + + // Step 4: non-generic rule dispatch (10 plain fn calls). + // Token-based rules (Step 6) receive &tokens + source; AST rules receive &module + &ctx. + run_rules(&module, &ctx, &tokens, source, filename, config, &mut builder); + + Ok(builder.build(is_standalone)) +} +``` + +Key invariants: +- The check gate (resolve+validate) runs ONCE in the public `mds::lint()` wrapper before calling `lint_source`. The engine never calls the resolver again. +- Per-file fresh resolve is intentional (v1). There is NO cross-file `ModuleCache` — it would be unsafe under per-file runtime vars. +- `run_rules` threads both `&[Token]` (the raw token stream) and `source: &str` through so token-based rules (currently `legacy-interpolation`) can operate without re-tokenizing. + +### AST: ElseifBranch, IfBlock, ForBlock, DefineBlock + +`ElseifBranch` carries a per-branch offset. `IfBlock`, `ForBlock`, and `DefineBlock` carry `end_offset` for fix span computation: + +```rust +pub struct ElseifBranch { + pub condition: Condition, + pub body: Vec, + pub offset: usize, // byte offset of @elseif token — used for diagnostic spans +} + +pub struct IfBlock { + pub condition: Condition, + pub then_body: Vec, + pub elseif_branches: Vec, + pub else_body: Option>, + pub offset: usize, + pub else_offset: Option, // byte offset of @else token, when present + pub end_offset: usize, // byte offset of @end token (for fix span computation) +} +``` + +`structural_eq.rs` compares `ElseifBranch` by `condition` and `body` only — `offset` is excluded. `end_offset` on all block types is excluded from structural equality. `DefineFact` in `facts.rs` mirrors this: `DefineFact { name, offset, end_offset }` — the facts walker copies `b.end_offset` from the AST so fix rules have it without re-walking. + +### The `--fix` Pipeline + +The fix pipeline is split into a pure core (`fix.rs`) and an I/O layer (`lint.rs`). + +**fix.rs** (pure, no I/O): + +Fix edits are driven by two complementary fields on `LintDiagnostic`: +- `fix_removals: Option>` — line-range removals. `FixLineSpan` encodes `from`/`to`/`to_inclusive`. Used by all Tier A/B rules except `legacy-interpolation`. +- `fix_edits: Option>` — in-place replacements. `TextEdit { start: usize, end: usize, new_text: String }` where `start` is inclusive, `end` is exclusive, and empty `new_text` is a pure deletion. Used by `legacy-interpolation`. + +Both paths go through `diag_to_edits(diag, source) -> Vec`. The `fix_removals` path produces `ByteEdit { replacement: String::new() }` (pure deletion); the `fix_edits` path produces `ByteEdit { replacement: edit.new_text.clone() }` with a char-boundary guard (fail-closed, ADR-001). `apply_plan_unchecked` applies edits right-to-left via `replace_range`, handling both deletions and replacements uniformly. + +Planning steps in `plan_fixes_with_options`: +1. Collect `ByteEdit`s from fixable diagnostics via `diag_to_edits`. +2. Sort edits by `(start ASC, end DESC)` — widest edit first among same-start edits. +3. **Containment coalescing** (`dedup_contained_or_identical`): drop any edit whose byte range is fully contained within (or identical to) an earlier retained edit. +4. **Overlap detection**: after containment deduplication, any remaining partial overlap causes the whole batch to be cleared (`overlap_rejected = true`). Fail-closed. + +`FixOutcome` (returned by `apply_fixes` and `apply_fixes_incremental`) has four variants: `Fixed { source, residual }`, `PartiallyFixed { source, residual, rejected }`, `Rejected { source, reason }`, `NothingToFix`. + +**Reverify gate (AC-F-20)**: After applying edits, a reverify callback checks three conditions: (1) recompile-success; (2) no-new-untargeted-diagnostics; (3) output byte-equality for standalone files **when all edits in the plan are output-neutral**. + +The output-equality sub-check (3) is **skipped** when the plan contains any edit from `legacy-interpolation` (the only non-output-neutral rule). Both `plan_and_apply_fixes` and `preview_fixes` in `lint.rs` compute `all_output_neutral = plan.edits.iter().all(|e| mds::fix::is_output_neutral(&e.rule))` and gate the equality check on it. This bypass applies to the whole batch — a mixed plan containing `legacy-interpolation` alongside output-neutral rules skips the equality check for co-batched neutral edits too (assessed P2 risk). + +**`reverify_failure_reason(err)` in `fix.rs`** — the ONLY construction site for `FixOutcome::Rejected.reason`. It WIRE-escapes `err.to_string()` via `sanitize_control_chars_wire` so the CLI can print `fix rejected: {reason}` as a bare status line without further escaping. Construction-time sanitization, not print-time. + +**CLI's `plan_and_apply_fixes`** (in lint.rs): The short-circuit for `NothingToFix` guards on `plan.edits.is_empty() && !plan.overlap_rejected`. When overlap was detected, `plan.edits` is cleared but `plan.overlap_rejected = true` — so the function falls through to `apply_fixes_incremental`, which immediately returns `Rejected`. + +**Atomic write** (`atomic_write_file` in `output.rs`, imported by both `lint.rs` and `fmt.rs`): TOCTOU guard + permissions restore + `sync_all()` + `persist()` (intra-filesystem rename). Temp prefix `.mds-tmp-`. `Fixed:` is printed only AFTER a successful write. + +### CLI Preview Pipeline (`preview_fixes`) + +`preview_fixes` in lint.rs routes through the same gated pipeline as the write path — it calls `apply_fixes_incremental` with the full reverify closure and does NOT write to disk. `PreviewOutcome` has three variants: `WouldFix(String)` (at least one edit applies; contains would-be source), `Rejected(String)` (every edit refused), `NothingToFix`. The three-way gate in both single-file and directory modes: `fix && !check && !diff` → write path; `fix && (check || diff)` → preview path; report-only (no `fix`) → `mds::lint` + render. + +### Directory Mode: Per-File Config Discovery + +In directory mode, `mds lint` no longer loads a single root `mds.json` — each file independently walks up to its nearest `mds.json`. This is managed by `LintDirCtx`: + +```rust +struct LintDirCtx<'a> { + flags: LintFlags, + runtime_vars: &'a Option>, + config_cache: RefCell>>, +} +``` + +A config-load failure for a nested `mds.json` is a per-file error: it emits the error and continues linting other files, but the overall exit code reflects the failure (exit 2). + +### Display Path Remapping (`set_diag_display_path`) + +`mds::lint(path, …)` sets each diagnostic's `file` field to the file's **basename**. In directory mode, `set_diag_display_path` replaces this with the relative path (relative to the lint root) immediately after every `mds::lint` call. + +### Cross-Surface Options Validation (`packages/mds`) + +`packages/mds/src/util/options.ts` exports `assertKnownKeys(options, method)`, which enforces strict unknown-option rejection at the wrapper layer before dispatching to any backend. + +- Error code: `'mds::invalid_options'` (satisfies `isMdsError`). +- Message format is byte-matched to the backend's `format_unknown_keys_error`. +- `CheckOptions { vars? }` is a separate interface from `CompileOptions` (which also carries `sourceMap`, `sourcesContent`). + +### Python Surface + +`crates/mds-python/src/lib.rs` exposes fully typed frozen result classes: + +- `LintDiagnostic`: `#[pyclass(frozen)]` with `.rule`, `.severity`, `.message`, `.help`, `.fixable`, `.span`, `.fix_edits` attributes. Supports pickling via `__reduce__`. +- `fix_edits` is a custom `#[getter]` (not `#[pyo3(get)]`) because `Vec` does not implement `IntoPy`. Returns `list[dict] | None`. Pickle round-trips `fix_edits` as a JSON string (`fix_edits_json: Option`) via `__reduce__`/`__new__`. +- `LintDiagnostic.as_json()` inserts keys in alphabetical order explicitly (serde_json::Map with insertion order): `fix_edits`, `fixable`, `help`, `message`, `rule`, `severity`, `span` — emitting `fix_edits`, `help`, and `span` unconditionally as `null` when `None` (PF-007 parity). +- `LintFileReport`: `#[pyclass(frozen)]` per-file findings group with `.file` and `.diagnostics`. + +### WASM Surface + +`parse_check_options` in `crates/mds-wasm/src/lib.rs` uses a strict allow-list: `reject_unknown_wasm_keys(&obj, &["filename", "modules", "vars"])?;` — the `check()` export rejects any option key not in `[filename, modules, vars]`. + +### Canonical JSON Wire Format + +`LintResult::to_canonical_json()` is THE single serializer. All five surfaces call it: + +```json +{ + "version": 1, + "files": [ + { + "file": "path/to/file.mds", + "diagnostics": [ + { + "rule": "legacy-interpolation", + "severity": "warn", + "message": "...", + "help": "...", + "fixable": true, + "span": { "offset": 6, "length": 6 }, + "fix_edits": [{ "start": 6, "end": 12, "new_text": "{{name}}" }] + } + ] + } + ], + "truncated": false +} +``` + +**SPAN-1**: `span.line` and `span.column` are NOT part of the stable wire format. All 10 lint rules pass `None` for `line`/`column`. + +**`fix_edits` field**: emitted unconditionally on every diagnostic — `null` when the rule uses `fix_removals` or has no fix, `[{start,end,new_text}]` when populated. Because `to_canonical_json()` builds its map via `serde_json::json!` and serde_json's `Map` defaults to `BTreeMap`, keys are sorted alphabetically: `fix_edits` appears before `fixable`. File ordering in `files` is deterministic (BTreeMap sorted by filename). `fixable` is NOT stored on the diagnostic struct. + +**Per-file cap**: `MAX_DIAGNOSTICS = 1000` (`crates/mds-core/src/limits.rs`). **File-less key**: `""`. **Analysis failure envelope** (JSON mode only): `{ "version": 1, "error": { "code": "...", "message": "...", "help": "...", "span": {...} } }`. **`--fix --format json --stdin` is a usage error** — exits 2 with a plain stderr message (AC-F-22b). + +## Sanitization Discipline + +This is the most important section for any agent working on security or error-output paths. Read it before touching anything in `output.rs`, `diagnostic.rs`, `error.rs`, or any file that calls `eprintln!`. + +### The Governing Principle: Per-Field, Not Per-Surface + +The design is **input-sanitizing, not output-sanitizing** (avoids PF-014). The single governing rule, normative in spec §7.5: + +> **On the diagnostic surfaces — the `"version": 1` JSON wire, CLI status and warning lines, `[file:line:col]` frame headers — untrusted identifiers, filenames, and error causes are WIRE-escaped, human terminal output included. Prose — a diagnostic message body or help body — stays HUMAN on terminal surfaces so multi-line frames keep rendering.** + +**The rule governs diagnostic output only.** Two categories are named carve-outs and are not escaped at all: + +1. **The command's product** — compiled template output (`mds build -o -`, `mds lint --fix -`). Escaping it would corrupt every redirect. +2. **Functional path references** — source-map `file` / `sources` / `sourcesContent` (both the `mds build --source-map` sidecar and the `sourceMap` embedded in `CompileResult::to_canonical_json()`), and `CompileResult.dependencies`. These paths are emitted **verbatim**, control bytes and all: devtools, bundlers and IDEs resolve them against the filesystem, so an escaped path would not exist. Consumers MUST treat them as untrusted and escape them for their own destination — JSON string encoding is not escaping, since a decoded `"\n"` is a real newline again. The CLI does not depend on this: `Compiled to …` / `Source map written to …` print through `safe_path`. See spec §7.5 "Carve-out: functional path references". + +A reviewer reproduced (2026-07-26) that a file named with a real `\n` and ESC yields a sidecar whose decoded `file` / `sources` carry those bytes while the CLI status line for the same run is escaped. That is the carve-out working as designed, not a gap — but the per-field sentence used to claim otherwise, which is why it is now scoped to diagnostic surfaces. + +The discriminator is whether the value is ever legitimately multi-line. A filename, an `mds.json` rule name, a `--format` argument, and an `io::Error` cause are each rendered on exactly one line (a status line or a `[file:line:col]` frame header) — a raw `\n` in one forges a standalone line byte-identical to genuine output (CWE-117). A diagnostic body genuinely is multi-line, so escaping its newlines breaks the frame. + +This rule supersedes the old "wire mode at exactly these four boundaries" enumeration. Enumerating boundaries went stale twice under review; the per-field rule makes each new site decidable without re-deriving the list. + +### The Escape Class and the Two Modes + +**The escape class** — identical for both modes: C0 (U+0000–U+001F), DEL (U+007F), C1 (U+0080–U+009F), all 12 Unicode `Bidi_Control=Yes` members (U+061C, U+200E/U+200F, U+202A–U+202E, U+2066–U+2069 — Trojan Source / CVE-2021-42574), the JS line/paragraph separators U+2028/U+2029, and U+FEFF (BOM). Each hostile character is replaced by an uppercase `\uXXXX` 6-char literal. `\t` (U+0009) is exempted from both modes. `\n` (U+000A) is IN the class — but whether it is escaped depends on mode, not on the class definition. + +Two modes, one shared implementation via `sanitize_with(s, EscapeMode)`: + +- **HUMAN** (`sanitize_control_chars(s)`) — preserves `\n`. For terminal/miette render output where multi-line frames must stay readable. +- **WIRE** (`sanitize_control_chars_wire(s)`) — also escapes `\n` → ` +`. For JSON wire, binding error objects, status lines, any line-oriented consumer of the string value. + +Both return `Cow<'_, str>`: borrowed on clean input (zero allocation), owned only when a hostile character is actually present. Both are idempotent. + +**Key byte-level detail**: The fast-path scan checks for bytes `< 0x20`, `0x7F`, `0xC2` (C1 prefix), **`0xD8`** (U+061C prefix), `0xE2` (U+200E/U+202E/U+2028 etc. prefix), `0xEF` (U+FEFF prefix). The `0xD8` byte must be in the fast-path scan or U+061C short-circuits to `Cow::Borrowed` without being inspected. + +### The `neutralize_source_for_render` Byte-Width Branches + +Source text passed to `NamedSource` uses a different function: `neutralize_source_for_render(s)` — byte-length-preserving substitution so span offsets and caret alignment stay exact: + +- **C0/DEL (1-byte)** → `?` (1 byte) +- **C1 (U+0080–U+009F) AND U+061C** (both 2-byte UTF-8) → U+00A0 NBSP (2 bytes). U+061C is in the 2-byte branch. +- **The other 11 format hazards** (U+200E/U+200F, U+2028/U+2029, U+202A–U+202E, U+2066–U+2069, U+FEFF — all 3-byte) → U+FFFD REPLACEMENT CHARACTER (3 bytes). + +The split is implemented via two private predicates: `is_two_byte_format_hazard(ch)` (only U+061C) and `is_three_byte_format_hazard(ch)` (the remaining 11). A `debug_assert_eq!` in `neutralize_source_for_render` catches byte-length violations immediately during development. + +### `named_source_for_render` — The Single NamedSource Builder + +`named_source_for_render(file: &str, source: &str) -> miette::NamedSource` is now the ONLY `NamedSource` builder in the codebase. Three callers: `MdsError::at()` (error.rs), `check_equivalence` (formatter.rs), `render_diag_human` (mds-cli/src/lint.rs). Contract: + +- **filename** → `sanitize_control_chars_wire` (WIRE — a filename is never legitimately multi-line). +- **source** → `neutralize_source_for_render` (byte-length-preserving — keeps every span offset and caret column exact). + +### `SanitizedReport` — The CLI stderr Choke-Point + +`eprint_error` wraps every `miette::Report` in a `SanitizedReport` before miette renders it. This covers **both** CLI error families: `MdsError` (compiler diagnostics) and CLI-authored `miette::miette!()` reports — wrapping at the `Report` level means any error type added later inherits the guarantee without touching a downcast ladder (avoids PF-004). + +`SanitizedReport` overrides every prose surface — the `Display` message, the `help` text, each `LabeledSpan`'s label text — with HUMAN-mode sanitized copies, while delegating `code`, `severity`, `url`, `source_code`, and each label's **byte span** to the inner report untouched (so `named_source_for_render`'s byte-length neutralization keeps every span exact). + +The auxiliary diagnostic graph (`source` cause chain, `related`, `diagnostic_source`) cannot be forwarded by reference; the wrapper materialises the whole graph into owned `SanitizedNode`s at construction, bounded by `MAX_AUX_DEPTH = 16` to prevent infinite loops from a cyclic `source()` (avoids PF-005). An earlier revision returned `None` for `source()`/`related()` behind a `debug_assert!` — the guarantee was real in tests and absent in the shipped binary. Now enforced by data transformation. + +### The Complete Boundary Table + +| Boundary | Mode | Fields | +|----------|------|--------| +| `eprint_error` (output.rs) via `SanitizedReport` | HUMAN for prose | message, help, label text, entire auxiliary graph — every report rendered to CLI stderr | +| `eprint_warning` (output.rs) | prose HUMAN; interpolated identifiers/paths WIRE | HUMAN for the warning body prose; `safe_path` / `safe_inline` for any untrusted value the caller interpolates into it | +| `safe_inline(value)` (output.rs) | WIRE | any single-line untrusted value interpolated into a status, warning, or error line: rule names, config paths, `--format` args, `io::Error` causes | +| `safe_path(p)` / `safe_file_display(name)` (output.rs) | WIRE | CLI status-line path display (`Clean:`, `Fixed:`, `Would fix:`, `Compiled to`, …) | +| `named_source_for_render(file, source)` (diagnostic.rs) | WIRE for filename; neutralize for source | the single `NamedSource` builder used by `MdsError::at()`, `check_equivalence`, `render_diag_human` | +| `render_diag_human` (lint.rs) | HUMAN | message/help (filename and source go through `named_source_for_render`) | +| `fix::FixOutcome::Rejected.reason` (fix.rs) | WIRE | construction-time via `reverify_failure_reason(err)` — the CLI prints `fix rejected: {reason}` as a bare status line | +| `MdsError::serialize()` (error.rs) | WIRE | message, help — covers all three bindings' error path | +| `LintResult::to_canonical_json()` (diagnostic.rs) | WIRE | message, help, `files[].file` key | +| `CompileResult::to_canonical_json()` (lib.rs) | WIRE | warning strings (distinct method, not a duplicate) | +| `emit_warnings()` (lib.rs) | HUMAN for prose; WIRE at construction for identifiers interpolated in `resolver.rs`/`evaluator.rs` | warnings printed to stderr | +| Python `LintResult::new()` via `sanitize_lint_value()` | WIRE | message, help, file — construction-time, so typed getters read pre-sanitized data (closes PF-004 parallel-path) | +| `--diff` preview output (output.rs) | neutralized on TTY, byte-faithful when piped | `preview_text_for(writer_is_tty, text)` — redirected diffs stay `patch`-applicable | + +**`--check` is NOT TTY-gated** — it emits only status lines, which are unconditionally sanitized via `safe_path`. Only `--diff` calls `preview_text_for`. This distinction matters. + +**Deliberate residual (not closed):** `MdsError` message bodies and CLI `miette!()` message construction interpolate untrusted text as prose (paths, `io::Error` causes, identifiers). `eprint_error` applies HUMAN mode before miette renders them, so no raw control byte reaches stderr — but a `\n` in an interpolated path or identifier survives *inside the rendered frame*. Frame content is `│`-prefixed and indented, so it cannot masquerade as a bare status line. Not closed because there are 110+ `MdsError::*(format!(…))` construction sites; fixing a few would leave the claim false at ~100 others. Named in spec §7.5. + +**Escaping is one-way** — the transformation is lossy and non-injective: a template literally containing `\u001B` and one containing an actual ESC byte produce identical output. Consumers MUST NOT un-escape `\uXXXX` sequences back into bytes. Round-tripping is an explicit non-goal. + +### The Print-Discipline Guard + +`crates/mds-cli/tests/print_discipline.rs` is a CI-enforced test that **lexes** `crates/mds-cli/src/**` and FAILS if any print macro interpolates a value that is not a call to one of the accepted sanitizing helpers. + +Accepted helpers (SANITIZERS): `safe_path`, `safe_file_display`, `safe_inline`, `sanitize_control_chars_wire`, `render_error_sanitized`. **HUMAN `sanitize_control_chars` is deliberately NOT in SANITIZERS** — it preserves `\n`, so it cannot make an identifier safe. That is the M2 finding: routing an `mds.json` rule name through `eprint_warning` (HUMAN) still forged three standalone status lines. + +The guard: +- Traces `let` bindings **one hop** in the same file — hoisting a `format!` out of the call is checked exactly as if written inline. +- **Fails closed on untraceable arguments** (function parameters, loop variables, unrecognised expression shapes) — false positives cost one allowlist entry with written justification; false negatives cost another review round. +- **Poisons non-`let` binder names** (`collect_non_let_binders`). `let`s are matched file-wide, so before this a `for` variable / parameter / closure param was resolved against unrelated `let`s of the same name and accepted if all of them were safe. Proven live: `for label in rules { eprint_warning(label) }` injected into the real `lint.rs` (which has three `let label = safe_path(…)`) passed the guard; it now fails with `lint.rs:1547: eprint_warning(untraced) interpolates unsanitized \`label\``. +- Covers `eprint_warning` arguments too — the whole `format!` nested inside must have every interpolation pass through an accepted helper. +- Allowlists are keyed by `(file, expression)` — an `every_allowlist_entry_is_live` test fails if an entry stops matching, preventing silent staleness. + +**Documented limits** (five, in the file's own rustdoc): sanitizers are matched by the last path segment of the callee (an alias or a local function named `safe_path` would pass); binding traces are one hop within one file; allowlist exemptions are anti-rot but not anti-reuse (a new variable reusing an exempted name in the same file inherits the exemption silently); `write!` stream detection is by name; and the poison set models only `for` / parameter / closure binders, not `if let` / `while let` / `match`-arm ones. These limits close *accidental* reintroduction (which is what all four review rounds of #176 involved) — not intentional defeat. + +**The one cross-crate precondition** — that `mds-core` WIRE-escapes the identifiers its warning producers interpolate, since `mds-cli` prints whole warning strings through HUMAN-mode `eprint_warning` — is pinned by `crates/mds-cli/tests/producer_discipline.rs`. `mds-core` has exactly three such producers: `resolver.rs`'s imported-module filename (testable, and tested — a module key is a filesystem path) and `evaluator.rs`'s two `@include` alias warnings (**not** testable: the parser restricts an alias to `[A-Za-z_][A-Za-z0-9_]*`, so a test would be vacuous per PF-013 — upheld by review, and stated as such). + +This guard exists because three consecutive review rounds of #176 each found a NEW bare `eprintln!` after the previous one was fixed. It converts an unbounded reviewer search into a bounded enforced invariant. + +### Cross-Surface ESC-Injection Test Anchors + +The test anchor inventory covers five surfaces across both error and lint paths. + +**T-1..T-3** `error_tests.rs` — serialize path: ESC/DEL/C1 +**T-4** `diagnostic.rs` — `to_canonical_json`: bidi override (U+202E) in message/help/file key; span offsets byte-accurate +**T-5..T-9** `cli_lint.rs` — single-file, dir, stdin, DEL/C1, JSON (T-9 rewritten non-vacuous: duplicate-import + U+0085 NEL vector) +**T-10a/b/c** `output.rs` — `neutralize_source_for_render` byte-length invariant, caret alignment, miette SGR survives hostile OSC +**T-11** `error.spec.mjs` universal JS — differential +**T-11a/b** `output.rs` — `safe_path` (ESC → escaped literal, clean passthrough) +**T-12/T-13** `index.spec.mjs` napi +**T-14** `test_errors.py` Python (DEL/NEL params) +**T-15** `web.rs` WASM (F5/F5-DEL/F6/F6-C1) +**T-16f** `diagnostic.rs` — U+2028 in wire message +**T-16g** `diagnostic.rs` — wire-mode newline escaping / HUMAN mode preserves `\n` +**T-16h** `diagnostic.rs` — WIRE and HUMAN modes differ only on `\n` +**T-16i** `diagnostic.rs` — WIRE mode: borrowed-on-clean, idempotent +**T-NS-1/2/3** `diagnostic.rs` — `named_source_for_render`: hostile filename WIRE, hostile filename bidi class, source neutralized without changing byte length +**T-AUX-1/2/3** `output.rs` — `SanitizedReport`: cause chain escaped+preserved, related diagnostics escaped+preserved, cyclic cause chain bounded at `MAX_AUX_DEPTH` +**T-ESC-5/6/7** `output.rs` — label text escaped/span preserved, PF-014 colour path, inert on clean input +**T-WARN-1/2/3** `output.rs` — `eprint_warning`: C0, clean passthrough, bidi +**T-REASON-1/2** `fix.rs` — `reverify_failure_reason` WIRE on both construction paths +**T-ESC-MSG-1/2** `security.rs` — `MdsError` and CLI-authored message escaping +**T-ESC-RULE-1** `security.rs` — unknown `mds.json` rule name with embedded control bytes +**T-ESC-FNAME-1/2** `security.rs` — `\n` in filename cannot forge standalone status line (build and lint) +**T-ESC-WALK-1** `security.rs` — walker depth-limit warning hostile directory name +**Print-discipline self-tests** `print_discipline.rs` — `the_guard_flags_a_bare_interpolating_print`, `the_guard_follows_a_hoisted_format_binding`, `the_guard_reports_an_untraceable_helper_argument`, `cli_print_sites_sanitize_every_interpolated_value`, `every_allowlist_entry_is_live` + +### CLI Exit Codes + +Lint uses **direct `std::process::exit`**, never the shared `exit_code()` function. + +| Code | Condition | +|------|-----------| +| 0 | Clean — no Warn or Error findings | +| 1 | Warn-severity findings only, no errors; also `--fix --check` when any file would change | +| 2 | Any Error-severity finding OR analysis failure (parse, syntax, nesting-overflow) OR usage error | +| 3 | ResourceLimit — `MAX_BLOCKS_PER_MODULE=256` exceeded in the resolver's `collect_block()` | + +`Info` severity never contributes to exit code. With `--fix`, residual post-fix findings determine the code. + +## State Transitions + +### How a finding becomes a fix (apply_fixes_incremental path) + +``` +LintDiagnostic.fix_removals (FixLineSpan) OR .fix_edits (TextEdit) + → diag_to_edits() → Vec + → plan_fixes_with_options(): + sort (start ASC, end DESC) + → dedup_contained_or_identical() — drop edits contained in wider edits + → overlap detection — any partial overlap: FixPlan { overlap_rejected: true, edits: [] } + → apply_fixes_incremental(): + all_output_neutral? → skips equality gate if any edit is legacy-interpolation + batch attempt (1 reverify call) → passes? → FixOutcome::Fixed + batch fails → per-edit right-to-left retry → accept | RejectedEdit + all rejected → FixOutcome::Rejected { reason: reverify_failure_reason(&err) } + ≥1 accepted, ≥1 rejected → FixOutcome::PartiallyFixed + → CLI: Fixed/PartiallyFixed → atomic_write_file() + | Rejected → "fix rejected: ..." + original diagnostics + | NothingToFix → pass through +``` + +## Anti-Patterns + +- **Re-inlining the tier table**: Adding tier logic to `fix.rs` or `diagnostic.rs` instead of importing from `tier.rs` recreates the circular dependency the leaf module was designed to prevent. + +- **Splitting a TextEdit into separate open/close edits**: `legacy-interpolation` must emit one atomic `TextEdit` replacing the entire `{expr}` span. Split edits allow per-edit fallback to accept only the close half, producing `{expr}}` garbage that compounds on repeated `--fix` runs. + +- **Forking a second escape map** (the anti-pattern `sanitize_control_chars_wire` was created to prevent): WIRE and HUMAN share one implementation via `sanitize_with(s, EscapeMode)`. A second table that duplicates the character class but changes one entry will diverge silently when the class is extended. + +- **Using HUMAN mode for an identifier or filename**: `sanitize_control_chars` (HUMAN) preserves `\n`, so a hostile filename routed through it can still forge a standalone status line. Use `safe_path`, `safe_file_display`, or `safe_inline` (all WIRE) for identifiers and filenames. + +- **Adding a bare interpolating `eprintln!`**: `crates/mds-cli/tests/print_discipline.rs` will fail CI immediately. The guard checks `crates/mds-cli/src/**` and fails on any interpolation not routed through a WIRE helper. + +- **Post-processing a rendered miette frame with any sanitizer** (PF-014): Sanitizing the rendered output escapes miette's own ANSI SGR colour codes into `\u001B[33m` noise on TTYs. CI uses `NO_COLOR=1` and piped stderr so this regression would stay green indefinitely. Pre-sanitize inputs before constructing the `Report`. + +- **Putting U+061C in the 3-byte neutralization branch**: U+061C is 2 bytes in UTF-8. Routing it through the 3-byte branch (`U+FFFD`) fires the byte-length `debug_assert_eq!` (13 vs 12 bytes). This was proven, not theorized, during the #176 development. U+061C belongs in `is_two_byte_format_hazard`. + +- **Omitting `0xD8` from the fast-path byte scan in `sanitize_with`**: U+061C is encoded as `0xD8 0x9C`. Without `0xD8` in the fast-path, `sanitize_control_chars("a\u{061C}b")` returns `Borrowed` and skips the character entirely. + +- **Resting a security invariant on `debug_assert!`** (PF-005): `debug_assert!` is compiled out of release. The old `SanitizedReport` returned `None` for `source()`/`related()` behind a `debug_assert!` that no CLI error populates the aux graph — real in tests, absent in the shipped binary. Enforce invariants with data transformation, not assertions. + +- **Calling `apply_plan_unchecked()` on a production write path**: Production code that writes back to disk MUST use `apply_fixes()` or `apply_fixes_incremental()`. The `_unchecked` suffix makes the bypass explicit at every call site. + +- **Adding a ModuleCache "optimization"**: Per-file fresh resolve is intentional. A shared cache would be unsafe because runtime vars are per-call. + +- **Calling `sanitize_control_chars` in a `LintDiagnostic` constructor, or on source text passed to `NamedSource`**: Constructors must keep raw bytes so span offsets and fix-edits remain accurate. Source text for miette must use `neutralize_source_for_render` (byte-length-preserving) — `sanitize_control_chars` expands 1–2-byte control chars to 6 bytes, desynchronising every span offset that follows. + +- **Using `format!()` to build the canonical JSON**: Always use `serde_json::json!()`. + +- **Sorting the file list after processing** in directory mode: `collect_mds_files()` does NOT return a sorted list. Sort with `files.sort()` before the loop (F1 invariant). + +- **Indexing source with `source[..offset]` in fix logic**: Panics on non-char-boundary offsets. Use `source.get(..offset)?` (fail-closed → None) as required by ADR-001 (REL-1). + +- **Omitting `set_diag_display_path` in directory mode**: Without it, every file in a directory lint run maps its diagnostics under the same basename key in JSON. + +- **Setting `fix_removals` on `unused-import`**: Partial-name removal from an import list is structurally ambiguous and unsafe. `unused-import` always leaves `fix_removals: None`. + +## Gotchas + +**WASM budget raised three times**: 700K→750K (S2 lint rules), 750K→800K (S4 full surface), 800K→850K (v0.4.0 dogfood remediation). Current guard in `ci.yml`: **850,000 bytes**. + +**Two-pass artifact for `\{x\}` remnants**: Fixing a `\{x\}` remnant removes the backslash, leaving bare `{x}`. On the NEXT lint pass, that `{x}` is flagged as a `legacy-interpolation` single-brace expression. Two `--fix` runs are needed to fully migrate. + +**Mixed batch and output-equality bypass**: When a fix plan contains any `legacy-interpolation` edit, `all_output_neutral = false` and the output byte-equality sub-check is skipped for the **entire batch**. Assessed P2 risk. + +**is_standalone requires BOTH conditions**: A file is standalone only when `!is_partial_or_extends && ctx.imports.is_empty()`. A file with `@extends` but no `@import` is NOT standalone. + +**Tier B unused-import cannot fire on standalone files**: A standalone file has `imports.is_empty()` by definition — the rule never fires in the only context where Tier B fixes would be attempted. + +**Containment coalescing resolves same-block multi-rule conflicts**: When two rules fire on the same block (e.g. `unreachable-branch` spans the full dead `@if`/`@end`, `empty-block` spans only the inner `@else` body), the containment step keeps only the wider edit. + +**Partial-overlap from 10 real rules is structurally impossible via CLI**: All current rules emit spans that are disjoint or containment-related. The `overlap_rejected` path is only reachable with synthetic diagnostics. Regression anchors: `fix.rs::a4_partial_overlap_still_rejected_after_dedup` and `lint.rs::preview_fixes_surfaces_rejected_on_overlap`. + +**span.line / span.column are never in lint diagnostic JSON**: All 10 rules pass `None` for `line`/`column`. + +**examples/stress-test/errors/ fixtures contain intentional errors**: Running `mds lint examples/` will exit 2 by design. + +**shadow-variable is info AND default-off**: Only fires when explicitly configured. `Info` findings never contribute to the exit code. + +**Unknown rule NAMES vs unknown severity VALUES behave differently**: An unknown rule name in `mds.json` is warned about and ignored. An unknown severity value fails loudly with a serde deserialization error (exits 2). + +**D2 mechanical ripple in resolver.rs**: The `..` in the three `ExportDirective` match arms in `resolver.rs` is intentional — it acknowledges the new `offset` field without reading it. + +**Frontmatter key span is approximate**: `FmVarFact.approx_offset` is `Option` — it can be `None` when substring search fails. + +**`assertKnownKeys` must be called before backend dispatch**: The validation runs synchronously in the wrapper, before `init()` is awaited or any backend is invoked. + +**atomic_write_file temp prefix**: The temp file prefix is `.mds-tmp-`. Both lint and fmt share the same `atomic_write_file` from `output.rs`. + +**Python `LintDiagnostic.fix_edits` getter vs `#[pyo3(get)]`**: `Vec` does not implement `IntoPy`. Use the custom `#[getter]` which calls `value_to_py`. Stored internally as `Option>`. + +**Python `LintDiagnostic.to_dict()` always includes `fix_edits`, `help`, and `span` keys**: All three are emitted as Python `None` (JSON `null`) when not set — never absent. This matches `to_canonical_json()` exactly (PF-007 guard). + +**Reverify rejection message**: Exact stable text: `"could not verify fix — the edited source did not re-parse cleanly ({err}); leaving the file unchanged"`. Test `A5` in `cli_lint.rs` pins this. + +**B1 attribution test pattern — use `MdsError::TypeMismatch { src, .. }`, not offsets**: `SerializedError` has no `file` field (PF-012). See `crates/mds-core/tests/virtual_fs.rs` B1 tests. + +**serde_yaml_ng rejects raw ESC/DEL in YAML double-quoted keys, but U+0085 NEL passes**: ESC (U+001B) and DEL (U+007F) in YAML double-quoted string keys raise a parser error. U+0085 NEL (`0xC2 0x85`) IS valid YAML `c-printable` and passes through — a reachable ESC-injection vector for `unused-variable` and rules that embed import paths. + +**`render_error_sanitized` is private and does no post-processing**: It is just `format!("{report:?}")` on the `SanitizedReport`. Do NOT expect it to sanitize content — sanitize inputs before constructing the `Report`. Use `eprint_error` for CLI output (which wraps in `SanitizedReport` before calling it). + +**`MdsError::Display` is explicitly unsanitized**: `e.to_string()` / `eprintln!("{e}")` may emit raw C0/DEL/C1 bytes. Use `e.display_sanitized()` for terminal output or `e.serialize().message` for JSON/binding output. In the CLI, `eprint_error` handles this; downstreams of the published crate should use `display_sanitized()`. + +**napi build script is `build:native`, NOT `build`**: Running `npm run build -w @mdscript/mds-napi` silently does nothing useful. Use `npm run build:native -w @mdscript/mds-napi`. + +**`packages/mds` prefers the dev WASM artifact**: `packages/mds/src/backend/wasm.ts` resolves to `crates/mds-wasm/pkg/` (the `wasm-pack` dev output) rather than `packages/mds-wasm/dist/node/`. Rebuilding only the `packages/mds-wasm` npm package leaves a STALE backend active, and the cross-surface differential test fails with convincing-looking divergence that isn't a real bug. Always rebuild via `wasm-pack build crates/mds-wasm` when working on WASM output. + +## Related Follow-ups / Known Limitations + +- **#173**: `run_lint_file` FixFileOutcome 3rd-copy duplication and dir-mode JSON per-file wrapper churn. +- **#179**: Entry file read 2–3× — a raw `std::fs::read` call in lint.rs bypasses `NativeFs`. +- **#180**: `LintOptions.basePath` vs `CompileOptions` asymmetry. +- **#202**: Diagnostic ordering — within a file, the order of diagnostics across rules is currently arbitrary. +- **#203**: `unused-import` span anchor — points to the full `@import` directive, not the specific unused name. + +## Key Files + +- `crates/mds-core/src/lint/mod.rs` — engine entry point; `lint_source()`, `run_rules()`, partial detection +- `crates/mds-core/src/lint/tier.rs` — fix tier table (leaf module); `is_output_neutral(rule)`; `first_occurrence` helper +- `crates/mds-core/src/lint/diagnostic.rs` — `LintDiagnostic`, `LintResult`, `to_canonical_json()` (WIRE: message/help/file key); `sanitize_control_chars` (HUMAN, `Cow`, `#[must_use]`, idempotent); `sanitize_control_chars_wire` (WIRE, new public API, shares one impl via `EscapeMode`); `neutralize_source_for_render` (byte-length-preserving: C0/DEL → `?`, C1+U+061C → NBSP, other 11 hazards → U+FFFD); `named_source_for_render` (new public API, the single `NamedSource` builder); `is_two_byte_format_hazard` / `is_three_byte_format_hazard` +- `crates/mds-core/src/error.rs` — `MdsError`: `serialize()` (WIRE message/help); `display_sanitized()` (HUMAN Display for TTY); raw `Display` documented as unsanitized; `at()` (uses `named_source_for_render` — inherited by all `*_at` constructors) +- `crates/mds-core/src/lib.rs` — `CompileResult::to_canonical_json()` (WIRE warnings, distinct from `LintResult::to_canonical_json`); `emit_warnings()` (HUMAN for prose; identifiers WIRE at construction) +- `crates/mds-core/src/lint/fix.rs` — `plan_fixes_with_options`, `diag_to_edits`, `ByteEdit`, `apply_plan_unchecked`, `dedup_contained_or_identical`, `apply_fixes_incremental`, `FixOutcome`; `reverify_failure_reason()` (sole construction site for `Rejected.reason`, WIRE) +- `crates/mds-core/src/lint/rules/legacy_interpolation.rs` — Tier A token-based rule; atomic single TextEdit per finding; two-pass artifact for backslash-escape fixing +- `crates/mds-core/src/lint/facts.rs` — `collect_facts()`, `AnalysisContext`, `DefineFact { name, offset, end_offset }` +- `crates/mds-core/src/lint/config.rs` — `LintConfig` (lives in mds-core; CLI converts to it) +- `crates/mds-core/src/ast.rs` — `ElseifBranch { offset }`, `IfBlock { else_offset, end_offset }`, `ForBlock/DefineBlock { end_offset }` +- `crates/mds-core/src/lint/rules/` — 10 rule modules + `structural_eq.rs` +- `crates/mds-cli/src/lint.rs` — CLI subcommand; `render_diag_human` (HUMAN for message/help; filename+source via `named_source_for_render`; all status lines via `safe_path`); `set_diag_display_path`, `LintDirCtx`, `KNOWN_RULES` +- `crates/mds-cli/src/output.rs` — `atomic_write_file`; `eprint_error` (single CLI stderr choke-point, wraps in `SanitizedReport`); `SanitizedReport` / `SanitizedNode` / `MAX_AUX_DEPTH`; `render_error_sanitized` (private, plain `format!("{report:?}")` on sanitized wrapper); `eprint_warning` (HUMAN, new); `safe_path` / `safe_file_display` / `safe_inline` (all WIRE, new); `preview_text_for` (TTY-gated source neutralization for `--diff`); `render_unified_diff` / `colorize_unified_diff` +- `crates/mds-cli/src/build.rs` — `LintCliConfig` struct, `into_core_config()`, `MdsConfig.lint` field +- `crates/mds-cli/src/watch.rs` — all 11 error prints route through `eprint_error`; lifecycle status lines route through `safe_path` / `safe_inline` / `eprint_warning` +- `crates/mds-cli/tests/print_discipline.rs` — CI-enforced lexical guard; SANITIZERS allowlist; `ALLOWED_UNSANITIZED` allowlist; `every_allowlist_entry_is_live` rot check +- `crates/mds-cli/tests/security.rs` — T-ESC-MSG-1/2, T-ESC-RULE-1, T-ESC-FNAME-1/2, T-ESC-WALK-1 +- `crates/mds-wasm/src/lib.rs` — `lint()` and `lintVirtual()` exports; `parse_check_options` strict allow-list +- `crates/mds-napi/src/lib.rs` — `lint`, `lintFile`, `lintVirtual`; `extract_rules_direct`; `parse_lint_file_opts` +- `crates/mds-python/src/lib.rs` — `LintDiagnostic` frozen pyclass; `LintResult::new()` calls `sanitize_lint_value()` (WIRE, construction-time — closes PF-004 parallel-path gap) +- `packages/mds/src/types.ts` — `LintDiagnostic.fix_edits`; `CheckOptions { vars? }` +- `packages/mds/src/util/options.ts` — `assertKnownKeys` (strict unknown-option rejection) +- `crates/mds-cli/tests/cli_lint.rs` — A5 (reverify rejection message prefix), L-CLI-RESOURCE (exit-3), L-CLI-DIR2 (file-order determinism); T-5..T-9 ESC-injection anchors + +## Related + +- **ADR-007** (sanitizer escape-map contract): `sanitize_control_chars` / `sanitize_control_chars_wire` share one implementation via `EscapeMode`; forking a second escape table is the explicit anti-pattern this prevents. +- **PF-004** (parallel-path enforcement gaps): `SanitizedReport` wraps at the `Report` level to cover both `MdsError` and CLI `miette!()` error families unconditionally. Python `LintResult::new()` calls `sanitize_lint_value()` construction-time. `print_discipline.rs` enforces the per-field rule mechanically. +- **PF-005** (debug_assert-only invariants absent in release): `SanitizedReport` materialises the auxiliary graph at construction instead of returning `None` behind a `debug_assert!` — the earlier revision's guarantee held in tests and was absent in the shipped binary. +- **PF-007** (cross-surface goldens can't catch divergence): `fix_edits` is emitted unconditionally (null when None) across all surfaces; differential tests cover cross-surface parity. +- **PF-013** (vacuous negative security tests): Every ESC-injection test now pairs a NEGATIVE assertion (raw byte absent) with a POSITIVE one (escaped form present) and a non-vacuity guard (diagnostics non-empty, expected rule matched). T-9 was rewritten from a vacuous YAML-rejection vector to a reachable duplicate-import + U+0085 NEL vector. +- **PF-014** (sanitize inputs, not rendered artifacts): The `SanitizedReport` design — pre-sanitize message/help/labels before miette renders — is the PF-014-correct boundary. Post-processing the rendered frame corrupts miette's own ANSI SGR codes; CI uses `NO_COLOR=1` and pipes stderr so the failure would stay green. T-ESC-6 pins this on the colour path. +- **ADR-001** (span-guided rewrite + compile-equivalence gate): All `--fix` edits are span-guided byte rewrites. `TextEdit` ranges are validated fail-closed. `apply_plan_unchecked` is explicitly named to make ADR-001 bypass visible. +- **ADR-004** (three-tier --fix safety model, reverify gate): `apply_fixes_incremental`'s batch-first strategy with bounded per-edit fallback is the AC-F-20 implementation. +- **ADR-002** (v0.4.0 whitespace contract, interior-verbatim): The `empty-block` rule's "whitespace-only-Text body" definition is directly downstream of this contract. +- **ADR-003** (@extends FM emission): The `unused-variable` rule is suppressed on `@extends` children. +- **PF-012** (span source-identity): For test attribution (B1 tests), `SerializedError` has no `file` field — use `MdsError::TypeMismatch { src, .. }` pattern-match. +- `crates/mds-core/tests/api_surface.rs` — pins the public lint API signatures. +- `.devflow/features/mds-fmt/KNOWLEDGE.md` — `mds fmt` knowledge base; `atomic_write_file` is shared between both subcommands via `output.rs`. +- `.devflow/features/source-map-security/KNOWLEDGE.md` — source map path-containment choke-point. diff --git a/CHANGELOG.md b/CHANGELOG.md index e5a6c52b..b46e3cd0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -62,6 +62,222 @@ field is present across all binding surfaces: CLI JSON output, napi project root (located via `.mdsroot` / `.git` walk-up), and `..`-escaping references outside the project root fall back to the basename. (#3) +- **Control-byte injection hardening (CWE-150 / #176):** Raw C0 / DEL / C1 + control bytes in `.mds` source content could reach terminal stderr and + JS / Python / WASM API error messages, enabling terminal escape-sequence + injection. The serialization and diagnostic-render boundaries hardened here are + `MdsError::serialize()` (inherited by all three binding layers), + `LintResult::to_canonical_json()` including the `"file"` group key, + `CompileResult::to_canonical_json()` warnings, and the CLI render path. That is + an audit list, **not a closed set**: the governing rule is the per-field one + below, and the residual it leaves is named there. Enumerating boundaries is + exactly the framing this changelog retires further down. + The CLI render path (PF-014 redesign) sanitizes the renderer's *source-excerpt* + input byte-length-preservingly — hostile C0/DEL/C1 bytes become `?` (C0/DEL) or + NBSP (C1) so span offsets and caret columns stay exact and miette's own SGR + colour codes survive intact on TTY. `message` and `help` are renderer inputs too, + but they are `\uXXXX`-escaped rather than length-preserved; only source text + carries the byte-length invariant. A new + `MdsError::display_sanitized()` public API is provided for Rust consumers; + the raw `Display` impl is preserved with an explicit unsafety contract in + its rustdoc. `span` byte offsets, `fix_edits` byte ranges, and `rule` + identifiers are deliberate exclusions — they carry position data, not + terminal-bound text. (#176) + +- **CLI error *message* text is now escaped too (#176).** The hardening above + covered rendered source excerpts, filenames, and the diagnostic wire boundaries, but a + diagnostic's own message and help text still reached stderr raw. Both CLI error + families interpolate untrusted input into their messages — compiler errors carry + template text (`invalid include alias: ''`) and CLI errors carry `mds.json` + values and filesystem paths (`mds.json output_dir '' must not contain '..'`) + — so a hostile `.mds` file or config value could still emit raw ANSI escape + sequences to a terminal. `mds build`, `check`, `fmt`, `lint`, and `watch` now escape + each report's message, help, and caret-label text at the single `eprint_error` + choke-point, *before* the diagnostic renderer runs. The rendered frame is still never + post-processed, so terminal colour and caret alignment are unaffected, and output for + well-formed input is byte-for-byte unchanged. (#176) + +- **Every CLI print now escapes what it interpolates, and CI enforces it (#176).** + Warning and status prints scattered across `main.rs`, `build.rs`, `fmt.rs`, `lint.rs`, + `watch.rs` and `output.rs` interpolated filenames, `mds.json` rule names, `--format` + arguments and `io::Error` causes into `eprintln!` raw, bypassing the + `sanitize_control_chars` call that `mds-core`'s `emit_warnings` applies on the primary + code paths (a PF-004 parallel-path gap). The two most directly reachable: + + - a rule NAME in `mds.json` is an arbitrary JSON object key, and a JSON `\uXXXX` + escape decodes to a real byte, so any repository could put a raw ESC on a + developer's stderr — or forge whole `Clean: …` / `0 problems found` status lines — + just by being linted; + - the shared directory walker's depth-limit warning named the directory it stopped at, + so one hostile directory name reached `mds build`, `check`, `fmt`, `lint` and + `watch` at once. + + All of them now apply the per-field rule below: the warning *body* goes through + `eprint_warning` (HUMAN), and every value interpolated into it goes through + `safe_path` / `safe_inline` (WIRE). `watch.rs`'s lifecycle status lines + (`Watching {}`, `Removed {}`, `warning: could not remove {}: {e}`) — previously + carved out as a pre-existing gap — are included. + + **This is now a machine-checked invariant, not an enumeration.** A new + `crates/mds-cli/tests/print_discipline.rs` fails CI if *any* print macro under + `crates/mds-cli/src/**` interpolates a value that is not passed through one of the + escape helpers. It applies the same rule to the argument of `eprint_warning` + (HUMAN-mode escaping alone is not sufficient — it preserves `\n`, which is the + line-forgery vector), including when the message has been hoisted into a local: + a bare identifier is traced one hop through its `let` binding and judged the same + way, and an argument the trace cannot resolve is **reported**, not trusted. Because + `let`s are matched file-wide, every `for` variable, function parameter and closure + parameter **poisons** its own name, so a value arriving through one of those is + reported rather than resolved against an unrelated `let` that happens to share the + name. It also + scans `write!` / `writeln!` to a stdout/stderr handle. Deliberate exceptions — the + compiled artefact written to stdout, `&'static str` labels, integer counters, and + whole warning strings produced by `mds-core` — live in explicit allowlists with a + written justification per entry, and a companion test fails if an entry ever stops + matching. The guard is a **lexical** scanner: it catches accidental reintroduction, + and its five known limits (name-matched sanitizers, anti-rot-not-anti-reuse + allowlists, the one-hop single-file trace, name-based stream detection, and the + `if let` / `while let` / `match`-arm binders the poison set does not model) are stated + in its own rustdoc rather than implied away. Four successive reviews of this change + each found a *different* unescaped print; the guard is what ends that. + + The one precondition the guard depends on and cannot check — that `mds-core` WIRE-escapes + the identifiers its warning producers interpolate, since `mds-cli` prints whole + warning strings — is now pinned by `crates/mds-cli/tests/producer_discipline.rs` for + the only producer whose input can carry a hostile character (`resolver.rs`'s + imported-module filename). The other two producers interpolate an `@include` alias, + which the parser restricts to `[A-Za-z_][A-Za-z0-9_]*`, so they are upheld by review + and stated as such rather than claimed to be tested. (#176) + +- **The escape mode is chosen per field, not per surface (#176).** Normative in spec + §7.5: **on the diagnostic surfaces — the `"version": 1` JSON wire, CLI status and + warning lines, `[file:line:col]` frame headers — untrusted identifiers, filenames and + error causes are WIRE-escaped, human terminal output included; prose — a diagnostic + message or help body — stays HUMAN so multi-line frames keep rendering.** The + rule governs *diagnostic* output; the two carve-outs below are not diagnostics and are + not escaped at all. The discriminator is whether the + value is ever legitimately multi-line: a filename, a config key, a `--format` + argument and an `io::Error` never are, so preserving a raw `\n` in one buys nothing + and lets it forge a standalone line byte-identical in form to genuine output + (CWE-117). This supersedes the earlier per-surface framing and the "wire mode at + exactly four boundaries" enumeration. A new `mds::sanitize_control_chars_wire` and + `mds::named_source_for_render` are public in `mds-core` for consumers that need to + apply the same rule. + + **Declared carve-out: functional path references are NOT escaped.** Source-map + documents (the `mds build --source-map` sidecar, and the `sourceMap` embedded in + `CompileResult.to_canonical_json()`) emit their `file`, `sources` and `sourcesContent` + values **verbatim**, as does the `dependencies` array. These are functional references + that devtools, bundlers and IDEs resolve against the filesystem — rewriting a path to a + `\uXXXX` literal would point at a path that does not exist, breaking source-map + resolution and dependency tracking to defend against a pathological filename. That is + the same product-versus-display distinction that keeps compiled output byte-faithful. + **Consumers of a source map or of `dependencies` must treat every path in them as + untrusted** and escape it for whatever destination they render it to; JSON string + encoding is not that escaping, since a decoded `"\n"` is a real newline again. The CLI + does not rely on this: its `Compiled to …` and `Source map written to …` lines print + through `safe_path` and carry the escaped form even though the sidecar does not. + Specified in spec §7.5 ("Carve-out: functional path references"). (#176) + + **Declared residual.** "Identifier / filename / cause" means the value occupies such a + *field* — a CLI status line, a `[file:line:col]` frame header, the JSON `file` key. A + path or identifier interpolated into a diagnostic **message body** is part of prose, + so it follows the message row and stays HUMAN on terminal surfaces. That applies at + both message-construction sites: the CLI's `miette::miette!()` reports **and** + `mds-core`'s `MdsError` message bodies (`fs.rs`'s `cannot read {path}: {e}`, + `parser_helpers.rs`'s `invalid import alias: '{alias}'`). A `\n` in one of those + survives into the rendered frame and takes a line there. It is a weaker surface than a + status line — frame content is indented and `│`-prefixed, and the prefix survives + `strip()`, so it cannot masquerade as genuine bare status output — and no raw control + byte reaches the terminal either way. Closing it means WIRE-escaping over a hundred + `MdsError` construction sites and changing the public message text seen by all three + binding layers; that is a separate change. Disclosed in spec §7.5 ("Residual: paths and + identifiers inside a message body") and in the boundary table in + `crates/mds-core/src/lint/diagnostic.rs`. (#176) + +- **Hostile filenames can no longer forge CLI status lines (CWE-117 / #176).** POSIX + permits a newline inside a filename, and directory-mode commands discover names by + walking the tree — the user never types them. Filename display used HUMAN mode, which + preserves newlines by design so that multi-line diagnostic *messages* keep rendering, + so a file named `evil.mdsClean: real.mdsOK: all-fine.mds` made + `mds build`/`lint`/`fmt`/`check` emit attacker-authored lines byte-identical in form + to genuine status output — unframed, unindented, and indistinguishable. **Diagnostic + filename fields are now escaped in WIRE mode on every surface that renders one, human + included** (source-map paths and `dependencies` are the declared carve-out): `safe_path` + and the status-line printers, and the `[file:line:col]` frame header via a new shared + `mds::named_source_for_render` builder that `MdsError::at()`, the formatter and the + lint renderer all call. Message and help text are unchanged (still HUMAN, still + multi-line). A filename is never legitimately multi-line, so nothing legitimate is + lost. (#176) + +- **Fix-rejection reasons are display-safe by construction (#176).** + `mds::fix::FixOutcome::Rejected.reason` interpolated an `MdsError`'s deliberately-raw + `Display` — whose variants embed template text (`syntax error: {message}`) and + filesystem paths (`file not found: {path}`) — and the CLI prints that value as an + unframed `fix rejected: {reason}` status line. The embedded error is now escaped in + WIRE mode at the single construction site in `fix.rs`, so the field is single-line and + control-byte-free for **every** consumer of the published `mds::fix` API, not just the + CLI's own print sites. (#176) + +- **Widened escape class: bidi / separator / BOM characters (#176).** The + escaped set now covers characters outside C0 / DEL / C1 that are still + display-hazardous, on every surface that escapes: + - **U+061C, U+200E, U+200F, U+202A–U+202E, U+2066–U+2069** — the complete + Unicode `Bidi_Control=Yes` set (all twelve codepoints), behind Trojan Source + (CVE-2021-42574). A single U+202E in a filename or diagnostic message + reverses how the rest of the line renders in any bidi-aware terminal, IDE, or + code-review UI. U+061C ARABIC LETTER MARK is the only member outside + U+200E–U+2069 and is easy to miss for exactly that reason. + - **U+2028, U+2029** — LINE / PARAGRAPH SEPARATOR, which terminate a + JavaScript string literal. + - **U+FEFF** — BOM / ZWNBSP, invisible in every renderer. + + Each becomes its uppercase six-character `\uXXXX` literal, exactly like the + existing C0 / DEL / C1 escapes. Source excerpts inside a rendered diagnostic + frame are neutralized to a **same-width** substitute instead, preserving the + byte-length invariant that keeps span offsets and caret columns exact: 1-byte + C0/DEL → `?`, 2-byte C1 and U+061C → U+00A0, 3-byte bidi controls, separators + and BOM → U+FFFD. (#176) + +- **BREAKING (wire format): machine-readable boundaries now escape `\n` (#176).** + `MdsError::serialize()`, `LintResult::to_canonical_json()` (message, help, and + the `"file"` group key), `CompileResult::to_canonical_json()` warnings, and the + Python typed lint surface now emit `\n` as the six-character `\u000A` literal. + A raw newline inside a diagnostic string is a line-forging vector: any consumer + that prints or line-splits the value can be made to render an attacker-authored + line as a genuine second finding. `\t` is unaffected. + + **Human-render output of diagnostic PROSE is unchanged** — the CLI renderer, + `MdsError::display_sanitized()`, and warning *bodies* on stderr still preserve + raw newlines so multi-line diagnostic frames stay readable. Human output of + diagnostic filenames, identifiers and causes **did** change, by design: under the + per-field rule above those are WIRE-escaped on every surface that renders a + diagnostic, human included, so a newline + in one now renders as the six-character `\u000A` literal instead of forging a + line. Source-map paths and `dependencies` are unaffected: they are the declared + carve-out and stay verbatim. See the two entries below. + + **Migration:** consumers that split a `message` / `help` / warning string on + `\n` will now see a single line containing the literal `\u000A` where a real + newline used to be. Split on that literal instead, or render the value verbatim. + +- **Escaping is one-way — consumers must not un-escape (#176).** The + transformation is lossy and non-injective by design: a template that literally + contains the six characters `\u001B` and one containing an actual ESC byte are + indistinguishable after serialization. **Do not** convert `\uXXXX` sequences + back into bytes — that reconstitutes the injection the escape prevents. Round-tripping is + an explicit non-goal; no backslash-escaping will be added to make the mapping + reversible. Consumers needing original bytes must read them from the source via + the raw `span` / `fix_edits` byte offsets, which stay unsanitized for this + purpose. Documented normatively in spec §7.5. + +- **`--diff` preview output is TTY-gated (#176).** Applies to both + `mds lint --fix --diff` and `mds fmt --diff`, which share one renderer. + Preview diff text is neutralized when stdout is a terminal (where control bytes + would execute) and emitted **byte-faithful when piped or redirected**, so a + redirected diff remains applicable. Preview output is not part of the + `"version": 1` JSON wire format. + ### **BREAKING** — Strict cross-type comparisons, merged `@extends` frontmatter, interior-verbatim whitespace, filesystem API These changes alter observable runtime behavior and compiled output. Templates relying @@ -156,6 +372,21 @@ directly via `ModuleCache::with_fs`. `source_map_base: None` or use the `..Default::default()` tail. Binding surfaces (napi, Python, WASM) are not affected. (#3) +### **BREAKING** — Error/lint messages now carry `\uXXXX` literals for embedded control bytes (#176) + +Across the JS / Python / WASM API surfaces, `err.message`, `err.help`, and lint +`LintDiagnostic.message` / `LintDiagnostic.help` now contain six-character `\uXXXX` +Unicode escape literals (e.g. `\u001B`, `\u007F`, `\u0085`) wherever MDS source +content caused raw C0-minus-`\n`/`\t`, DEL (U+007F), or C1 (U+0080–U+009F) control +bytes to appear in error or diagnostic messages. + +**Not affected:** `span.offset`, `span.length`, and `fix_edits` byte ranges are raw +byte offsets and are never sanitized. The `rule` field is a fixed ASCII identifier. +The `"file"` key in lint JSON output is sanitized on the same pass as `message`/`help`. + +**Migration:** consumers that test for exact control byte sequences in error or +diagnostic messages must update to check for the `\uXXXX` literal form instead. + ### Added - **`--set-string KEY=VALUE`** CLI flag for `mds build`, `mds check`, and `mds watch`. diff --git a/Cargo.lock b/Cargo.lock index 8dd9cc68..a02a045d 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -570,6 +570,7 @@ dependencies = [ "serde_json", "similar", "tempfile", + "thiserror", ] [[package]] diff --git a/crates/mds-cli/Cargo.toml b/crates/mds-cli/Cargo.toml index 83345bf9..f0bc020f 100644 --- a/crates/mds-cli/Cargo.toml +++ b/crates/mds-cli/Cargo.toml @@ -25,5 +25,8 @@ ctrlc = { workspace = true } similar = { workspace = true } tempfile = { workspace = true } +[dev-dependencies] +thiserror = { workspace = true } + [target.'cfg(unix)'.dev-dependencies] libc = "0.2" diff --git a/crates/mds-cli/src/build.rs b/crates/mds-cli/src/build.rs index a2b32e98..f9c71334 100644 --- a/crates/mds-cli/src/build.rs +++ b/crates/mds-cli/src/build.rs @@ -8,8 +8,8 @@ use std::io::Read; use std::path::{Path, PathBuf}; use mds::{ - effective_parent, sanitize_control_chars, CompiledOutput, MdsError, MAX_FILE_SIZE, - MAX_TRAVERSAL_DEPTH, STRING_SOURCE_MAP_LABEL, + effective_parent, CompiledOutput, MdsError, MAX_FILE_SIZE, MAX_TRAVERSAL_DEPTH, + STRING_SOURCE_MAP_LABEL, }; use miette::Result; use serde::Deserialize; @@ -189,6 +189,18 @@ impl OutputKind { } } +/// Human-readable label for an [`OutputKind`], for the extension-mismatch warning. +/// +/// Returns one of two `&'static str` literals and carries no runtime data — which is +/// why `kind_label(kind)` is allowlisted in `tests/print_discipline.rs` instead of +/// being wrapped in a sanitizer. +fn kind_label(kind: OutputKind) -> &'static str { + match kind { + OutputKind::Markdown => "markdown (.md)", + OutputKind::Messages => "messages JSON (.json)", + } +} + impl From<&CompiledOutput> for OutputKind { fn from(output: &CompiledOutput) -> Self { match output { @@ -282,13 +294,19 @@ pub(crate) fn resolve_output_path_for_kind( if let Some(ext) = path.extension().and_then(|e| e.to_str()) { let expected = kind.extension(); if ext != expected { + // The `-o` value and the extension derived from it occupy a + // diagnostic `file` field on a status line: WIRE (spec §7.5 + // per-field rule). + // The escape call is repeated rather than bound to a local so it + // is visible at each interpolation — the print-discipline guard + // reads call sites, not bindings. eprintln!( - "warning: output path '{o}' has extension '.{ext}' but compiled \ - output is {kind_name}; writing to '{o}' anyway", - kind_name = match kind { - OutputKind::Markdown => "markdown (.md)", - OutputKind::Messages => "messages JSON (.json)", - } + "warning: output path '{}' has extension '.{}' but compiled \ + output is {}; writing to '{}' anyway", + crate::output::safe_inline(o), + crate::output::safe_inline(ext), + kind_label(kind), + crate::output::safe_inline(o) ); } } @@ -561,7 +579,7 @@ pub(crate) fn write_output( std::fs::write(&path, compiled) .map_err(|e| miette::miette!("cannot write {}: {e}", path.display()))?; if !quiet && announce { - eprintln!("Compiled to {}", path.display()); + eprintln!("Compiled to {}", crate::output::safe_path(&path)); } } None => { @@ -691,7 +709,7 @@ pub(crate) fn compile_to_content( if !quiet { for w in &result.warnings { - eprintln!("{w}"); + crate::output::eprint_warning(w); } } @@ -1003,7 +1021,7 @@ pub(crate) fn verify_then_delete_map(map_path: &Path, expected_basename: &str, q if !quiet { eprintln!( "warning: leaving {} in place — not a tool-generated SMv3 map (version/file mismatch)", - map_path.display() + crate::output::safe_path(map_path) ); } return; @@ -1012,7 +1030,7 @@ pub(crate) fn verify_then_delete_map(map_path: &Path, expected_basename: &str, q if !quiet { eprintln!( "warning: leaving {} in place — not a tool-generated SMv3 map (version/file mismatch)", - map_path.display() + crate::output::safe_path(map_path) ); } return; @@ -1020,12 +1038,13 @@ pub(crate) fn verify_then_delete_map(map_path: &Path, expected_basename: &str, q if let Err(e) = std::fs::remove_file(map_path) { if !quiet { eprintln!( - "warning: could not remove stale map {}: {e}", - map_path.display() + "warning: could not remove stale map {}: {}", + crate::output::safe_path(map_path), + crate::output::safe_inline(&e) ); } } else if !quiet { - eprintln!("Removed stale map {}", map_path.display()); + eprintln!("Removed stale map {}", crate::output::safe_path(map_path)); } } @@ -1053,7 +1072,7 @@ pub(crate) fn run_build(args: BuildArgs) -> Result<()> { // When auto-detected, print a "Building {path}" banner so users know which file was selected. let (input, auto_detected) = resolve_input(input, "build")?; if auto_detected && !quiet { - eprintln!("Building {}", input.display()); + eprintln!("Building {}", crate::output::safe_path(&input)); } // Directory mode: compile every non-partial .mds file in the tree. @@ -1145,7 +1164,7 @@ pub(crate) fn run_build(args: BuildArgs) -> Result<()> { .map_err(miette::Error::from)?; if !quiet { for w in &result.warnings { - eprintln!("{w}"); + crate::output::eprint_warning(w); } } @@ -1192,12 +1211,22 @@ pub(crate) fn run_build(args: BuildArgs) -> Result<()> { write_output(Some(out.clone()), &content, quiet, true)?; if let Some(ref sm) = source_map { let map_path = map_path_for(out); + // The sidecar's `file` / `sources` / `sourcesContent` are + // written VERBATIM, by decision — spec §7.5 "Carve-out: + // functional path references". They are resolved against the + // filesystem by devtools and bundlers, so a `\uXXXX`-escaped + // path would not exist. Consumers must treat them as + // untrusted; see the `mds::SourceMap` rustdoc. The status + // line below is a diagnostic surface and IS escaped. let map_json = sm.to_json(); std::fs::write(&map_path, &map_json).map_err(|e| { miette::miette!("cannot write {}: {e}", map_path.display()) })?; if !quiet { - eprintln!("Source map written to {}", map_path.display()); + eprintln!( + "Source map written to {}", + crate::output::safe_path(&map_path) + ); } } return Ok(()); @@ -1310,9 +1339,10 @@ pub(crate) fn run_build(args: BuildArgs) -> Result<()> { .map(|(_, p)| p.display().to_string()) .unwrap_or_else(|| "mds.json".to_owned()); eprintln!( - "warning: source_map in {cfg_path} has no effect when writing to \ + "warning: source_map in {} has no effect when writing to \ stdout (sidecar requires -o or --out-dir); use --inline to \ - embed the map, or --no-source-map to silence this warning" + embed the map, or --no-source-map to silence this warning", + crate::output::safe_inline(&cfg_path) ); } return write_output(None, &compiled.content, quiet, true); @@ -1326,7 +1356,10 @@ pub(crate) fn run_build(args: BuildArgs) -> Result<()> { miette::miette!("cannot write {}: {e}", map_path.display()) })?; if !quiet { - eprintln!("Source map written to {}", map_path.display()); + eprintln!( + "Source map written to {}", + crate::output::safe_path(&map_path) + ); } } } @@ -1418,7 +1451,7 @@ fn run_build_directory( std::process::exit(1); } if !quiet { - eprintln!("No .mds files found in {}", dir.display()); + eprintln!("No .mds files found in {}", crate::output::safe_path(dir)); } return Ok(()); } @@ -1466,8 +1499,9 @@ fn run_build_directory( if !parent.as_os_str().is_empty() { if let Err(e) = std::fs::create_dir_all(parent) { eprintln!( - "error: cannot create output directory {}: {e}", - parent.display() + "error: cannot create output directory {}: {}", + crate::output::safe_path(parent), + crate::output::safe_inline(&e) ); fail_count += 1; continue; @@ -1494,7 +1528,7 @@ fn run_build_directory( match std::fs::write(&out_path, &final_content) { Ok(()) => { if !quiet { - eprintln!("Compiled to {}", out_path.display()); + eprintln!("Compiled to {}", crate::output::safe_path(&out_path)); } written_this_run.insert(out_path.clone()); @@ -1504,12 +1538,19 @@ fn run_build_directory( let map_path = map_path_for(&out_path); let map_json = sm.to_json(); if let Err(e) = std::fs::write(&map_path, &map_json) { - eprintln!("error: cannot write {}: {e}", map_path.display()); + eprintln!( + "error: cannot write {}: {}", + crate::output::safe_path(&map_path), + crate::output::safe_inline(&e) + ); fail_count += 1; continue; } if !quiet { - eprintln!("Source map written to {}", map_path.display()); + eprintln!( + "Source map written to {}", + crate::output::safe_path(&map_path) + ); } } } @@ -1537,15 +1578,19 @@ fn run_build_directory( ok_count += 1; } Err(e) => { - eprintln!("error: cannot write {}: {e}", out_path.display()); + eprintln!( + "error: cannot write {}: {}", + crate::output::safe_path(&out_path), + crate::output::safe_inline(&e) + ); fail_count += 1; } } } Err(e) => { - // Sanitize at the render boundary: MdsError::Syntax embeds user-controlled - // source fragments that may contain raw ESC bytes (avoids terminal escape injection). - eprintln!("{}", sanitize_control_chars(&format!("{e:?}"))); + // Route through the single render choke point (avoids PF-004 / + // architecture-6: hand-rolled sanitize_control_chars bypass). + crate::output::eprint_error(e); fail_count += 1; } } diff --git a/crates/mds-cli/src/fmt.rs b/crates/mds-cli/src/fmt.rs index 89c79ccf..4aa7b1cf 100644 --- a/crates/mds-cli/src/fmt.rs +++ b/crates/mds-cli/src/fmt.rs @@ -22,15 +22,14 @@ //! - 2: file not found / not `.mds` / I/O / bad UTF-8 //! - 3: oversized source -use std::io::{IsTerminal, Write as _}; +use std::io::Write as _; use std::path::{Path, PathBuf}; use mds::{effective_parent, FileSystem}; use miette::Result; use crate::build::{ensure_existing_mds_file, load_config, read_stdin, resolve_input}; -use crate::output::atomic_write_file; -use crate::output::collect_mds_files_detailed; +use crate::output::{atomic_write_file, collect_mds_files_detailed, render_unified_diff}; pub(crate) struct FmtArgs { pub(crate) input: Option, @@ -58,7 +57,7 @@ pub(crate) fn run_fmt(args: FmtArgs) -> Result<()> { let (input, auto_detected) = resolve_input(input, "fmt")?; if auto_detected && !quiet { - eprintln!("Formatting {}", input.display()); + eprintln!("Formatting {}", crate::output::safe_path(&input)); } let flags = FmtFlags { check, diff, quiet }; @@ -134,7 +133,7 @@ fn run_fmt_stdin(flags: FmtFlags) -> Result<()> { let result = format_source_named(&source, Some(&cwd), "")?; if diff { - print_diff(&render_diff(&source, &result.formatted, ""))?; + print_diff(&render_unified_diff(&source, &result.formatted, ""))?; } else if !check { // Plain filter mode: formatted content is the output. write_stdout(&result.formatted)?; @@ -163,8 +162,8 @@ fn run_fmt_file(path: &Path, flags: FmtFlags) -> Result<()> { let result = format_source_named(&source, base_dir, &file_name)?; if diff && result.changed { - let label = path.display().to_string(); - print_diff(&render_diff(&source, &result.formatted, &label))?; + let label = crate::output::safe_path(path); + print_diff(&render_unified_diff(&source, &result.formatted, &label))?; } let read_only = check || diff; @@ -175,10 +174,10 @@ fn run_fmt_file(path: &Path, flags: FmtFlags) -> Result<()> { // commit c5aa086 — both write paths now share the same helper). atomic_write_file(path, &result.formatted)?; if !quiet { - eprintln!("Formatted: {}", path.display()); + eprintln!("Formatted: {}", crate::output::safe_path(path)); } } else if !quiet { - eprintln!("Unchanged: {}", path.display()); + eprintln!("Unchanged: {}", crate::output::safe_path(path)); } return Ok(()); } @@ -186,12 +185,12 @@ fn run_fmt_file(path: &Path, flags: FmtFlags) -> Result<()> { if check { if result.changed { if !quiet { - eprintln!("Would reformat: {}", path.display()); + eprintln!("Would reformat: {}", crate::output::safe_path(path)); } std::process::exit(1); } if !quiet { - eprintln!("Unchanged: {}", path.display()); + eprintln!("Unchanged: {}", crate::output::safe_path(path)); } } Ok(()) @@ -247,7 +246,7 @@ fn format_one_file(file: &Path, flags: FmtFlags) -> FileOutcome { if diff && result.changed { let label = file_name.clone(); - if let Err(e) = print_diff(&render_diff(&source, &result.formatted, &label)) { + if let Err(e) = print_diff(&render_unified_diff(&source, &result.formatted, &label)) { crate::output::eprint_error(e); return FileOutcome::Failed; } @@ -269,7 +268,7 @@ fn format_one_file(file: &Path, flags: FmtFlags) -> FileOutcome { match atomic_write_file(file, &result.formatted) { Ok(()) => { if !quiet { - eprintln!("Formatted: {}", file.display()); + eprintln!("Formatted: {}", crate::output::safe_path(file)); } FileOutcome::Formatted } @@ -315,7 +314,7 @@ fn run_fmt_directory(dir: &Path, flags: FmtFlags) -> Result<()> { std::process::exit(1); } if !flags.quiet { - eprintln!("No .mds files found in {}", dir.display()); + eprintln!("No .mds files found in {}", crate::output::safe_path(dir)); } return Ok(()); } @@ -383,74 +382,6 @@ fn print_diff(rendered: &str) -> Result<()> { write_stdout(rendered) } -// ── unified diff rendering ─────────────────────────────────────────────────── - -/// Render a unified diff between `original` and `formatted`, colorized with -/// raw ANSI escapes ONLY when stdout is a terminal (mirrors `watch.rs`'s -/// `clear_terminal`, which does the same TTY gate for its own raw escapes; -/// this repo has no color crate dependency). -fn render_diff(original: &str, formatted: &str, label: &str) -> String { - let diff = similar::TextDiff::from_lines(original, formatted); - let unified = diff - .unified_diff() - .context_radius(3) - .header(label, label) - .to_string(); - - if unified.is_empty() || !std::io::stdout().is_terminal() { - return unified; - } - colorize_unified_diff(&unified) -} - -fn colorize_unified_diff(unified: &str) -> String { - const RED: &str = "\x1b[31m"; - const GREEN: &str = "\x1b[32m"; - const CYAN: &str = "\x1b[36m"; - const RESET: &str = "\x1b[0m"; - - let mut out = String::with_capacity(unified.len() + 64); - // `---`/`+++` file headers appear before the first `@@` hunk marker. - // Inside a hunk a removed line starts with `-` and an added line starts - // with `+` — but if that line's *content* itself starts with `-` or `+`, - // the rendered diff line looks like `---` or `+++`, which the old global - // prefix check mis-colored as CYAN (file header) instead of RED/GREEN. - // A state machine keyed on the first `@@` avoids the ambiguity. - let mut in_hunk = false; - for line in unified.split_inclusive('\n') { - let color = if line.starts_with("@@") { - in_hunk = true; - CYAN - } else if !in_hunk && (line.starts_with("---") || line.starts_with("+++")) { - // File header — always precedes the first @@ hunk. - CYAN - } else if in_hunk && line.starts_with('+') { - GREEN - } else if in_hunk && line.starts_with('-') { - RED - } else { - "" - }; - if color.is_empty() { - out.push_str(line); - continue; - } - out.push_str(color); - match line.strip_suffix('\n') { - Some(stripped) => { - out.push_str(stripped); - out.push_str(RESET); - out.push('\n'); - } - None => { - out.push_str(line); - out.push_str(RESET); - } - } - } - out -} - // ── Unit tests ──────────────────────────────────────────────────────────────── #[cfg(test)] @@ -521,46 +452,4 @@ mod tests { assert!(result.changed); assert!(!result.formatted.contains('\r')); } - - #[test] - fn render_diff_empty_for_identical_input() { - let rendered = render_diff("same\n", "same\n", "label"); - assert!(rendered.is_empty()); - } - - #[test] - fn render_diff_contains_unified_markers_for_changed_input() { - let rendered = render_diff("a\n", "b\n", "label"); - assert!(rendered.contains("---")); - assert!(rendered.contains("+++")); - assert!(rendered.contains("-a")); - assert!(rendered.contains("+b")); - } - - #[test] - fn colorize_unified_diff_wraps_add_remove_lines_with_ansi_when_requested() { - let unified = "--- a\n+++ b\n@@ -1 +1 @@\n-old\n+new\n"; - let colorized = colorize_unified_diff(unified); - assert!(colorized.contains("\x1b[32m+new\x1b[0m")); - assert!(colorized.contains("\x1b[31m-old\x1b[0m")); - assert!(colorized.contains("\x1b[36m--- a\x1b[0m")); - } - - #[test] - fn colorize_unified_diff_correctly_colors_content_starting_with_dashes_or_pluses() { - // Regression: a removed line whose content starts with "-- " produces - // "--- ..." in the rendered unified diff. The old global prefix check - // matched `starts_with("---")` and mis-colored it CYAN (file header) - // instead of RED (removal). Same defect for "++" content → "+++ " → - // mis-colored CYAN instead of GREEN. - let unified = "--- a\n+++ b\n@@ -1,2 +1,2 @@\n--- dashes content\n+++ plus content\n"; - let colorized = colorize_unified_diff(unified); - // Inside the hunk: removal of a line whose content starts with "-- " - assert!(colorized.contains("\x1b[31m--- dashes content\x1b[0m")); - // Inside the hunk: addition of a line whose content starts with "++" - assert!(colorized.contains("\x1b[32m+++ plus content\x1b[0m")); - // File headers (before @@) must still be CYAN - assert!(colorized.contains("\x1b[36m--- a\x1b[0m")); - assert!(colorized.contains("\x1b[36m+++ b\x1b[0m")); - } } diff --git a/crates/mds-cli/src/lint.rs b/crates/mds-cli/src/lint.rs index 2545d39e..2ad8113c 100644 --- a/crates/mds-cli/src/lint.rs +++ b/crates/mds-cli/src/lint.rs @@ -26,7 +26,7 @@ use std::cell::RefCell; use std::collections::HashMap; -use std::io::{IsTerminal, Write as _}; +use std::io::Write as _; use std::path::{Path, PathBuf}; use std::rc::Rc; @@ -37,7 +37,10 @@ use crate::build::{ build_runtime_vars, ensure_existing_mds_file, load_config, read_stdin, resolve_input, RuntimeVarArgs, }; -use crate::output::{atomic_write_file, collect_mds_files_detailed}; +use crate::output::{ + atomic_write_file, collect_mds_files_detailed, eprint_error, eprint_warning, + render_unified_diff, safe_file_display, safe_inline, safe_path, +}; /// Known lint rule names — used to warn about unknown names in mds.json config. const KNOWN_RULES: &[&str] = &[ @@ -92,7 +95,9 @@ pub(crate) fn run_lint(args: LintArgs) -> Result<()> { match do_lint(args) { Ok(()) => Ok(()), Err(e) => { - eprintln!("{}", mds::sanitize_control_chars(&format!("{e:?}"))); + // Route through the single render choke point (avoids PF-004 / + // architecture-6: hand-rolled sanitize_control_chars bypass). + eprint_error(e); std::process::exit(2); } } @@ -187,9 +192,24 @@ fn load_lint_config(dir: &Path) -> Result { None => Ok(mds::LintConfig::default()), Some((mds_config, _config_dir)) => { // Warn on unknown rule names — unknown NAMES are ignored for forward compat. + // + // `name` is an arbitrary attacker-supplied JSON object key: `mds.json` is + // read from the working tree, and JSON `\uXXXX` escapes decode to real + // control bytes. + // + // Two escapes, deliberately different (spec §7.5 per-field rule): the warning + // PROSE goes through `eprint_warning` (HUMAN — it is the message body), while + // the rule NAME goes through `safe_inline` (WIRE). Routing the whole line + // through `eprint_warning` alone was NOT sufficient: HUMAN mode preserves + // `\n`, so a rule name of `x\nClean: totally-real.mds\n0 problems found\n` + // still emitted three standalone lines byte-identical to genuine status + // output (CWE-117). A JSON object key is never legitimately multi-line. for name in mds_config.lint.rules.keys() { if !KNOWN_RULES.contains(&name.as_str()) { - eprintln!("warning: unknown lint rule '{name}' in mds.json; ignoring"); + eprint_warning(&format!( + "warning: unknown lint rule '{}' in mds.json; ignoring", + safe_inline(name) + )); } } Ok(mds_config.lint.into_core_config()) @@ -253,44 +273,53 @@ fn write_stdout(s: &str) -> Result<()> { // ── Human diagnostic rendering ──────────────────────────────────────────────── -/// Render one lint diagnostic to stderr, applying `sanitize_control_chars` at the boundary. +/// Render one lint diagnostic to stderr. All user-controlled text — message, help, +/// filename, and source — is sanitized at the input boundary so miette renders from +/// safe inputs; the frame itself is not post-processed (PF-014). /// /// `--quiet` suppresses Warn and Info; Error always renders. -/// `named_source`: optional `(filename, source_text)` pair attached to the miette Report so -/// the renderer can show the offending source line + caret (span highlighting). When absent, -/// diagnostics are still rendered but without source context. -fn render_diag_human(diag: &mds::LintDiagnostic, quiet: bool, named_source: Option<(&str, &str)>) { +/// `named_source`: `(filename, source_text)` pair for span context rendering. +/// +/// Sanitization strategy (PF-014): +/// - message/help: HUMAN-mode `sanitize_control_chars` → \\uXXXX escapes. `\n` is +/// preserved so a multi-line diagnostic frame keeps rendering. +/// - filename + source: `mds::named_source_for_render`, the shared boundary +/// `MdsError::at()` and the formatter also use — WIRE-mode escaping for the +/// single-line filename, byte-length-preserving neutralization for the span-indexed +/// source so miette's byte-offset slices stay valid. +/// +/// The rendered miette frame is NOT post-processed, so miette's own SGR colour codes +/// are never corrupted. +fn render_diag_human(diag: &mds::LintDiagnostic, quiet: bool, named_source: (&str, &str)) { if quiet && matches!(diag.severity, Severity::Info | Severity::Warn) { return; } - // Sanitize at the render boundary (AC-F-16): message and help only; raw bytes - // are preserved in the stored diagnostic and in JSON output via to_canonical_json(). + // Sanitize message and help at the input boundary. + // fix_removals/fix_edits are not read by miette's Diagnostic impl — set to None + // to avoid unnecessary allocations (architecture-3 / rust-1). let sanitized = mds::LintDiagnostic { rule: diag.rule.clone(), severity: diag.severity, - message: mds::sanitize_control_chars(&diag.message), - help: diag.help.as_deref().map(mds::sanitize_control_chars), + message: mds::sanitize_control_chars(&diag.message).into_owned(), + help: diag + .help + .as_deref() + .map(|h| mds::sanitize_control_chars(h).into_owned()), span: diag.span.clone(), file: diag.file.clone(), - fix_removals: diag.fix_removals.clone(), - fix_edits: diag.fix_edits.clone(), + fix_removals: None, + fix_edits: None, }; - // Attach the source code so miette can render the span with source context - // (source line + caret underline). The labels() implementation on LintDiagnostic - // returns the span; with_source_code() provides the text miette reads to render it. - if let Some((filename, src)) = named_source { - let report = miette::Report::from(sanitized) - .with_source_code(miette::NamedSource::new(filename, src.to_string())); - eprintln!("{report:?}"); - } else { - eprintln!("{:?}", miette::Report::from(sanitized)); - } + let (filename, src) = named_source; + let report = miette::Report::from(sanitized) + .with_source_code(mds::named_source_for_render(filename, src)); + eprint_error(report); } /// Render all diagnostics in a `LintResult` to stderr. /// /// `named_source` is forwarded to `render_diag_human` for span context rendering. -fn render_result_human(result: &mds::LintResult, quiet: bool, named_source: Option<(&str, &str)>) { +fn render_result_human(result: &mds::LintResult, quiet: bool, named_source: (&str, &str)) { for diag in &result.diagnostics { render_diag_human(diag, quiet, named_source); } @@ -642,7 +671,7 @@ fn run_lint_stdin( match preview { PreviewOutcome::WouldFix(ref fixed) => { if diff { - let diff_str = render_diff_lint(&source, fixed, "stdin"); + let diff_str = render_unified_diff(&source, fixed, "stdin"); let _ = write_stdout(&diff_str); } if check { @@ -654,7 +683,7 @@ fn run_lint_stdin( } PreviewOutcome::Rejected(ref reason) => { if !quiet { - eprintln!("fix rejected: {reason}"); + eprintln!("fix rejected: {}", safe_inline(reason)); } } PreviewOutcome::NothingToFix => {} @@ -692,13 +721,13 @@ fn run_lint_stdin( (new_source, residual) } FixFileOutcome::Rejected { reason, original } => { - eprintln!("fix rejected: {reason}"); + eprintln!("fix rejected: {}", safe_inline(&reason)); (source, original) } FixFileOutcome::NothingToFix { original } => (source, original), }; // Stdin diagnostics: pass source text for span context rendering. - let named_source = Some((mds::STRING_SOURCE_MAP_LABEL, output_src.as_str())); + let named_source = (mds::STRING_SOURCE_MAP_LABEL, output_src.as_str()); render_result_human(&diag_result, quiet, named_source); let _ = write_stdout(&output_src); exit_by_severity(&diag_result); @@ -787,7 +816,7 @@ fn run_lint_file( emit_result(format, &residual, quiet, named_source); atomic_write_file(path, &new_source)?; if !quiet { - eprintln!("Fixed: {}", path.display()); + eprintln!("Fixed: {}", safe_path(path)); } exit_by_severity(&residual); } @@ -800,7 +829,7 @@ fn run_lint_file( if !quiet { eprintln!( "Partially fixed: {} ({applied_count} of {total_count} fixes applied)", - path.display() + safe_path(path) ); } emit_result(format, &residual, quiet, named_source); @@ -808,14 +837,14 @@ fn run_lint_file( exit_by_severity(&residual); } FixFileOutcome::Rejected { reason, original } => { - eprintln!("fix rejected: {reason}"); + eprintln!("fix rejected: {}", safe_inline(&reason)); emit_result(format, &original, quiet, named_source); exit_by_severity(&original); } FixFileOutcome::NothingToFix { original } => { emit_result(format, &original, quiet, named_source); if !quiet && format == LintFormat::Human && original.diagnostics.is_empty() { - eprintln!("Clean: {filename}"); + eprintln!("Clean: {}", safe_file_display(filename)); } exit_by_severity(&original); } @@ -823,7 +852,7 @@ fn run_lint_file( return Ok(()); } - // ── Preview path: --fix --check and/or --fix --diff ─────────────────────── + // ── Preview path: --fix --check and/or --fix --diff ───────────────── // Route preview through the same gated pipeline as the write path. // Previously called apply_plan_unchecked directly, bypassing the reverify gate — // a diff or check result could misrepresent what --fix would actually do. @@ -832,13 +861,13 @@ fn run_lint_file( match preview { PreviewOutcome::WouldFix(ref fixed) => { if diff { - let label = path.display().to_string(); - let diff_str = render_diff_lint(&source, fixed, &label); + let label = safe_path(path); + let diff_str = render_unified_diff(&source, fixed, &label); let _ = write_stdout(&diff_str); } if check { if !quiet { - eprintln!("Would fix: {}", path.display()); + eprintln!("Would fix: {}", safe_path(path)); } std::process::exit(1); } @@ -846,7 +875,7 @@ fn run_lint_file( PreviewOutcome::Rejected(ref reason) => { // Surface the rejection reason so --fix --check is as honest as --fix. if !quiet { - eprintln!("fix rejected: {reason}"); + eprintln!("fix rejected: {}", safe_inline(reason)); } } PreviewOutcome::NothingToFix => {} @@ -861,7 +890,7 @@ fn run_lint_file( // ── Report-only mode (no --fix) ─────────────────────────────────────────── emit_result(format, &result, quiet, named_source); if !quiet && format == LintFormat::Human && result.diagnostics.is_empty() { - eprintln!("Clean: {filename}"); + eprintln!("Clean: {}", safe_file_display(filename)); } exit_by_severity(&result); Ok(()) @@ -969,7 +998,7 @@ fn run_lint_directory( std::process::exit(2); } if !quiet { - eprintln!("No .mds files found in {}", dir.display()); + eprintln!("No .mds files found in {}", safe_path(dir)); } return Ok(()); } @@ -1100,7 +1129,7 @@ fn lint_one_file_accumulating( eprintln!( "{}: diagnostic cap ({}) reached; further findings were suppressed — \ re-run --fix to continue", - file.display(), + safe_path(file), mds::MAX_DIAGNOSTICS ); } @@ -1131,7 +1160,7 @@ fn lint_one_file_accumulating( set_diag_display_path(&mut residual, &display_path); accumulate_result_json(&residual, json_files); if let Err(e) = atomic_write_file(file, &new_source) { - eprintln!("error writing {}: {e}", file.display()); + eprintln!("error writing {}: {}", safe_path(file), safe_inline(&e)); return FileTally::Error; } tally_from_result(&residual) @@ -1146,19 +1175,23 @@ fn lint_one_file_accumulating( if !quiet { eprintln!( "Partially fixed: {} ({applied_count} of {total_count} fixes applied)", - file.display() + safe_path(file) ); } set_diag_display_path(&mut residual, &display_path); accumulate_result_json(&residual, json_files); if let Err(e) = atomic_write_file(file, &new_source) { - eprintln!("error writing {}: {e}", file.display()); + eprintln!("error writing {}: {}", safe_path(file), safe_inline(&e)); return FileTally::Error; } tally_from_result(&residual) } FixFileOutcome::Rejected { reason, original } => { - eprintln!("{}: fix rejected: {reason}", file.display()); + eprintln!( + "{}: fix rejected: {}", + safe_path(file), + safe_inline(&reason) + ); accumulate_result_json(&original, json_files); tally_from_result(&original) } @@ -1189,13 +1222,13 @@ fn lint_one_file_accumulating( PreviewOutcome::WouldFix(ref fixed) => { *any_would_fix = true; if diff { - let label = file.display().to_string(); - let diff_str = render_diff_lint(&source, fixed, &label); + let label = safe_path(file); + let diff_str = render_unified_diff(&source, fixed, &label); let _ = write_stdout(&diff_str); } } PreviewOutcome::Rejected(ref reason) => { - eprintln!("{}: fix rejected: {reason}", file.display()); + eprintln!("{}: fix rejected: {}", safe_path(file), safe_inline(reason)); } PreviewOutcome::NothingToFix => {} } @@ -1232,7 +1265,7 @@ fn lint_one_file_human( let source = match read_source_file(file) { Ok(s) => s, Err(e) => { - crate::output::eprint_error(miette::Report::from(e)); + eprint_error(miette::Report::from(e)); return FileTally::Error; } }; @@ -1246,18 +1279,18 @@ fn lint_one_file_human( let config = match ctx.config_for(base_dir) { Ok(c) => c, Err(e) => { - crate::output::eprint_error(miette::Report::from(e)); + eprint_error(miette::Report::from(e)); return FileTally::Error; } }; // Named source for span rendering: relative display path + source text. - let named_source = Some((display_path.as_str(), source.as_str())); + let named_source = (display_path.as_str(), source.as_str()); let mut result = match mds::lint(file, ctx.runtime_vars.clone(), &config) { Ok(r) => r, Err(ref e) => { - crate::output::eprint_error(miette::Report::from(e.clone())); + eprint_error(miette::Report::from(e.clone())); return if matches!(e, MdsError::ResourceLimit { .. }) { FileTally::ResourceLimit } else { @@ -1274,7 +1307,7 @@ fn lint_one_file_human( eprintln!( "{}: diagnostic cap ({}) reached; further findings were suppressed — \ re-run --fix to continue", - file.display(), + safe_path(file), mds::MAX_DIAGNOSTICS ); } @@ -1291,11 +1324,11 @@ fn lint_one_file_human( set_diag_display_path(&mut residual, &display_path); render_result_human(&residual, quiet, named_source); if let Err(e) = atomic_write_file(file, &new_source) { - eprintln!("error writing {}: {e}", file.display()); + eprintln!("error writing {}: {}", safe_path(file), safe_inline(&e)); return FileTally::Error; } if !quiet { - eprintln!("Fixed: {}", file.display()); + eprintln!("Fixed: {}", safe_path(file)); } tally_from_result(&residual) } @@ -1309,19 +1342,23 @@ fn lint_one_file_human( if !quiet { eprintln!( "Partially fixed: {} ({applied_count} of {total_count} fixes applied)", - file.display() + safe_path(file) ); } set_diag_display_path(&mut residual, &display_path); render_result_human(&residual, quiet, named_source); if let Err(e) = atomic_write_file(file, &new_source) { - eprintln!("error writing {}: {e}", file.display()); + eprintln!("error writing {}: {}", safe_path(file), safe_inline(&e)); return FileTally::Error; } tally_from_result(&residual) } FixFileOutcome::Rejected { reason, original } => { - eprintln!("{}: fix rejected: {reason}", file.display()); + eprintln!( + "{}: fix rejected: {}", + safe_path(file), + safe_inline(&reason) + ); render_result_human(&original, quiet, named_source); tally_from_result(&original) } @@ -1342,17 +1379,17 @@ fn lint_one_file_human( PreviewOutcome::WouldFix(ref fixed) => { *any_would_fix = true; if diff { - let label = file.display().to_string(); - let diff_str = render_diff_lint(&source, fixed, &label); + let label = safe_path(file); + let diff_str = render_unified_diff(&source, fixed, &label); let _ = write_stdout(&diff_str); } if check && !quiet { - eprintln!("Would fix: {}", file.display()); + eprintln!("Would fix: {}", safe_path(file)); } } PreviewOutcome::Rejected(ref reason) => { if !quiet { - eprintln!("{}: fix rejected: {reason}", file.display()); + eprintln!("{}: fix rejected: {}", safe_path(file), safe_inline(reason)); } } PreviewOutcome::NothingToFix => {} @@ -1383,7 +1420,11 @@ fn emit_result( serde_json::to_string(&json).expect("canonical lint JSON is always serializable") )); } else { - render_result_human(result, quiet, named_source); + render_result_human( + result, + quiet, + named_source.expect("Human format requires named_source"), + ); } } @@ -1400,10 +1441,9 @@ fn emit_analysis_failure_json_or_stderr(e: &MdsError, format: LintFormat) { serde_json::to_string(&envelope).expect("canonical lint JSON is always serializable") )); } else { - // Sanitize at the render boundary: MdsError::Syntax embeds user-controlled - // source fragments that may contain raw ESC bytes (avoids terminal escape injection). - let rendered = format!("{:?}", miette::Report::from(e.clone())); - eprintln!("{}", mds::sanitize_control_chars(&rendered)); + // Route through the single render choke point (avoids PF-004 / + // architecture-6: hand-rolled sanitize_control_chars bypass). + eprint_error(miette::Report::from(e.clone())); } } @@ -1415,64 +1455,6 @@ fn accumulate_result_json(result: &mds::LintResult, json_files: &mut Vec String { - let diff = similar::TextDiff::from_lines(original, fixed); - let unified = diff - .unified_diff() - .context_radius(3) - .header(label, label) - .to_string(); - - if unified.is_empty() || !std::io::stdout().is_terminal() { - return unified; - } - colorize_unified_diff(&unified) -} - -fn colorize_unified_diff(unified: &str) -> String { - const RED: &str = "\x1b[31m"; - const GREEN: &str = "\x1b[32m"; - const CYAN: &str = "\x1b[36m"; - const RESET: &str = "\x1b[0m"; - - let mut out = String::with_capacity(unified.len() + 64); - let mut in_hunk = false; - for line in unified.split_inclusive('\n') { - let color = if line.starts_with("@@") { - in_hunk = true; - CYAN - } else if !in_hunk && (line.starts_with("---") || line.starts_with("+++")) { - CYAN - } else if in_hunk && line.starts_with('+') { - GREEN - } else if in_hunk && line.starts_with('-') { - RED - } else { - "" - }; - if color.is_empty() { - out.push_str(line); - continue; - } - out.push_str(color); - match line.strip_suffix('\n') { - Some(stripped) => { - out.push_str(stripped); - out.push_str(RESET); - out.push('\n'); - } - None => { - out.push_str(line); - out.push_str(RESET); - } - } - } - out -} - // ── Unit tests ──────────────────────────────────────────────────────────────── #[cfg(test)] diff --git a/crates/mds-cli/src/main.rs b/crates/mds-cli/src/main.rs index 87b308a7..3a55377a 100644 --- a/crates/mds-cli/src/main.rs +++ b/crates/mds-cli/src/main.rs @@ -225,12 +225,13 @@ fn main() { let result = run(cli); if let Err(e) = result { - // Sanitize at the last-resort render boundary: every subcommand's error propagates - // here, and MdsError::Syntax embeds user-controlled source fragments that may contain - // raw ESC bytes. Guarding here makes the protection hold by construction for any - // future error path, not just the ones we remember to sanitize individually (PF-004). - eprintln!("{}", mds::sanitize_control_chars(&format!("{e:?}"))); - process::exit(exit_code(&e)); + // Route through the single render choke point (avoids PF-004 / + // architecture-6: hand-rolled sanitize_control_chars bypass). Every subcommand's + // error propagates here; guarding here makes the protection hold by construction + // for any future error path, not just the ones we remember to sanitize individually. + let code = exit_code(&e); + output::eprint_error(e); + process::exit(code); } } @@ -275,7 +276,7 @@ fn run_check( .map_err(miette::Error::from)?; if !quiet { for w in &warnings { - eprintln!("{w}"); + output::eprint_warning(w); } eprintln!("OK: "); } @@ -284,9 +285,9 @@ fn run_check( mds::check_collecting_warnings(&input, runtime_vars).map_err(miette::Error::from)?; if !quiet { for w in &warnings { - eprintln!("{w}"); + output::eprint_warning(w); } - eprintln!("OK: {}", input.display()); + eprintln!("OK: {}", output::safe_path(&input)); } } Ok(()) @@ -319,7 +320,7 @@ fn run_check_directory( std::process::exit(1); } if !quiet { - eprintln!("No .mds files found in {}", dir.display()); + eprintln!("No .mds files found in {}", output::safe_path(dir)); } return Ok(()); } @@ -337,15 +338,15 @@ fn run_check_directory( Ok(((), warnings)) => { if !quiet { for w in &warnings { - eprintln!("{w}"); + output::eprint_warning(w); } } ok_count += 1; } Err(e) => { - // Sanitize at the render boundary: MdsError::Syntax embeds user-controlled - // source fragments that may contain raw ESC bytes (PF-004 parallel-path guard). - eprintln!("{}", mds::sanitize_control_chars(&format!("{e:?}"))); + // Route through the single render choke point (avoids PF-004 / + // architecture-6: hand-rolled sanitize_control_chars bypass). + output::eprint_error(e); fail_count += 1; } } @@ -395,8 +396,8 @@ Your items: if !quiet { eprintln!( "Created {}\n Try: mds build {}", - filename.display(), - filename.display() + output::safe_path(&filename), + output::safe_path(&filename) ); } Ok(()) @@ -456,7 +457,8 @@ fn run(cli: Cli) -> Result<()> { "json" => lint::LintFormat::Json, other => { eprintln!( - "error: unknown --format value '{other}'; expected 'human' or 'json'" + "error: unknown --format value '{}'; expected 'human' or 'json'", + output::safe_inline(other) ); std::process::exit(2); } diff --git a/crates/mds-cli/src/output.rs b/crates/mds-cli/src/output.rs index 6b7934ab..ab7dbb8e 100644 --- a/crates/mds-cli/src/output.rs +++ b/crates/mds-cli/src/output.rs @@ -6,14 +6,18 @@ //! path resolution used by watch and build-directory. //! - [`collect_mds_files`] / [`is_partial`]: directory traversal helpers. //! - [`probe_and_remove_stale`]: stale-output cleanup for format-flip (AC-FUNC-23). -//! - [`eprint_error`]: sanitized stderr render for directory-mode error loops (PF-004). +//! - [`eprint_error`]: the single CLI stderr choke-point — escapes every report's +//! message, help, and label text before miette renders it (CWE-150 / PF-014). //! - [`atomic_write_file`]: temp-file-then-rename writer shared by `fmt` and `lint --fix`. +//! - [`preview_text_for`]: `--diff` preview output — neutralized on TTY, byte-faithful +//! when piped, so redirected diffs stay applicable by `patch`/tooling. //! //! Single-file path helpers (`OutputKind`, `compile_to_content`, `compile_and_write`, //! `resolve_output_path_for_kind`) remain in `build.rs`; they are imported here when //! callers need both single-file and directory logic. -use std::io::Write as _; +use std::borrow::Cow; +use std::io::{IsTerminal, Write as _}; use std::path::{Path, PathBuf}; use miette::Result; @@ -254,11 +258,16 @@ fn collect_mds_files_inner( excluded_count: &mut usize, ) { if depth > max_depth { - eprintln!( + // The directory name is discovered by the walk — the user never types it — so it + // is untrusted on every subcommand that shares this walker (build / check / fmt / + // lint / watch). Prose through `eprint_warning` (HUMAN), the path through + // `safe_path` (WIRE): a directory name is never legitimately multi-line, and a + // raw one here forged a standalone `Clean: …` line as well as emitting raw ESC. + eprint_warning(&format!( "warning: directory depth limit ({max_depth}) reached at {}; \ deeper files will not be processed", - dir.display() - ); + safe_path(dir) + )); return; } let read_dir = match std::fs::read_dir(dir) { @@ -374,10 +383,13 @@ pub(crate) fn probe_and_remove_stale(base_no_ext: &Path, kind: OutputKind) { // user normally needs to know about (mirrors watch "Removed …" style). } Err(e) => { - eprintln!( - "warning: could not remove stale output {}: {e}", - stale_path.display() - ); + // Same shape as the depth-limit warning above: the path is walker-derived + // and the `io::Error` Display embeds a path of its own, so both are WIRE. + eprint_warning(&format!( + "warning: could not remove stale output {}: {}", + safe_path(&stale_path), + safe_inline(&e) + )); } } } @@ -485,21 +497,650 @@ pub(crate) fn atomic_write_file(path: &Path, content: &str) -> Result<()> { // ── Sanitized stderr render ─────────────────────────────────────────────────── -/// Render a miette Report to stderr with control-character sanitization applied. +/// Re-export: byte-length-preserving source neutralization before miette rendering. +/// +/// Lives in `mds-core` so both `MdsError::at()` (compiler error path) and +/// `render_diag_human` (lint diagnostic path) use a single canonical implementation +/// (avoids PF-004 / PF-014 parallel-path drift). +pub(crate) use mds::neutralize_source_for_render; + +/// A terminal-safe view of a [`miette::Report`], built **before** rendering. +/// +/// Overrides every prose surface the frame can render — the `Display` message, the +/// `help` text, each [`miette::LabeledSpan`]'s label, and the whole auxiliary +/// diagnostic graph (`source` cause chain, `related`, `diagnostic_source`) — with +/// [`mds::sanitize_control_chars`]-escaped copies (HUMAN mode, so `\n` and `\t` survive +/// and multi-line frames stay readable). Everything the frame's geometry depends on — +/// `code`, `severity`, `url`, `source_code`, and each label's byte span — is delegated +/// to the inner report untouched, so the byte-length-preserving neutralization already +/// applied to source excerpts (via `mds::named_source_for_render`) keeps every span +/// offset and caret column exact. +/// +/// # Why this is the PF-014-correct boundary +/// +/// The rendered frame is never post-processed. Running a sanitizer over an +/// already-rendered miette frame would escape miette's *own* ANSI SGR colour codes +/// into literal `\u001b[33m` noise on any colour-capable TTY — on completely benign +/// input — and desynchronise caret alignment. CI cannot catch that regression +/// because the CLI tests pin `NO_COLOR=1` and pipe stderr. Sanitizing the renderer's +/// *inputs* has neither failure mode. +/// +/// # Why it wraps the whole `Report` rather than just `MdsError` +/// +/// The CLI produces two error families: `MdsError` (compiler diagnostics) and +/// CLI-authored `miette::miette!()` reports, which do **not** downcast to `MdsError` +/// (see `build::exit_code`). Both interpolate untrusted text — a template's include +/// alias in the first case, an `mds.json` value or a hostile filename in the second. +/// Wrapping at the `Report` level covers both, and keeps covering any error type added +/// later, so the guarantee holds by construction rather than by remembering to extend +/// a downcast ladder (avoids PF-004). +/// +/// # The auxiliary graph (`source` / `related` / `diagnostic_source`) +/// +/// A `Diagnostic` can hang three further diagnostic graphs off itself, all of which +/// miette renders: the `std::error::Error` cause chain, the `related()` siblings, and +/// `diagnostic_source()`. All three carry prose, so all three must be escaped. +/// +/// They cannot be forwarded by reference the way `code`/`severity`/`url` are: those +/// accessors return borrows of the *inner* report, so handing back a sanitized view +/// would require the wrapper to own it and hand out a borrow of itself. This wrapper +/// therefore materialises the whole graph into owned [`SanitizedNode`]s at construction +/// (bounded by [`MAX_AUX_DEPTH`]) and forwards borrows of those. +/// +/// Enforcing this with a `debug_assert!` that the graph is empty — which is what an +/// earlier revision did, on the grounds that no CLI error populates it today — is +/// exactly PF-005: `debug_assert!` is compiled out of release, so the invariant would +/// hold under test and be absent in the shipped binary. The first error type to grow a +/// `#[source]` field would have silently dropped its cause chain from release stderr +/// while CI stayed green. +struct SanitizedReport { + inner: miette::Report, + message: String, + help: Option, + source: Option, + related: Vec, + diagnostic_source: Option, +} + +/// Depth bound for materialising a report's cause / related / diagnostic-source graph. +/// +/// Cause chains are finite in practice, but nothing in the `Error` trait forbids a cycle +/// (a `source()` that returns a sibling of itself would loop forever). Rendering is not +/// a place to discover that, so the walk is explicitly bounded; a graph deeper than this +/// is truncated, which drops trailing context but never hangs or overflows the stack. +const MAX_AUX_DEPTH: usize = 16; + +/// An owned, control-character-escaped snapshot of one node in a report's auxiliary +/// diagnostic graph. +/// +/// Every prose surface (`message`, `help`, `code`, `url`, label text) is escaped with +/// HUMAN-mode [`mds::sanitize_control_chars`] when the node is built. Label byte spans +/// are copied verbatim, exactly as [`SanitizedReport::labels`] does, so caret geometry +/// against the parent's already-neutralized source stays exact. +/// +/// `source_code()` returns `None` by design: a `&dyn miette::SourceCode` cannot be +/// cloned out of the inner diagnostic, and miette falls back to the *parent* report's +/// source — which [`SanitizedReport::source_code`] forwards — when a nested diagnostic +/// supplies none. So a nested diagnostic still renders against neutralized source. +struct SanitizedNode { + message: String, + help: Option, + code: Option, + url: Option, + severity: Option, + labels: Option>, + source: Option>, + related: Vec, + diagnostic_source: Option>, +} + +/// Escape one optional `Display` surface to an owned `String`. +fn escape_display(d: Option>) -> Option { + d.map(|v| mds::sanitize_control_chars(&v.to_string()).into_owned()) +} + +/// Escape a `Diagnostic`'s label text, keeping each byte span verbatim. +/// +/// `None` is preserved as `None` (rather than collapsed to an empty vector) so miette +/// distinguishes "no labels" from "labels, but none of them" exactly as it did before +/// the wrapper was introduced. +fn escape_labels(d: &dyn miette::Diagnostic) -> Option> { + Some( + d.labels()? + .map(|label| { + let text = label + .label() + .map(|t| mds::sanitize_control_chars(t).into_owned()); + let span = *label.inner(); + if label.primary() { + miette::LabeledSpan::new_primary_with_span(text, span) + } else { + miette::LabeledSpan::new_with_span(text, span) + } + }) + .collect(), + ) +} + +impl SanitizedNode { + /// Build from a `Diagnostic` node (used for `related` / `diagnostic_source`). + fn from_diagnostic(d: &dyn miette::Diagnostic, depth: usize) -> Self { + Self { + message: mds::sanitize_control_chars(&d.to_string()).into_owned(), + help: escape_display(d.help()), + code: escape_display(d.code()), + url: escape_display(d.url()), + severity: d.severity(), + labels: escape_labels(d), + source: Self::chain_from_error(std::error::Error::source(d), depth), + related: Self::related_from(d, depth), + diagnostic_source: Self::boxed_from_diagnostic(d.diagnostic_source(), depth), + } + } + + /// Build from a plain `Error` node (used for the `source()` cause chain, whose links + /// expose no `Diagnostic` data — only a `Display` message and a further `source()`). + fn from_error(e: &(dyn std::error::Error + 'static), depth: usize) -> Self { + Self { + message: mds::sanitize_control_chars(&e.to_string()).into_owned(), + help: None, + code: None, + url: None, + severity: None, + labels: None, + source: Self::chain_from_error(e.source(), depth), + related: Vec::new(), + diagnostic_source: None, + } + } + + fn chain_from_error( + e: Option<&(dyn std::error::Error + 'static)>, + depth: usize, + ) -> Option> { + if depth >= MAX_AUX_DEPTH { + return None; + } + e.map(|e| Box::new(Self::from_error(e, depth + 1))) + } + + fn boxed_from_diagnostic( + d: Option<&dyn miette::Diagnostic>, + depth: usize, + ) -> Option> { + if depth >= MAX_AUX_DEPTH { + return None; + } + d.map(|d| Box::new(Self::from_diagnostic(d, depth + 1))) + } + + fn related_from(d: &dyn miette::Diagnostic, depth: usize) -> Vec { + if depth >= MAX_AUX_DEPTH { + return Vec::new(); + } + d.related() + .map(|rs| { + rs.map(|r| SanitizedNode::from_diagnostic(r, depth + 1)) + .collect() + }) + .unwrap_or_default() + } +} + +impl std::fmt::Display for SanitizedNode { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.message) + } +} + +/// Hand-written for the same reason as [`SanitizedReport`]'s: the derived `Debug` of a +/// miette type can be its full graphical render, which would bypass the escaping. +impl std::fmt::Debug for SanitizedNode { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("SanitizedNode") + .field("message", &self.message) + .field("help", &self.help) + .finish_non_exhaustive() + } +} + +impl std::error::Error for SanitizedNode { + fn source(&self) -> Option<&(dyn std::error::Error + 'static)> { + self.source + .as_deref() + .map(|n| n as &(dyn std::error::Error + 'static)) + } +} + +impl miette::Diagnostic for SanitizedNode { + fn code<'a>(&'a self) -> Option> { + boxed_str(self.code.as_deref()) + } + + fn severity(&self) -> Option { + self.severity + } + + fn help<'a>(&'a self) -> Option> { + boxed_str(self.help.as_deref()) + } + + fn url<'a>(&'a self) -> Option> { + boxed_str(self.url.as_deref()) + } + + fn labels(&self) -> Option + '_>> { + Some(Box::new(self.labels.as_ref()?.iter().cloned())) + } + + fn related<'a>(&'a self) -> Option + 'a>> { + related_iter(&self.related) + } + + fn diagnostic_source(&self) -> Option<&dyn miette::Diagnostic> { + self.diagnostic_source + .as_deref() + .map(|n| n as &dyn miette::Diagnostic) + } +} + +/// Box an optional `&str` as the `Display` trait object miette's accessors return. +fn boxed_str(s: Option<&str>) -> Option> { + s.map(|s| -> Box { Box::new(s) }) +} + +/// Shared `related()` body: `None` when empty, so miette omits the section entirely. +fn related_iter( + nodes: &[SanitizedNode], +) -> Option + '_>> { + if nodes.is_empty() { + return None; + } + Some(Box::new(nodes.iter().map(|n| n as &dyn miette::Diagnostic))) +} + +impl SanitizedReport { + fn new(inner: miette::Report) -> Self { + let message = mds::sanitize_control_chars(&inner.to_string()).into_owned(); + let help = escape_display(inner.help()); + let source = SanitizedNode::chain_from_error(std::error::Error::source(&*inner), 0) + .map(|boxed| *boxed); + let related = SanitizedNode::related_from(inner.as_ref(), 0); + let diagnostic_source = + SanitizedNode::boxed_from_diagnostic(inner.diagnostic_source(), 0).map(|boxed| *boxed); + + Self { + inner, + message, + help, + source, + related, + diagnostic_source, + } + } +} + +impl std::fmt::Display for SanitizedReport { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(&self.message) + } +} + +/// Hand-written so the wrapper never surfaces the inner report's own `Debug`, which +/// under miette's `fancy` feature is the full graphical render of the *unsanitized* +/// error. +impl std::fmt::Debug for SanitizedReport { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("SanitizedReport") + .field("message", &self.message) + .field("help", &self.help) + .finish_non_exhaustive() + } +} + +/// Forwards the **sanitized** cause chain, never the inner report's own. +impl std::error::Error for SanitizedReport { + fn source(&self) -> Option<&(dyn std::error::Error + 'static)> { + self.source + .as_ref() + .map(|n| n as &(dyn std::error::Error + 'static)) + } +} + +impl miette::Diagnostic for SanitizedReport { + fn code<'a>(&'a self) -> Option> { + self.inner.code() + } + + fn severity(&self) -> Option { + self.inner.severity() + } + + fn help<'a>(&'a self) -> Option> { + boxed_str(self.help.as_deref()) + } + + fn url<'a>(&'a self) -> Option> { + self.inner.url() + } + + fn source_code(&self) -> Option<&dyn miette::SourceCode> { + self.inner.source_code() + } + + /// Byte spans are forwarded verbatim (they index the neutralized, byte-length- + /// preserved source); only the label *text* is escaped. `LintDiagnostic` uses its + /// own message as the label text, so this is a real untrusted-text surface. + fn labels(&self) -> Option + '_>> { + Some(Box::new(escape_labels(self.inner.as_ref())?.into_iter())) + } + + /// The **sanitized** related-diagnostic snapshots, not the inner report's. + fn related<'a>(&'a self) -> Option + 'a>> { + related_iter(&self.related) + } + + /// The **sanitized** diagnostic source, not the inner report's. + fn diagnostic_source(&self) -> Option<&dyn miette::Diagnostic> { + self.diagnostic_source + .as_ref() + .map(|n| n as &dyn miette::Diagnostic) + } +} + +/// Wrap `report` so every prose surface it renders is control-character-escaped. /// -/// Per-file error handlers in directory-mode loops (e.g. `lint_one_file_human`, -/// `format_one_file`) MUST use this helper instead of bare -/// `eprintln!("{:?}", miette::Report::from(e))`. Centralising the render here -/// means the sanitizer cannot be forgotten on any future parallel path -/// (avoids PF-004: a check enforced on the primary path silently absent on a -/// sibling path). +/// This is the construction-boundary half of the PF-014 design: callers hand it a +/// `Report` built from raw values, and it produces one whose message, help, and label +/// text are safe to hand to miette. See [`SanitizedReport`] for the full rationale. +fn sanitize_report(report: miette::Report) -> miette::Report { + miette::Report::new(SanitizedReport::new(report)) +} + +/// Sanitize `report`'s inputs and render it to a `String` — the pure transformation +/// extracted from [`eprint_error`] so it can be tested without touching stderr. +/// +/// The rendered frame itself is never post-processed (PF-014); all escaping happens +/// in [`sanitize_report`], before miette sees the values. +/// +/// Note: idempotency is a property of [`mds::sanitize_control_chars`] (calling it +/// twice on already-sanitized input is a no-op), not of this function (each call +/// re-renders the `Report` from scratch). That idempotency is what lets +/// `render_diag_human` keep sanitizing its own inputs — it must, because it also +/// neutralizes the source excerpt and filename, which this boundary cannot do. +fn render_error_sanitized(report: miette::Report) -> String { + let report = sanitize_report(report); + format!("{report:?}") +} + +/// Render a miette `Report` to stderr — the single choke-point for all CLI error +/// output. /// -/// The `mds::sanitize_control_chars` function strips C0, C1, and DEL codepoints -/// while preserving `\n`, `\t`, and printable Unicode — miette box-drawing and -/// carets therefore survive intact; only raw ESC bytes and other non-printing -/// controls are escaped to `\uXXXX` literals. +/// All per-file error handlers in `main`, `build`, `fmt`, `lint`, and `watch` route +/// error rendering through this helper, so there is exactly one site to audit for +/// escape-injection safety (architecture-6 / avoids PF-004: a check enforced on the +/// primary path silently absent on a sibling path). +/// +/// **This function escapes the report's message, help, label text, and auxiliary +/// diagnostic graph itself**, via [`sanitize_report`], for every report it is given — +/// `MdsError` and CLI-authored `miette::miette!()` alike. Callers do not need to +/// pre-sanitize prose. +/// +/// What callers still owe, because this boundary cannot supply it: the +/// [`miette::NamedSource`] attached to the report, whose two halves need different +/// treatments (byte-length-preserving neutralization for the span-indexed source, WIRE +/// escaping for the single-line filename). Build it with +/// [`mds::named_source_for_render`] — `MdsError::at()` (compiler path), +/// `check_equivalence` (formatter) and `render_diag_human` (lint path) all do. +/// +/// miette's own ANSI SGR styling is passed through untouched — carets and box-drawing +/// survive intact. +/// +/// Note: status-line path display (`Clean:`, `Fixed:`, etc.) is handled by the +/// separate [`safe_path`] helper, not by this function. pub(crate) fn eprint_error(report: miette::Report) { - eprintln!("{}", mds::sanitize_control_chars(&format!("{report:?}"))); + eprintln!("{}", render_error_sanitized(report)); +} + +/// Print a CLI warning to stderr with HUMAN-mode escaping applied to the whole line +/// (CWE-150 / PF-004 / #176). +/// +/// Applies [`mds::sanitize_control_chars`] (preserves `\n` and `\t`; escapes all other +/// C0 / DEL / C1 controls and bidi / separator / BOM characters to their six-character +/// `\uXXXX` literals) before printing, so a hostile warning message cannot inject ANSI +/// terminal commands into stderr. +/// +/// # This helper alone is NOT sufficient — it is the prose half of the rule +/// +/// HUMAN mode preserves `\n` on purpose, so that a multi-line warning body still renders +/// as multiple lines. That means routing a hostile *identifier* through this function +/// closes CWE-150 on it and leaves CWE-117 open: `lint.rs`'s unknown-`mds.json`-rule +/// warning already called this helper, and a rule name of +/// `xClean: totally-real.mds0 problems found` still emitted three standalone +/// lines byte-identical in form to genuine status output. +/// +/// The governing rule is per FIELD, not per surface (spec §7.5): +/// +/// | Part of the line | Mode | Helper | +/// |------------------|------|--------| +/// | warning prose / body | HUMAN | this function | +/// | interpolated filename or path | WIRE | [`safe_path`] | +/// | interpolated identifier, config value, or error cause | WIRE | [`safe_inline`] | +/// +/// So the correct shape is `eprint_warning(&format!("warning: … {} …", safe_path(p)))` — +/// both escapes, not either one. +/// +/// # Enumeration is not a guarantee — the guard is +/// +/// Two earlier revisions of this rustdoc asserted coverage by listing the sites that had +/// been routed. Each list was correct when written and stale by the next review: first +/// `lint.rs`'s rule warning was missed, then the walker's own depth-limit warning inside +/// this very file. A guarantee stated as a property of a code path is only ever a +/// property of the sites someone remembered to enumerate — that is PF-004. +/// +/// The property is therefore no longer asserted in prose here. It is enforced by +/// `crates/mds-cli/tests/print_discipline.rs`, which fails if any print macro under +/// `crates/mds-cli/src/**` interpolates a value that is not passed through one of the +/// escape helpers, and which applies the same rule to `format!` invocations nested +/// inside `eprint_warning` calls. `watch.rs`'s lifecycle status lines — previously +/// carved out as a pre-existing gap — are in scope and now routed like everything else. +pub(crate) fn eprint_warning(w: &str) { + eprintln!("{}", mds::sanitize_control_chars(w)); +} + +/// Neutralize hostile control bytes in source text for `--diff` preview output. +/// +/// `--diff` preview output: neutralized on TTY, byte-faithful when piped. +/// +/// **`--diff` only.** `--check` on its own emits no preview text — just `Would fix:` / +/// `Would reformat:` status lines, which are sanitized unconditionally via +/// [`safe_path`] and never reach this function. The only caller is +/// [`render_unified_diff`], shared by `mds fmt --diff` and `mds lint --fix --diff`. +/// +/// When `writer_is_tty` is `true`, returns [`neutralize_source_for_render`]`(text)` — +/// a byte-length-preserving substitution that maps C0/DEL/C1 controls and the widened +/// bidi/format hazard class (added in #176) to `?` or U+00A0/U+FFFD so hostile template +/// source cannot inject ANSI terminal commands into the rendered diff (CWE-150). +/// +/// When `writer_is_tty` is `false`, returns `Cow::Borrowed(text)` unchanged so +/// redirected diff output (e.g. `mds fmt --diff > patch.diff`) stays byte-faithful +/// and applicable by `patch`/tooling. +/// +/// This is a pure, allocation-free helper for clean inputs; the TTY detection +/// (`std::io::stdout().is_terminal()`) is performed at the call site so this function +/// is testable without a real TTY (avoids PF-014: sanitize renderer inputs, not the +/// rendered frame). +/// +/// # Byte-length invariant +/// +/// `neutralize_source_for_render` is byte-length-preserving — every substitution +/// produces a replacement of the same UTF-8 byte count — so diff hunk byte offsets +/// remain coherent after substitution on the TTY path. Never use +/// [`mds::sanitize_control_chars`] here: it expands 1–2-byte controls to 6 bytes, +/// desynchronising all offsets that follow. +/// +/// # Boundary table entry (consistent with `crates/mds-core/src/lint/diagnostic.rs`) +/// +/// | Boundary | Mode | Content | +/// |----------|------|---------| +/// | `--diff` preview output | neutralized, TTY-gated | source excerpts via `neutralize_source_for_render`; piped path returns `Cow::Borrowed` | +#[must_use] +pub(crate) fn preview_text_for(writer_is_tty: bool, text: &str) -> Cow<'_, str> { + if writer_is_tty { + neutralize_source_for_render(text) + } else { + Cow::Borrowed(text) + } +} + +// ── Diff rendering ─────────────────────────────────────────────────────────── + +/// Render a unified diff between `original` and `modified` with optional colorization. +/// +/// The single shared implementation used by both `fmt --diff` and `lint --diff`. +/// +/// When stdout is a TTY, both strings are neutralized via [`preview_text_for`] before +/// diffing so hostile ESC/control bytes in template source cannot inject ANSI commands +/// into the rendered diff (CWE-150, security-11). When piped, the strings pass through +/// unchanged so redirected diff output remains byte-faithful and applicable by +/// `patch`/tooling. +#[must_use] +pub(crate) fn render_unified_diff(original: &str, modified: &str, label: &str) -> String { + let is_tty = std::io::stdout().is_terminal(); + let original = preview_text_for(is_tty, original); + let modified = preview_text_for(is_tty, modified); + let diff = similar::TextDiff::from_lines(original.as_ref(), modified.as_ref()); + let unified = diff + .unified_diff() + .context_radius(3) + .header(label, label) + .to_string(); + + if unified.is_empty() || !is_tty { + return unified; + } + colorize_unified_diff(&unified) +} + +/// Colorize a unified diff string with ANSI codes. +/// +/// Uses a state machine keyed on the first `@@` hunk marker to correctly color +/// removed (`-`) and added (`+`) lines inside hunks without miscoloring file-header +/// lines (`---`/`+++`) as hunk content when a removed or added line's own content +/// starts with `--` or `++`. +pub(crate) fn colorize_unified_diff(unified: &str) -> String { + const RED: &str = "\x1b[31m"; + const GREEN: &str = "\x1b[32m"; + const CYAN: &str = "\x1b[36m"; + const RESET: &str = "\x1b[0m"; + + let mut out = String::with_capacity(unified.len() + 64); + let mut in_hunk = false; + for line in unified.split_inclusive('\n') { + let color = if line.starts_with("@@") { + in_hunk = true; + CYAN + } else if !in_hunk && (line.starts_with("---") || line.starts_with("+++")) { + CYAN + } else if in_hunk && line.starts_with('+') { + GREEN + } else if in_hunk && line.starts_with('-') { + RED + } else { + "" + }; + if color.is_empty() { + out.push_str(line); + continue; + } + out.push_str(color); + match line.strip_suffix('\n') { + Some(stripped) => { + out.push_str(stripped); + out.push_str(RESET); + out.push('\n'); + } + None => { + out.push_str(line); + out.push_str(RESET); + } + } + } + out +} + +/// Sanitize a filesystem path for terminal display (CWE-150 / CWE-117 guard). +/// +/// Converts the path to a display string and applies WIRE-mode +/// [`mds::sanitize_control_chars_wire`] so hostile filenames cannot inject ANSI +/// terminal commands (e.g. `ESC[2J`) *or* forge additional status lines. +/// +/// # Why WIRE and not HUMAN +/// +/// Status lines are single-line by construction: `Clean: {path}`, `Fixed: {path}`, +/// `Compiled to {path}` are emitted unframed and unindented, one per file. POSIX +/// permits a newline inside a filename, and the user never types the name — `mds lint .` +/// discovers it by directory walk. HUMAN mode preserves newlines (so that multi-line +/// diagnostic *messages* keep rendering), which would let a file whose name embeds +/// `Clean: real.mdsOK: all-fine.mds` emit two attacker-authored lines +/// byte-identical in form to genuine output. A filename is never legitimately +/// multi-line, so escaping the newline to its six-character literal costs nothing and +/// closes the forgery. +/// +/// This matches the treatment `files[].file` already gets on the JSON wire surface +/// (`LintResult::to_canonical_json`) and that miette frame headers get via +/// [`mds::named_source_for_render`] — the human status path was the inconsistent one. +/// +/// All status-line path interpolations (`Clean:`, `Fixed:`, `Compiled to:`, etc.) in +/// `lint`, `fmt`, and `build` must route through this helper (avoids PF-004 / +/// security-5: unsanitized filename vector in status output). +pub(crate) fn safe_path(p: &std::path::Path) -> String { + safe_inline(p.display()) +} + +/// [`safe_path`] for a filename that is already a `&str` (e.g. a `LintDiagnostic::file` +/// basename that never became a `Path`). +/// +/// Exists so those sites cannot drift into open-coding a different escape mode — the +/// exact PF-004 shape that left `Clean: {filename}` on HUMAN mode while every other +/// status line was on WIRE. +pub(crate) fn safe_file_display(name: &str) -> String { + safe_inline(name) +} + +/// WIRE-escape any untrusted value that is interpolated into a **single-line** status, +/// warning, or error line. +/// +/// This is the general form of [`safe_path`] / [`safe_file_display`]: the same WIRE +/// escape, for values that are neither a `Path` nor a filename — an `io::Error` +/// `Display` (which embeds a filesystem path), an `mds.json` rule name or config value, +/// a `--format` argument, a fix-rejection reason. +/// +/// # Why WIRE, on human surfaces too +/// +/// Per the governing per-field rule (spec §7.5): **on the diagnostic surfaces — the +/// `"version": 1` JSON wire, CLI status and warning lines, `[file:line:col]` frame +/// headers — untrusted identifiers, filenames and causes are WIRE-escaped, human output +/// included; only *prose* — a diagnostic message or help body — stays HUMAN.** (Source-map +/// paths and `CompileResult.dependencies` are the named carve-out: not diagnostics, not +/// escaped.) The discriminator is whether the +/// value is legitimately multi-line. A rule name, a path, a `--format` value and an +/// `io::Error` never are; a diagnostic body genuinely is. Leaving `\n` raw in the first +/// group buys nothing and lets the value forge a standalone status line that is +/// byte-identical in form to genuine output (CWE-117). +/// +/// The surrounding warning prose stays HUMAN — pass the assembled string to +/// [`eprint_warning`], and WIRE-escape each interpolated value with this helper. +/// +/// Idempotent (a property of [`mds::sanitize_control_chars_wire`]), so wrapping a value +/// that was already escaped at construction — e.g. +/// `mds::fix::FixOutcome::Rejected.reason` — is a no-op rather than a double escape. +/// That matters: it lets every call site apply the rule unconditionally instead of +/// tracking which values arrived pre-escaped (PF-004). +/// +/// Enforced mechanically by `tests/print_discipline.rs`. +pub(crate) fn safe_inline(value: impl std::fmt::Display) -> String { + mds::sanitize_control_chars_wire(&value.to_string()).into_owned() } // ── Unit tests ──────────────────────────────────────────────────────────────── @@ -787,4 +1428,716 @@ mod tests { let result = output_base_no_ext(&source, &root, &base); assert_eq!(result, PathBuf::from("/root/src/chat")); } + + // ── preview_text_for: TTY-gated source neutralization (security-11) ────────── + + /// T-12a [security-11 / PF-013]: preview_text_for(true, hostile) neutralizes ESC + /// and preserves byte length (so diff hunks remain coherent after substitution). + /// + /// Positive assertion required by PF-013: the neutralized form MUST be present, + /// not merely asserted absent in the raw form. + #[test] + fn preview_text_for_tty_neutralizes_esc_preserves_byte_length() { + let hostile = "hello\x1bworld"; // ESC (U+001B) is 1-byte C0 + let result = preview_text_for(true, hostile); + // Byte-length invariant: neutralize_source_for_render is byte-length-preserving. + assert_eq!( + result.len(), + hostile.len(), + "byte length must be preserved on TTY path" + ); + // ESC must be absent from TTY output (security gate). + assert!( + !result.contains('\x1b'), + "raw ESC must be absent from TTY output; got: {result:?}" + ); + // PF-013 positive assertion: the substituted '?' must be present. + assert!( + result.contains('?'), + "ESC must be replaced with '?' on TTY path; got: {result:?}" + ); + } + + /// T-12b [security-11 / PF-013]: preview_text_for(false, ...) returns Cow::Borrowed + /// so piped diff output is byte-identical to the raw source (patch/tooling safety). + #[test] + fn preview_text_for_not_tty_returns_borrowed_passthrough() { + let hostile = "hello\x1bworld"; + let result = preview_text_for(false, hostile); + // Must be Cow::Borrowed — no allocation, byte-identical to input. + assert!( + matches!(result, Cow::Borrowed(_)), + "piped path must return Cow::Borrowed (no allocation); got Owned" + ); + assert_eq!( + result.as_ref(), + hostile, + "piped path must be byte-identical to input" + ); + } + + /// T-12c [security-11 / PF-013]: preview_text_for(true, clean) returns unchanged content + /// because neutralize_source_for_render returns Cow::Borrowed for clean inputs. + #[test] + fn preview_text_for_tty_clean_string_unchanged() { + let clean = "hello world\n"; + let result = preview_text_for(true, clean); + assert_eq!( + result.as_ref(), + clean, + "clean string must be unchanged on TTY path" + ); + } + + // ── safe_path: CWE-150 status-line guard (security-5/6) ────────────────────── + + /// T-11a [security-5 / PF-013]: safe_path escapes a raw ESC byte in a filename + /// to the 6-char \\uXXXX literal so it cannot inject ANSI into a status line. + #[test] + fn safe_path_sanitizes_esc_byte() { + let raw = format!("dir/fo{}o.mds", '\x1b'); + let p = std::path::Path::new(&raw); + let result = safe_path(p); + assert!( + !result.contains('\x1b'), + "raw ESC must be absent from safe_path output" + ); + assert!( + result.contains("\\u001B"), + "ESC must be escaped to \\u001B; got: {result:?}" + ); + } + + /// T-11b [security-5 / PF-013]: safe_path passes a clean path through unchanged + /// (no unnecessary allocation or mutation). + #[test] + fn safe_path_passes_clean_path_unchanged() { + let p = std::path::Path::new("dir/normal.mds"); + assert_eq!(safe_path(p), "dir/normal.mds"); + } + + // ── T-10a/b/c: neutralize_source_for_render + colour path (── PF-014) ──────── + + /// T-10a [PF-014 / AC-F2]: neutralize_source_for_render removes C0 controls + /// (except \n/\t), DEL, and 2-byte C1 controls while preserving total byte length + /// so that miette span offsets remain valid after substitution. + #[test] + fn neutralize_source_removes_c0_del_c1_preserving_byte_length() { + // ASCII NUL (C0), DEL (0x7F), and U+0085 NEL (C1, 2-byte UTF-8) are hostile. + // \n and \t are allowed through unchanged. + let raw = "a\x00b\x7fc\u{0085}d\ne\tf"; + let out = neutralize_source_for_render(raw); + // Byte length must be identical (span safety invariant). + assert_eq!(out.len(), raw.len(), "byte length preserved"); + // Hostile bytes are replaced; safe bytes survive. + assert!(!out.contains('\x00'), "NUL removed"); + assert!(!out.contains('\x7f'), "DEL removed"); + assert!(!out.contains('\u{0085}'), "C1 NEL removed"); + assert!(out.contains('\n'), "LF preserved"); + assert!(out.contains('\t'), "TAB preserved"); + } + + /// T-10b [reliability-8 / PF-014]: When the span starts AFTER a control character + /// the substituted byte at that position must still form a valid char boundary so + /// miette can slice the excerpt without panicking. + #[test] + fn neutralize_source_caret_alignment_with_span_after_control_char() { + use miette::{GraphicalReportHandler, GraphicalTheme, NamedSource, SourceSpan}; + + // "abc" where span covers 'b' (byte offset 2..3, AFTER the ESC byte). + // If neutralization breaks the byte-length invariant, miette panics here. + let raw = "a\x1bbc"; + let clean = neutralize_source_for_render(raw); + assert_eq!(clean.len(), raw.len(), "byte length invariant"); + + // Build a minimal miette report whose source excerpt exercises the span. + #[derive(Debug, thiserror::Error, miette::Diagnostic)] + #[error("test")] + struct SpanErr { + #[source_code] + src: NamedSource, + #[label("here")] + span: SourceSpan, + } + + let report = miette::Report::new(SpanErr { + src: NamedSource::new("test.mds", clean.into_owned()), + span: (2, 1).into(), // byte 2..3 = 'b' + }); + + let mut buf = String::new(); + GraphicalReportHandler::new_themed(GraphicalTheme::unicode_nocolor()) + .render_report(&mut buf, report.as_ref()) + .expect("render must not panic after neutralization"); + // The caret must point at 'b', not produce garbage. + assert!(buf.contains('b'), "caret points at b"); + // No raw ESC byte survives into the rendered output. + assert!(!buf.contains('\x1b'), "no raw ESC in rendered output"); + } + + /// T-10c [testing-2 / PF-014]: miette's own SGR colour codes survive + /// (render_error_sanitized never post-processes the frame) while a hostile + /// OSC sequence embedded in the source is neutralised at the input stage. + #[test] + fn colour_path_miette_sgr_survives_hostile_osc_is_removed() { + use miette::{GraphicalReportHandler, GraphicalTheme, NamedSource, SourceSpan}; + + // A 2-byte C1 U+009D = OSC opener (hostile). After neutralize it becomes + // U+00A0 NBSP, same byte length; no OSC survives into the render. + // U+009D in UTF-8 is 0xC2 0x9D (2 bytes). We use the char directly. + let hostile_char = '\u{009D}'; // C1 OSC opener + let raw = format!("good{}text", hostile_char); + let clean = neutralize_source_for_render(&raw); + assert_eq!(clean.len(), raw.len(), "byte length preserved"); + assert!( + !clean.contains(hostile_char), + "C1 OSC neutralised in source" + ); + + #[derive(Debug, thiserror::Error, miette::Diagnostic)] + #[error("colour test")] + struct ColourErr { + #[source_code] + src: NamedSource, + #[label("here")] + span: SourceSpan, + } + + let src_len = clean.len(); + let report = miette::Report::new(ColourErr { + src: NamedSource::new("colour.mds", clean.into_owned()), + span: (0, src_len).into(), + }); + + // Colour-enabled renderer — miette will emit real SGR codes. + let mut coloured = String::new(); + GraphicalReportHandler::new_themed(GraphicalTheme::unicode()) + .render_report(&mut coloured, report.as_ref()) + .expect("render must not panic"); + + // miette's own ANSI codes must survive in the coloured output. + assert!( + coloured.contains('\x1b'), + "miette SGR codes present in coloured output" + ); + // But the hostile C1 byte is gone from the rendered string. + assert!( + !coloured.contains(hostile_char), + "hostile C1 absent from rendered output" + ); + } + + // ── sanitize_report: the CLI human message/help boundary (CWE-150 / #176) ──── + // + // These are in-process unit tests so they can pin the COLOUR path, which the + // subprocess e2e tests in `tests/security.rs` are structurally blind to (they set + // `NO_COLOR=1` and pipe stderr). Per the H4 test-determinism decision, colour is + // selected explicitly via `GraphicalTheme` rather than inherited from the + // environment. + + /// Render `report` through a colour-neutral handler — the deterministic in-process + /// equivalent of what `render_error_sanitized` emits under `NO_COLOR=1`. + fn render_nocolor(report: &miette::Report) -> String { + use miette::{GraphicalReportHandler, GraphicalTheme}; + let mut buf = String::new(); + GraphicalReportHandler::new_themed(GraphicalTheme::unicode_nocolor()) + .render_report(&mut buf, report.as_ref()) + .expect("render must not panic"); + buf + } + + /// T-ESC-1 [security-11 / PF-013 / #176]: an `MdsError` whose message *and* help + /// both interpolate attacker-controlled text is escaped on both surfaces before + /// miette renders it. + /// + /// `UndefinedVariable` is the vector because `name` lands in the `#[error(...)]` + /// message and in the `#[diagnostic(help(...))]` text, so one input exercises both. + #[test] + fn sanitize_report_escapes_message_and_help() { + let hostile_name = format!("bad{}[31mNAME", '\u{1b}'); + let report = sanitize_report(miette::Report::new(mds::MdsError::UndefinedVariable { + name: hostile_name, + span: None, + src: None, + })); + let rendered = render_nocolor(&report); + + // Non-vacuity: the diagnostic actually rendered, with its help line. + assert!( + rendered.contains("undefined variable"), + "non-vacuity: expected the undefined-variable message; got: {rendered:?}" + ); + assert!( + rendered.contains("help:"), + "non-vacuity: expected a help line; got: {rendered:?}" + ); + // Negative. + assert!( + !rendered.contains('\u{1b}'), + "raw ESC must not survive into the rendered frame; got: {rendered:?}" + ); + // Positive: escaped on BOTH surfaces, so the count is at least two. + assert!( + rendered.matches("\\u001B").count() >= 2, + "ESC must be escaped in the message AND the help text; got: {rendered:?}" + ); + } + + /// T-ESC-2 [security-11 / PF-004 / #176]: a CLI-authored `miette::miette!()` report + /// — which does NOT downcast to `MdsError` — is escaped by the same boundary. + #[test] + fn sanitize_report_escapes_cli_authored_miette_message() { + let hostile = format!("cannot write fo{}[2Jo.mds", '\u{1b}'); + let report = sanitize_report(miette::miette!("{hostile}")); + let rendered = render_nocolor(&report); + + assert!( + rendered.contains("cannot write"), + "non-vacuity: expected the miette!() message; got: {rendered:?}" + ); + assert!( + !rendered.contains('\u{1b}'), + "raw ESC must not survive a miette!() report; got: {rendered:?}" + ); + assert!( + rendered.contains("\\u001B"), + "ESC must be escaped to the \\u001B literal; got: {rendered:?}" + ); + } + + /// T-ESC-3 [security-11 / #176]: the widened hazard class (bidi controls, Trojan + /// Source CVE-2021-42574) is escaped on this boundary too, not just C0/DEL/C1. + #[test] + fn sanitize_report_escapes_bidi_override_in_message() { + let hostile = format!("alias{}reversed", '\u{202e}'); + let report = sanitize_report(miette::miette!("{hostile}")); + let rendered = render_nocolor(&report); + + assert!( + !rendered.contains('\u{202e}'), + "raw U+202E must not survive; got: {rendered:?}" + ); + assert!( + rendered.contains("\\u202E"), + "U+202E must be escaped to the \\u202E literal; got: {rendered:?}" + ); + } + + /// T-ESC-4 [PF-014 / #176]: HUMAN mode — a real newline in the message survives, so + /// multi-line diagnostic frames stay readable. This is the one deliberate + /// difference from the WIRE boundary (`MdsError::serialize`), which escapes `\n`. + #[test] + fn sanitize_report_preserves_newline_in_message() { + let report = sanitize_report(miette::miette!("line one\nline two")); + let rendered = render_nocolor(&report); + + assert!( + rendered.contains("line one"), + "non-vacuity: message must render; got: {rendered:?}" + ); + assert!( + !rendered.contains("\\u000A"), + "HUMAN mode must NOT escape newlines; got: {rendered:?}" + ); + } + + /// T-ESC-5 [security-11 / #176]: label TEXT is escaped while the label's byte SPAN + /// is forwarded verbatim, so the caret still lands on the right columns. + /// + /// `LintDiagnostic::labels()` uses its own message as the label text, so this is a + /// real untrusted-text surface and not a hypothetical one. + #[test] + fn sanitize_report_escapes_label_text_but_keeps_span() { + let hostile_label = format!("here{}[31m", '\u{1b}'); + let diag = miette::MietteDiagnostic::new("outer message") + .with_label(miette::LabeledSpan::at(2..5, hostile_label)); + let report = + sanitize_report(miette::Report::new(diag).with_source_code("abcdefgh".to_string())); + let rendered = render_nocolor(&report); + + assert!( + rendered.contains("here"), + "non-vacuity: the label must render; got: {rendered:?}" + ); + assert!( + !rendered.contains('\u{1b}'), + "raw ESC must not survive in label text; got: {rendered:?}" + ); + assert!( + rendered.contains("\\u001B"), + "label ESC must be escaped; got: {rendered:?}" + ); + // Span geometry preserved: the source line and its caret still render. + assert!( + rendered.contains("abcdefgh"), + "the labelled source excerpt must still render; got: {rendered:?}" + ); + } + + /// T-ESC-6 [PF-014 / #176]: the regression this boundary exists to avoid. + /// + /// Rendering the sanitized report with a COLOUR theme must leave miette's own ANSI + /// SGR codes intact (real ESC bytes present) while the hostile input is escaped to + /// a literal. An implementation that sanitized the rendered frame instead would + /// strip miette's SGR into literal `\u001b[...` noise — this test fails loudly in + /// that case, and the subprocess e2e tests cannot (they pin `NO_COLOR=1`). + #[test] + fn sanitize_report_colour_path_keeps_miette_sgr_and_escapes_hostile_input() { + use miette::{GraphicalReportHandler, GraphicalTheme}; + + let hostile = format!("hostile{}[31mtext", '\u{1b}'); + let report = sanitize_report(miette::miette!("{hostile}")); + + let mut coloured = String::new(); + GraphicalReportHandler::new_themed(GraphicalTheme::unicode()) + .render_report(&mut coloured, report.as_ref()) + .expect("render must not panic"); + + // miette's own SGR codes survive — the frame was never post-processed. + assert!( + coloured.contains('\u{1b}'), + "miette's own SGR codes must survive on the colour path; got: {coloured:?}" + ); + // The hostile input is still escaped. + assert!( + coloured.contains("\\u001B"), + "hostile ESC must be escaped even on the colour path; got: {coloured:?}" + ); + // And miette's SGR was NOT itself escaped: the only escaped literal is the one + // hostile byte, not the ~10 SGR sequences the frame contains. + assert_eq!( + coloured.matches("\\u001B").count(), + 1, + "exactly one escaped literal (the hostile byte) — more means miette's own \ + SGR codes were escaped, which is the PF-014 regression; got: {coloured:?}" + ); + } + + /// T-ESC-7 [#176]: clean input renders byte-identically with and without the + /// sanitizing wrapper, so the boundary is inert on the overwhelmingly common path. + #[test] + fn sanitize_report_is_inert_on_clean_input() { + let raw = miette::Report::new(mds::MdsError::UndefinedVariable { + name: "user_name".to_string(), + span: None, + src: None, + }); + let before = render_nocolor(&raw); + let after = render_nocolor(&sanitize_report(raw)); + assert_eq!( + before, after, + "sanitizing must not alter the render of a clean diagnostic" + ); + } + + // ── sanitize_report: the auxiliary diagnostic graph (PF-005 / #176) ───────── + // + // `source()` / `related()` / `diagnostic_source()` used to be reported as absent, + // guarded only by a `debug_assert!` that no CLI error populates them. `debug_assert!` + // is compiled out of release, so that invariant was real in tests and ABSENT in the + // shipped binary: the first error type to grow a `#[source]` field would have had + // its cause chain silently dropped from release stderr while CI stayed green. + // + // These tests are pure functional assertions on the `Diagnostic` impl, so they hold + // identically in debug and release — which is the point. + + /// A two-link cause chain: an outer diagnostic whose `#[source]` carries hostile text. + #[derive(Debug, thiserror::Error, miette::Diagnostic)] + #[error("outer failure")] + struct OuterWithCause { + #[source] + cause: InnerCause, + } + + #[derive(Debug, thiserror::Error)] + #[error("inner cause: {0}")] + struct InnerCause(String); + + /// A diagnostic that hangs a hostile `related` diagnostic off itself. + #[derive(Debug, thiserror::Error, miette::Diagnostic)] + #[error("parent diagnostic")] + struct ParentWithRelated { + #[related] + related: Vec, + } + + #[derive(Debug, thiserror::Error, miette::Diagnostic)] + #[error("related child: {0}")] + #[diagnostic(help("child help: {0}"))] + struct RelatedChild(String); + + /// The hostile fragment shared by the auxiliary-graph tests: one C0 byte, one + /// 3-byte bidi control, and the 2-byte bidi control #176 added. + fn hostile_fragment() -> String { + format!("bad{}[31m{}{}end", '\u{1b}', '\u{202e}', '\u{061c}') + } + + /// Negative + positive assertions shared by T-AUX-1/2/3. + fn assert_aux_text_escaped(rendered: &str, surface: &str) { + for raw in ['\u{1b}', '\u{202e}', '\u{061c}'] { + assert!( + !rendered.contains(raw), + "{surface}: raw U+{:04X} must not survive into the rendered frame; \ + got: {rendered:?}", + raw as u32 + ); + } + // Uppercase literals from a lowercase-hex source vector: proof the byte really + // decoded and was really escaped rather than passing through as literal text. + for escaped in ["\\u001B", "\\u202E", "\\u061C"] { + assert!( + rendered.contains(escaped), + "{surface}: {escaped} must appear in the rendered frame; got: {rendered:?}" + ); + } + } + + /// T-AUX-1 [PF-005 / security-11 / PF-013 / #176]: a report carrying a `#[source]` + /// cause chain renders that cause, escaped — neither leaked raw nor dropped. + #[test] + fn sanitize_report_escapes_and_preserves_the_cause_chain() { + let report = sanitize_report(miette::Report::new(OuterWithCause { + cause: InnerCause(hostile_fragment()), + })); + let rendered = render_nocolor(&report); + + // Non-vacuity: the cause is actually rendered. This is the assertion that fails + // if `source()` reverts to returning `None` — the release-build silent drop. + assert!( + rendered.contains("inner cause"), + "non-vacuity: the cause chain must still render; got: {rendered:?}" + ); + assert!( + rendered.contains("outer failure"), + "non-vacuity: the outer message must still render; got: {rendered:?}" + ); + assert_aux_text_escaped(&rendered, "cause chain"); + } + + /// T-AUX-2 [PF-005 / #176]: `related()` diagnostics — message AND help — are + /// escaped and preserved. + #[test] + fn sanitize_report_escapes_and_preserves_related_diagnostics() { + let report = sanitize_report(miette::Report::new(ParentWithRelated { + related: vec![RelatedChild(hostile_fragment())], + })); + let rendered = render_nocolor(&report); + + assert!( + rendered.contains("parent diagnostic"), + "non-vacuity: the parent message must render; got: {rendered:?}" + ); + assert!( + rendered.contains("related child"), + "non-vacuity: the related diagnostic must still render; got: {rendered:?}" + ); + assert!( + rendered.contains("child help"), + "non-vacuity: the related diagnostic's help must still render; got: {rendered:?}" + ); + assert_aux_text_escaped(&rendered, "related diagnostics"); + } + + /// T-AUX-3 [PF-005 / #176]: the walk is depth-bounded, so a self-referential + /// `source()` cannot hang or overflow the stack during rendering. + /// + /// `Cycle::source()` returns `self`, an infinite chain. Construction must terminate. + #[test] + fn sanitize_report_bounds_a_cyclic_cause_chain() { + #[derive(Debug)] + struct Cycle; + impl std::fmt::Display for Cycle { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str("cyclic cause") + } + } + impl std::error::Error for Cycle { + fn source(&self) -> Option<&(dyn std::error::Error + 'static)> { + Some(&Cycle) + } + } + #[derive(Debug, thiserror::Error, miette::Diagnostic)] + #[error("outer")] + struct Outer(#[source] Cycle); + + let wrapped = SanitizedReport::new(miette::Report::new(Outer(Cycle))); + + // Walk the materialised chain and assert it is finite and within the bound. + let mut depth = 0usize; + let mut node = wrapped.source.as_ref(); + while let Some(n) = node { + depth += 1; + assert!( + depth <= MAX_AUX_DEPTH, + "the cause-chain walk must be bounded by MAX_AUX_DEPTH" + ); + node = n.source.as_deref(); + } + // Non-vacuity: the bound was actually reached, so this is not passing because + // the chain was empty. + assert_eq!( + depth, MAX_AUX_DEPTH, + "an infinite chain must be truncated at exactly MAX_AUX_DEPTH" + ); + } + + // ── eprint_warning: the CLI warning sanitization boundary (CWE-150 / PF-004 / #176) ── + // + // eprint_warning is a thin wrapper around mds::sanitize_control_chars + eprintln!. + // The tests below exercise the transformation directly (the pure function that the + // wrapper applies) to keep assertions deterministic without capturing stderr. + // + // Test strategy — corrected. + // + // An earlier revision of this block claimed that "the only warning that interpolates + // untrusted text is the resolver warning" (which needs a MAX_SOURCEMAP_SEGMENTS + // overflow in a hostile-named module, so an e2e vector for it would be contrived), + // and concluded that no e2e test was required because "every warning print in + // main.rs and build.rs now calls eprint_warning". Both halves were false, and the + // unreachability argument was the load-bearing one: + // + // * lint.rs's unknown-`mds.json`-rule warning (added in e145e41) interpolates an + // arbitrary JSON object key — reachable in about thirty seconds by writing an + // mds.json into any linted directory. It is covered e2e by T-ESC-RULE-1 in + // tests/security.rs, whose vector now carries newlines as well as C0 and bidi + // controls, because routing it through eprint_warning (HUMAN, `\n` preserved) + // closed CWE-150 on it while leaving the CWE-117 line forgery open. + // * The walker's own depth-limit warning, ~40 lines up in THIS file, printed + // `dir.display()` through a bare eprintln!. It is neither main.rs nor build.rs, + // so the enumeration above walked straight past it. Covered e2e by T-ESC-WALK-1. + // + // The unit tests below stay — they pin the transformation deterministically without + // capturing stderr — but they are no longer offered as a substitute for e2e vectors. + // + // Coverage is now enforced rather than enumerated: tests/print_discipline.rs fails if + // ANY print macro under crates/mds-cli/src interpolates a value that is not passed + // through safe_path / safe_file_display / safe_inline / sanitize_control_chars*, and + // applies the same rule to `format!`s nested inside eprint_warning calls. That is + // what makes a claim about warning-path coverage checkable instead of remembered. + // + // Guard-removal RED evidence (T-WARN-1): + // Replace `mds::sanitize_control_chars(&hostile)` with + // `std::borrow::Cow::Borrowed(hostile.as_str())` in the test body. + // The test fails with: + // assertion `left == right` failed: raw ESC must be absent from warning output + // left: true + // right: false + // because the unsanitized string still contains '\u{1b}'. + // Restoring the call makes the test green. + + /// T-WARN-1 [security-11 / PF-004 / PF-013 / #176]: a hostile warning string + /// (one that interpolates a filename containing a raw ESC byte) is sanitized to + /// its `\uXXXX` literal form — raw ESC is absent and the escaped form is present. + /// + /// This tests the exact transformation that `eprint_warning` applies before printing. + #[test] + fn eprint_warning_sanitizes_hostile_control_chars() { + // Simulate the only real-world hostile vector: the resolver warning that + // interpolates an untrusted module filename (mds-core/src/resolver.rs). + let hostile = format!( + "MAX_SOURCEMAP_SEGMENTS exceeded in imported module 'lib{}[2Jbar.mds'", + '\u{1b}' + ); + // This is exactly what eprint_warning applies before calling eprintln!. + let result = mds::sanitize_control_chars(&hostile); + + // Non-vacuity: the plain-text content is preserved. + assert!( + result.contains("MAX_SOURCEMAP_SEGMENTS"), + "non-vacuity: warning text must be preserved; got: {result:?}" + ); + assert!( + result.contains("lib"), + "non-vacuity: filename prefix must be preserved; got: {result:?}" + ); + // Negative: raw ESC absent. + assert!( + !result.contains('\u{1b}'), + "raw ESC must be absent from warning output; hostile = {hostile:?}" + ); + // Positive: escaped form present. + assert!( + result.contains("\\u001B"), + "ESC must be escaped to \\u001B literal; got: {result:?}" + ); + } + + /// T-WARN-2 [PF-013 / #176]: a clean warning string passes through unchanged — + /// the sanitization is inert on the overwhelmingly common path. + /// + /// This corresponds to the "clean string passes through unchanged" requirement from + /// the task brief: eprint_warning must never mutate a warning that contains no + /// hazardous bytes. + #[test] + fn eprint_warning_clean_string_passes_through_unchanged() { + let clean = "MAX_SOURCEMAP_SEGMENTS exceeded in imported module 'lib.mds'"; + let result = mds::sanitize_control_chars(clean); + assert_eq!( + result.as_ref(), + clean, + "clean warning must pass through unchanged; got: {result:?}" + ); + } + + /// T-WARN-3 [PF-013 / #176]: the widened hazard class (bidi controls, Trojan + /// Source CVE-2021-42574) is escaped by eprint_warning too — not just C0/ESC. + #[test] + fn eprint_warning_sanitizes_bidi_control_in_warning_text() { + // U+202E RIGHT-TO-LEFT OVERRIDE: injecting this into a warning message + // reverses how the rest of the terminal line renders in bidi-aware terminals. + let hostile = format!("warning: module 'lib{}evil.mds'", '\u{202e}'); + let result = mds::sanitize_control_chars(&hostile); + assert!( + !result.contains('\u{202e}'), + "raw U+202E must be absent from warning output; got: {result:?}" + ); + assert!( + result.contains("\\u202E"), + "U+202E must be escaped to \\u202E literal; got: {result:?}" + ); + } + + // ── render_unified_diff / colorize_unified_diff ─────────────────────────── + + #[test] + fn render_unified_diff_empty_for_identical_input() { + let rendered = render_unified_diff("same\n", "same\n", "label"); + assert!(rendered.is_empty()); + } + + #[test] + fn render_unified_diff_contains_unified_markers_for_changed_input() { + let rendered = render_unified_diff("a\n", "b\n", "label"); + assert!(rendered.contains("---")); + assert!(rendered.contains("+++")); + assert!(rendered.contains("-a")); + assert!(rendered.contains("+b")); + } + + #[test] + fn colorize_unified_diff_wraps_add_remove_lines_with_ansi_when_requested() { + let unified = "--- a\n+++ b\n@@ -1 +1 @@\n-old\n+new\n"; + let colorized = colorize_unified_diff(unified); + assert!(colorized.contains("\x1b[32m+new\x1b[0m")); + assert!(colorized.contains("\x1b[31m-old\x1b[0m")); + assert!(colorized.contains("\x1b[36m--- a\x1b[0m")); + } + + #[test] + fn colorize_unified_diff_correctly_colors_content_starting_with_dashes_or_pluses() { + // Regression: a removed line whose content starts with "-- " produces + // "--- ..." in the rendered unified diff. The old global prefix check + // matched `starts_with("---")` and mis-colored it CYAN (file header) + // instead of RED (removal). Same defect for "++" content → "+++ " → + // mis-colored CYAN instead of GREEN. + let unified = "--- a\n+++ b\n@@ -1,2 +1,2 @@\n--- dashes content\n+++ plus content\n"; + let colorized = colorize_unified_diff(unified); + // Inside the hunk: removal of a line whose content starts with "-- " + assert!(colorized.contains("\x1b[31m--- dashes content\x1b[0m")); + // Inside the hunk: addition of a line whose content starts with "++" + assert!(colorized.contains("\x1b[32m+++ plus content\x1b[0m")); + // File headers (before @@) must still be CYAN + assert!(colorized.contains("\x1b[36m--- a\x1b[0m")); + assert!(colorized.contains("\x1b[36m+++ b\x1b[0m")); + } } diff --git a/crates/mds-cli/src/watch.rs b/crates/mds-cli/src/watch.rs index 92232b67..e278193f 100644 --- a/crates/mds-cli/src/watch.rs +++ b/crates/mds-cli/src/watch.rs @@ -39,8 +39,9 @@ use crate::build::{ resolve_output_path_for_kind, write_output, OutputKind, RuntimeVarArgs, }; use crate::output::{ - canonicalize_out_dir, collect_mds_files, is_partial, is_within_default_excluded_dir, - output_base_no_ext, output_path_for, probe_and_remove_stale, resolve_output_base, OutputBase, + canonicalize_out_dir, collect_mds_files, eprint_error, eprint_warning, is_partial, + is_within_default_excluded_dir, output_base_no_ext, output_path_for, probe_and_remove_stale, + resolve_output_base, safe_inline, safe_path, OutputBase, }; // ── Public args struct ──────────────────────────────────────────────────────── @@ -382,7 +383,11 @@ pub(crate) fn resync_watches( // Watch new directories. for dir in new_dirs.difference(current_dirs) { if let Err(e) = watcher.watch(dir, RecursiveMode::NonRecursive) { - eprintln!("warning: failed to watch {}: {e}", dir.display()); + eprint_warning(&format!( + "warning: failed to watch {}: {}", + safe_path(dir), + safe_inline(&e) + )); } else { result.insert(dir.clone()); } @@ -471,7 +476,10 @@ fn drain_debounce(rx: &mpsc::Receiver, debounce_ms: u64) -> (BTreeSet { - eprintln!("warning: watch error during debounce: {e}"); + eprint_warning(&format!( + "warning: watch error during debounce: {}", + safe_inline(&e) + )); } Ok(Msg::Interrupt) => return (paths, true), Err(mpsc::RecvTimeoutError::Timeout) => break, @@ -726,7 +734,7 @@ fn handle_fs_event_file( let interrupted = match msg { Msg::Interrupt => true, Msg::Fs(Err(e)) => { - eprintln!("warning: watch error: {e}"); + eprint_warning(&format!("warning: watch error: {}", safe_inline(&e))); // Non-fatal watch error — skip but don't rebuild. return FileEventAction::Skip; } @@ -786,7 +794,7 @@ fn rebuild_file( }) { Ok(v) => v, Err(e) => { - eprintln!("{e:?}"); + eprint_error(e); state.last_mtimes = snapshot_state(&state.foi); return; } @@ -860,12 +868,14 @@ fn rebuild_file( if !ctx.quiet { eprintln!( "Recompiled {} ({} deps) in {}ms", - out_display, dep_count, elapsed + safe_inline(&out_display), + dep_count, + elapsed ); } } Err(e) => { - eprintln!("{e:?}"); + eprint_error(e); // Error-settle: update snapshot so we don't re-fire. state.last_mtimes = snapshot_state(&state.foi); } @@ -873,7 +883,7 @@ fn rebuild_file( } } Err(e) => { - eprintln!("{e:?}"); + eprint_error(e); // Error-settle: snapshot current state so the tick gate // won't re-fire on the same unchanged files (AC-R7/W6). state.last_mtimes = snapshot_state(&state.foi); @@ -912,7 +922,7 @@ fn run_watch_file( set_string_vars: static_set_string_vars.clone(), })?; if !quiet { - eprintln!("Watching {}", entry.display()); + eprintln!("Watching {}", safe_path(&entry)); } // Load project config (for output_dir) — used if no explicit -o / --out-dir. @@ -933,7 +943,7 @@ fn run_watch_file( Ok(result) => result, Err(e) => { // Initial compile error: print and continue watching (entry dir still watched). - eprintln!("{e:?}"); + eprint_error(e); // Fall back: resolve output path with Markdown kind as a placeholder so we // know where to watch. This path may not match a later successful compile if // the template has @message blocks, but it will correct on first successful rebuild. @@ -1237,7 +1247,7 @@ fn compile_one_source( if !quiet { eprintln!( "Recompiled {} ({} deps) in {}ms", - out.display(), + safe_path(&out), dep_count, elapsed ); @@ -1276,7 +1286,7 @@ fn compile_one_source( ); } Err(e) => { - eprintln!("{e:?}"); + eprint_error(e); // Error-settle: update mtime so the gate won't re-fire. state.record_error(src); } @@ -1288,7 +1298,7 @@ fn compile_one_source( } } Err(e) => { - eprintln!("{e:?}"); + eprint_error(e); state.record_error(src); } } @@ -1456,7 +1466,7 @@ fn liveness_probe_dir( }) { Ok(v) => v, Err(e) => { - eprintln!("{e:?}"); + eprint_error(e); state.last_mtimes = snapshot_state(&state.known_set()); return; } @@ -1507,7 +1517,7 @@ fn handle_fs_event_dir( let interrupted = match msg { Msg::Interrupt => true, Msg::Fs(Err(e)) => { - eprintln!("warning: watch error: {e}"); + eprint_warning(&format!("warning: watch error: {}", safe_inline(&e))); return DirEventOutcome::Skip; } Msg::Fs(Ok(event)) => { @@ -1584,7 +1594,7 @@ fn handle_fs_event_dir( }) { Ok(v) => v, Err(e) => { - eprintln!("{e:?}"); + eprint_error(e); state.last_mtimes = snapshot_state(&state.known_set()); return DirEventOutcome::Done; } @@ -1646,7 +1656,7 @@ fn dir_watch_startup( }; if !quiet { - eprintln!("Watching directory {}", root.display()); + eprintln!("Watching directory {}", safe_path(&root)); } // Startup compile: compile all .mds files found under root. @@ -1699,14 +1709,14 @@ fn dir_watch_startup( let out = output_path_for(&key, &root, &output_base, ext); if let Err(e) = write_output(Some(out.clone()), &compiled.content, quiet, true) { - eprintln!("{e:?}"); + eprint_error(e); } else { state.last_written.insert(out, compiled.content); } } } Err(e) => { - eprintln!("{e:?}"); + eprint_error(e); state.forward_deps.insert(key.clone(), vec![]); state.errored.insert(key.clone()); state.known_files.insert(key); @@ -1745,10 +1755,11 @@ fn dir_watch_startup( // Watch external dep dirs NonRecursive (DD3). for ext_dir in &state.external_dep_dirs { if let Err(e) = watcher.watch(ext_dir, RecursiveMode::NonRecursive) { - eprintln!( - "warning: failed to watch external dep dir {}: {e}", - ext_dir.display() - ); + eprint_warning(&format!( + "warning: failed to watch external dep dir {}: {}", + safe_path(ext_dir), + safe_inline(&e) + )); } } @@ -1767,10 +1778,11 @@ fn dir_watch_startup( // a transient failure must not abort the session, applies ADR-021 / consistency fix). if let Some(ref vd) = vars_dir_extra { if let Err(e) = watcher.watch(vd, RecursiveMode::NonRecursive) { - eprintln!( - "warning: failed to watch vars directory {}: {e}", - vd.display() - ); + eprint_warning(&format!( + "warning: failed to watch vars directory {}: {}", + safe_path(vd), + safe_inline(&e) + )); } } @@ -1971,11 +1983,15 @@ fn process_dir_batch_vars_changed( match std::fs::remove_file(&out) { Ok(()) => { if !quiet { - eprintln!("Removed {} (source deleted)", out.display()); + eprintln!("Removed {} (source deleted)", safe_path(&out)); } } Err(e) => { - eprintln!("warning: could not remove {}: {e}", out.display()); + eprint_warning(&format!( + "warning: could not remove {}: {}", + safe_path(&out), + safe_inline(&e) + )); } } // Use the canonical forget() helper so ALL state maps are cleaned up uniformly @@ -2095,7 +2111,7 @@ fn process_dir_batch_incremental( state.errored.remove(src); } Err(e) => { - eprintln!("{e:?}"); + eprint_error(e); state.errored.insert(src.clone()); settle_mtime(src, &mut state.last_mtimes); } @@ -2117,11 +2133,15 @@ fn process_dir_batch_incremental( match std::fs::remove_file(&out) { Ok(()) => { if !quiet { - eprintln!("Removed {} (source deleted)", out.display()); + eprintln!("Removed {} (source deleted)", safe_path(&out)); } } Err(e) => { - eprintln!("warning: could not remove {}: {e}", out.display()); + eprint_warning(&format!( + "warning: could not remove {}: {}", + safe_path(&out), + safe_inline(&e) + )); } } } diff --git a/crates/mds-cli/tests/cli_build.rs b/crates/mds-cli/tests/cli_build.rs index cb8b6f2b..cec4c650 100644 --- a/crates/mds-cli/tests/cli_build.rs +++ b/crates/mds-cli/tests/cli_build.rs @@ -1088,7 +1088,7 @@ fn build_load_config_finds_grandparent_mds_json() { ); } -// ── ESC injection regression (issue #5 / ESC-INJECTION) ────────────────────── +// ── ESC injection regression (issue #176 / ESC-INJECTION) ───────────────────── /// Regression gate: `mds build` (single-file mode) must not emit raw ESC bytes to /// stderr when the source file embeds a raw ESC byte (U+001B) in content that reaches diff --git a/crates/mds-cli/tests/cli_commands.rs b/crates/mds-cli/tests/cli_commands.rs index d7a211ba..ec3d8095 100644 --- a/crates/mds-cli/tests/cli_commands.rs +++ b/crates/mds-cli/tests/cli_commands.rs @@ -609,7 +609,7 @@ fn cli_init_rejects_path_traversal() { ); } -// ── ESC injection regression — mds check (issue #5 / ESC-INJECTION) ────────── +// ── ESC injection regression — mds check (issue #176 / ESC-INJECTION) ──────── /// Regression gate: `mds check ` (single-file mode) must not emit raw /// ESC bytes to stderr when the source file contains a raw ESC byte that reaches diff --git a/crates/mds-cli/tests/cli_fmt.rs b/crates/mds-cli/tests/cli_fmt.rs index 93dc5b37..779f51a4 100644 --- a/crates/mds-cli/tests/cli_fmt.rs +++ b/crates/mds-cli/tests/cli_fmt.rs @@ -1149,7 +1149,7 @@ fn fmt_bare_filename_propagates_syntax_error() { assert_eq!(after, src, "broken file must be left untouched"); } -// ── ESC injection regression — mds fmt (issue #5 / ESC-INJECTION) ──────────── +// ── ESC injection regression — mds fmt (issue #176 / ESC-INJECTION) ────────── /// Regression gate: `mds fmt ` (single-file mode) must not emit raw ESC /// bytes to stderr when the source file contains a raw ESC byte that reaches diff --git a/crates/mds-cli/tests/cli_lint.rs b/crates/mds-cli/tests/cli_lint.rs index bd26507d..ca302a57 100644 --- a/crates/mds-cli/tests/cli_lint.rs +++ b/crates/mds-cli/tests/cli_lint.rs @@ -24,7 +24,7 @@ //! - I-26: shadow-variable Info severity emits diagnostic and exits 0 (Info never affects exit) mod common; -use common::{fixture, mds_bin}; +use common::{assert_no_control_chars, fixture, mds_bin}; use std::fs; use std::path::Path; @@ -1551,7 +1551,7 @@ fn lint_fix_bare_filename_applies_fix() { ); } -// ── ESC injection regression (issue #5 / ESC-INJECTION) ────────────────────── +// ── ESC injection regression (issue #176 / ESC-INJECTION) ───────────────────── /// Regression gate: a .mds file containing a raw ESC byte (U+001B) that reaches /// `MdsError::Syntax` must not emit raw ESC bytes to stderr — single-file mode. @@ -1585,6 +1585,445 @@ fn lint_esc_byte_in_syntax_error_is_sanitized_on_stderr() { ); } +// ── T-5..T-9: complete ESC-injection hardening for lint output (issue #176) ─── +// +// The vector used by T-5..T-8 is a .mds file whose template body line contains a +// raw ESC byte (U+001B) as part of a `legacy-interpolation` finding. The line +// "Hello \x1b{name} world" contains both the friendly word "Hello" (guard against +// vacuous passes) and the raw ESC byte; `legacy-interpolation` fires because `{name}` +// matches the old single-brace syntax. +// +// C1 note (T-8): raw 0x80–0x9F bytes are invalid UTF-8 so they are rejected by the +// MDS lexer before any lint rule runs. The C1 representative used here is U+0085 +// (NEL, UTF-8 0xC2 0x85), which is valid UTF-8 and is neutralized by +// `neutralize_source_for_render` before miette renders the source frame. +// +// Neutralization model (PF-014): control bytes in source are substituted at the INPUT +// boundary before miette renders, not post-processed over the rendered frame. +// - C0/DEL (1-byte) → '?' (1 byte, byte-length-preserving) +// - C1 (2-byte UTF-8 0x80–0x9F) → U+00A0 NBSP (2 bytes, byte-length-preserving) +// `sanitize_control_chars` (\\uXXXX expansion) is used only for message and help text. +// +// All tests use `NO_COLOR=1` so ANSI colour codes in miette output don't interfere +// with the raw-byte search. + +/// T-5 [AC-F1]: single-file mode — human-format lint on a file whose diagnostic +/// source line contains a raw ESC byte must produce no raw 0x1B byte on stderr. +#[test] +fn lint_single_file_esc_in_diagnostic_frame_is_sanitized() { + let dir = tempfile::tempdir().unwrap(); + let file = dir.path().join("esc_lint.mds"); + // "Hello \x1b{name} world" — legacy-interpolation fires on `{name}`; + // the raw ESC byte on the same line reaches the source frame. + fs::write(&file, b"Hello \x1b{name} world\n").unwrap(); + + let out = lint_path(&file, &[]); + + let stderr_str = String::from_utf8_lossy(&out.stderr); + // Exit 1 — legacy-interpolation is Warn severity. + assert_eq!( + out.status.code(), + Some(1), + "legacy-interpolation is warn-only; expected exit 1; stderr: {stderr_str}" + ); + // No raw control bytes on stderr (char-based check — catches ESC, DEL, and C1). + assert_no_control_chars(&stderr_str, "T-5 stderr"); + // Under input-neutralization (PF-014): ESC → '?' in the source frame. + // "Hello" must be visible, confirming source context is non-vacuous. + assert!( + stderr_str.contains("Hello"), + "source context word 'Hello' must be visible in output; got: {stderr_str:?}" + ); + // Stdout must be empty (human mode only writes to stderr). + assert!( + out.stdout.is_empty(), + "human mode must not write to stdout; got: {}", + String::from_utf8_lossy(&out.stdout) + ); +} + +/// T-6 [AC-F1]: directory mode — same vector via dir walk. +#[test] +fn lint_directory_esc_in_diagnostic_frame_is_sanitized() { + let dir = tempfile::tempdir().unwrap(); + fs::write(dir.path().join("esc_lint.mds"), b"Hello \x1b{name} world\n").unwrap(); + + let out = lint_path(dir.path(), &[]); + + let stderr_str = String::from_utf8_lossy(&out.stderr); + assert_eq!( + out.status.code(), + Some(1), + "directory mode: expected exit 1; stderr: {stderr_str}" + ); + // No raw control bytes on stderr (char-based check). + assert_no_control_chars(&stderr_str, "T-6 stderr"); + // Under input-neutralization (PF-014): ESC → '?' in the source frame. + assert!( + stderr_str.contains("Hello"), + "directory mode: 'Hello' must be visible in output; got: {stderr_str:?}" + ); + // Stdout must be empty (human mode). + assert!( + out.stdout.is_empty(), + "directory mode: stdout must be empty; got: {}", + String::from_utf8_lossy(&out.stdout) + ); +} + +/// T-7 [AC-F1]: stdin mode — same vector via the BrokenPipe-safe `lint_stdin` helper. +#[test] +fn lint_stdin_esc_in_diagnostic_frame_is_sanitized() { + let input = "Hello \x1b{name} world\n"; + let out = lint_stdin(input, &[]); + + let stderr_str = String::from_utf8_lossy(&out.stderr); + assert_eq!( + out.status.code(), + Some(1), + "stdin mode: expected exit 1; stderr: {stderr_str}" + ); + // No raw control bytes on stderr (char-based check). + assert_no_control_chars(&stderr_str, "T-7 stderr"); + // Under input-neutralization (PF-014): ESC → '?' in the source frame. + assert!( + stderr_str.contains("Hello"), + "stdin mode: 'Hello' must be visible in output; got: {stderr_str:?}" + ); +} + +/// T-8 [AC-F1]: DEL (U+007F) and C1 NEL (U+0085, UTF-8 0xC2 0x85) must both be +/// sanitized in the rendered source frame. +#[test] +fn lint_del_and_c1_in_diagnostic_frame_is_sanitized() { + let dir = tempfile::tempdir().unwrap(); + // U+007F = DEL; U+0085 = C1 NEL (valid UTF-8, 0xC2 0x85). + // `{name}` trips legacy-interpolation so we get a diagnostic and a source frame. + let content = "Hello\u{007F}and\u{0085}{name}world\n"; + fs::write(dir.path().join("del_c1_lint.mds"), content.as_bytes()).unwrap(); + + let out = lint_path(dir.path(), &[]); + + let stderr_str = String::from_utf8_lossy(&out.stderr); + assert_eq!( + out.status.code(), + Some(1), + "del/c1 test: expected exit 1; stderr: {stderr_str}" + ); + // DEL must be neutralized: 0x7F → '?' (1-byte → 1-byte, PF-014). + assert!( + !stderr_str.contains('\u{007F}'), + "raw DEL must not appear on stderr; got: {stderr_str:?}" + ); + // C1 NEL must be neutralized: U+0085 → U+00A0 NBSP (2-byte → 2-byte, PF-014). + assert!( + !stderr_str.contains('\u{0085}'), + "raw C1 NEL must not appear on stderr; got: {stderr_str:?}" + ); + assert!( + stderr_str.contains('\u{00A0}'), + "neutralized C1 NEL must appear as NBSP (U+00A0) in source frame; got: {stderr_str:?}" + ); + // Source context guard: "Hello" confirms output is non-vacuous. + assert!( + stderr_str.contains("Hello"), + "source context word 'Hello' must be visible; got: {stderr_str:?}" + ); +} + +/// T-9 [AC-C3]: `mds lint --format json` on a source whose `duplicate-import` +/// diagnostic message embeds a raw C1 control character (U+0085 NEL) must emit +/// valid JSON with no raw control bytes anywhere — in particular the embedded +/// path must be escaped to the 6-char literal `…`. +/// +/// ## Why this vector? +/// +/// The original T-9 used a YAML frontmatter key containing a raw ESC byte +/// (`"a\x1Bb": 1`). `serde_yaml_ng` rejects raw ESC/DEL bytes inside +/// double-quoted YAML keys, so the test never reached the lint code path at all +/// — the YAML parser rejected the input before any sanitizer ran, making every +/// assertion vacuous (PF-013 shape). +/// +/// U+0085 (NEL, C1 NEL) is a C1 control character whose UTF-8 encoding +/// (0xC2 0x85) **passes** `serde_yaml_ng` YAML parsing. We exploit a +/// different route: the `duplicate-import` rule fires when the same module is +/// imported twice and embeds the raw import path in its message. A module +/// whose file *name* contains U+0085 therefore injects that byte into the +/// diagnostic message. When `to_canonical_json` serializes the result, it +/// must sanitize U+0085 → `…` (6-char ASCII literal); if that +/// sanitization is removed the raw 0xC2 0x85 bytes appear in the JSON wire. +/// +/// ## Failure mode (regression guard) +/// +/// Removing the `sanitize_control_chars` call in `to_canonical_json` causes: +/// - Gate 1 still passes (the JSON is syntactically valid) +/// - Gate 2 FAILS: `assert_no_control_chars` finds U+0085 (a C1 char) in +/// the JSON wire output +/// - Gate 3 FAILS: the per-message check finds U+0085 in the diagnostic message +/// - The positive assertion FAILS: `…` is not present when raw bytes leak +#[test] +fn lint_json_hostile_source_output_contains_no_raw_control_bytes() { + let dir = tempfile::tempdir().unwrap(); + + // Helper module whose file name contains U+0085 (NEL, C1). This character + // survives serde_yaml_ng YAML parsing (unlike ESC/DEL which are rejected). + let nel_name = "fo\u{0085}o.mds"; + let nel_module = dir.path().join(nel_name); + fs::write(&nel_module, "hi\n").unwrap(); + + // main.mds imports the NEL-named module twice → duplicate-import fires. + // The diagnostic message embeds the raw import path (including U+0085). + let import_line = format!("@import \"./{nel_name}\"\n"); + let main_content = format!("{import_line}{import_line}"); + let main_file = dir.path().join("main.mds"); + fs::write(&main_file, main_content.as_bytes()).unwrap(); + + let out = lint_path(&main_file, &["--format", "json"]); + + let stdout_str = String::from_utf8_lossy(&out.stdout); + + // Gate 1 (fail-closed): stdout must be valid JSON with version 1. + let json: serde_json::Value = serde_json::from_str(&stdout_str).unwrap_or_else(|e| { + panic!( + "T-9: lint --format json must emit valid JSON; parse error: {e}; \ + stdout: {stdout_str:?}" + ) + }); + assert_eq!(json["version"], 1, "T-9: JSON version must be 1"); + + // Gate 2 (fail-closed, char-based): the entire JSON wire must contain no raw + // C0 (excl. \t \n), DEL, or C1 control codepoints. + // Uses `assert_no_control_chars` which iterates over Unicode codepoints, not + // raw bytes, to avoid false-positives on continuation bytes of non-C1 chars. + assert_no_control_chars(&stdout_str, "T-9 JSON wire output"); + + // Gate 3 (fail-closed): assert diagnostics ARE present and the rule fired. + let files = json["files"] + .as_array() + .unwrap_or_else(|| panic!("T-9: JSON must have 'files' array; got: {json}")); + assert!( + !files.is_empty(), + "T-9: files array must be non-empty (duplicate-import should fire); got: {json}" + ); + let all_diags: Vec<&serde_json::Value> = files + .iter() + .flat_map(|f| { + f["diagnostics"].as_array().unwrap_or_else(|| { + panic!( + "T-9: every file entry must have a 'diagnostics' array; \ + got: {f}" + ) + }) + }) + .collect(); + assert!( + !all_diags.is_empty(), + "T-9: must have at least one diagnostic; got: {files:?}" + ); + let has_dup_import = all_diags.iter().any(|d| d["rule"] == "duplicate-import"); + assert!( + has_dup_import, + "T-9: duplicate-import must be among the diagnostics; got rules: {:?}", + all_diags.iter().map(|d| &d["rule"]).collect::>() + ); + // Per-message sanitization check (char-based). + for diag in &all_diags { + let msg = diag["message"] + .as_str() + .unwrap_or_else(|| panic!("T-9: diagnostic must have a string 'message'; got: {diag}")); + assert_no_control_chars(msg, "T-9 diagnostic message"); + } + + // Positive assertion (non-vacuous, PF-013): the sanitized literal `…` + // must appear in at least one message. If sanitization is removed the raw + // U+0085 character leaks and this assertion fails because the 6-char literal + // is absent while the raw codepoint (caught by Gate 2/3) is present. + // + // After JSON deserialisation by serde_json the string value is `…` + // (6 chars: backslash, u, 0, 0, 8, 5). + let has_sanitized_nel = all_diags.iter().any(|d| { + d["message"] + .as_str() + .map(|m| m.contains("\\u0085")) + .unwrap_or(false) + }); + assert!( + has_sanitized_nel, + "T-9: sanitized \\u0085 literal must appear in at least one diagnostic message; \ + got messages: {:?}", + all_diags.iter().map(|d| &d["message"]).collect::>() + ); +} + +/// T-9b [AC-C3]: the Trojan Source vector (CVE-2021-42574) end-to-end through +/// `mds lint --format json`. +/// +/// U+202E RIGHT-TO-LEFT OVERRIDE is not a C0/DEL/C1 control character, so before +/// the escape class was widened it passed through the wire untouched. A single RLO +/// inside a filename makes every downstream renderer (terminal, IDE, code-review UI) +/// display the remainder of the line in reverse — a benign-looking diagnostic can be +/// made to read as something entirely different from its bytes. +/// +/// Reachability: the same `duplicate-import` route T-9 uses. The rule embeds the raw +/// import path in its message, and the path is a plain string (not YAML), so U+202E +/// reaches the lint engine intact. +/// +/// Non-vacuity (PF-013): asserts the `files` array is non-empty, that +/// `duplicate-import` actually fired, AND the POSITIVE assertion that the escaped +/// `\\u202E` literal is present — all three fail if the widened class is reverted. +#[test] +fn lint_json_bidi_override_in_import_path_is_escaped() { + let dir = tempfile::tempdir().unwrap(); + + // "fognp.mds" renders as "fopng.mds" in a bidi-aware terminal. + let rlo_name = "fo\u{202E}gnp.mds"; + fs::write(dir.path().join(rlo_name), "hi\n").unwrap(); + + let import_line = format!("@import \"./{rlo_name}\"\n"); + let main_file = dir.path().join("main.mds"); + fs::write(&main_file, format!("{import_line}{import_line}").as_bytes()).unwrap(); + + let out = lint_path(&main_file, &["--format", "json"]); + let stdout_str = String::from_utf8_lossy(&out.stdout); + + // Gate 1 (fail-closed): valid JSON, version 1. + let json: serde_json::Value = serde_json::from_str(&stdout_str).unwrap_or_else(|e| { + panic!("T-9b: lint --format json must emit valid JSON; parse error: {e}; stdout: {stdout_str:?}") + }); + assert_eq!(json["version"], 1, "T-9b: JSON version must be 1"); + + // Gate 2 (fail-closed): no raw hostile codepoint anywhere in the wire output. + assert_no_control_chars(&stdout_str, "T-9b JSON wire output"); + + // Gate 3 (non-vacuity): the rule actually fired. + let files = json["files"] + .as_array() + .unwrap_or_else(|| panic!("T-9b: JSON must have 'files' array; got: {json}")); + assert!( + !files.is_empty(), + "T-9b: files array must be non-empty (duplicate-import should fire); got: {json}" + ); + let all_diags: Vec<&serde_json::Value> = files + .iter() + .flat_map(|f| { + f["diagnostics"] + .as_array() + .unwrap_or_else(|| panic!("T-9b: every file entry needs 'diagnostics'; got: {f}")) + }) + .collect(); + assert!( + all_diags.iter().any(|d| d["rule"] == "duplicate-import"), + "T-9b: duplicate-import must be among the diagnostics; got rules: {:?}", + all_diags.iter().map(|d| &d["rule"]).collect::>() + ); + + // Positive assertion (PF-013): the escaped form must be present. + let has_escaped_rlo = all_diags + .iter() + .any(|d| d["message"].as_str().is_some_and(|m| m.contains("\\u202E"))); + assert!( + has_escaped_rlo, + "T-9b: escaped \\u202E literal must appear in at least one diagnostic message; \ + got messages: {:?}", + all_diags.iter().map(|d| &d["message"]).collect::>() + ); +} + +/// T-9c [AC-C3]: wire-mode newline escaping end-to-end — the log/YAML-key forging +/// vector. +/// +/// A raw newline inside a diagnostic message lets an attacker forge what reads as a +/// second, independent finding in any consumer that prints or line-splits the JSON +/// string value. On the wire that newline (U+000A) must arrive as its 6-char escape +/// literal. +/// +/// ## Why this vector? +/// +/// The obvious route — a POSIX filename containing a newline, imported twice — is +/// **not reachable**: the lexer rejects a newline inside an `@import "..."` path with +/// `syntax error: unclosed quote in path` before any lint rule runs, which would make +/// every assertion below vacuous (PF-013 shape). +/// +/// A YAML **double-quoted frontmatter key** does reach it: `serde_yaml_ng` decodes the +/// `\n` escape into a real newline, and `unused-variable` embeds the decoded key +/// verbatim in its message. The forged payload mimics a real diagnostic header, so a +/// naive line-oriented consumer would render it as a genuine second finding. +#[test] +fn lint_json_newline_in_frontmatter_key_is_escaped_on_the_wire() { + let dir = tempfile::tempdir().unwrap(); + let file = dir.path().join("forge.mds"); + + // The `\n` sequences below are YAML escapes inside a double-quoted key — after + // YAML parsing the key contains two REAL newline characters. + fs::write( + &file, + b"---\n\"a\\nerror[mds::forged]: FAKE\\nb\": 1\n---\nHello\n", + ) + .unwrap(); + + let out = lint_path(&file, &["--format", "json"]); + let stdout_str = String::from_utf8_lossy(&out.stdout); + + let json: serde_json::Value = serde_json::from_str(&stdout_str).unwrap_or_else(|e| { + panic!("T-9c: lint --format json must emit valid JSON; parse error: {e}; stdout: {stdout_str:?}") + }); + assert_eq!(json["version"], 1, "T-9c: JSON version must be 1"); + + let files = json["files"] + .as_array() + .unwrap_or_else(|| panic!("T-9c: JSON must have 'files' array; got: {json}")); + assert!( + !files.is_empty(), + "T-9c: files array must be non-empty; got: {json}" + ); + let all_diags: Vec<&serde_json::Value> = files + .iter() + .flat_map(|f| { + f["diagnostics"] + .as_array() + .unwrap_or_else(|| panic!("T-9c: every file entry needs 'diagnostics'; got: {f}")) + }) + .collect(); + + // Non-vacuity: the vector really did reach the lint engine. + assert!( + all_diags.iter().any(|d| d["rule"] == "unused-variable"), + "T-9c: unused-variable must fire; got rules: {:?}", + all_diags.iter().map(|d| &d["rule"]).collect::>() + ); + + // No parsed message may contain a raw newline … + for diag in &all_diags { + let msg = diag["message"].as_str().unwrap_or_else(|| { + panic!("T-9c: diagnostic must have a string 'message'; got: {diag}") + }); + assert!( + !msg.contains('\n'), + "T-9c: raw newline must not survive into a wire message; got: {msg:?}" + ); + } + + // … and the escaped form must be present (positive, non-vacuous). + assert!( + all_diags + .iter() + .any(|d| d["message"].as_str().is_some_and(|m| m.contains("\\u000A"))), + "T-9c: escaped \\u000A literal must appear in at least one diagnostic message; \ + got messages: {:?}", + all_diags.iter().map(|d| &d["message"]).collect::>() + ); + + // The forged payload text itself is preserved verbatim (only the newline changed), + // proving the guard escapes rather than strips. + assert!( + all_diags.iter().any(|d| d["message"] + .as_str() + .is_some_and(|m| m.contains("error[mds::forged]"))), + "T-9c: message body must be preserved verbatim; got messages: {:?}", + all_diags.iter().map(|d| &d["message"]).collect::>() + ); +} + // ── atomic_write_file: mode preservation and error-message coverage ────────── /// Regression gate: `mds lint --fix ` must preserve the original Unix file diff --git a/crates/mds-cli/tests/cli_watch.rs b/crates/mds-cli/tests/cli_watch.rs index fe1c43e1..5b32f4ed 100644 --- a/crates/mds-cli/tests/cli_watch.rs +++ b/crates/mds-cli/tests/cli_watch.rs @@ -3696,3 +3696,74 @@ fn watch_dir_skips_symlinked_source_file() { drop(child); } + +// ── ESC-injection: watch initial-compile-error stderr sanitization ──────────── + +/// T-Watch-ESC [AC-F-W1]: the `eprint_error` call at the non-loop +/// initial-compile-error path in `run_watch_file` (watch.rs line ~937) must +/// sanitize any raw control bytes before writing to stderr. +/// +/// Vector: a `.mds` file containing a raw ESC byte (U+001B) in an unclosed +/// `@define` — guaranteed syntax error — so the initial compile fails and +/// `eprint_error` is called before the watch loop begins. +/// +/// Determinism: the initial compile is a synchronous call that completes before +/// the watch loop starts. We wait a fixed bounded interval (500 ms, >> any +/// realistic compile time) then kill the process. No file-system events are +/// polled, so this test is immune to FSEvents/inotify timing flakiness. +/// +/// Assertions: +/// 1. stderr is non-empty (the error IS present — non-vacuous). +/// 2. No raw ESC byte (0x1B) appears anywhere in stderr. +#[test] +fn watch_esc_in_initial_compile_error_is_sanitized() { + let dir = tempfile::tempdir().unwrap(); + let src = dir.path().join("esc_watch.mds"); + // Unclosed @define with a raw ESC byte (0x1B) on the directive line so that + // miette renders it inside the source context frame. + std::fs::write(&src, b"@define \x1bfoo:\nhello\n").unwrap(); + + let mut child = ChildGuard( + mds_bin() + .args(["watch", src.to_str().unwrap(), "--debounce", "0"]) + .stdout(Stdio::null()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(), + ); + + // Move the stderr handle to a reader thread so the pipe never fills and + // so read_to_end completes once the process is killed. + let stderr_handle = child.0.stderr.take().expect("piped stderr"); + let reader = std::thread::spawn(move || { + let mut buf = Vec::new(); + let mut h = stderr_handle; + let _ = std::io::Read::read_to_end(&mut h, &mut buf); + buf + }); + + // Give the synchronous initial compile time to run and write its error. + // 500 ms >> typical compile time; no polling of file-system events. + std::thread::sleep(Duration::from_millis(500)); + + // Kill the watch process (ChildGuard.drop → kill + wait) to close the pipe. + drop(child); + + let stderr_bytes = reader.join().expect("stderr reader thread panicked"); + + // Assertion 1: error IS present (the initial-compile-error path ran). + assert!( + !stderr_bytes.is_empty(), + "watch initial-compile-error must emit something to stderr (non-vacuous)" + ); + + // Assertion 2: no raw ESC byte (0x1B) anywhere in stderr. + // eprint_error sanitizes via render_error_sanitized before writing; + // if sanitization is removed the raw 0x1B byte leaks here. + assert!( + !stderr_bytes.contains(&0x1Bu8), + "raw ESC byte (0x1B) must be sanitized in watch initial-compile-error stderr; \ + got (hex first 512): {:02x?}", + &stderr_bytes[..stderr_bytes.len().min(512)] + ); +} diff --git a/crates/mds-cli/tests/common/mod.rs b/crates/mds-cli/tests/common/mod.rs index 89553ade..706b0eed 100644 --- a/crates/mds-cli/tests/common/mod.rs +++ b/crates/mds-cli/tests/common/mod.rs @@ -8,7 +8,55 @@ pub fn fixture(name: &str) -> PathBuf { .join(name) } +/// Return a `Command` for the `mds` binary with `NO_COLOR=1` set so that +/// miette does not emit ANSI SGR codes. Tests that inspect raw stderr/stdout +/// bytes for control-character sanitization must not see miette's own escape +/// sequences, and suppressing colour globally is the safest way to ensure that. #[allow(dead_code)] pub fn mds_bin() -> std::process::Command { - std::process::Command::new(env!("CARGO_BIN_EXE_mds")) + let mut cmd = std::process::Command::new(env!("CARGO_BIN_EXE_mds")); + cmd.env("NO_COLOR", "1"); + cmd +} + +/// Assert that `s` contains no raw C0 (excluding `\t` and `\n`), DEL, C1, bidi +/// control, line/paragraph separator, or BOM codepoint. +/// +/// The predicate iterates over *chars* (Unicode codepoints), not raw bytes, +/// so it correctly identifies C1 characters encoded as two-byte UTF-8 +/// sequences (0xC2 0x80–0xC2 0x9F) without false-positives on continuation +/// bytes inside ordinary multi-byte codepoints. +/// +/// `\n` is permitted because this helper is used on HUMAN-mode output too, where +/// newlines are preserved by design. Wire-mode newline escaping is asserted +/// explicitly at the call sites that need it. +/// +/// # Panics +/// Panics on the first offending codepoint with a human-readable message that +/// includes `label`, the codepoint, its byte offset, and the full string. +#[allow(dead_code)] +pub fn assert_no_control_chars(s: &str, label: &str) { + for (byte_offset, ch) in s.char_indices() { + let code = ch as u32; + let is_c0 = code < 0x20 && code != 0x09 && code != 0x0a; + let is_del = code == 0x7f; + let is_c1 = (0x80..=0x9f).contains(&code); + // All twelve Unicode `Bidi_Control=Yes` codepoints (Trojan Source, + // CVE-2021-42574) — note U+061C, the only one outside U+200E–U+2069 — plus the + // JS line/paragraph separators and the invisible BOM. All escaped by the + // sanitizers. + let is_format_hazard = matches!(ch, + '\u{061C}' + | '\u{200E}' | '\u{200F}' + | '\u{2028}' | '\u{2029}' + | '\u{202A}'..='\u{202E}' + | '\u{2066}'..='\u{2069}' + | '\u{FEFF}' + ); + assert!( + !is_c0 && !is_del && !is_c1 && !is_format_hazard, + "{label}: raw hostile char U+{code:04X} at byte offset {byte_offset}; \ + full string: {s:?}" + ); + } } diff --git a/crates/mds-cli/tests/print_discipline.rs b/crates/mds-cli/tests/print_discipline.rs new file mode 100644 index 00000000..5f8c3c49 --- /dev/null +++ b/crates/mds-cli/tests/print_discipline.rs @@ -0,0 +1,1754 @@ +//! Print-discipline guard — a CI-enforced invariant over `crates/mds-cli/src/**` +//! (CWE-150 / CWE-117 / PF-004 / #176). +//! +//! # Why this exists +//! +//! Three review rounds of #176 each found a *different* bare `eprintln!` that +//! interpolated an untrusted value onto a terminal: the `*_collecting_warnings` sites, +//! then `lint.rs`'s unknown-`mds.json`-rule warning, then the walker's depth-limit +//! warning inside `output.rs` itself. Each round fixed its findings correctly and each +//! time the next reviewer found another one, because the property was only ever +//! asserted about the sites someone remembered to enumerate. That is the PF-004 failure +//! mode, and no amount of careful reading closes it — the search is unbounded. +//! +//! This test converts the unbounded search into a bounded, machine-checked invariant: +//! +//! > **Every value that reaches a terminal stream from `crates/mds-cli/src/**` — whether +//! > interpolated by a print macro or carried into a sanitizing print helper — is passed +//! > through one of the escape helpers, or appears in an explicit allowlist below with a +//! > written justification.** +//! +//! A new `eprintln!("… {}", path.display())` anywhere in the crate fails this test and +//! names the file, line, and offending expression. So does the same interpolation hoisted +//! into a local and handed to `eprint_warning`, and so does a value whose provenance the +//! scanner cannot establish at all. +//! +//! # Scope — what this guard does and does not cover +//! +//! **Covered:** +//! - `println!` / `eprintln!` / `print!` / `eprint!` in `crates/mds-cli/src/**`, for the +//! union of their inline captures (`{name}`) and their positional arguments. +//! - `write!` / `writeln!` whose first argument names a terminal stream — `std::io::stderr()`, +//! `io::stdout()`, or a local whose `let` initialiser names one. The crate contains no +//! such call today; the rule is here so that the first one cannot arrive unnoticed. +//! - **The argument of every [`SANITIZING_PRINT_HELPERS`] call** (`eprint_warning`). +//! `eprint_warning` applies HUMAN-mode escaping, which preserves `\n` by design so +//! multi-line frames render — so routing a hostile filename through it is **not** +//! sufficient on its own. That was M2: `lint.rs` already called `eprint_warning`, and an +//! `mds.json` rule name of `x\nClean: totally-real.mds\n` still forged three standalone +//! status lines. The governing rule (spec §7.5) is per FIELD: prose HUMAN, interpolated +//! identifiers/filenames/causes WIRE. This guard enforces the second half. +//! +//! The argument is accepted only in one of three shapes: a string literal, a +//! whole-expression sanitizer call, or a `format!` whose every interpolation is itself +//! accepted. A **bare local** is traced one hop through its `let` binding in the same +//! file and judged by the same rule — so hoisting the message out of the call +//! (`let msg = format!("… {name}"); eprint_warning(&msg);`) is checked exactly as if it +//! had been written inline. An expression shape not listed above, and a name with no +//! visible `let`, are **reported**, not assumed safe. +//! +//! Because `let` bindings are matched by name file-wide, a name that is *also* +//! introduced by a non-`let` binder would otherwise be judged by whatever unrelated +//! `let` of that name happens to exist elsewhere in the file. [`collect_non_let_binders`] +//! closes that: every `for`-loop variable, function parameter and closure parameter in +//! the file **poisons** its name, so such an argument is reported rather than resolved. +//! [`the_guard_refuses_to_resolve_a_non_let_binder`] is the proof. Pattern binders the +//! collector does not model — `if let` / `while let` / `match`-arm bindings — are +//! limit 5 under "Accepted limits". +//! +//! The guard fails closed: a false positive costs one allowlist entry with +//! a written justification, a false negative costs another review round. +//! +//! **Not covered, deliberately, and not claimed to be:** +//! - `miette::miette!(…)` report construction, and `MdsError` message bodies built in +//! `crates/mds-core/**`. Both are rendered by `eprint_error`, which escapes message, +//! help, and label text in HUMAN mode before miette sees them — so no raw control byte +//! reaches stderr from either path — but a `\n` in an interpolated path or identifier +//! survives inside the rendered frame. Frame content is indented and `│`-prefixed rather +//! than emitted as a bare status line, so it is a weaker surface than the ones above; it +//! is a known residual, not a closed one. Both halves of that residual are disclosed in +//! spec §7.5 and in the boundary table in `crates/mds-core/src/lint/diagnostic.rs`. +//! - `write!` / `writeln!` into an in-memory `String`, and stdout writes via +//! `crate::build::write_stdout`. Compiled template output is the command's *product* +//! and must stay byte-faithful. +//! - `crates/mds-core/**` warning *producers*. Core does not print except through +//! `emit_warnings`, which escapes in HUMAN mode; the identifiers its warning producers +//! interpolate are WIRE-escaped at construction instead. This guard is lexical and +//! cannot follow a value across a crate boundary, so that is a **precondition it +//! depends on and does not check**; [`ALLOWED_UNTRACED_HELPER_ARGS`] is where the +//! dependency is written down. +//! +//! `mds-core` has exactly three warning producers that interpolate a runtime value. +//! Their status differs and is worth stating exactly, because "upheld by tests" was +//! claimed here once when it was not true: +//! - `resolver.rs`'s imported-module filename (the source-map segment-cap warning) — +//! the only one whose input can actually carry a hostile character, since a module +//! key is a filesystem path. **Pinned by a test**: +//! `crates/mds-cli/tests/producer_discipline.rs`, which compiles a module named with +//! a real ESC byte and asserts the warning that reaches this crate is WIRE-escaped. +//! - `evaluator.rs`'s two `@include` alias warnings — **upheld by review only, and not +//! testable today.** The parser admits an `@include` alias only if it matches +//! `[A-Za-z_][A-Za-z0-9_]*` (`parser.rs`'s `is_valid_identifier` check), so no +//! hostile character can reach either site; the WIRE call there is defence in depth +//! against a future parser relaxation. A behavioural test of it would assert on an +//! input the parser rejects, i.e. it would be vacuous — the PF-013 failure mode — so +//! none is written. +//! +//! # Accepted limits +//! +//! This is a lexical scanner over Rust text, not a compiler. It is defeated by anyone +//! who sets out to defeat it, and stating the limits plainly is worth more than +//! implying they are closed: +//! +//! 1. **Sanitizers are matched by the last path segment of the callee.** `use +//! evil::passthrough as safe_path;`, or a locally-defined `fn safe_path` that returns +//! its input, both satisfy the check while escaping nothing. +//! 2. **Allowlist entries are anti-rot, not anti-reuse.** They are keyed by `(file, +//! expression)` with no macro or stream constraint, so [`every_allowlist_entry_is_live`] +//! catches an entry that stops matching, but a *new* variable that reuses an exempted +//! name in the same file (`compiled`, `ok_count`, `max_depth`) inherits the exemption +//! silently. The names were chosen to be specific for that reason. +//! 3. **The binding trace is one hop, within one file.** A local initialised from another +//! local is not followed; it is reported instead. Because bindings are matched by name +//! across the whole file rather than within the enclosing function, a name bound more +//! than once is accepted only if *every* binding of it is accepted. +//! 4. **Stream detection for `write!` is by name.** `let out = std::io::stderr()` is +//! followed, but a handle whose name and initialiser both avoid the words `stdout` and +//! `stderr` (passed in as a parameter, say) is not recognised as a terminal. +//! 5. **Only three non-`let` binder shapes poison a name.** [`collect_non_let_binders`] +//! models `for` variables, function parameters and closure parameters — the shapes a +//! hostile value plausibly arrives in. It does **not** model `if let` / `while let` / +//! `match`-arm bindings, so a name introduced by one of those and passed bare to +//! `eprint_warning` is still resolved against the file's `let`s. This is the narrowed +//! remnant of a wider hole: before limit 5 existed, *every* non-`let` binder was +//! resolved that way, and `for label in rules { eprint_warning(label) }` in `lint.rs` +//! passed the guard because the file's three unrelated `let label = safe_path(…)` +//! bindings were all safe. +//! +//! Every one of these requires writing code that looks wrong on purpose. The bar this +//! guard is built to meet is **accidental** reintroduction — the four times #176 was +//! reopened, it was an ordinary `eprintln!` or an ordinary hoisted `format!`, never an +//! alias. Closing the lexical gaps beyond that bar would need a rustc lint or a +//! `syn`-based analysis over expanded HIR, which is a different tool. +//! +//! # The escape helpers are not special-cased +//! +//! `eprint_error` and `eprint_warning` are the two functions that actually write to the +//! stream, and neither gets a blanket exemption. `eprint_error` passes with no allowlist +//! entry at all: its single interpolated argument *is* `render_error_sanitized(report)`. +//! `eprint_warning` passes via one narrow, written-out allowlist entry, because its +//! argument is HUMAN-escaped prose — and HUMAN mode is deliberately *not* in +//! [`SANITIZERS`], since it preserves `\n` and so cannot make an identifier safe. +//! +//! # PF-013 evidence +//! +//! - **Positive:** [`the_guard_flags_a_bare_interpolating_print`] proves the scanner +//! reports the exact expression from a synthetic violation; +//! [`the_guard_follows_a_hoisted_format_binding`] proves the same for a message hoisted +//! into a local, [`the_guard_reports_an_untraceable_helper_argument`] for one it +//! cannot resolve at all, and [`the_guard_refuses_to_resolve_a_non_let_binder`] for one +//! whose name is shadowed by a `for` / parameter / closure binder. +//! - **Negative:** [`cli_print_sites_sanitize_every_interpolated_value`] proves the real +//! sources are clean. +//! - **Non-vacuity:** the same test asserts the scanner actually found the crate's +//! modules, its print sites, its interpolations, its `let` bindings, the non-`let` +//! binders that poison a name, and its calls into the sanitizing print helpers, so it +//! cannot pass because the parser silently returned nothing. +//! - **Allowlist rot:** [`every_allowlist_entry_is_live`] fails if an entry in either +//! allowlist stops matching anything, so exemptions cannot outlive the code that +//! needed them. + +use std::path::{Path, PathBuf}; + +// ── Configuration ───────────────────────────────────────────────────────────── + +/// Macros that write directly to a terminal stream. +const PRINT_MACROS: &[&str] = &["eprintln!", "println!", "eprint!", "print!"]; + +/// Macros that write to whatever sink they are handed. Scanned only when that sink is a +/// terminal stream (see `is_stream_target`) — a `write!` into an in-memory `String` is +/// not a print, and compiled output written to stdout is the command's product. +const STREAM_WRITE_MACROS: &[&str] = &["writeln!", "write!"]; + +/// Functions that escape their whole argument in HUMAN mode and write it to a stream as +/// one warning. +/// +/// HUMAN mode preserves `\n`, so the helper makes *prose* safe and does nothing for an +/// identifier interpolated into that prose. Every argument handed to one of these is +/// therefore classified in its own right (`classify_helper_arg`), including through one +/// hop of `let`-binding — see the module doc. +const SANITIZING_PRINT_HELPERS: &[&str] = &["eprint_warning"]; + +/// Functions whose return value is escape-safe by construction. An interpolated +/// expression is accepted when it *is* a call to one of these (with any module path +/// prefix, and through any number of leading `&`). +/// +/// **WIRE only.** HUMAN-mode `mds::sanitize_control_chars` is deliberately absent: it +/// preserves `\n`, so it does not make an interpolated identifier safe (that was M2). +/// The one place HUMAN mode is correct for a whole line — `eprint_warning`'s own body, +/// which escapes *prose* — is an explicit allowlist entry below, so the exception is +/// visible instead of blanket. +const SANITIZERS: &[&str] = &[ + // crates/mds-cli/src/output.rs — WIRE, for single-line values. + "safe_path", + "safe_file_display", + "safe_inline", + // crates/mds-core — the WIRE escape entry point. + "sanitize_control_chars_wire", + // crates/mds-cli/src/output.rs — renders a report whose inputs were escaped first. + "render_error_sanitized", +]; + +/// Values that are printed unescaped **on purpose**, keyed by `(file, expression)`. +/// +/// Every entry carries the reason it is safe. An entry without a justification, or one +/// that stops matching (see [`every_allowlist_entry_is_live`]), is the same defect this +/// guard exists to prevent, wearing a different hat. +/// +/// Deliberately keyed by *expression*, not by line number, so the list does not rot as +/// code moves — and so an entry cannot silently start covering a different print. +const ALLOWED_UNSANITIZED: &[(&str, &str, &str)] = &[ + // ── The one place HUMAN mode is the correct mode ───────────────────────── + ( + "output.rs", + "mds::sanitize_control_chars(w)", + "`eprint_warning`'s own body. HUMAN mode is correct here and only here: the \ + argument is warning PROSE, which is legitimately multi-line, and escaping its \ + newlines would break multi-line warning bodies. Values interpolated INTO that \ + prose are WIRE-escaped by the caller — which this guard checks separately, by \ + scanning the `format!`s nested inside `eprint_warning` calls.", + ), + // ── The compiled artefact itself ───────────────────────────────────────── + ( + "build.rs", + "compiled", + "`print!(\"{compiled}\")` writes the compiled template to STDOUT. This is the \ + command's product, not a diagnostic: `mds build -o - > out.md` must reproduce \ + the artefact byte for byte, so escaping it would corrupt every redirect. \ + Terminal-hazard bytes here originate in the user's own template and are the \ + same bytes `mds build -o file.md` would write to disk.", + ), + // ── `&'static str` labels — no runtime data reaches these ──────────────── + ( + "build.rs", + "kind_label(kind)", + "Returns one of exactly two `&'static str` literals (`build.rs::kind_label`); \ + it is a compile-time label for an `OutputKind`, not user data.", + ), + // ── Integer counters — a `usize`/`u128` cannot carry a control byte ─────── + ( + "build.rs", + "walk.excluded_by_default", + "`usize` count of `.mds` files the default-exclusion walker skipped \ + (hidden dirs, node_modules); produced by `collect_mds_files_detailed`.", + ), + ( + "build.rs", + "ok_count", + "`usize` tally of successful compilations in `mds build ` summary output.", + ), + ( + "build.rs", + "fail_count", + "`usize` tally of failed compilations in `mds build ` summary output.", + ), + ( + "fmt.rs", + "walk.excluded_by_default", + "`usize` count of `.mds` files the default-exclusion walker skipped \ + (hidden dirs, node_modules); produced by `collect_mds_files_detailed`.", + ), + ( + "fmt.rs", + "changed_count", + "`usize` tally of reformatted files in the `mds fmt ` summary line.", + ), + ( + "fmt.rs", + "unchanged_count", + "`usize` tally of already-formatted files in the `mds fmt ` summary line.", + ), + ( + "fmt.rs", + "fail_count", + "`usize` tally of files `mds fmt ` could not process, in its summary line.", + ), + ( + "lint.rs", + "walk.excluded_by_default", + "`usize` count of `.mds` files the default-exclusion walker skipped \ + (hidden dirs, node_modules); produced by `collect_mds_files_detailed`.", + ), + ( + "lint.rs", + "applied_count", + "`usize` tally of lint fixes actually applied, in the `Partially fixed:` line.", + ), + ( + "lint.rs", + "total_count", + "`usize` tally of lint fixes planned, in the `Partially fixed:` line.", + ), + ( + "lint.rs", + "mds::MAX_DIAGNOSTICS", + "`usize` compile-time constant `mds::MAX_DIAGNOSTICS` (the per-file diagnostic cap).", + ), + ( + "main.rs", + "walk.excluded_by_default", + "`usize` count of `.mds` files the default-exclusion walker skipped \ + (hidden dirs, node_modules); produced by `collect_mds_files_detailed`.", + ), + ( + "main.rs", + "ok_count", + "`usize` tally of files that passed `mds check `, in its summary line.", + ), + ( + "main.rs", + "fail_count", + "`usize` tally of files that failed `mds check `, in its summary line.", + ), + ( + "output.rs", + "max_depth", + "`usize` recursion bound; every caller passes the compile-time `MAX_DEPTH` constant.", + ), + ( + "watch.rs", + "dep_count", + "`usize` count of a compiled template's dependencies, in the `Recompiled` line.", + ), + ( + "watch.rs", + "elapsed", + "`u128` elapsed milliseconds from `Instant::elapsed().as_millis()` — pure arithmetic.", + ), +]; + +/// Arguments to a [`SANITIZING_PRINT_HELPERS`] call that the one-hop binding trace cannot +/// resolve, and that are accepted anyway, keyed by `(file, expression)`. +/// +/// Kept separate from [`ALLOWED_UNSANITIZED`] on purpose. An entry here exempts a value +/// **only** in the argument position of a sanitizing print helper; the same name appearing +/// in an `eprintln!` in the same file is still a violation. That is a narrower exemption +/// than the general allowlist grants, which matters because the names in this position are +/// short loop variables. +/// +/// Every entry here is a dependency on discipline the guard cannot check. Say so. +const ALLOWED_UNTRACED_HELPER_ARGS: &[(&str, &str, &str)] = &[ + ( + "build.rs", + "w", + "`for w in &result.warnings` — `w` is a whole warning string produced by \ + `mds-core`, not a value this crate interpolates. HUMAN mode is the correct mode \ + for it: it is prose, legitimately multi-line. What makes it safe is that \ + `mds-core`'s three untrusted-value warning producers WIRE-escape at construction: \ + `resolver.rs`'s imported-module filename, and `evaluator.rs`'s two `@include` \ + alias warnings — see the boundary table in \ + `crates/mds-core/src/lint/diagnostic.rs`. That is PRODUCER DISCIPLINE, which this \ + lexical guard cannot follow across a crate boundary to confirm. It is upheld by \ + review, plus one test on the only producer whose input can carry a hostile \ + character: `producer_discipline.rs` in this crate. The two alias sites are upheld \ + by review alone — the parser restricts an alias to `[A-Za-z_][A-Za-z0-9_]*`, so \ + testing them would be vacuous (PF-013). See this file's module doc.", + ), + ( + "main.rs", + "w", + "`for w in &warnings` on the `mds check` file, stdin and directory paths — same \ + value and same reasoning as the `build.rs` entry above: a whole `mds-core` \ + warning string, prose, HUMAN by design, safe because mds-core WIRE-escapes the \ + identifiers it interpolates at construction rather than because this lexical \ + guard checks it. Two entries — this one and `build.rs`'s — cover all five live \ + bare-`w` sites, because the list is keyed by (file, expression).", + ), +]; + +// ── The guard ───────────────────────────────────────────────────────────────── + +#[test] +fn cli_print_sites_sanitize_every_interpolated_value() { + let src_dir = Path::new(env!("CARGO_MANIFEST_DIR")).join("src"); + let files = rust_files(&src_dir); + + // Non-vacuity #1: the crate's modules were actually found and read. + assert!( + files.len() >= 6, + "non-vacuity: expected at least the 6 mds-cli modules under {}, found {}", + src_dir.display(), + files.len() + ); + + let mut violations: Vec = Vec::new(); + let mut total_sites = 0usize; + let mut total_exprs = 0usize; + let mut total_bindings = 0usize; + let mut total_non_let = 0usize; + let mut total_helper_calls = 0usize; + + for file in &files { + let name = file_key(file); + let src = std::fs::read_to_string(file).expect("mds-cli source must be readable"); + let masked = mask_comments(&src); + total_bindings += collect_let_bindings(&masked).len(); + total_non_let += collect_non_let_binders(&masked).len(); + total_helper_calls += find_invocations(&masked, SANITIZING_PRINT_HELPERS).len(); + for site in collect_sites(&src) { + total_sites += 1; + for expr in &site.exprs { + total_exprs += 1; + if is_sanitizer_call(expr) { + continue; + } + if justification(&name, expr, &site.kind).is_some() { + continue; + } + violations.push(format!( + " {}:{}: {} interpolates unsanitized `{}`", + name, site.line, site.kind, expr + )); + } + } + } + + // Non-vacuity #2–#5: the scanner really parsed print sites, their interpolations, the + // `let` bindings the trace depends on, and the helper calls it classifies. Without + // these, a broken parser would make this test pass by finding nothing. + assert!( + total_sites >= 80, + "non-vacuity: expected at least 80 print sites across mds-cli/src, found {total_sites}" + ); + assert!( + total_exprs >= 60, + "non-vacuity: expected at least 60 interpolated expressions, found {total_exprs}" + ); + assert!( + total_bindings >= 100, + "non-vacuity: the binding trace is only as good as the bindings it finds; \ + expected at least 100 `let` bindings across mds-cli/src, found {total_bindings}" + ); + assert!( + total_non_let >= 50, + "non-vacuity: the poison set is only as good as the binders it finds; expected at \ + least 50 non-`let` binders (for-loop vars, fn params, closure params) across \ + mds-cli/src, found {total_non_let}" + ); + assert!( + total_helper_calls >= 10, + "non-vacuity: expected at least 10 calls into {SANITIZING_PRINT_HELPERS:?}, \ + found {total_helper_calls}" + ); + + assert!( + violations.is_empty(), + "print-discipline violation: {} interpolation(s) reach a terminal stream unescaped.\n\ + \n{}\n\n\ + Fix by wrapping the value in one of {:?} (see crates/mds-cli/src/output.rs), or — \ + if the value genuinely must not be escaped — add it to ALLOWED_UNSANITIZED (or, \ + for an argument the helper trace cannot resolve, ALLOWED_UNTRACED_HELPER_ARGS) in \ + this file with a written justification.", + violations.len(), + violations.join("\n"), + SANITIZERS + ); +} + +/// An allowlist entry that no longer matches anything is dead weight that quietly widens +/// the exemption surface for whatever gets written next. Fail on it. +#[test] +fn every_allowlist_entry_is_live() { + let src_dir = Path::new(env!("CARGO_MANIFEST_DIR")).join("src"); + // `(file, expr, is_untraced_helper_arg)` for every interpolation the scanner saw. + let mut seen: Vec<(String, String, bool)> = Vec::new(); + for file in rust_files(&src_dir) { + let name = file_key(&file); + let src = std::fs::read_to_string(&file).expect("mds-cli source must be readable"); + for site in collect_sites(&src) { + let untraced = site.kind.ends_with("(untraced)"); + for expr in site.exprs { + seen.push((name.clone(), expr, untraced)); + } + } + } + + for (list_name, list, want_untraced) in [ + ("ALLOWED_UNSANITIZED", ALLOWED_UNSANITIZED, false), + ( + "ALLOWED_UNTRACED_HELPER_ARGS", + ALLOWED_UNTRACED_HELPER_ARGS, + true, + ), + ] { + let dead: Vec<&str> = list + .iter() + .filter(|(file, expr, _)| { + !seen + .iter() + .any(|(f, e, u)| f == file && e == expr && *u == want_untraced) + }) + .map(|(_, expr, _)| *expr) + .collect(); + + assert!( + dead.is_empty(), + "these {list_name} entries no longer match any site in the position they \ + exempt, and must be deleted: {dead:?}" + ); + + // Every entry must carry a non-trivial justification. + for (file, expr, why) in list { + assert!( + why.len() >= 40, + "{list_name} entry {file}/{expr} needs a real justification, got {why:?}" + ); + } + } +} + +// ── Scanner self-tests (PF-013 positive / negative / robustness) ────────────── + +#[test] +fn the_guard_flags_a_bare_interpolating_print() { + let src = r#" + fn f(path: &std::path::Path, e: std::io::Error) { + eprintln!("warning: could not remove {}: {e}", path.display()); + } + "#; + let exprs = only_site(src); + // Positive: BOTH the inline capture and the positional argument are reported. + assert!( + exprs.contains(&"e".to_string()), + "the inline `{{e}}` capture must be reported; got {exprs:?}" + ); + assert!( + exprs.contains(&"path.display()".to_string()), + "the positional `path.display()` argument must be reported; got {exprs:?}" + ); + for expr in &exprs { + assert!( + !is_sanitizer_call(expr), + "`{expr}` must not be mistaken for a sanitizer call" + ); + } +} + +#[test] +fn the_guard_accepts_sanitized_and_literal_prints() { + // A sanitized interpolation, through a module path and a leading `&`. + let exprs = only_site(r#"fn f() { eprintln!("Clean: {}", crate::output::safe_path(p)); }"#); + assert_eq!(exprs, vec!["crate::output::safe_path(p)".to_string()]); + assert!(is_sanitizer_call(&exprs[0])); + + // A literal-only print interpolates nothing and is safe by construction. + assert_eq!( + only_site(r#"fn f() { eprintln!("Stopped watching."); }"#), + Vec::::new() + ); + + // `{{` is an escaped brace, not a placeholder. + assert_eq!( + only_site(r#"fn f() { println!("use {{x}} to interpolate"); }"#), + Vec::::new() + ); + + // A sanitizer nested inside another call is NOT accepted — the outer call could + // undo the escape. + assert!(!is_sanitizer_call("wrap(safe_path(p))")); + assert!(!is_sanitizer_call("format!(\"{}\", safe_path(p))")); + // A method named like a sanitizer on some other receiver is not accepted either. + assert!(!is_sanitizer_call("thing.safe_path()")); + + // …and neither is anything that CONTINUES after the sanitizer call. The escape is + // only worth something if it is the last thing that happens to the value, so the + // suffix direction must be rejected exactly like the prefix direction above. + assert!( + !is_sanitizer_call("safe_path(p) + &evil"), + "concatenating onto a sanitized value must not be accepted" + ); + assert!( + !is_sanitizer_call("safe_path(p).replace(\"a\", &evil)"), + "a postfix method on a sanitized value must not be accepted" + ); + assert!( + !is_sanitizer_call("safe_path(p).to_string() + evil"), + "a postfix method plus concatenation must not be accepted" + ); + // A trailing `?`, `.as_str()` or index is the same hazard shape. + assert!(!is_sanitizer_call("safe_inline(x)[1..]")); + // The bare call, with and without a leading `&`, is still accepted — the tightened + // check must not have closed the legitimate form. + assert!(is_sanitizer_call("safe_path(p)")); + assert!(is_sanitizer_call("&crate::output::safe_inline(&e)")); + // A `)` inside a string argument must not be mistaken for the closing paren. + assert!(is_sanitizer_call("safe_inline(\"a)b\")")); +} + +#[test] +fn the_guard_follows_a_hoisted_format_binding() { + // B1: hoisting the message into a local is completely idiomatic, and it used to make + // the whole interpolation invisible — `collect_sites` only looked for `format!` + // lexically INSIDE the `eprint_warning(…)` parens. This is M2 reintroduced verbatim. + let src = r#" + fn f(name: &str) { + let msg = format!("warning: unknown lint rule '{name}'; ignoring"); + eprint_warning(&msg); + } + "#; + let sites = collect_sites(src); + assert_eq!(sites.len(), 1, "the hoisted format! must still be a site"); + assert_eq!(sites[0].exprs, vec!["name".to_string()]); + assert!( + sites[0].kind.contains("let msg"), + "the report must name the binding it traced; got {:?}", + sites[0].kind + ); + + // …and it passes once the identifier is WIRE-escaped, exactly as the inline form does. + let fixed = r#" + fn f(name: &str) { + let msg = format!("warning: unknown lint rule '{}'", safe_inline(name)); + eprint_warning(&msg); + } + "#; + let sites = collect_sites(fixed); + assert_eq!(sites[0].exprs, vec!["safe_inline(name)".to_string()]); + assert!(is_sanitizer_call(&sites[0].exprs[0])); + + // A binding that is itself a whole sanitizer call needs no further checking. + assert!( + collect_sites(r#"fn f(p: &Path) { let m = safe_path(p); eprint_warning(&m); }"#).is_empty(), + "a binding that IS a sanitizer call must be accepted outright" + ); +} + +#[test] +fn the_guard_reports_an_untraceable_helper_argument() { + // B2: `eprint_warning()` used to produce zero sites, so the argument + // was trusted without anything checking it. It must fail closed instead. + let src = r#" + fn f(warnings: &[String]) { + for w in warnings { + eprint_warning(w); + } + } + "#; + let sites = collect_sites(src); + assert_eq!(sites.len(), 1, "the loop variable must be reported"); + assert_eq!(sites[0].exprs, vec!["w".to_string()]); + assert!( + sites[0].kind.ends_with("(untraced)"), + "an unresolved argument must be reported as untraced so it is judged against \ + ALLOWED_UNTRACED_HELPER_ARGS, not the general allowlist; got {:?}", + sites[0].kind + ); + + // A binding the trace CAN reach but does not recognise is reported too — the trace + // never falls back to trusting the value. + let opaque = r#" + fn f(name: &str) { + let msg = mk_msg(name); + eprint_warning(&msg); + } + "#; + let sites = collect_sites(opaque); + assert_eq!(sites.len(), 1); + assert!(sites[0].kind.ends_with("(untraced)")); + + // Two bindings of one name, only one of them safe: the unsafe one poisons the trace. + let mixed = r#" + fn a(p: &Path) { let m = safe_path(p); eprint_warning(&m); } + fn b(p: &Path) { let m = mk_msg(p); } + "#; + let sites = collect_sites(mixed); + assert_eq!(sites.len(), 1); + assert!( + sites[0].kind.ends_with("(untraced)"), + "a name bound unsafely anywhere in the file must not be accepted; got {:?}", + sites[0].kind + ); + + // A literal argument is safe by construction and produces no site at all. + assert!(collect_sites(r#"fn f() { eprint_warning("done."); }"#).is_empty()); + + // The helper's own DEFINITION is not one of its call sites. + assert!( + collect_sites(r#"pub(crate) fn eprint_warning(w: &str) { let _ = w; }"#).is_empty(), + "`fn eprint_warning(w: &str)` is a definition, not a call" + ); +} + +#[test] +fn the_guard_refuses_to_resolve_a_non_let_binder() { + // The bypass this closes: `let` bindings are matched file-wide, so a name introduced + // by a `for` variable / parameter / closure param used to be judged by whatever + // unrelated `let`s of that name the file contained — and accepted if all of them were + // safe. This is the exact construct, against the exact shape `lint.rs` carries + // (`let label = safe_path(…)`, three times). Before `collect_non_let_binders` it + // produced ZERO sites. + let for_var = r#" + fn render(p: &Path, source: &str, fixed: &str) -> String { + let label = safe_path(p); + render_unified_diff(source, fixed, &label) + } + fn atk_v12(rules: &[String]) { + for label in rules { + eprint_warning(label); + } + } + "#; + let sites = collect_sites(for_var); + assert_eq!( + sites.len(), + 1, + "the `for label in rules` binder must be reported even though every `let label` \ + in the file is safe; got {sites:?}" + ); + assert_eq!(sites[0].exprs, vec!["label".to_string()]); + assert!( + sites[0].kind.ends_with("(untraced)"), + "it must land in the untraced position so ALLOWED_UNTRACED_HELPER_ARGS is what \ + exempts it, not the general allowlist; got {:?}", + sites[0].kind + ); + + // Same hole through a function parameter and through a closure parameter. + for src in [ + r#" + fn render(p: &Path) -> String { let note = safe_path(p); wrap(note) } + fn atk(note: &str) { eprint_warning(note); } + "#, + r#" + fn render(p: &Path) -> String { let note = safe_path(p); wrap(note) } + fn atk(v: &[String]) { v.iter().for_each(|note| eprint_warning(note)); } + "#, + ] { + let sites = collect_sites(src); + assert_eq!( + sites.len(), + 1, + "a parameter / closure param must not be resolved through an unrelated \ + `let` of the same name; got {sites:?}" + ); + assert!(sites[0].kind.ends_with("(untraced)")); + } + + // The collector must find each shape it claims to model. + let binders = collect_non_let_binders( + "fn f(alpha: &str, beta: usize) { for gamma in xs { xs.map(|delta| delta); } }", + ); + for want in ["alpha", "beta", "gamma", "delta"] { + assert!( + binders.iter().any(|b| b == want), + "`{want}` must be collected as a non-`let` binder; got {binders:?}" + ); + } + + // …and must not read a bitwise / logical `|` as a closure, which would poison the + // names of arbitrary operands and turn the guard into noise. + let bitwise = collect_non_let_binders("fn f() { let m = flag_a | flag_b; let n = x || y; }"); + assert!( + !bitwise + .iter() + .any(|b| b == "flag_a" || b == "flag_b" || b == "x" || b == "y"), + "an operand of `|` / `||` is not a closure parameter; got {bitwise:?}" + ); + + // A name that is ONLY `let`-bound is still resolved — the fix must not have made the + // trace useless. + assert!( + collect_sites( + r#"fn f(p: &Path) { let only_let = safe_path(p); eprint_warning(&only_let); }"# + ) + .is_empty(), + "a purely `let`-bound safe local must still be accepted" + ); +} + +#[test] +fn the_guard_scans_writes_to_a_stream_but_not_to_a_buffer() { + // B4: `writeln!(std::io::stderr(), …)` reaches a terminal exactly like `eprintln!`. + // There are none in the crate today; this pins the rule before the first one lands. + let sites = collect_sites( + r#"fn f(p: &Path) { writeln!(std::io::stderr(), "warning: {}", p.display()); }"#, + ); + assert_eq!(sites.len(), 1, "a write to stderr must be a print site"); + assert_eq!(sites[0].exprs, vec!["p.display()".to_string()]); + + // A handle bound to a local is followed through its `let`. + let via_local = r#" + fn f(p: &Path) { + let out = std::io::stdout(); + writeln!(out, "Clean: {}", p.display()); + } + "#; + assert_eq!( + collect_sites(via_local)[0].exprs, + vec!["p.display()".to_string()] + ); + + // A write into an in-memory buffer is NOT a print — compiled output and assembled + // strings must stay byte-faithful, and scanning them would be a false positive. + let to_buffer = r#" + fn f(p: &Path) { + let mut buf = String::new(); + write!(buf, "{}", p.display()); + } + "#; + assert!( + collect_sites(to_buffer).is_empty(), + "a write into a String buffer must not be scanned; got {:?}", + collect_sites(to_buffer) + ); + + // A sanitized write to a stream passes, so the rule is satisfiable. + assert!( + collect_sites(r#"fn f(p: &Path) { writeln!(std::io::stderr(), "{}", safe_path(p)); }"#)[0] + .exprs + .iter() + .all(|e| is_sanitizer_call(e)), + "a WIRE-escaped write to stderr must pass" + ); +} + +#[test] +fn the_guard_covers_format_inside_eprint_warning() { + // HUMAN-mode `eprint_warning` does not make an interpolated identifier safe (M2). + let src = r#" + fn f(name: &str) { + eprint_warning(&format!("warning: unknown lint rule '{name}'")); + } + "#; + let sites = collect_sites(src); + assert_eq!( + sites.len(), + 1, + "the format! inside eprint_warning must be a site" + ); + assert_eq!(sites[0].exprs, vec!["name".to_string()]); + assert!(sites[0].kind.contains("eprint_warning")); + + // …and it passes once the identifier is WIRE-escaped. + let fixed = r#" + fn f(name: &str) { + eprint_warning(&format!( + "warning: unknown lint rule '{}'", + safe_inline(name) + )); + } + "#; + let sites = collect_sites(fixed); + assert_eq!(sites[0].exprs, vec!["safe_inline(name)".to_string()]); + assert!(is_sanitizer_call(&sites[0].exprs[0])); +} + +#[test] +fn the_guard_ignores_comments_and_string_literals() { + // A print macro named inside a comment or a string must not be scanned — otherwise + // the rustdoc that *documents* this rule would trip it. + let src = r#" + /// Never write `eprintln!("{}", path.display())` — use safe_path. + // eprintln!("{}", dir.display()); + fn f() { + let s = "eprintln!(\"{}\", nope.display())"; + /* block: eprintln!("{}", also_nope.display()); */ + let _ = s; + } + "#; + assert!( + collect_sites(src).is_empty(), + "comments and string literals must not be scanned as code; got {:?}", + collect_sites(src) + ); +} + +#[test] +fn the_guard_rejects_a_dynamic_format_string() { + // If the first argument is not a string literal we cannot see the placeholders, so + // the whole invocation is reported rather than skipped. Fail safe, not open. + let exprs = only_site(r#"fn f() { eprintln!(FMT, y); }"#); + assert_eq!(exprs, vec!["FMT, y".to_string()]); + assert!(!is_sanitizer_call(&exprs[0])); +} + +// ── Implementation ──────────────────────────────────────────────────────────── + +/// One print-like invocation and the expressions it interpolates. +#[derive(Debug)] +struct Site { + line: usize, + /// `eprintln!`, `print!`, or `eprint_warning(format!)`. + kind: String, + exprs: Vec, +} + +fn only_site(src: &str) -> Vec { + let sites = collect_sites(src); + assert_eq!( + sites.len(), + 1, + "expected exactly one print site in the fixture" + ); + sites.into_iter().next().expect("checked above").exprs +} + +fn file_key(path: &Path) -> String { + path.file_name() + .and_then(|n| n.to_str()) + .unwrap_or_default() + .to_string() +} + +fn rust_files(dir: &Path) -> Vec { + let mut out = Vec::new(); + let mut stack = vec![dir.to_path_buf()]; + // Bounded: the source tree is finite and acyclic (no symlinks are followed because + // `read_dir` entries are checked with `file_type`, which does not traverse). + while let Some(d) = stack.pop() { + let Ok(rd) = std::fs::read_dir(&d) else { + continue; + }; + for entry in rd.flatten() { + let p = entry.path(); + let Ok(ft) = entry.file_type() else { continue }; + if ft.is_dir() { + stack.push(p); + } else if ft.is_file() && p.extension().and_then(|e| e.to_str()) == Some("rs") { + out.push(p); + } + } + } + out.sort(); + out +} + +/// Collect every print-like site in one Rust source file. +fn collect_sites(src: &str) -> Vec { + let masked = mask_comments(src); + let bindings = collect_let_bindings(&masked); + let non_let = collect_non_let_binders(&masked); + let mut sites = Vec::new(); + + for inv in find_invocations(&masked, PRINT_MACROS) { + sites.push(Site { + line: inv.line, + kind: inv.name.clone(), + exprs: interpolated_exprs(&inv.body), + }); + } + + // `write!` / `writeln!` are only prints when their sink is a terminal. + for inv in find_invocations(&masked, STREAM_WRITE_MACROS) { + let Some((target, rest)) = split_first_arg(&inv.body) else { + continue; + }; + if !is_stream_target(target, &bindings) { + continue; + } + sites.push(Site { + line: inv.line, + kind: format!("{}()", inv.name), + exprs: interpolated_exprs(rest), + }); + } + + // `eprint_warning` escapes its argument in HUMAN mode, which preserves `\n`. Any + // value interpolated into the string it is handed must therefore be WIRE-escaped in + // its own right — and the argument may be a local rather than an inline `format!`, + // so classify it, tracing one hop through its `let` binding. + for call in find_invocations(&masked, SANITIZING_PRINT_HELPERS) { + match classify_helper_arg(&call.body, &bindings, &non_let, TRACE_BUDGET) { + ArgVerdict::Safe => {} + // A `format!` with nothing interpolated, or a binding that resolved wholly to + // sanitizer calls, has nothing left to judge — do not record an empty site. + ArgVerdict::Checked { exprs, .. } if exprs.is_empty() => {} + ArgVerdict::Checked { via, exprs } => sites.push(Site { + line: call.line, + kind: format!("{}({via})", call.name), + exprs, + }), + // Fail closed: an argument shape the trace cannot resolve is reported + // verbatim, so it must be fixed or justified rather than silently trusted. + ArgVerdict::Unchecked => sites.push(Site { + line: call.line, + kind: format!("{}(untraced)", call.name), + exprs: vec![normalize(&call.body)], + }), + } + } + + sites +} + +/// One `let` binding: the name it introduces and the text of its initialiser. +#[derive(Debug)] +struct Binding { + name: String, + init: String, +} + +/// How the argument handed to a sanitizing print helper is judged. +#[derive(Debug)] +enum ArgVerdict { + /// A string literal, or a whole-expression sanitizer call. Nothing further to check. + Safe, + /// Resolved to one or more `format!`s; `exprs` is everything they interpolate. + Checked { via: String, exprs: Vec }, + /// Not a recognised shape. Report it. + Unchecked, +} + +/// How many `let` hops `classify_helper_arg` will follow. +/// +/// One. A local initialised from another local is reported rather than followed — an +/// explicit bound, so the trace cannot loop on `let a = b; let b = a;`. +const TRACE_BUDGET: u8 = 1; + +/// Collect every `let = ;` binding in already-masked source. +/// +/// Destructuring patterns (`let Some(x) = …`, `let (a, b) = …`) are skipped: the name is +/// required to be a plain identifier followed by `=` or a `:` type annotation. Names are +/// collected file-wide rather than per-function, which is why `classify_helper_arg` +/// requires *every* binding of a name to be acceptable. +fn collect_let_bindings(text: &str) -> Vec { + let b = text.as_bytes(); + let mut out = Vec::new(); + let mut i = 0usize; + while i < b.len() { + if let Some(next) = skip_literal(text, b, i) { + i = next; + continue; + } + if !(text[i..].starts_with("let") + && !prev_is_ident(b, i) + && b.get(i + 3).is_some_and(u8::is_ascii_whitespace)) + { + i += 1; + continue; + } + let mut j = skip_ws(b, i + 3); + if text[j..].starts_with("mut") && b.get(j + 3).is_some_and(u8::is_ascii_whitespace) { + j = skip_ws(b, j + 3); + } + let start = j; + while j < b.len() && (b[j].is_ascii_alphanumeric() || b[j] == b'_') { + j += 1; + } + let name = text[start..j].to_string(); + let after = skip_ws(b, j); + // A plain binding is followed by `=` (or `: Type =`); anything else is a pattern. + if !is_ident(&name) || !matches!(b.get(after), Some(b'=') | Some(b':')) { + i += 3; + continue; + } + let Some((eq, semi)) = find_init_bounds(text, b, after) else { + i += 3; + continue; + }; + out.push(Binding { + name, + init: text[eq + 1..semi].trim().to_string(), + }); + i = semi; + } + out +} + +/// Collect every name in already-masked source that is introduced by something *other* +/// than a `let` — a `for`-loop variable, a function parameter, or a closure parameter. +/// +/// # Why +/// +/// `collect_let_bindings` matches names file-wide, not per scope. Without this set, a +/// name bound by one of the shapes above was resolved against whatever unrelated `let`s +/// of the same name the file happened to contain, and was accepted if all of them were +/// safe. On real source that was a live bypass: +/// +/// ```ignore +/// // in lint.rs, which has three unrelated `let label = safe_path(…);` bindings +/// fn atk(rules: &[String]) { for label in rules { eprint_warning(label); } } +/// ``` +/// +/// Every name returned here **poisons** itself for [`classify_helper_arg`]: a bare +/// argument with that name is reported instead of resolved, whatever its `let`s say. A +/// name collected here that is genuinely safe costs one allowlist entry; the opposite +/// mistake costs a review round. +/// +/// Over-collection is the safe direction, so the shapes are matched loosely: for a `for` +/// pattern and a parameter pattern, *every* identifier-shaped token in the pattern is +/// taken, keywords aside. `if let` / `while let` / `match`-arm binders are **not** +/// modelled — limit 5 in the module doc. +fn collect_non_let_binders(text: &str) -> Vec { + let b = text.as_bytes(); + let mut out = Vec::new(); + let mut i = 0usize; + while i < b.len() { + if let Some(next) = skip_literal(text, b, i) { + i = next; + continue; + } + // `for in …` — the pattern ends at the ` in ` that follows it. + if text[i..].starts_with("for") + && !prev_is_ident(b, i) + && b.get(i + 3).is_some_and(u8::is_ascii_whitespace) + { + let tail = &text[i + 3..]; + // Bound the search: a `for` header never runs past its opening brace. + let head = &tail[..tail.find('{').unwrap_or(tail.len()).min(400)]; + if let Some(kw) = head.find(" in ") { + push_pattern_idents(&head[..kw], &mut out); + } + i += 3; + continue; + } + // `fn name()` — one entry per parameter. + if text[i..].starts_with("fn") + && !prev_is_ident(b, i) + && b.get(i + 2).is_some_and(u8::is_ascii_whitespace) + { + if let Some(rel) = text[i..].find('(') { + let open = i + rel; + if let Some(close) = matching_paren(text, b, open) { + for param in split_top_level(&text[open + 1..close]) { + // `name: Type` — the pattern is everything before the top-level `:`. + let pat = param.split(':').next().unwrap_or(¶m); + push_pattern_idents(pat, &mut out); + } + i = close; + continue; + } + } + i += 2; + continue; + } + // Closure parameters, `|a, b|` / `|a: &T|`. A `|` is only read as the opening + // delimiter when what follows, up to the next `|`, is parameter-shaped: nothing + // but identifiers, commas, `&`, `mut`, `ref` and type annotations. That excludes + // `a | b` (bitwise or) and `a || b`, whose operands are arbitrary expressions. + if b[i] == b'|' && b.get(i + 1) != Some(&b'|') { + let tail = &text[i + 1..]; + if let Some(rel) = tail.find('|') { + let params = &tail[..rel]; + if is_closure_param_list(params) { + for param in split_top_level(params) { + let pat = param.split(':').next().unwrap_or(¶m); + push_pattern_idents(pat, &mut out); + } + // Resume *past* the closing `|`, so the text after a closure is never + // read as the parameter list of the next one. + i += rel + 2; + continue; + } + } + } + i += 1; + } + out.sort(); + out.dedup(); + out +} + +/// Binding-position keywords and the receiver, none of which name a value a caller +/// controls. +const PATTERN_KEYWORDS: &[&str] = &["mut", "ref", "self", "impl", "dyn", "in"]; + +/// Push every identifier-shaped token in a binding pattern. +fn push_pattern_idents(pattern: &str, out: &mut Vec) { + for tok in pattern.split(|c: char| !(c.is_ascii_alphanumeric() || c == '_')) { + if is_ident(tok) && !PATTERN_KEYWORDS.contains(&tok) { + out.push(tok.to_string()); + } + } +} + +/// Does `s` look like the inside of a closure's `|…|`, rather than the right-hand side +/// of a bitwise `|`? Empty is a closure (`||` is handled by the caller as an early-out, +/// so this only sees `| |`); otherwise every character must be pattern-shaped. +fn is_closure_param_list(s: &str) -> bool { + !s.is_empty() + && s.chars().all(|c| { + c.is_ascii_alphanumeric() + || c.is_ascii_whitespace() + || matches!(c, '_' | ',' | ':' | '&' | '<' | '>' | '\'' | '[' | ']') + }) +} + +/// From the start of a binding's `: Type = init;` tail, find the top-level `=` and the +/// top-level `;` that closes it. +fn find_init_bounds(text: &str, b: &[u8], from: usize) -> Option<(usize, usize)> { + let mut depth = 0i32; + let mut eq: Option = None; + let mut i = from; + while i < b.len() { + if let Some(next) = skip_literal(text, b, i) { + i = next; + continue; + } + match b[i] { + b'(' | b'[' | b'{' => depth += 1, + b')' | b']' | b'}' => { + depth -= 1; + if depth < 0 { + return None; + } + } + // `=` but not `==` / `=>` / `!=` / `<=` / `>=`. + b'=' if depth == 0 + && eq.is_none() + && b.get(i + 1) != Some(&b'=') + && b.get(i + 1) != Some(&b'>') + && !matches!( + b.get(i.wrapping_sub(1)), + Some(b'=' | b'!' | b'<' | b'>' | b'+' | b'-' | b'*' | b'/' | b'%') + ) => + { + eq = Some(i); + } + b';' if depth == 0 => return eq.map(|e| (e, i)), + _ => {} + } + i += 1; + } + None +} + +fn skip_ws(b: &[u8], mut i: usize) -> usize { + while i < b.len() && b[i].is_ascii_whitespace() { + i += 1; + } + i +} + +/// Judge the single argument handed to a sanitizing print helper. +/// +/// Accepts a string literal, a whole-expression sanitizer call, or a whole-expression +/// `format!` (whose interpolations are returned for the caller to check). A bare +/// identifier is resolved through its `let` bindings, up to `budget` hops; a name with no +/// visible binding, with any binding that is itself unrecognised, or that appears in +/// `non_let` (see [`collect_non_let_binders`]), is `Unchecked`. +fn classify_helper_arg( + arg: &str, + bindings: &[Binding], + non_let: &[String], + budget: u8, +) -> ArgVerdict { + let e = arg.trim().trim_start_matches(['&', ' ']).trim(); + if e.is_empty() { + return ArgVerdict::Unchecked; + } + + // A whole string literal — `eprint_warning("Stopped watching.")`. + if let Some((_, end)) = parse_string_literal(e) { + if e[end..].trim().is_empty() { + return ArgVerdict::Safe; + } + } + + // A whole-expression sanitizer call. + if is_sanitizer_call(e) { + return ArgVerdict::Safe; + } + + // A whole-expression `format!(…)` — check what it interpolates. + if let Some(body) = whole_invocation_body(e, "format!") { + return ArgVerdict::Checked { + via: "format!".to_string(), + exprs: interpolated_exprs(&body), + }; + } + + // A bare local: follow its binding(s) — unless the name is also introduced by a + // `for` variable, a parameter or a closure param somewhere in the file, in which case + // the file's `let`s of that name say nothing about this value. Fail closed. + if is_ident(e) { + if budget == 0 || non_let.iter().any(|n| n == e) { + return ArgVerdict::Unchecked; + } + let mut matched = false; + let mut exprs = Vec::new(); + for binding in bindings.iter().filter(|b| b.name == e) { + matched = true; + match classify_helper_arg(&binding.init, bindings, non_let, budget - 1) { + ArgVerdict::Safe => {} + ArgVerdict::Checked { exprs: mut v, .. } => exprs.append(&mut v), + // One unrecognised binding of this name poisons the whole trace. + ArgVerdict::Unchecked => return ArgVerdict::Unchecked, + } + } + if !matched { + return ArgVerdict::Unchecked; + } + return ArgVerdict::Checked { + via: format!("let {e} = …"), + exprs, + }; + } + + ArgVerdict::Unchecked +} + +/// Body of `name(…)` when it spans the *entire* expression — nothing may trail the +/// closing paren, or a postfix continuation could undo whatever the call did. +fn whole_invocation_body(expr: &str, name: &str) -> Option { + let rest = expr.strip_prefix(name)?; + let open = expr.len() - rest.trim_start().len(); + if expr.as_bytes().get(open) != Some(&b'(') { + return None; + } + let close = matching_paren(expr, expr.as_bytes(), open)?; + expr[close + 1..] + .trim() + .is_empty() + .then(|| expr[open + 1..close].to_string()) +} + +/// Does this `write!` / `writeln!` target a terminal stream rather than a buffer? +/// +/// True when the target expression names stdout/stderr, or is a local whose `let` +/// initialiser does. See "Accepted limits" in the module doc for what this misses. +fn is_stream_target(target: &str, bindings: &[Binding]) -> bool { + let t = target.trim(); + if names_stream(t) { + return true; + } + let ident = t.trim_start_matches(['&', ' ']).trim(); + let ident = ident.strip_prefix("mut ").unwrap_or(ident).trim(); + is_ident(ident) + && bindings + .iter() + .any(|b| b.name == ident && names_stream(&b.init)) +} + +fn names_stream(s: &str) -> bool { + let lower = s.to_ascii_lowercase(); + lower.contains("stdout") || lower.contains("stderr") +} + +/// Split off the first top-level argument, returning it and the text after its comma. +fn split_first_arg(body: &str) -> Option<(&str, &str)> { + let b = body.as_bytes(); + let mut depth = 0i32; + let mut i = 0usize; + while i < b.len() { + if let Some(next) = skip_literal(body, b, i) { + i = next; + continue; + } + match b[i] { + b'(' | b'[' | b'{' => depth += 1, + b')' | b']' | b'}' => depth -= 1, + b',' if depth == 0 => return Some((&body[..i], &body[i + 1..])), + _ => {} + } + i += 1; + } + None +} + +/// Is `expr` *exactly* a call to one of [`SANITIZERS`], possibly module-qualified and +/// possibly behind leading `&`? +/// +/// Both ends are checked. Nothing may precede the callee but `&` and whitespace, so a +/// sanitizer nested in an outer call (`wrap(safe_path(p))`) is rejected — the outer call +/// could undo the escape. And nothing may follow the closing paren, so a postfix +/// continuation (`safe_path(p) + &evil`, `safe_path(p).replace("a", &evil)`) is rejected +/// for the same reason. +fn is_sanitizer_call(expr: &str) -> bool { + let e = expr.trim_start_matches(['&', ' ']).trim(); + let Some(open) = e.find('(') else { + return false; + }; + let callee = e[..open].trim(); + if callee.is_empty() { + return false; + } + // Every path segment must be a plain identifier — this rejects `thing.safe_path`, + // `wrap(safe_path`, `format!` and friends. + let segments: Vec<&str> = callee.split("::").collect(); + if !segments.iter().all(|s| is_ident(s)) { + return false; + } + if !segments + .last() + .is_some_and(|last| SANITIZERS.contains(last)) + { + return false; + } + // The call must be the whole expression. + matching_paren(e, e.as_bytes(), open).is_some_and(|close| e[close + 1..].trim().is_empty()) +} + +/// The written justification exempting `expr` at a site of this `kind`, if any. +/// +/// [`ALLOWED_UNTRACED_HELPER_ARGS`] applies *only* to the untraced-helper-argument +/// position; [`ALLOWED_UNSANITIZED`] applies to every other site. The two lists never +/// cover for each other, so exempting a loop variable as a warning body does not also +/// exempt that name in an `eprintln!`. +fn justification(file: &str, expr: &str, kind: &str) -> Option<&'static str> { + let list = if kind.ends_with("(untraced)") { + ALLOWED_UNTRACED_HELPER_ARGS + } else { + ALLOWED_UNSANITIZED + }; + list.iter() + .find(|(f, e, _)| *f == file && *e == expr) + .map(|(_, _, why)| *why) +} + +fn is_ident(s: &str) -> bool { + !s.is_empty() + && s.chars() + .next() + .is_some_and(|c| c.is_ascii_alphabetic() || c == '_') + && s.chars().all(|c| c.is_ascii_alphanumeric() || c == '_') +} + +struct Invocation { + /// 1-based line of the macro/function name within the scanned text. + line: usize, + name: String, + body: String, +} + +/// Find every `name(...)` / `name!(...)` invocation, with balanced-paren bodies. +/// +/// `text` must already have had its comments masked; string, raw-string, and char +/// literals are skipped so parentheses inside them do not unbalance the scan. +fn find_invocations(text: &str, names: &[&str]) -> Vec { + let b = text.as_bytes(); + let mut out = Vec::new(); + let mut i = 0usize; + while i < b.len() { + // Skip literals so a `"("` inside a string is not treated as code. + if let Some(next) = skip_literal(text, b, i) { + i = next; + continue; + } + let mut matched: Option<&str> = None; + for name in names { + if text[i..].starts_with(name) && !prev_is_ident(b, i) && !is_fn_definition(text, i) { + matched = Some(name); + break; + } + } + let Some(name) = matched else { + i += 1; + continue; + }; + let mut j = i + name.len(); + while j < b.len() && (b[j] as char).is_ascii_whitespace() { + j += 1; + } + if j >= b.len() || b[j] != b'(' { + i += name.len(); + continue; + } + let Some(close) = matching_paren(text, b, j) else { + i += name.len(); + continue; + }; + out.push(Invocation { + line: text[..i].matches('\n').count() + 1, + name: (*name).to_string(), + body: text[j + 1..close].to_string(), + }); + i = close + 1; + } + out +} + +/// Union of a format invocation's inline captures and its positional arguments. +fn interpolated_exprs(body: &str) -> Vec { + let trimmed = body.trim_start(); + let Some((fmt_inner, fmt_end)) = parse_string_literal(trimmed) else { + // Not a literal format string — we cannot see the placeholders, so report the + // whole invocation rather than assume it is safe. + let whole = normalize(body); + return if whole.is_empty() { + Vec::new() + } else { + vec![whole] + }; + }; + let mut exprs = placeholder_exprs(&fmt_inner); + let rest = trimmed[fmt_end..].trim_start(); + if let Some(args) = rest.strip_prefix(',') { + for arg in split_top_level(args) { + let a = strip_named_arg(arg.trim()); + let a = normalize(a); + if !a.is_empty() { + exprs.push(a); + } + } + } + exprs +} + +/// Named captures written inline in the format string (`{e}`, `{max_depth}`). +/// +/// Positional `{}` / `{0}` placeholders consume an argument instead and are collected +/// from the argument list. A placeholder whose format spec uses a `$` reference +/// (`{:>width$}`) is reported verbatim so it must be justified rather than silently +/// skipped. +fn placeholder_exprs(fmt: &str) -> Vec { + let mut out = Vec::new(); + let bytes = fmt.as_bytes(); + let mut i = 0usize; + while i < bytes.len() { + match bytes[i] { + b'{' if i + 1 < bytes.len() && bytes[i + 1] == b'{' => i += 2, + b'}' if i + 1 < bytes.len() && bytes[i + 1] == b'}' => i += 2, + b'{' => { + let Some(rel) = fmt[i..].find('}') else { break }; + let inner = &fmt[i + 1..i + rel]; + let (name, spec) = match inner.find(':') { + Some(c) => (&inner[..c], &inner[c + 1..]), + None => (inner, ""), + }; + if spec.contains('$') { + out.push(format!("{{{inner}}}")); + } else if !name.is_empty() && !name.chars().all(|c| c.is_ascii_digit()) { + out.push(name.to_string()); + } + i += rel + 1; + } + _ => i += 1, + } + } + out +} + +/// `kind_name = expr` → `expr`; anything else is returned unchanged. +fn strip_named_arg(arg: &str) -> &str { + let Some(eq) = arg.find('=') else { return arg }; + // Not `==`, `!=`, `>=`, `<=`, `+=` … + if arg.as_bytes().get(eq + 1) == Some(&b'=') { + return arg; + } + if eq > 0 + && matches!( + arg.as_bytes()[eq - 1], + b'=' | b'!' | b'<' | b'>' | b'+' | b'-' | b'*' | b'/' | b'%' | b'&' | b'|' | b'^' + ) + { + return arg; + } + if !is_ident(arg[..eq].trim()) { + return arg; + } + arg[eq + 1..].trim() +} + +fn normalize(s: &str) -> String { + s.split_whitespace().collect::>().join(" ") +} + +/// Split on top-level commas, honouring nesting and literals. +fn split_top_level(s: &str) -> Vec { + let b = s.as_bytes(); + let mut out = Vec::new(); + let mut depth = 0i32; + let mut start = 0usize; + let mut i = 0usize; + while i < b.len() { + if let Some(next) = skip_literal(s, b, i) { + i = next; + continue; + } + match b[i] { + b'(' | b'[' | b'{' => depth += 1, + b')' | b']' | b'}' => depth -= 1, + b',' if depth == 0 => { + out.push(s[start..i].to_string()); + start = i + 1; + } + _ => {} + } + i += 1; + } + out.push(s[start..].to_string()); + out.retain(|a| !a.trim().is_empty()); + out +} + +/// Parse a Rust string literal at the start of `s`, returning its inner text and the +/// byte index just past the closing delimiter. +fn parse_string_literal(s: &str) -> Option<(String, usize)> { + let b = s.as_bytes(); + if b.is_empty() { + return None; + } + if b[0] == b'r' { + let mut j = 1usize; + let mut hashes = 0usize; + while j < b.len() && b[j] == b'#' { + hashes += 1; + j += 1; + } + if j < b.len() && b[j] == b'"' { + let close = format!("\"{}", "#".repeat(hashes)); + let rel = s[j + 1..].find(&close)?; + return Some((s[j + 1..j + 1 + rel].to_string(), j + 1 + rel + close.len())); + } + return None; + } + if b[0] != b'"' { + return None; + } + let mut i = 1usize; + while i < b.len() { + match b[i] { + b'\\' => i += 2, + b'"' => return Some((s[1..i].to_string(), i + 1)), + _ => i += 1, + } + } + None +} + +/// Replace every comment byte with a space (newlines preserved) so line numbers and +/// byte offsets stay stable while comment text disappears from the scan. +fn mask_comments(src: &str) -> String { + let b = src.as_bytes(); + let mut out = b.to_vec(); + let mut i = 0usize; + while i < b.len() { + if let Some(next) = skip_literal(src, b, i) { + i = next; + continue; + } + if b[i] == b'/' && i + 1 < b.len() && b[i + 1] == b'/' { + while i < b.len() && b[i] != b'\n' { + out[i] = b' '; + i += 1; + } + continue; + } + if b[i] == b'/' && i + 1 < b.len() && b[i + 1] == b'*' { + let mut depth = 0usize; + while i < b.len() { + if b[i] == b'/' && i + 1 < b.len() && b[i + 1] == b'*' { + depth += 1; + out[i] = b' '; + out[i + 1] = b' '; + i += 2; + } else if b[i] == b'*' && i + 1 < b.len() && b[i + 1] == b'/' { + depth -= 1; + out[i] = b' '; + out[i + 1] = b' '; + i += 2; + if depth == 0 { + break; + } + } else { + if b[i] != b'\n' { + out[i] = b' '; + } + i += 1; + } + } + continue; + } + i += 1; + } + String::from_utf8(out).expect("masking replaces comment bytes with ASCII spaces") +} + +/// If a string / raw-string / char literal starts at `i`, return the index just past it. +fn skip_literal(src: &str, b: &[u8], i: usize) -> Option { + // Raw string, optionally byte-prefixed: r"…", r#"…"#, br#"…"# + let raw_start = if b[i] == b'r' && !prev_is_ident(b, i) { + Some(i) + } else if b[i] == b'b' && !prev_is_ident(b, i) && b.get(i + 1) == Some(&b'r') { + Some(i + 1) + } else { + None + }; + if let Some(r) = raw_start { + let mut j = r + 1; + let mut hashes = 0usize; + while j < b.len() && b[j] == b'#' { + hashes += 1; + j += 1; + } + if j < b.len() && b[j] == b'"' { + let close = format!("\"{}", "#".repeat(hashes)); + return Some(match src[j + 1..].find(&close) { + Some(rel) => j + 1 + rel + close.len(), + None => b.len(), + }); + } + } + if b[i] == b'"' { + let mut j = i + 1; + while j < b.len() { + match b[j] { + b'\\' => j += 2, + b'"' => return Some(j + 1), + _ => j += 1, + } + } + return Some(b.len()); + } + if b[i] == b'\'' { + // Escaped char literal: '\n', '\u{1b}', '\'' + if b.get(i + 1) == Some(&b'\\') { + let mut j = i + 2; + while j < b.len() && b[j] != b'\'' { + j += 1; + } + return Some((j + 1).min(b.len())); + } + // Plain char literal: 'x' (any codepoint width). Otherwise it is a lifetime or + // a loop label, which carries no literal text to skip. + if let Some(ch) = src[i + 1..].chars().next() { + let after = i + 1 + ch.len_utf8(); + if b.get(after) == Some(&b'\'') { + return Some(after + 1); + } + } + return None; + } + None +} + +fn prev_is_ident(b: &[u8], i: usize) -> bool { + i > 0 && (b[i - 1].is_ascii_alphanumeric() || b[i - 1] == b'_') +} + +/// Is the name at `i` introduced by `fn`, i.e. a definition rather than a call? +/// +/// Without this, `fn eprint_warning(w: &str)` in `output.rs` would be scanned as a call +/// to itself whose argument is the parameter list. +fn is_fn_definition(text: &str, i: usize) -> bool { + let head = text[..i].trim_end(); + let Some(before) = head.strip_suffix("fn") else { + return false; + }; + !before.ends_with(|c: char| c.is_ascii_alphanumeric() || c == '_') +} + +/// Index of the `)` matching the `(` at `open`. +fn matching_paren(src: &str, b: &[u8], open: usize) -> Option { + let mut depth = 0i32; + let mut i = open; + while i < b.len() { + if let Some(next) = skip_literal(src, b, i) { + i = next; + continue; + } + match b[i] { + b'(' => depth += 1, + b')' => { + depth -= 1; + if depth == 0 { + return Some(i); + } + } + _ => {} + } + i += 1; + } + None +} diff --git a/crates/mds-cli/tests/producer_discipline.rs b/crates/mds-cli/tests/producer_discipline.rs new file mode 100644 index 00000000..d8058395 --- /dev/null +++ b/crates/mds-cli/tests/producer_discipline.rs @@ -0,0 +1,136 @@ +//! Producer-discipline test — the one cross-crate precondition +//! `crates/mds-cli/tests/print_discipline.rs` depends on and cannot check +//! (CWE-117 / PF-013 / #176). +//! +//! # Why this lives here, and why it is narrow +//! +//! `build.rs` and `main.rs` print whole `mds-core` warning strings with +//! `for w in &result.warnings { eprint_warning(w) }`. `eprint_warning` escapes in HUMAN +//! mode, which preserves `\n` by design so multi-line prose renders — so what stops a +//! warning from forging a bare `Clean: …` status line is **not** the print helper. It is +//! that `mds-core` WIRE-escapes every untrusted value it interpolates at construction. +//! `print_discipline.rs` is a lexical scanner over `crates/mds-cli/src/**`; it cannot +//! follow a value across the crate boundary, so those two sites sit in its +//! `ALLOWED_UNTRACED_HELPER_ARGS` with that dependency written down. +//! +//! `mds-core` has exactly three warning producers that interpolate a runtime value — +//! `resolver.rs`'s imported-module filename and `evaluator.rs`'s two `@include` alias +//! warnings. Only the first can receive a hostile character: a module key is a filesystem +//! path, and POSIX permits any byte but `/` and NUL in a filename. The parser admits an +//! `@include` alias only if it matches `[A-Za-z_][A-Za-z0-9_]*`, so a test of the other +//! two would assert on an input the parser rejects — vacuous, which is the PF-013 failure +//! mode — and none is written. That asymmetry is stated in `print_discipline.rs`'s module +//! doc rather than papered over. +//! +//! # PF-013 evidence +//! +//! - **Reachable vector:** the module map handed to `compile_virtual_with_deps_opts` is +//! keyed by path, and no layer between the caller and the warning rejects a control +//! byte in that key. [`hostile_module_name_reaches_the_warning`] proves the hostile +//! name really does reach this producer by finding it, escaped, in the warning text. +//! - **Positive:** the escaped `\u001B` literal must be present in the warning. +//! - **Negative:** no raw ESC byte, and no raw `\n`, may appear anywhere in it. +//! - **Non-vacuity:** the warning must exist and must be the segment-cap warning. Without +//! that, a compile that silently stopped emitting the warning would pass every +//! assertion above by producing nothing to assert on. +//! - **Guard-removal:** deleting the `sanitize_control_chars_wire(…)` call at +//! `crates/mds-core/src/resolver.rs`'s segment-cap `warnings.push` makes the positive +//! assertion fail on the missing `\u001B` literal and the negative assertion fail on +//! the raw ESC byte. + +/// A filename that carries two members of the escape class: ESC (U+001B, the CSI +/// introducer of an ANSI escape sequence) and U+202E RIGHT-TO-LEFT OVERRIDE (Trojan +/// Source, CVE-2021-42574). Written with `\u{…}` escapes so this file holds no raw +/// control bytes. +const HOSTILE_MODULE: &str = "big\u{1b}[31m\u{202e}.mds"; + +/// The WIRE form the two hostile characters must arrive in. +const ESCAPED_ESC: &str = "\\u001B"; +const ESCAPED_RLO: &str = "\\u202E"; + +/// AC / #176: the imported-module filename in `mds-core`'s source-map segment-cap warning +/// is WIRE-escaped at construction, so the string `mds-cli` hands to `eprint_warning` — +/// which preserves `\n` — cannot carry a terminal-hazardous byte. +/// +/// The vector: a module whose evaluation exceeds `MAX_SOURCEMAP_SEGMENTS` (1 000 000), +/// imported by an entry module. 100 000 iterations x 11 segment-producing nodes = +/// 1 100 000 segments, which trips the cap inside the *imported* module and takes the +/// `resolver.rs` branch that names the module in its warning. +#[test] +fn hostile_module_name_reaches_the_warning() { + use mds::{CompileOptions, Value}; + + let items: Vec = (0..100_000) + .map(|_| Value::String("x".to_string())) + .collect(); + let mut vars = std::collections::HashMap::new(); + vars.insert("items".to_string(), Value::Array(items)); + + let mut modules = std::collections::HashMap::new(); + // The resolver requires an explicitly relative specifier; it normalizes back to the + // bare module key, which is what `ctx.file_str` — and therefore the warning — carries. + modules.insert( + "entry.mds".to_string(), + format!("@import \"./{HOSTILE_MODULE}\" as big\n@include big\n"), + ); + // 6 Text nodes + 5 Interpolation nodes = 11 segments per iteration. + modules.insert( + HOSTILE_MODULE.to_string(), + "@for item in items:\nA{{item}}B{{item}}C{{item}}D{{item}}E{{item}}F\n@end\n".to_string(), + ); + + let result = mds::compile_virtual_with_deps_opts( + modules, + "entry.mds", + Some(vars), + CompileOptions { + source_map: true, + ..Default::default() + }, + ) + .expect("compilation must succeed even when the segment cap is hit"); + + // Non-vacuity: the producer under test must actually have run. If the cap warning + // stops being emitted, every assertion below would hold over an empty haystack. + let cap_warning = result + .warnings + .iter() + .find(|w| w.contains("segment cap") && w.contains("imported module")) + .unwrap_or_else(|| { + panic!( + "non-vacuity: the imported-module segment-cap warning must be emitted; \ + got warnings: {:?}", + result.warnings + ) + }); + + // Reachability + positive: the hostile name reached the producer, and both hostile + // characters arrived as their six-character WIRE literals. + assert!( + cap_warning.contains(ESCAPED_ESC), + "the ESC byte in the module name must arrive as the literal {ESCAPED_ESC}; \ + got {cap_warning:?}" + ); + assert!( + cap_warning.contains(ESCAPED_RLO), + "U+202E in the module name must arrive as the literal {ESCAPED_RLO}; \ + got {cap_warning:?}" + ); + + // Negative: no member of the escape class survives raw. `\n` is checked explicitly — + // it is the line-forgery vector (CWE-117) that HUMAN-mode `eprint_warning` would + // preserve, and the whole reason this producer uses WIRE mode. + assert!( + !cap_warning.contains('\u{1b}'), + "a raw ESC byte must not survive into a warning string: {cap_warning:?}" + ); + assert!( + !cap_warning.contains('\u{202e}'), + "a raw U+202E must not survive into a warning string: {cap_warning:?}" + ); + assert!( + !cap_warning.contains('\n'), + "a raw newline must not survive into a warning string — `eprint_warning` \ + preserves it, so it would forge a standalone status line: {cap_warning:?}" + ); +} diff --git a/crates/mds-cli/tests/security.rs b/crates/mds-cli/tests/security.rs index 2008e1ec..fb3004b3 100644 --- a/crates/mds-cli/tests/security.rs +++ b/crates/mds-cli/tests/security.rs @@ -1,5 +1,5 @@ mod common; -use common::{fixture, mds_bin}; +use common::{assert_no_control_chars, fixture, mds_bin}; use std::collections::HashMap; #[test] @@ -420,3 +420,444 @@ fn load_vars_file_rejects_symlinked_path() { "error must mention symlink restriction; got: {err}" ); } + +// ── CWE-150: control bytes in the error MESSAGE on the CLI human path (#176) ── +// +// The pre-existing ESC e2e tests in `cli_build.rs` / `cli_fmt.rs` / `cli_lint.rs` +// all use the `@define \x1bfoo:` vector, whose message is the *fixed* string +// "syntax error: @define must have parameter list: @define name(params):" — the +// hostile byte only ever reaches the rendered **source excerpt**, which +// `MdsError::at()` already neutralized. Those tests therefore never exercised the +// message text at all (PF-013: a hostile-input test that cannot fail for the +// reason it claims). The two tests below use vectors that put attacker-controlled +// bytes into the message itself, on both of the CLI's error families: +// +// 1. `MdsError` — `parser.rs` formats the raw alias into +// `invalid include alias: '{alias}'`. +// 2. CLI-authored `miette::miette!()` — `output.rs` formats the raw `mds.json` +// value into `mds.json output_dir '{}' must not contain '..' components`. +// +// Both render through `eprint_error`, the single CLI stderr choke-point. + +/// T-ESC-MSG-1 [security-11 / CWE-150 / PF-013 / #176]: an `MdsError` whose message +/// interpolates attacker-controlled template text must reach stderr escaped. +/// +/// Non-vacuity guard: the command must actually fail *and* stderr must contain the +/// expected error text, so the escape assertions cannot pass by the build silently +/// succeeding or failing for an unrelated reason. +#[test] +fn build_mds_error_message_escapes_control_bytes() { + let dir = tempfile::tempdir().unwrap(); + let src = dir.path().join("inc.mds"); + // ESC (U+001B) and RIGHT-TO-LEFT OVERRIDE (U+202E) inside the include alias. + // `is_valid_identifier` rejects it, and the raw alias is formatted into the message. + std::fs::write(&src, "@include \u{1b}[31mBAD\u{202e}alias\n").unwrap(); + + let out = mds_bin() + .arg("build") + .arg(&src) + .stdout(std::process::Stdio::piped()) + .stderr(std::process::Stdio::piped()) + .output() + .unwrap(); + + // ── Non-vacuity ────────────────────────────────────────────────────────── + assert!( + !out.status.success(), + "build of an invalid include alias must fail" + ); + let stderr = String::from_utf8_lossy(&out.stderr); + assert!( + stderr.contains("invalid include alias"), + "non-vacuity: the alias error must be the one rendered; got: {stderr}" + ); + + // ── Negative: no raw hostile bytes survive ─────────────────────────────── + assert!( + !out.stderr.contains(&0x1Bu8), + "raw ESC byte (0x1B) must not reach stderr in the error message; got: {stderr}" + ); + assert_no_control_chars(&stderr, "mds build MdsError message"); + + // ── Positive: the escaped literals are present ─────────────────────────── + assert!( + stderr.contains("\\u001B"), + "ESC must be rendered as the \\u001B literal in the message; got: {stderr}" + ); + assert!( + stderr.contains("\\u202E"), + "U+202E must be rendered as the \\u202E literal in the message; got: {stderr}" + ); +} + +/// T-ESC-MSG-2 [security-11 / CWE-150 / PF-004 / #176]: a CLI-authored +/// `miette::miette!()` error that interpolates attacker-controlled config text must +/// reach stderr escaped too. +/// +/// This is the PF-004 sibling path: `miette!()` reports do **not** downcast to +/// `MdsError` (see `exit_code`'s rustdoc), so a fix that only handled `MdsError` +/// would leave this vector open while claiming the boundary was closed. +#[test] +fn build_cli_authored_error_message_escapes_control_bytes() { + let dir = tempfile::tempdir().unwrap(); + std::fs::write(dir.path().join("hello.mds"), "Hello!\n").unwrap(); + // JSON `\u001b` decodes to a raw ESC byte in the parsed config value. The `..` + // component trips the traversal guard in `resolve_output_base`, which formats the + // raw value into its message. + std::fs::write( + dir.path().join("mds.json"), + r#"{"build": {"output_dir": "../\u001b[31mBAD"}}"#, + ) + .unwrap(); + + let out = mds_bin() + .arg("build") + .arg(dir.path()) + .stdout(std::process::Stdio::piped()) + .stderr(std::process::Stdio::piped()) + .output() + .unwrap(); + + // ── Non-vacuity ────────────────────────────────────────────────────────── + assert!( + !out.status.success(), + "build with output_dir containing '..' must fail" + ); + let stderr = String::from_utf8_lossy(&out.stderr); + assert!( + stderr.contains("output_dir"), + "non-vacuity: the output_dir traversal error must be the one rendered; got: {stderr}" + ); + + // ── Negative + positive ────────────────────────────────────────────────── + assert!( + !out.stderr.contains(&0x1Bu8), + "raw ESC byte (0x1B) must not reach stderr from a miette!() message; got: {stderr}" + ); + assert_no_control_chars(&stderr, "mds build miette!() message"); + assert!( + stderr.contains("\\u001B"), + "ESC must be rendered as the \\u001B literal; got: {stderr}" + ); +} + +// ── S14 / PF-004: the two boundaries the #176 alignment review found still open ── +// +// Both are reproduced-first vectors: on the pre-fix binary each command below emits +// the raw hostile bytes (verified with `od -c`). + +/// T-ESC-RULE-1 [security-11 / CWE-150 / PF-004 / PF-013 / #176]: an unknown lint rule +/// NAME from `mds.json` reaches stderr escaped. +/// +/// Vector: `mds.json` is read from the working tree and its rule names are arbitrary +/// JSON object keys. A JSON `\uXXXX` escape decodes to a real byte, so a repository can +/// carry a rule name containing a raw ESC without the file itself looking hostile. +/// This print site sat on a bare `eprintln!`, bypassing both `eprint_error` and +/// `eprint_warning` — the "sixth warning print" that `eprint_warning`'s own rustdoc +/// claimed was foreclosed. +/// +/// `serde_json` writes the ESC as a LOWERCASE-hex JSON escape; the sanitizer emits +/// UPPERCASE. Asserting the uppercase form proves the byte genuinely decoded on the way +/// in and was genuinely escaped on the way out, rather than passing through as literal +/// text. +/// +/// The vector also carries **newlines**, and the assertions below include a standalone +/// forged-line check. That combination is deliberate: routing this print through +/// `eprint_warning` closed CWE-150 on this vector (no raw control byte) while leaving +/// CWE-117 wide open, because `eprint_warning` is HUMAN mode and HUMAN mode preserves +/// `\n` by design. The earlier version of this test used a newline-free rule name and +/// `assert_no_control_chars`, which permits `\n` — so it certified the fix while the +/// forgery still worked. A rule name is a JSON object key: never legitimately +/// multi-line, so it is WIRE per the spec §7.5 per-field rule. +#[test] +fn lint_unknown_rule_name_escapes_control_bytes() { + let dir = tempfile::tempdir().unwrap(); + std::fs::write(dir.path().join("a.mds"), "Hello!\n").unwrap(); + + // One member of each escape sub-class: a C0 byte, a 3-byte bidi control, the 2-byte + // bidi control (U+061C) that the class originally missed — and newlines carrying two + // complete forged status lines. + let rule_name = format!( + "{}[31mEVIL{}RULE{}ALM\nClean: totally-real.mds\nOK: all-fine.mds", + '\u{1b}', '\u{202e}', '\u{061c}' + ); + let mut rules = serde_json::Map::new(); + rules.insert(rule_name, serde_json::Value::String("warn".to_string())); + let config = serde_json::json!({ "lint": { "rules": rules } }); + std::fs::write( + dir.path().join("mds.json"), + serde_json::to_string(&config).unwrap(), + ) + .unwrap(); + + let out = mds_bin() + .arg("lint") + .arg(dir.path()) + .stdout(std::process::Stdio::piped()) + .stderr(std::process::Stdio::piped()) + .output() + .unwrap(); + + let stderr = String::from_utf8_lossy(&out.stderr); + + // ── Non-vacuity: the warning actually fired, naming the rule ───────────── + assert!( + stderr.contains("unknown lint rule"), + "non-vacuity: the unknown-rule warning must be the one rendered; got: {stderr}" + ); + assert!( + stderr.contains("EVIL"), + "non-vacuity: the rule name itself must be printed; got: {stderr}" + ); + + // ── Negative: no raw hostile byte survives ─────────────────────────────── + assert!( + !out.stderr.contains(&0x1Bu8), + "raw ESC byte (0x1B) must not reach stderr from an mds.json rule name; got: {stderr}" + ); + assert_no_control_chars(&stderr, "mds lint unknown-rule warning"); + + // ── Negative: neither forged line appears on a line of its own ─────────── + // + // `assert_no_control_chars` deliberately permits `\n` so it can be used on HUMAN-mode + // prose, so it cannot see this. Mirrors T-ESC-FNAME-1's standalone-line assertion. + for forged in ["Clean: totally-real.mds", "OK: all-fine.mds"] { + assert!( + !stderr.lines().any(|l| l.trim() == forged), + "an mds.json rule name must not be able to forge the standalone status line \ + {forged:?}; got: {stderr}" + ); + } + + // ── Positive: the escaped literals are present ─────────────────────────── + for escaped in ["\\u001B", "\\u202E", "\\u061C"] { + assert!( + stderr.contains(escaped), + "{escaped} must appear in the unknown-rule warning; got: {stderr}" + ); + } + assert_eq!( + stderr.matches("\\u000A").count(), + 2, + "both embedded newlines must be escaped to their WIRE literal; got: {stderr}" + ); + + // ── Non-vacuity: the whole rule name landed on ONE line ────────────────── + let warning_lines: Vec<&str> = stderr + .lines() + .filter(|l| l.contains("unknown lint rule")) + .collect(); + assert_eq!( + warning_lines.len(), + 1, + "the warning must occupy exactly one line; got: {stderr}" + ); + assert!( + warning_lines[0].contains("EVIL") && warning_lines[0].ends_with("; ignoring"), + "the single warning line must carry the whole rule name and the trailing prose; \ + got: {stderr}" + ); +} + +/// T-ESC-FNAME-1 [S14 / CWE-117 / PF-013 / #176]: a filename containing newlines +/// cannot forge CLI status lines. +/// +/// Vector: POSIX permits a newline inside a filename and the user never types the name +/// — `mds build ` discovers it by directory walk. `safe_path` used HUMAN mode, +/// which preserves newlines by design so that multi-line diagnostic *messages* keep +/// rendering, so a file named `evil.mdsClean: real.mdsOK: all-fine.mds` emitted +/// two attacker-authored lines byte-identical in form to genuine status output, +/// unframed and unindented. +/// +/// Unix-only: Windows filesystems reject a newline in a filename outright. +#[cfg(unix)] +#[test] +fn build_status_line_cannot_be_forged_by_a_newline_in_a_filename() { + let dir = tempfile::tempdir().unwrap(); + let hostile = "evil.mds\nClean: real.mds\nOK: all-fine.mds"; + std::fs::write(dir.path().join(hostile), "Hello!\n").unwrap(); + let out_dir = dir.path().join("out"); + + let out = mds_bin() + .arg("build") + .arg(dir.path()) + .arg("--out-dir") + .arg(&out_dir) + .stdout(std::process::Stdio::piped()) + .stderr(std::process::Stdio::piped()) + .output() + .unwrap(); + + let combined = format!( + "{}{}", + String::from_utf8_lossy(&out.stdout), + String::from_utf8_lossy(&out.stderr) + ); + + // ── Non-vacuity: the build ran and printed a status line naming the file ─ + assert!( + out.status.success(), + "the build must succeed; stdout+stderr: {combined}" + ); + assert!( + combined.contains("evil.mds"), + "non-vacuity: the status line must name the built file; got: {combined}" + ); + + // ── Negative: neither forged line appears on a line of its own ─────────── + for forged in ["Clean: real.mds", "OK: all-fine.mds"] { + assert!( + !combined.lines().any(|l| l.trim() == forged), + "a filename must not be able to forge the standalone status line \ + {forged:?}; got: {combined}" + ); + } + + // ── Positive: both newlines escaped to their literal form ──────────────── + assert_eq!( + combined.matches("\\u000A").count(), + 2, + "both embedded newlines must be escaped; got: {combined}" + ); +} + +/// T-ESC-FNAME-2 [S14 / PF-004 / #176]: the `Clean:` status line — which sanitized its +/// filename inline rather than through `safe_path` — is covered by the same guarantee. +/// +/// This is the PF-004 sibling of T-ESC-FNAME-1: two status-line printers, one of which +/// open-coded its own escape call and so would have kept HUMAN mode when `safe_path` +/// moved to WIRE. The vector also carries U+061C, the bidi control the escape class +/// originally missed, so this pins both fixes on one line of output. +/// +/// Unix-only: Windows filesystems reject a newline in a filename outright. +#[cfg(unix)] +#[test] +fn lint_clean_status_line_cannot_be_forged_by_a_newline_in_a_filename() { + let dir = tempfile::tempdir().unwrap(); + let hostile = "ok.mds\nClean: real\u{061c}.mds"; + let path = dir.path().join(hostile); + std::fs::write(&path, "Hello!\n").unwrap(); + + let out = mds_bin() + .arg("lint") + .arg(&path) + .stdout(std::process::Stdio::piped()) + .stderr(std::process::Stdio::piped()) + .output() + .unwrap(); + + let stderr = String::from_utf8_lossy(&out.stderr); + + // ── Non-vacuity: the Clean: line actually printed, naming the file ─────── + assert!( + stderr.contains("Clean: ok.mds"), + "non-vacuity: the Clean: status line must name the linted file; got: {stderr}" + ); + + // ── Negative: no forged standalone line, no raw hostile codepoint ──────── + assert!( + !stderr.lines().any(|l| l.trim().starts_with("Clean: real")), + "a filename must not be able to forge a second Clean: line; got: {stderr}" + ); + assert_no_control_chars(&stderr, "mds lint Clean: status line"); + + // ── Positive: newline and U+061C both escaped ─────────────────────────── + assert!( + stderr.contains("\\u000A"), + "the embedded newline must be escaped; got: {stderr}" + ); + assert!( + stderr.contains("\\u061C"), + "U+061C must be escaped; got: {stderr}" + ); +} + +/// T-ESC-WALK-1 [security-11 / CWE-150 / CWE-117 / PF-004 / PF-013 / #176]: the shared +/// walker's depth-limit warning escapes the directory name it interpolates. +/// +/// Vector: `collect_mds_files_inner` in `mds-cli/src/output.rs` warns when recursion +/// exceeds `MAX_DEPTH` (64) and names the directory it stopped at. The name is +/// *discovered by the walk* — the user never types it — and every directory-mode +/// subcommand shares this walker, so one hostile directory name reached `mds build`, +/// `check`, `fmt`, `lint` and `watch` at once. +/// +/// This print sat on a bare `eprintln!` inside `output.rs` itself: it was neither one of +/// the `*_collecting_warnings` sites nor `watch.rs`, so two rounds of "route the warning +/// prints through `eprint_warning`" walked straight past it while the CHANGELOG claimed +/// no raw control byte reached human stderr on any warning path. +/// +/// The vector carries all three sub-classes at once — a C0 byte (ESC), a 3-byte bidi +/// control (U+202E), and a newline that forges a complete status line. The ESC goes into +/// the directory name as a RAW byte and is asserted on the way out as the uppercase +/// six-character literal, which proves a genuine decode-then-escape rather than literal +/// passthrough. +/// +/// Unix-only: Windows filesystems reject a newline in a path component outright. +#[cfg(unix)] +#[test] +fn walker_depth_limit_warning_cannot_be_forged_by_a_hostile_directory_name() { + let dir = tempfile::tempdir().unwrap(); + let root = dir.path().join("root"); + std::fs::create_dir(&root).unwrap(); + std::fs::write(root.join("a.mds"), "Hello!\n").unwrap(); + + // MAX_DEPTH is 64; the warning fires on the first directory at depth 65. + let mut deep = root.clone(); + for i in 1..=64 { + deep = deep.join(format!("d{i}")); + } + let hostile = format!( + "evil{}[31m{}dir\nClean: totally-real.mds", + '\u{1b}', '\u{202e}' + ); + deep = deep.join(&hostile); + std::fs::create_dir_all(&deep).unwrap(); + std::fs::write(deep.join("deep.mds"), "Hello!\n").unwrap(); + + // The walker is shared, so assert on all three directory-mode subcommands rather + // than trusting that one of them stands in for the others (PF-004). + for subcommand in ["lint", "check", "fmt"] { + let out = mds_bin() + .arg(subcommand) + .arg(&root) + .stdout(std::process::Stdio::piped()) + .stderr(std::process::Stdio::piped()) + .output() + .unwrap(); + let stderr = String::from_utf8_lossy(&out.stderr); + let label = format!("mds {subcommand} depth-limit warning"); + + // ── Non-vacuity: the warning actually fired and named the directory ── + assert!( + stderr.contains("directory depth limit"), + "non-vacuity [{label}]: the depth-limit warning must fire; got: {stderr}" + ); + assert!( + stderr.contains("evil"), + "non-vacuity [{label}]: the directory name must be printed; got: {stderr}" + ); + + // ── Negative: no raw hostile byte, no forged standalone line ───────── + assert!( + !out.stderr.contains(&0x1Bu8), + "raw ESC byte (0x1B) must not reach stderr from a directory name [{label}]; \ + got: {stderr}" + ); + assert_no_control_chars(&stderr, &label); + assert!( + !stderr + .lines() + .any(|l| l.trim() == "Clean: totally-real.mds"), + "a directory name must not be able to forge a standalone status line \ + [{label}]; got: {stderr}" + ); + + // ── Positive: each hostile codepoint appears in escaped form ───────── + for escaped in ["\\u001B", "\\u202E", "\\u000A"] { + assert!( + stderr.contains(escaped), + "{escaped} must appear in the depth-limit warning [{label}]; got: {stderr}" + ); + } + } +} diff --git a/crates/mds-core/src/error.rs b/crates/mds-core/src/error.rs index e86b1787..029f1bdb 100644 --- a/crates/mds-core/src/error.rs +++ b/crates/mds-core/src/error.rs @@ -3,6 +3,8 @@ use std::sync::Arc; use miette::{Diagnostic, SourceSpan}; use thiserror::Error; +use crate::lint::{named_source_for_render, sanitize_control_chars, sanitize_control_chars_wire}; + // ── Serializable error types ────────────────────────────────────────────────── /// A serializable representation of a source span. @@ -119,13 +121,39 @@ fn at( return (Some(SourceSpan::new(offset.into(), len)), None); } + // `named_source_for_render` applies the per-half sanitization: byte-length-preserving + // neutralization for the span-indexed source (PF-014), WIRE-mode \uXXXX escaping for + // the single-line filename (a newline-bearing filename must not forge a line). ( Some(SourceSpan::new(offset.into(), len)), - Some(Arc::new(miette::NamedSource::new(file, source.to_string()))), + Some(Arc::new(named_source_for_render(file, source))), ) } /// All errors produced by the MDS compiler. +/// +/// ## Display contract — sanitization split (CWE-150 / issue #176) +/// +/// `MdsError` implements `std::fmt::Display` via `thiserror`. The `Display` +/// output is the **raw, unsanitized** message string: it may contain C0/DEL/C1 +/// control bytes from untrusted `.mds` source input. +/// +/// | Method / path | Sanitized | Intended context | +/// |---|---|---| +/// | `e.to_string()` / `format!("{e}")` | **No** | Machine-readable pipelines, structured loggers | +/// | [`MdsError::display_sanitized()`] | **Yes** | Terminal output, user-facing render | +/// | [`MdsError::serialize()`]`.message` | **Yes** | JSON API / binding surfaces | +/// +/// **Do not write `eprintln!("{e}")` or `e.to_string()` when the output goes +/// to a TTY or is embedded in a user-visible string without further escaping.** +/// Use [`MdsError::display_sanitized()`] instead to avoid terminal injection +/// (CWE-150). All three published binding surfaces (napi, WASM, Python) +/// already use `serialize()` and are unaffected. +/// +/// This contract governs **direct** rendering by a consumer of this crate. The `mds` +/// CLI never renders an `MdsError` directly: it hands the `miette::Report` to +/// `eprint_error`, which escapes the message, help, and label text of every report it +/// prints. See the boundary table in `mds-core/src/lint/diagnostic.rs` for the full set. #[must_use] #[non_exhaustive] #[derive(Error, Debug, Diagnostic, Clone)] @@ -823,8 +851,13 @@ impl MdsError { let code = Diagnostic::code(self) .map(|c| c.to_string()) .unwrap_or_default(); - let message = self.to_string(); - let help = Diagnostic::help(self).map(|h| h.to_string()); + // WIRE mode: this value is consumed as JSON / as a binding error object, so + // `\n` is escaped too — an embedded newline would let a hostile message forge + // an extra line in any line-oriented consumer. The HUMAN counterpart is + // `display_sanitized()`, which keeps newlines raw for terminal rendering. + let message = sanitize_control_chars_wire(&self.to_string()).into_owned(); + let help = Diagnostic::help(self) + .map(|h| sanitize_control_chars_wire(&h.to_string()).into_owned()); // Extract (span, src) from each span-bearing variant; no-span variants // use the wildcard arm and produce span: None. @@ -880,6 +913,44 @@ impl MdsError { span: serialized_span, } } + + /// Return a terminal-safe, sanitized version of this error's `Display` text. + /// + /// The escaped class is the one spec §7.5 defines: C0 (U+0000–U+001F) with `\t` + /// (U+0009) as the sole exemption, DEL (U+007F), C1 (U+0080–U+009F), all twelve + /// Unicode bidi controls (U+061C, U+200E/U+200F, U+202A–U+202E, U+2066–U+2069), + /// U+2028/U+2029, and U+FEFF. Each is replaced by its six-character `\uXXXX` escape + /// literal. + /// + /// This is **HUMAN mode**, so `\n` — which *is* in the class — is preserved, keeping + /// multi-line miette renders readable. Whether `\n` is escaped is a property of the + /// mode, not of the class; describing the class as "C0 except `\t` and `\n`" folds + /// the two together and is exactly the framing spec §7.5 retired. That mode choice + /// is the one deliberate difference from [`MdsError::serialize`], which is a + /// machine-readable (wire) boundary and escapes `\n` as well. `\t` is preserved in + /// both modes. + /// + /// The escaping is one-way: see [`mds::sanitize_control_chars`][crate::sanitize_control_chars] + /// — consumers must not un-escape `\uXXXX` sequences back into bytes. + /// + /// Use this method — not `e.to_string()` / `eprintln!("{e}")` — whenever the + /// string will be written to a TTY or embedded in a user-visible context + /// without further escaping. See the [type-level doc][MdsError] for the full + /// Display-contract table. + /// + /// # Audience + /// + /// This is an affordance for **downstream Rust consumers** of the published crate + /// who render an `MdsError` themselves. It is deliberately not on the `mds` CLI's + /// render path: the CLI hands `miette::Report`s to `eprint_error`, which sanitizes + /// the message, help, and label text of *every* report — including CLI-authored + /// `miette::miette!()` errors that are not `MdsError`s at all, and so could never + /// be covered by this method. Routing the CLI through here instead would leave + /// that second family unescaped (PF-004). + #[must_use] + pub fn display_sanitized(&self) -> String { + sanitize_control_chars(&self.to_string()).into_owned() + } } #[cfg(test)] diff --git a/crates/mds-core/src/error_tests.rs b/crates/mds-core/src/error_tests.rs index 8e1db14d..5d8f7713 100644 --- a/crates/mds-core/src/error_tests.rs +++ b/crates/mds-core/src/error_tests.rs @@ -232,6 +232,228 @@ fn serialized_error_to_json_null_span() { assert!(v["span"].is_null(), "span should be null in JSON when None"); } +// ── T-1..T-3: serialize() sanitizes control chars (issue #176 / ESC-INJECTION) ── + +/// T-1 [AC-F3]: serialize() sanitizes raw ESC (U+001B) in the message field. +/// The message string must not contain the raw ESC byte; the sanitized 6-char +/// literal `\u001B` must appear instead. +#[test] +fn serialize_sanitizes_esc_in_message() { + // Build a Syntax error whose message embeds a raw ESC byte mid-string. + let e = MdsError::syntax("bad\x1Btoken"); + let s = e.serialize(); + assert!( + !s.message.contains('\x1B'), + "raw ESC byte must not appear in serialized message; got: {:?}", + s.message + ); + assert!( + s.message.contains("\\u001B"), + "sanitized literal \\u001B must appear in message; got: {:?}", + s.message + ); +} + +/// T-2 [AC-F3]: serialize() sanitizes raw ESC in both message and help fields. +/// UndefinedVariable embeds `name` in both the message ("undefined variable 'name'") +/// and the help ("define 'name' in frontmatter or imports"). +#[test] +fn serialize_sanitizes_esc_in_help() { + // The name `a\x1Bb` embeds an ESC byte — miette uses it in both fields. + let e = MdsError::undefined_var("a\x1Bb"); + let s = e.serialize(); + // Message must be sanitized. + assert!( + !s.message.contains('\x1B'), + "raw ESC byte must not appear in serialized message; got: {:?}", + s.message + ); + assert!( + s.message.contains("\\u001B"), + "sanitized literal \\u001B must appear in message; got: {:?}", + s.message + ); + // Help must also be sanitized. + let help = s.help.expect("UndefinedVariable should carry help text"); + assert!( + !help.contains('\x1B'), + "raw ESC byte must not appear in serialized help; got: {:?}", + help + ); + assert!( + help.contains("\\u001B"), + "sanitized literal \\u001B must appear in help; got: {:?}", + help + ); +} + +/// T-3 [AC-F3]: serialize() sanitizes DEL (U+007F) and C1 NEL (U+0085) in addition +/// to ESC, producing the corresponding `\uXXXX` literals. +#[test] +fn serialize_sanitizes_del_and_c1() { + let e = MdsError::syntax("del\u{007F}and\u{0085}nel"); + let s = e.serialize(); + // Raw DEL byte must be sanitized. + assert!( + !s.message.contains('\u{007F}'), + "raw DEL must not appear in serialized message; got: {:?}", + s.message + ); + assert!( + s.message.contains("\\u007F"), + "sanitized \\u007F must appear in message; got: {:?}", + s.message + ); + // Raw C1 NEL (U+0085) must be sanitized. + assert!( + !s.message.contains('\u{0085}'), + "raw C1 NEL must not appear in serialized message; got: {:?}", + s.message + ); + assert!( + s.message.contains("\\u0085"), + "sanitized \\u0085 must appear in message; got: {:?}", + s.message + ); +} + +/// T-3b [AC-F3]: `serialize()` escapes the widened class — bidi overrides +/// (Trojan Source, CVE-2021-42574), U+2028/U+2029, and U+FEFF — none of which +/// are C0/DEL/C1 and all of which previously passed straight through. +#[test] +fn serialize_sanitizes_bidi_separators_and_bom() { + let e = MdsError::syntax("rlo\u{202E}iso\u{2066}ls\u{2028}ps\u{2029}bom\u{FEFF}end"); + let s = e.serialize(); + for (ch, escaped) in [ + ('\u{202E}', "\\u202E"), + ('\u{2066}', "\\u2066"), + ('\u{2028}', "\\u2028"), + ('\u{2029}', "\\u2029"), + ('\u{FEFF}', "\\uFEFF"), + ] { + assert!( + !s.message.contains(ch), + "raw U+{:04X} must not appear in serialized message; got: {:?}", + ch as u32, + s.message + ); + assert!( + s.message.contains(escaped), + "sanitized {escaped} must appear in message; got: {:?}", + s.message + ); + } + // Non-vacuity: the surrounding prose survives. + assert!( + s.message.contains("rlo") && s.message.contains("end"), + "clean text must be preserved; got: {:?}", + s.message + ); +} + +/// T-3c [AC-F3]: `serialize()` is a WIRE boundary — an embedded newline (U+000A) +/// becomes its 6-char escape literal so a hostile message cannot forge an extra +/// line in a line-oriented consumer of `SerializedError.message`. +#[test] +fn serialize_escapes_newline_on_the_wire() { + let e = MdsError::syntax("a\nerror[mds::forged]: FAKE\nb"); + let s = e.serialize(); + assert!( + !s.message.contains('\n'), + "raw newline must not appear in serialized message; got: {:?}", + s.message + ); + assert!( + s.message.contains("\\u000A"), + "sanitized \\u000A must appear in serialized message; got: {:?}", + s.message + ); + // Non-vacuity: the message body itself is untouched. + assert!( + s.message.contains("error[mds::forged]"), + "message body must be preserved verbatim; got: {:?}", + s.message + ); +} + +// ── display_sanitized() ─────────────────────────────────────────────────── + +/// T-DS: `display_sanitized()` escapes raw ESC (U+001B) bytes in the terminal- +/// safe output while `to_string()` / `Display` leaves them raw. +/// +/// This test is the regression anchor for rust-5/architecture-2 (PF-004 on +/// the published API, CWE-150 / issue #176). It FAILS if `display_sanitized()` +/// is removed or reverted to a bare `self.to_string()` call (avoids PF-013). +#[test] +fn display_sanitized_escapes_esc_byte() { + let e = MdsError::syntax("bad\x1Btoken"); + let displayed = e.display_sanitized(); + assert!( + !displayed.contains('\x1B'), + "raw ESC byte must not appear in display_sanitized(); got: {:?}", + displayed + ); + // Positive assertion — FAILS if display_sanitized() reverts to to_string(). + assert!( + displayed.contains("\\u001B"), + "sanitized literal \\u001B must appear in display_sanitized(); got: {:?}", + displayed + ); +} + +/// T-DS-BIDI: `display_sanitized()` also covers the widened class — a bidi +/// override reaching a TTY reverses the visible order of the rest of the line. +#[test] +fn display_sanitized_escapes_bidi_override() { + let e = MdsError::syntax("bad\u{202E}token"); + let displayed = e.display_sanitized(); + assert!( + !displayed.contains('\u{202E}'), + "raw U+202E must not appear in display_sanitized(); got: {displayed:?}" + ); + assert!( + displayed.contains("\\u202E"), + "sanitized literal \\u202E must appear in display_sanitized(); got: {displayed:?}" + ); +} + +/// T-DS-NL: `display_sanitized()` is the HUMAN boundary — newlines stay raw so +/// multi-line miette frames remain readable. This is the deliberate asymmetry +/// with `serialize()` (see T-3c); pinning it here prevents an accidental +/// "sanitize everything the same way" regression. +#[test] +fn display_sanitized_preserves_newline() { + let e = MdsError::syntax("line one\nline two"); + let displayed = e.display_sanitized(); + assert!( + displayed.contains('\n'), + "display_sanitized() must preserve raw newlines; got: {displayed:?}" + ); + assert!( + !displayed.contains("\\u000A"), + "display_sanitized() must not escape newlines; got: {displayed:?}" + ); +} + +/// `display_sanitized()` and `to_string()` differ on ESC-bearing input, proving +/// that `display_sanitized()` is not a trivial alias for the raw Display impl. +#[test] +fn display_sanitized_differs_from_to_string_on_esc() { + let e = MdsError::syntax("msg\x1Bend"); + // Raw Display keeps the ESC byte. + assert!( + e.to_string().contains('\x1B'), + "to_string() must keep raw ESC (display contract); got: {:?}", + e.to_string() + ); + // Sanitized form must not. + assert!( + !e.display_sanitized().contains('\x1B'), + "display_sanitized() must not keep raw ESC; got: {:?}", + e.display_sanitized() + ); +} + // ── Display output ──────────────────────────────────────────────────────── #[test] diff --git a/crates/mds-core/src/evaluator.rs b/crates/mds-core/src/evaluator.rs index a94da7e2..6e8ffab2 100644 --- a/crates/mds-core/src/evaluator.rs +++ b/crates/mds-core/src/evaluator.rs @@ -1144,9 +1144,10 @@ fn evaluate_include( Some(body) => body.clone(), None => { if ctx.warnings.len() < MAX_WARNINGS { + // Untrusted identifier from template source — WIRE (spec 7.5). ctx.warnings.push(format!( "warning: @include of '{}' produced empty output — module has no body text", - inc.alias + crate::lint::sanitize_control_chars_wire(&inc.alias) )); } return Ok(String::new()); @@ -1300,9 +1301,10 @@ fn collect_messages_strict( Node::Include(inc) => { // @include in messages mode is not meaningful — warn. if ctx.warnings.len() < MAX_WARNINGS { + // Untrusted identifier from template source — WIRE (spec 7.5). ctx.warnings.push(format!( "warning: @include '{}' inside messages mode is ignored", - inc.alias + crate::lint::sanitize_control_chars_wire(&inc.alias) )); } } diff --git a/crates/mds-core/src/formatter.rs b/crates/mds-core/src/formatter.rs index cf9406c5..8876a41d 100644 --- a/crates/mds-core/src/formatter.rs +++ b/crates/mds-core/src/formatter.rs @@ -78,6 +78,7 @@ use std::sync::Arc; use crate::error::MdsError; use crate::lexer::{self, Token}; +use crate::lint::named_source_for_render; /// Format MDS source code, returning the rewritten source string. /// @@ -470,10 +471,9 @@ fn assert_equivalent( Err(MdsError::Syntax { message, span, .. }) => Err(MdsError::Syntax { message, span, - src: Some(Arc::new(miette::NamedSource::new( - file_name, - source.to_string(), - ))), + // Same shared boundary `MdsError::at()` uses: byte-length-preserving source + // neutralization (PF-014) plus WIRE-mode filename escaping. + src: Some(Arc::new(named_source_for_render(file_name, source))), }), // Any other compile failure (undefined var/fn, unresolved import, …) // means the token stream is well-formed and only later analysis failed, diff --git a/crates/mds-core/src/lib.rs b/crates/mds-core/src/lib.rs index e010bec1..fda9f46c 100644 --- a/crates/mds-core/src/lib.rs +++ b/crates/mds-core/src/lib.rs @@ -61,7 +61,8 @@ pub(crate) mod value; pub use formatter::{format_str, format_str_named, format_str_with}; pub use fs::{effective_parent, FileSystem, NativeFs, VirtualFs}; pub use lint::{ - fix, sanitize_control_chars, FixLineSpan, LintConfig, LintDiagnostic, LintResult, Severity, + fix, named_source_for_render, neutralize_source_for_render, sanitize_control_chars, + sanitize_control_chars_wire, FixLineSpan, LintConfig, LintDiagnostic, LintResult, Severity, }; pub use options::{ format_unknown_keys_error, json_type_name, parse_json_vars, reject_unknown_json_keys, VarsError, @@ -140,6 +141,14 @@ pub struct CompileResult { pub warnings: Vec, /// Normalized keys of all modules imported during compilation, in /// first-resolution (depth-first) order. Excludes the entry module. + /// + /// These are **functional path references**, not display text: bundler plugins feed + /// them straight back into a watcher. They are a named carve-out from the + /// sanitization rule (spec §7.5, "Carve-out: functional path references") and are + /// emitted verbatim by [`CompileResult::to_canonical_json`] — a key may contain any + /// byte a filesystem permits, control characters and `\n` included. Consumers that + /// display one must escape it themselves; [`sanitize_control_chars_wire`] applies the + /// same escaping the diagnostic surfaces use. pub dependencies: Vec, /// Source map for the compiled output. Present only when /// `CompileOptions::source_map` is `true` (AC-API-02: absent, not null, when off). @@ -179,11 +188,27 @@ impl CompileResult { /// The **inactive payload field is ABSENT** — a markdown result has no `messages` /// key; a messages result has no `output` key. Explicit field-by-field construction /// via `serde_json::json!()` prevents serde derive from injecting unwanted keys. + /// + /// # Sanitization + /// + /// `warnings` entries are WIRE-escaped here (spec §7.5): a warning is display text, + /// and an embedded `\n` would forge an extra warning line in a line-oriented + /// consumer. `output` / `messages` are the command's **product** and are byte-faithful. + /// `dependencies` and the embedded `sourceMap` are the **functional path reference** + /// carve-out — emitted verbatim, so their paths reach the consumer with whatever + /// bytes the filesystem allowed. See [`CompileResult::dependencies`] and + /// [`crate::SourceMap`]; consumers that display those paths must escape them. pub fn to_canonical_json(self) -> serde_json::Value { + // Sanitize warnings at the serialization boundary (issue #176 / CWE-150): + // warning strings may embed a hostile filename. WIRE mode — this value is + // consumed as JSON by the bindings, so an embedded newline could forge an + // extra warning line downstream. let warnings: serde_json::Value = self .warnings .into_iter() - .map(serde_json::Value::String) + .map(|w| { + serde_json::Value::String(crate::lint::sanitize_control_chars_wire(&w).into_owned()) + }) .collect::>() .into(); let dependencies: serde_json::Value = self @@ -504,9 +529,14 @@ fn path_to_str(path: &Path) -> Result<&str, MdsError> { } /// Print warnings to stderr. Each warning is printed on its own line. +/// +/// Sanitizes each warning before printing (issue #176 / CWE-150): warning strings +/// may embed a hostile filename, so control/bidi/separator characters are escaped to +/// `\uXXXX` literals. HUMAN mode — this is terminal output, so `\n` stays raw +/// (the wire counterpart is `CompileResult::to_canonical_json`, which escapes it). fn emit_warnings(warnings: &[String]) { for w in warnings { - eprintln!("{w}"); + eprintln!("{}", crate::lint::sanitize_control_chars(w)); } } diff --git a/crates/mds-core/src/lint/diagnostic.rs b/crates/mds-core/src/lint/diagnostic.rs index 9081fa15..0d938ada 100644 --- a/crates/mds-core/src/lint/diagnostic.rs +++ b/crates/mds-core/src/lint/diagnostic.rs @@ -4,13 +4,181 @@ //! be rendered by miette at the CLI human-render boundary. The `severity()` override //! maps our `Severity` enum to miette's rendering tiers (Error/Warning/Advice). //! -//! **Sanitization discipline**: `sanitize_control_chars` is a render-boundary helper. -//! It is NOT called in `LintDiagnostic` constructors — the raw message is preserved -//! intact for `LintResult::to_canonical_json()` (typed serialization is safe; C0/C1 -//! bytes in JSON string values are legal and the consumer can handle them). Apply -//! `sanitize_control_chars` only at the CLI human-render step (mds-cli/src/lint.rs). +//! **Sanitization discipline**: `message` and `help` are sanitized at every output +//! boundary. The escaped class — stated exactly as spec §7.5 states it — is C0 +//! (U+0000–U+001F) **including `\n`**, with `\t` (U+0009) as the sole exemption; DEL +//! (U+007F); C1 (U+0080–U+009F); the complete Unicode `Bidi_Control=Yes` set — all twelve +//! of U+061C, U+200E/U+200F, U+202A–U+202E, U+2066–U+2069 (Trojan Source, +//! CVE-2021-42574) — the JS line/paragraph separators U+2028/U+2029, and U+FEFF; +//! each becomes an uppercase 6-char `\uXXXX` literal. +//! +//! `\n` is *in* the class. Whether a given boundary escapes it is the HUMAN/WIRE mode +//! choice below, not a property of the class. Describing the class as "C0 except +//! `\n`/`\t`" folds the mode into the class definition and makes the two documents +//! disagree about what the class contains; they must not. +//! +//! Two escape modes share one implementation, differing only on `\n`: +//! +//! - **HUMAN** ([`sanitize_control_chars`]) — `\n` preserved. For terminal / miette +//! render output, where multi-line frames must stay readable. +//! - **WIRE** ([`sanitize_control_chars_wire`]) — `\n` escaped too, so a hostile +//! message cannot forge an extra line in a line-oriented consumer of the value +//! (log forging, YAML key injection). +//! +//! `\t` is preserved in both modes. +//! +//! **Mode is chosen per FIELD, not per surface** — the governing rule, re-ratified +//! 2026-07-26 and normative in spec §7.5: +//! +//! > On the **diagnostic** surfaces — the `"version": 1` JSON wire, CLI status and +//! > warning lines, and `[file:line:col]` frame headers — untrusted **identifiers, +//! > filenames and error causes are WIRE**, human terminal output included. **Prose** — +//! > a diagnostic message body or help body — stays **HUMAN** on terminal surfaces so +//! > multi-line frames keep rendering. +//! +//! The rule governs diagnostic output. Two categories of output are named carve-outs and +//! are not escaped at all — the command's **product** (compiled template output) and +//! **functional path references** (source-map `file`/`sources`/`sourcesContent`, +//! `CompileResult.dependencies`). Both are in "Scope of the table" below. +//! +//! The discriminator is whether the value is ever legitimately multi-line. A filename, an +//! `mds.json` rule name, a `--format` argument and an `io::Error` cause are each rendered +//! on exactly one line — a status line, or a `[file:line:col]` frame header — so a raw +//! `\n` in one only lets it forge a standalone line byte-identical in form to genuine +//! output (CWE-117); POSIX permits a newline inside a filename and the user never types +//! it. A diagnostic body genuinely is multi-line, so escaping its newlines would break +//! the frame. +//! +//! This rule supersedes the earlier "wire mode at exactly these four boundaries" +//! enumeration. Enumerations of boundaries went stale twice under review; a per-field +//! rule makes each new site decidable without re-deriving the list. +//! +//! ## Boundary table +//! +//! This table is an audit list of the sanitizing boundaries, not a proof of closure. +//! Read it together with "Scope of the table" below, which names the one category of +//! CLI output that is deliberately outside it. +//! +//! | Boundary | Mode | Fields | +//! |----------|------|--------| +//! | `eprint_error` (mds-cli/src/output.rs) | HUMAN | `message`, `help`, `LabeledSpan` text, and the whole auxiliary diagnostic graph (`source` cause chain, `related`, `diagnostic_source`) of **every** report rendered to stderr — see "CLI terminal path" below | +//! | `eprint_warning` (mds-cli/src/output.rs) | **prose HUMAN, interpolated identifiers/paths WIRE** | the warning body is escaped HUMAN by the helper; each value interpolated into it must additionally be WIRE-escaped by the caller (`safe_path` for paths, `safe_inline` for identifiers / config values / `io::Error` causes). HUMAN alone is not sufficient — it preserves `\n`, which is the CWE-117 forgery vector | +//! | `safe_inline()` (mds-cli/src/output.rs) | WIRE | any single-line untrusted value interpolated into a status, warning or error line: `mds.json` rule names and config paths, `--format` arguments, fix-rejection reasons, `io::Error` causes | +//! | `tests/print_discipline.rs` (mds-cli) | *enforcement, not a boundary* | fails CI if any print macro under `crates/mds-cli/src/**` interpolates a value that is not passed through one of the escape helpers, and applies the same rule to `format!`s nested inside `eprint_warning` calls. Exceptions live in an explicit allowlist with per-entry justifications | +//! | `emit_warnings()` (lib.rs) | HUMAN | warning strings printed to stderr on the non-collecting paths. The identifiers its producers interpolate — `resolver.rs`'s imported-module filename, `evaluator.rs`'s `@include` alias — are WIRE-escaped at construction, per the per-field rule | +//! | `named_source_for_render()` (this module) | **per field** | the single `NamedSource` builder used by `MdsError::at()` (error.rs), `check_equivalence` (formatter.rs) and `render_diag_human` (mds-cli/src/lint.rs): filename WIRE, source via `neutralize_source_for_render` (byte-length-preserving; avoids PF-014 caret desync) | +//! | `render_diag_human` (mds-cli/src/lint.rs) | HUMAN | `message`/`help` (its filename and source go through `named_source_for_render`) | +//! | `safe_path()` / `safe_file_display()` (mds-cli/src/output.rs) | WIRE | CLI status-line path display (`Clean:`, `Fixed:`, `Would fix:`, `Compiled to`, …) | +//! | `fix::FixOutcome::Rejected.reason` (fix.rs) | WIRE | construction-time: the `MdsError` `Display` embedded in a reverify-failure reason, so the value is display-safe for every consumer of the published `mds::fix` API (PF-004) | +//! | `MdsError::serialize()` (error.rs) | WIRE | `message`, `help` — covers all three bindings' error path | +//! | `LintResult::to_canonical_json()` (this module) | WIRE | `message`, `help`, `files[].file` key | +//! | `CompileResult::to_canonical_json()` (lib.rs) | WIRE | warning strings; *distinct method from `LintResult::to_canonical_json`, not a duplicate* | +//! | Python `LintResult::new()` via `sanitize_lint_value()` | WIRE | `message`, `help`, `file` — construction-time, so typed getters read pre-sanitized data (PF-004) | +//! | `--diff` preview output (mds-cli/src/output.rs) | neutralized, TTY-gated | source excerpts neutralized when stdout is a TTY; byte-faithful when piped, so redirected diffs stay applicable. **`--check` alone emits no preview text** — only status lines, which are unconditionally sanitized via `safe_path`. | +//! +//! **Scope of the table.** It covers every path that carries *untrusted text* — +//! diagnostic prose, warnings, filenames, identifiers, and rejection reasons. +//! +//! `watch.rs`'s lifecycle status lines (`Watching {}`, `Removed {} (source deleted)`, +//! `warning: could not remove {}: {e}`) were previously carved out here as a +//! pre-existing gap. **That carve-out is gone**: they now route through `safe_path` / +//! `safe_inline` / `eprint_warning` like every other CLI print, because leaving them out +//! would have meant an allowlist entry in `print_discipline.rs` — a deliberate hole in +//! the guard rather than a documented one. +//! +//! Three categories remain outside the table, deliberately and without a coverage claim: +//! +//! - **Untrusted values interpolated into a diagnostic MESSAGE BODY**, at either of its +//! two construction sites: +//! - `miette::miette!(…)` in the CLI, which interpolates `mds.json` values and +//! filesystem paths; and +//! - **`MdsError` message bodies in this crate**, which interpolate paths and +//! `io::Error` causes (`fs.rs`'s `cannot read {normalized}: {e}` and `invalid UTF-8 +//! in {normalized}: {e}`) and template identifiers (`parser_helpers.rs`'s `invalid +//! import alias: '{alias}'`). +//! +//! Both are rendered through `eprint_error`, which escapes message, help and label text +//! in HUMAN mode before miette sees them, so no raw control byte reaches stderr — but a +//! `\n` in an interpolated path or identifier survives *inside the rendered frame*. +//! Frame content is indented and `│`-prefixed by the renderer rather than emitted as a +//! bare status line, and that prefix survives `strip()`, so forged frame content cannot +//! masquerade as genuine status output the way an unescaped filename in a `Clean: …` +//! line could. It is a weaker surface than the status lines above — a known residual, +//! not a closed one, disclosed here and in spec §7.5 ("Residual: paths and identifiers +//! inside a message body"). +//! +//! Note the asymmetry this preserves: a path in a **diagnostic** `file` field — a +//! status line, a `[file:line:col]` header, the JSON `file` key — *is* WIRE-escaped on +//! every surface that renders one. The residual is specifically a path or identifier +//! that has been interpolated into prose, where the per-field rule makes the +//! surrounding body HUMAN. +//! - **Compiled template output** (`mds build -o -`, `mds lint --fix -`). That is the +//! command's product, not a diagnostic; escaping it would corrupt every redirect. +//! - **Functional path references**: the source-map `file`, `sources` and +//! `sourcesContent` fields — in the `mds build --source-map` sidecar and in the +//! `sourceMap` object embedded in [`crate::CompileResult::to_canonical_json`] — and the +//! `dependencies` array of that same method. These are emitted **verbatim**: a +//! filename containing a control byte, a `\n` or a bidi control reaches the consumer +//! unmodified, and JSON string encoding is not escaping (a decoded `"\n"` is a real +//! newline again). Devtools, bundlers and IDEs resolve `file` / `sources` against the +//! filesystem and the bundler plugins watch `dependencies`, so rewriting a path to a +//! `\uXXXX` literal would point at a path that does not exist — breaking resolution to +//! defend against a pathological filename. **Consumers MUST treat these paths as +//! untrusted and escape them for whatever destination they render them to.** Specified +//! in spec §7.5 ("Carve-out: functional path references"). The CLI does not depend on +//! this contract for its own output: `Compiled to …` and `Source map written to …` +//! print through `safe_path` and so carry the WIRE-escaped form. +//! +//! **CLI terminal path.** `mds build` / `check` / `fmt` / `lint` / `watch` all render +//! errors through the single `eprint_error` choke-point, which wraps the `miette::Report` +//! in a sanitizing view *before* miette renders it. This covers both CLI error families: +//! `MdsError` (whose messages interpolate template text, e.g. `parser.rs`'s +//! `invalid include alias: '{alias}'`) and CLI-authored `miette::miette!()` reports +//! (which interpolate `mds.json` values and filesystem paths, and do **not** downcast to +//! `MdsError`). Because it wraps at the `Report` level, an error type added later inherits +//! the guarantee without touching the boundary — the PF-004 failure mode of a check that +//! holds on one path and silently lapses on a sibling path cannot recur here. +//! +//! Sanitizing the renderer's *inputs* is mandatory; sanitizing its *output* is forbidden. +//! Running an escaper over an already-rendered miette frame escapes miette's own ANSI SGR +//! codes into literal noise on any colour-capable TTY and desynchronises caret alignment, +//! on entirely benign input — and CI cannot catch it, because the CLI tests pin +//! `NO_COLOR=1` and pipe stderr. That is PF-014, and an earlier round of #176 shipped and +//! reverted exactly that defect. The colour path is pinned instead by in-process unit +//! tests that select a theme explicitly (`output.rs`, `sanitize_report_*`). +//! +//! **Deliberate exclusions** (documented, not gaps): +//! - `LintDiagnostic::fmt` and `MdsError`'s derived `Display` (raw) — unsanitized by +//! design so machine-readable pipelines see exact bytes. No in-tree path renders +//! either one to a terminal *unescaped*: diagnostics go through `eprint_error`, and +//! the one place that embeds an `MdsError` `Display` into another user-visible string +//! — `fix::FixOutcome::Rejected.reason` — escapes it at construction (see the table). +//! Downstream Rust consumers of the published crate that print an `MdsError` +//! themselves should use [`MdsError::display_sanitized`], which applies this module's +//! HUMAN mode to the `Display` string. That helper is a **consumer-facing API, not a +//! CLI boundary** — the CLI's own guarantee comes from `eprint_error`. +//! - napi `err.detail` — populated only under the `debug-panics` Cargo feature, +//! which CLAUDE.md forbids shipping +//! +//! Fields NOT sanitized: `rule` (fixed identifiers), `span`/`fix_edits` byte offsets +//! (raw byte accuracy required for fix pipelines and span highlighting). +//! +//! Raw byte values are preserved in the stored `LintDiagnostic` struct so that span +//! offsets and fix-edits remain byte-accurate. Neither sanitizer is called in +//! `LintDiagnostic` constructors. +//! +//! **Escaping is one-way.** The transformation is lossy and non-injective: a template +//! that literally contains the six characters `\`,`u`,`0`,`0`,`1`,`B` and one that +//! contains an actual ESC byte are indistinguishable in the output. Consumers MUST NOT +//! un-escape `\uXXXX` sequences back into bytes — that reconstitutes exactly the +//! injection this guard prevents. Round-tripping is an explicit non-goal; no +//! backslash-escaping will be added to make the mapping reversible. A consumer that +//! needs original bytes must read them from the source via the raw `span` / +//! `fix_edits` byte offsets. +use std::borrow::Cow; use std::fmt; +use std::fmt::Write as _; use crate::error::SerializedSpan; use crate::limits::MAX_DIAGNOSTICS; @@ -114,19 +282,21 @@ impl FixLineSpan { /// A single lint finding. /// /// Implements `std::error::Error + miette::Diagnostic` so it can be rendered by -/// miette at the CLI boundary: `eprintln!("{:?}", miette::Report::from(diag))`. -/// The `severity()` override maps `Severity::Info` → Advice, `Warn` → Warning, +/// miette. The `severity()` override maps `Severity::Info` → Advice, `Warn` → Warning, /// `Error` → Error; `Off` diagnostics are never constructed (the lint engine filters /// them before collecting). /// -/// Attach a named source for miette span rendering: -/// ```rust,no_run -/// // At the CLI render boundary: -/// // let diag = diag.with_source(Arc::new(miette::NamedSource::new(filename, src))); -/// ``` +/// **CLI render**: always use `mds_cli::output::eprint_error` to render diagnostics on +/// a TTY — never call `eprintln!("{report:?}")` on a raw `miette::Report`. Writing the +/// rendered frame directly bypasses input-level sanitization and can inject C0/C1 +/// control bytes from hostile source content into the terminal (CWE-150 / PF-014). +/// Input fields (`message`, `help`, source excerpts) are sanitized before the Report +/// is constructed; post-rendering the frame must not be re-sanitized. /// /// **JSON**: use `LintResult::to_canonical_json()` — never construct JSON manually. -/// **Sanitization**: apply `sanitize_control_chars` at the CLI render boundary only. +/// **Sanitization**: see the module-level "Sanitization discipline" note — `message` +/// and `help` are sanitized at every output boundary; constructors keep raw bytes so +/// span offsets and `fix_edits` stay byte-accurate. pub struct LintDiagnostic { /// Short rule identifier, e.g. `"unused-variable"`. Becomes the miette code /// `mds::lint::`. @@ -134,7 +304,8 @@ pub struct LintDiagnostic { /// Effective severity of this finding (never `Off` — `Off` diagnostics are not /// collected). pub severity: Severity, - /// Human-readable finding description. Raw — do not sanitize in the constructor. + /// Human-readable finding description. Raw — do not sanitize in the constructor + /// (sanitized at output boundaries — see module docs). pub message: String, /// Optional fix hint shown below the message. pub help: Option, @@ -275,6 +446,7 @@ impl LintResult { /// /// **NEVER** build this JSON via `format!()` — use `serde_json::json!()` so /// control characters in message/help are serialized safely. + #[must_use] pub fn to_canonical_json(&self) -> serde_json::Value { use std::collections::BTreeMap; @@ -311,11 +483,17 @@ impl LintResult { .collect::>() }); + // Sanitize at the serialization boundary (issue #176 / CWE-150) in WIRE + // mode: message and help carry sanitized \uXXXX literals for control, + // bidi, separator and BOM characters — and for `\n`, so a hostile message + // cannot forge an extra line in a line-oriented consumer of this JSON. + // Spans and fix_edits reference raw byte offsets into the source — left + // untouched so fix pipelines and span highlighting stay accurate. let d = serde_json::json!({ "rule": diag.rule, "severity": diag.severity.to_string(), - "message": diag.message, - "help": diag.help, + "message": sanitize_control_chars_wire(&diag.message), + "help": diag.help.as_deref().map(sanitize_control_chars_wire), "fixable": (diag.fix_removals.is_some() || diag.fix_edits.is_some()) && super::tier::is_fixable(&diag.rule, self.is_standalone), "span": span_json, "fix_edits": fix_edits_json, @@ -324,11 +502,15 @@ impl LintResult { by_file.entry(key).or_default().push(d); } + // Sanitize the file key at the serialization boundary (issue #176 / CWE-150), + // WIRE mode: POSIX filenames may legally contain C0/DEL/C1 bytes, bidi + // controls, and even newlines. All surfaces that call to_canonical_json() + // inherit this fix without further changes. let files: Vec = by_file .into_iter() .map(|(file, diagnostics)| { serde_json::json!({ - "file": file, + "file": sanitize_control_chars_wire(&file), "diagnostics": diagnostics, }) }) @@ -388,34 +570,340 @@ impl LintResultBuilder { // ── sanitize_control_chars ──────────────────────────────────────────────────── -/// Strip or escape C0 (U+0000–U+001F incl. ESC), DEL (U+007F), and C1 -/// (U+0080–U+009F) control characters from a string, except `\n` (U+000A) -/// and `\t` (U+0009). -/// -/// Applied ONLY at the CLI human-render boundary — NOT in `LintDiagnostic` -/// constructors. The raw message is preserved in `to_canonical_json()` output -/// because typed JSON serialization escapes control characters safely, and mutating -/// the constructor would corrupt the LSP-stable wire format. -/// -/// Replacement strategy: replace each control character with its Unicode escape -/// `\uXXXX` to make the rendered text visually safe on terminals without silently -/// dropping information that a developer might need to diagnose rule logic. -/// DEL (U+007F) is included because some terminals interpret it as a backspace, -/// which can corrupt human-readable output. -pub fn sanitize_control_chars(s: &str) -> String { +/// Which escape class a sanitizer call applies. +/// +/// Both variants escape the same hostile character class (see [`is_control_char`]); +/// they differ only in how they treat `\n` (U+000A). `\t` (U+0009) is preserved by +/// both — a tab cannot forge a line and cannot reposition a cursor destructively. +#[derive(Clone, Copy, PartialEq, Eq)] +enum EscapeMode { + /// Terminal / miette render output. `\n` is preserved so multi-line diagnostic + /// frames stay readable. + Human, + /// Machine-readable output (JSON wire, binding error objects). `\n` is escaped + /// as well, so a hostile message cannot forge an extra line in a line-oriented + /// consumer of the string value (log forging, YAML key injection). + Wire, +} + +/// Escape hostile characters, preserving `\n` (HUMAN mode). +/// +/// Escapes the full class — C0 (U+0000–U+001F incl. ESC) with `\t` exempt, DEL +/// (U+007F), C1 (U+0080–U+009F), all twelve Unicode bidi controls (U+061C, +/// U+200E/U+200F, U+202A–U+202E, U+2066–U+2069), the line/paragraph separators +/// U+2028/U+2029, and U+FEFF — **except `\n`, which this mode preserves**. `\n` is in +/// the class; HUMAN mode is the choice not to escape it. See the module doc. +/// +/// Use this at **render** boundaries (anything bound for a terminal or a miette +/// frame). Use [`sanitize_control_chars_wire`] at **wire** boundaries (JSON, binding +/// error objects), where an embedded newline is itself an injection vector. +/// +/// Returns a borrowed view of the input when no escaping is needed (zero +/// allocation for the overwhelmingly-common clean case). Allocates only when +/// a hostile character is actually present, reserving exact capacity. +/// +/// Applied at every output boundary — see the module-level "Sanitization discipline" +/// note for the authoritative list. NOT called in `LintDiagnostic` constructors so that +/// span offsets and fix-edit byte ranges remain accurate against the raw source. Raw +/// bytes in the stored struct; sanitized literals in all output. +/// +/// Replacement strategy: replace each hostile character with its Unicode escape +/// `\uXXXX` (uppercase hex, 4 digits, literal backslash) to make the rendered +/// text visually safe on terminals without silently dropping information that a +/// developer might need to diagnose rule logic. DEL (U+007F) is included because +/// some terminals interpret it as a backspace, which can corrupt human-readable +/// output. The bidi controls are included because they can visually reorder a +/// diagnostic line (Trojan Source, CVE-2021-42574). The function is idempotent — +/// calling it twice on already-sanitized text is a no-op. +/// +/// # Escaping is one-way +/// +/// This transformation is **lossy and non-injective**: a template that literally +/// contains the six characters `\`,`u`,`0`,`0`,`1`,`B` and an actual ESC byte both +/// serialize to the same `\u001B` output. Consumers **MUST NOT** un-escape `\uXXXX` +/// sequences back into bytes — doing so re-creates exactly the injection this guard +/// exists to prevent. The escape is for display only; when a consumer needs the +/// original bytes it must read the source through `span`/`fix_edits` byte offsets, +/// which are deliberately left raw. No backslash-escaping (`\` → `\\`) will be added +/// to make the mapping reversible; round-tripping is an explicit non-goal. +/// +/// # Examples +/// +/// ``` +/// use mds::sanitize_control_chars; +/// +/// // ESC (U+001B) is escaped to the 6-char uppercase literal. +/// assert_eq!(&*sanitize_control_chars("\x1B[33m"), "\\u001B[33m"); +/// +/// // \n and \t are preserved in HUMAN mode. +/// assert_eq!(&*sanitize_control_chars("hello\nworld"), "hello\nworld"); +/// +/// // DEL (U+007F) is escaped. +/// assert_eq!(&*sanitize_control_chars("a\x7Fb"), "a\\u007Fb"); +/// +/// // C1 control NEL (U+0085) is escaped. +/// assert_eq!(&*sanitize_control_chars("a\u{0085}b"), "a\\u0085b"); +/// +/// // Bidi override (U+202E RLO — Trojan Source) is escaped. +/// assert_eq!(&*sanitize_control_chars("a\u{202E}b"), "a\\u202Eb"); +/// +/// // So is U+061C ARABIC LETTER MARK — the one Bidi_Control codepoint outside +/// // U+200E–U+2069, and the only 2-byte member of the class. +/// assert_eq!(&*sanitize_control_chars("a\u{061C}b"), "a\\u061Cb"); +/// +/// // JS line separator (U+2028) and BOM (U+FEFF) are escaped. +/// assert_eq!(&*sanitize_control_chars("a\u{2028}b"), "a\\u2028b"); +/// assert_eq!(&*sanitize_control_chars("a\u{FEFF}b"), "a\\uFEFFb"); +/// +/// // Clean input is borrowed — zero allocation. +/// let s = "normal text"; +/// let cow = sanitize_control_chars(s); +/// assert!(matches!(cow, std::borrow::Cow::Borrowed(_))); +/// +/// // Idempotent: a second call on already-sanitized output is a no-op. +/// let once = sanitize_control_chars("a\x1Bb"); +/// let twice = sanitize_control_chars(&once); +/// assert_eq!(once, twice); +/// ``` +#[must_use] +pub fn sanitize_control_chars(s: &str) -> Cow<'_, str> { + sanitize_with(s, EscapeMode::Human) +} + +/// Escape hostile characters **including `\n`** (WIRE mode). +/// +/// Identical to [`sanitize_control_chars`] except that `\n` (U+000A) is also escaped +/// to its 6-character literal. `\t` is still preserved. +/// +/// Use this at machine-readable boundaries — `MdsError::serialize()`, +/// `LintResult::to_canonical_json()`, `CompileResult::to_canonical_json()` warnings, +/// and the Python typed-surface construction path. A raw newline inside a JSON string +/// value is legal JSON, but once a consumer prints or line-splits that value a hostile +/// message can forge an entire extra diagnostic line (log forging / YAML key +/// injection). Escaping it makes the value single-line by construction. +/// +/// The one-way-escaping contract in [`sanitize_control_chars`] applies verbatim here. +/// +/// # Examples +/// +/// ``` +/// use mds::{sanitize_control_chars, sanitize_control_chars_wire}; +/// +/// // WIRE escapes the newline; HUMAN keeps it. +/// assert_eq!(&*sanitize_control_chars_wire("a\nb"), "a\\u000Ab"); +/// assert_eq!(&*sanitize_control_chars("a\nb"), "a\nb"); +/// +/// // \t is preserved in both modes. +/// assert_eq!(&*sanitize_control_chars_wire("a\tb"), "a\tb"); +/// +/// // Everything else escapes identically in both modes. +/// assert_eq!(&*sanitize_control_chars_wire("a\u{202E}b"), "a\\u202Eb"); +/// +/// // Clean input is borrowed — zero allocation. +/// assert!(matches!( +/// sanitize_control_chars_wire("normal text"), +/// std::borrow::Cow::Borrowed(_) +/// )); +/// ``` +#[must_use] +pub fn sanitize_control_chars_wire(s: &str) -> Cow<'_, str> { + sanitize_with(s, EscapeMode::Wire) +} + +/// Single escape implementation shared by both public entry points. +/// +/// Kept as one function on purpose: a second, forked escape map would be a PF-004 +/// parallel path — the two would drift and one boundary would silently stop +/// enforcing what the other does. +fn sanitize_with(s: &str, mode: EscapeMode) -> Cow<'_, str> { + // Byte-level fast path: scan for any byte that can start an escaped character. + // - C0 (U+0000–U+001F) and DEL (U+007F) are single bytes: b < 0x20 or b == 0x7F. + // - C1 (U+0080–U+009F) in UTF-8 is encoded as 0xC2 0x80–0xC2 0x9F. + // - U+061C is encoded as 0xD8 0x9C. + // - U+200E/U+200F, U+2028/U+2029, U+202A–U+202E and U+2066–U+2069 all start + // with 0xE2; U+FEFF starts with 0xEF. + // The scan is a deliberate over-approximation (0xC2/0xD8/0xE2/0xEF also lead many + // benign codepoints); false positives only cost a trip through the char loop + // below, which leaves non-hostile characters unchanged. + let needs_work = s + .bytes() + .any(|b| b < 0x20 || b == 0x7F || b == 0xC2 || b == 0xD8 || b == 0xE2 || b == 0xEF); + if !needs_work { + return Cow::Borrowed(s); + } + + // Count the escapes so we can reserve exactly: each escaped char takes 6 output + // bytes (\uXXXX) instead of 1–3 input bytes, a net growth of at most 5 bytes. + let n_escaped = s.chars().filter(|&ch| escapes_in(ch, mode)).count(); + let mut out = String::with_capacity(s.len() + 5 * n_escaped); + + // Bulk-copy clean runs; replace each hostile char with its \uXXXX literal. + let mut bulk_start = 0; + for (i, ch) in s.char_indices() { + if escapes_in(ch, mode) { + out.push_str(&s[bulk_start..i]); + // \uXXXX: literal backslash + u + 4 uppercase hex digits. Every escaped + // codepoint is in the BMP, so 4 digits is always exact. + write!(out, "\\u{:04X}", ch as u32).expect("writing to a String is infallible"); + bulk_start = i + ch.len_utf8(); + } + } + // Flush the final clean segment. + out.push_str(&s[bulk_start..]); + Cow::Owned(out) +} + +/// Returns `true` when `ch` must be escaped under `mode`. +#[inline] +fn escapes_in(ch: char, mode: EscapeMode) -> bool { + is_control_char(ch) || (mode == EscapeMode::Wire && ch == '\n') +} + +/// Returns `true` for codepoints escaped in **both** modes. +/// +/// Also the predicate driving [`neutralize_source_for_render`], so the render path +/// and the escape path can never diverge on which characters are hostile (PF-004). +#[inline] +fn is_control_char(ch: char) -> bool { + (ch < '\u{0020}' && ch != '\n' && ch != '\t') + || ch == '\u{007F}' + || ('\u{0080}'..='\u{009F}').contains(&ch) + || is_format_hazard_char(ch) +} + +/// Returns `true` for the non-C0/C1 codepoints that are still display-hazardous. +/// +/// The class is split by **UTF-8 byte width**, not by hazard category, because +/// [`neutralize_source_for_render`] must substitute a replacement of identical byte +/// length and therefore needs a different replacement per width. Splitting the +/// predicate is what keeps that invariant checkable by reading the code rather than +/// by trusting a comment: a member added to the wrong helper is a byte-width bug the +/// `debug_assert_eq!` in `neutralize_source_for_render` catches immediately. +/// +/// See [`is_two_byte_format_hazard`] and [`is_three_byte_format_hazard`] for the +/// per-width membership and the rationale for each codepoint. +#[inline] +fn is_format_hazard_char(ch: char) -> bool { + is_two_byte_format_hazard(ch) || is_three_byte_format_hazard(ch) +} + +/// Display-hazardous codepoints that occupy **2 bytes** in UTF-8 (U+0080–U+07FF). +/// +/// - **U+061C** — ARABIC LETTER MARK. One of the twelve codepoints with the Unicode +/// `Bidi_Control=Yes` property, and the only one outside the U+200E–U+2069 range. +/// It reorders how the rest of a line renders exactly like its U+200E/U+200F +/// siblings (Trojan Source, CVE-2021-42574), so omitting it would leave a hole in +/// the bidi class that the other eleven members close. +/// +/// Members here are neutralized to U+00A0 NBSP (also 2 bytes), the same replacement +/// the C1 range uses — **not** U+FFFD, which is 3 bytes and would break the +/// byte-length invariant. +#[inline] +fn is_two_byte_format_hazard(ch: char) -> bool { + ch == '\u{061C}' +} + +/// Display-hazardous codepoints that occupy **3 bytes** in UTF-8 (U+0800–U+FFFF). +/// +/// - **U+200E/U+200F, U+202A–U+202E, U+2066–U+2069** — the remaining eleven Unicode +/// bidirectional controls (marks, embeddings, overrides, isolates). They reorder how +/// the rest of a line renders, which is the Trojan Source attack (CVE-2021-42574): a +/// diagnostic or filename can be made to display as something entirely different from +/// its bytes. +/// - **U+2028/U+2029** — LINE SEPARATOR / PARAGRAPH SEPARATOR. Both terminate a +/// JavaScript string literal, so an unescaped one can break out of generated JS. +/// - **U+FEFF** — BOM / ZERO WIDTH NO-BREAK SPACE. Invisible in every renderer, so it +/// can hide or split content the reader believes is contiguous. +/// +/// Members here are neutralized to U+FFFD (also 3 bytes). +#[inline] +fn is_three_byte_format_hazard(ch: char) -> bool { + matches!(ch, + '\u{200E}' | '\u{200F}' + | '\u{2028}' | '\u{2029}' + | '\u{202A}'..='\u{202E}' + | '\u{2066}'..='\u{2069}' + | '\u{FEFF}' + ) +} + +/// Replace control characters in source text with byte-length-preserving substitutes so +/// that miette's span byte-offsets and caret columns remain accurate (avoids PF-014). +/// +/// This function is the input-sanitization companion to [`sanitize_control_chars`]. +/// It MUST be applied to any source string passed to [`miette::NamedSource`] before +/// the Report is rendered. Applying [`sanitize_control_chars`] instead would expand +/// each control char to 6 bytes (`\uXXXX`), desynchronising every span byte-offset +/// that follows the substitution point and producing misaligned carets. +/// +/// It neutralizes exactly the character class [`sanitize_control_chars`] escapes — +/// the two are kept symmetric on purpose so the render path can never lag the wire +/// path on a newly-recognised hostile character (PF-004). +/// +/// Substitution rules (byte-length-preserving): +/// - C0 bytes (U+0000–U+001F) except `\n`/`\t`: 1-byte → `?` (U+003F, 1 byte) +/// - DEL (U+007F): 1-byte → `?` +/// - C1 range (U+0080–U+009F) and U+061C (both 2-byte UTF-8): → U+00A0 NBSP (2 bytes) +/// - Remaining bidi controls, U+2028/U+2029, U+FEFF (3-byte UTF-8): → U+FFFD (3 bytes) +/// +/// Returns [`Cow::Borrowed`] when no substitution is needed (fast path). +pub fn neutralize_source_for_render(s: &str) -> Cow<'_, str> { + let needs_neutralize = s.chars().any(is_control_char); + if !needs_neutralize { + return Cow::Borrowed(s); + } + // Allocate once; capacity is exact because every substitution preserves byte length. let mut out = String::with_capacity(s.len()); - for ch in s.chars() { - let is_c0 = ch < '\u{0020}' && ch != '\n' && ch != '\t'; - let is_del = ch == '\u{007F}'; - let is_c1 = ('\u{0080}'..='\u{009F}').contains(&ch); - if is_c0 || is_del || is_c1 { - // Replace with Unicode escape so the byte is visible but harmless. - let _ = fmt::write(&mut out, format_args!("\\u{:04X}", ch as u32)); + for c in s.chars() { + let u = c as u32; + if (u < 0x20 && c != '\n' && c != '\t') || u == 0x7F { + out.push('?'); // 1-byte C0/DEL → '?' (1 byte) — byte-length-preserving + } else if (0x80..=0x9F).contains(&u) || is_two_byte_format_hazard(c) { + // 2-byte C1 / U+061C → U+00A0 NBSP (2 bytes) — byte-length-preserving. + out.push('\u{00A0}'); + } else if is_three_byte_format_hazard(c) { + // 3-byte bidi/separator/BOM → U+FFFD (3 bytes) — byte-length-preserving. + out.push('\u{FFFD}'); } else { - out.push(ch); + out.push(c); } } - out + debug_assert_eq!( + out.len(), + s.len(), + "neutralize_source_for_render must preserve byte length" + ); + Cow::Owned(out) +} + +/// Build the [`miette::NamedSource`] for a diagnostic frame, applying the sanitization +/// each of its two halves requires. +/// +/// The two halves need **different** treatments, and getting them the wrong way round +/// is a real defect in both directions — which is why every in-tree site that hands a +/// filename plus source text to miette goes through this one function instead of +/// open-coding the pair (avoids PF-004 parallel-path drift): +/// +/// - **`file`** — [`sanitize_control_chars_wire`] (WIRE). A filename is prose that is +/// rendered on a single line, both in miette's `[file:line:col]` frame header and in +/// the CLI's own status lines. POSIX permits `\n` inside a filename, so HUMAN mode — +/// which preserves `\n` so multi-line diagnostic *messages* stay readable — would let +/// a file named `evil.mds\nClean: real.mds` emit an attacker-authored line that is +/// byte-identical in form to genuine status output (CWE-117 log forging). A filename +/// is never legitimately multi-line, so escaping `\n` costs nothing. +/// - **`source`** — [`neutralize_source_for_render`] (byte-length-preserving). The +/// source text is span-indexed: `sanitize_control_chars*` expands a 1–2-byte control +/// to a 6-byte `\uXXXX` literal and desynchronises every following span offset and +/// caret column (PF-014). +/// +/// Both halves are sanitized *before* the `Report` is built. The rendered frame is +/// never post-processed — see the module-level "Sanitization discipline" note. +#[must_use] +pub fn named_source_for_render(file: &str, source: &str) -> miette::NamedSource { + miette::NamedSource::new( + sanitize_control_chars_wire(file).as_ref(), + neutralize_source_for_render(source).into_owned(), + ) } // ── Unit tests ──────────────────────────────────────────────────────────────── @@ -470,18 +958,443 @@ mod tests { assert!(output.contains("\\u0085")); } - /// L-U-H2 regression: raw message is NOT sanitized in to_canonical_json — - /// JSON serialization handles control characters safely via `serde_json`. + /// T-4 [AC-F4, AC-C3]: `to_canonical_json()` sanitizes control chars in diagnostic + /// message and help. Simulates a `unused-variable` diagnostic whose message embeds + /// a raw ESC byte (e.g. from a hostile frontmatter key like `"a\u001Bb"`). + /// + /// After the fix: the JSON message AND help carry the sanitized `\uXXXX` literal + /// (uppercase, exactly 4 digits). Span offsets (raw byte positions) are unchanged. #[test] - fn canonical_json_raw_message_preserved() { + fn to_canonical_json_sanitizes_diagnostic_message() { + // Simulate an unused-variable diagnostic whose variable name contains U+001B. + let hostile_name = "a\x1Bb"; let result = LintResult { diagnostics: vec![LintDiagnostic { - rule: "test-rule".to_string(), + rule: "unused-variable".to_string(), severity: Severity::Warn, - message: "msg\x1Bwith\x00controls".to_string(), + message: format!( + "Variable '{}' is defined in frontmatter but never referenced in the body.", + hostile_name + ), + // Help also embeds the hostile name: removing the `map(sanitize_control_chars)` + // call for help would leave the raw ESC byte in the output and fail below. + help: Some(format!( + "Remove the key '{}' from frontmatter or reference it in the template body.", + hostile_name + )), + span: Some(crate::error::SerializedSpan { + offset: 4, + length: 3, + line: None, + column: None, + }), + file: Some("test.mds".to_string()), + fix_removals: None, + fix_edits: None, + }], + truncated: false, + is_standalone: false, + }; + + let json = result.to_canonical_json(); + let diag = &json["files"][0]["diagnostics"][0]; + + // Wire format shape must be intact. + assert_eq!(json["version"], 1, "version field must be 1"); + assert_eq!(json["truncated"], false, "truncated must be false"); + assert_eq!(json["files"][0]["file"], "test.mds"); + + let msg = diag["message"].as_str().unwrap(); + // Raw ESC byte must not appear in the serialized message. + assert!( + !msg.contains('\x1B'), + "raw ESC byte must not appear in to_canonical_json message; got: {msg:?}" + ); + // Sanitized 6-char uppercase literal must appear (no lowercase alternative). + assert!( + msg.contains("\\u001B"), + "sanitized literal \\u001B must appear in to_canonical_json message; got: {msg:?}" + ); + + let help = diag["help"].as_str().unwrap(); + // Help field must also be sanitized — pins the help-sanitize call (avoids PF-013). + assert!( + !help.contains('\x1B'), + "raw ESC byte must not appear in to_canonical_json help; got: {help:?}" + ); + assert!( + help.contains("\\u001B"), + "sanitized literal \\u001B must appear in to_canonical_json help; got: {help:?}" + ); + + // Span offset must be byte-accurate (not corrupted by sanitization). + assert_eq!( + diag["span"]["offset"], 4, + "span offset must be unchanged after message sanitization" + ); + assert_eq!( + diag["span"]["length"], 3, + "span length must be unchanged after message sanitization" + ); + } + + /// [testing-7]: `sanitize_control_chars` is idempotent — a second call on + /// already-sanitized output is always a no-op. This property is relied on by the + /// double-pass in `render_diag_human` (field-level + whole-frame). + #[test] + fn sanitize_is_idempotent() { + let cases: &[&str] = &[ + "\x1B", + "a\x1Bb", + "a\u{007F}b", + "a\u{0085}b", + "\x00\x01\x02\x1F", + "hello world", + "", + ]; + for &input in cases { + let once = sanitize_control_chars(input); + let twice = sanitize_control_chars(&once); + assert_eq!( + once, twice, + "sanitize_control_chars is not idempotent for input {input:?}" + ); + } + } + + // ── T-16: widened escape class — bidi / separator / BOM (issue #176) ───── + // + // These codepoints are outside C0/DEL/C1 but are still display-hazardous: + // - The twelve Unicode `Bidi_Control=Yes` codepoints — U+061C, U+200E/U+200F, + // U+202A–U+202E and U+2066–U+2069 — are the controls behind Trojan Source + // (CVE-2021-42574): they can visually reorder a diagnostic so a benign-looking + // line renders as something else entirely. + // - U+2028/U+2029 terminate a JavaScript string literal, so an unescaped one + // inside a diagnostic message can break out of generated JS. + // - U+FEFF (BOM / ZWNBSP) is invisible and can hide content in any consumer. + + /// The complete Unicode `Bidi_Control=Yes` set (12 codepoints) with the escaped + /// literal each must produce. Shared by the escape and neutralize tests so the two + /// paths are pinned against one list and cannot diverge on a member. + const BIDI_CONTROLS: &[(char, &str)] = &[ + ('\u{061C}', "\\u061C"), // ARABIC LETTER MARK (2 bytes in UTF-8) + ('\u{200E}', "\\u200E"), // LEFT-TO-RIGHT MARK + ('\u{200F}', "\\u200F"), // RIGHT-TO-LEFT MARK + ('\u{202A}', "\\u202A"), // LEFT-TO-RIGHT EMBEDDING + ('\u{202B}', "\\u202B"), // RIGHT-TO-LEFT EMBEDDING + ('\u{202C}', "\\u202C"), // POP DIRECTIONAL FORMATTING + ('\u{202D}', "\\u202D"), // LEFT-TO-RIGHT OVERRIDE + ('\u{202E}', "\\u202E"), // RIGHT-TO-LEFT OVERRIDE (Trojan Source) + ('\u{2066}', "\\u2066"), // LEFT-TO-RIGHT ISOLATE + ('\u{2067}', "\\u2067"), // RIGHT-TO-LEFT ISOLATE + ('\u{2068}', "\\u2068"), // FIRST STRONG ISOLATE + ('\u{2069}', "\\u2069"), // POP DIRECTIONAL ISOLATE + ]; + + /// T-16a: every bidi override / isolate / mark codepoint is escaped to its + /// uppercase 6-char `\uXXXX` literal, and the raw codepoint is gone. + /// + /// Covers the whole `Bidi_Control=Yes` property, including U+061C — the only member + /// outside U+200E–U+2069, and the one the class originally missed (#176). + #[test] + fn sanitize_escapes_bidi_control_chars() { + // Non-vacuity: the table really is the complete Unicode property, not a subset + // that happens to match whatever the implementation covers. + assert_eq!( + BIDI_CONTROLS.len(), + 12, + "Unicode defines exactly 12 Bidi_Control=Yes codepoints" + ); + for &(ch, expected) in BIDI_CONTROLS { + for out in [ + sanitize_control_chars(&format!("a{ch}b")), + sanitize_control_chars_wire(&format!("a{ch}b")), + ] { + assert!( + !out.contains(ch), + "raw U+{:04X} must not survive sanitization; got: {out:?}", + ch as u32 + ); + assert_eq!( + &*out, + format!("a{expected}b"), + "U+{:04X} must escape to {expected}", + ch as u32 + ); + } + } + } + + /// T-16a-WIDTH [#176]: `neutralize_source_for_render` preserves byte length for + /// every bidi control — the invariant the widened class most easily breaks. + /// + /// U+061C is 2 bytes in UTF-8 while the other eleven are 3. Routing it through the + /// 3-byte branch (→ U+FFFD) would grow the string by one byte per occurrence, + /// desynchronising every following span offset. The `debug_assert_eq!` inside + /// `neutralize_source_for_render` fires on that; this test pins it from outside so + /// the guarantee is also checked as an observable output property. + #[test] + fn neutralize_preserves_byte_length_for_every_bidi_control() { + for &(ch, _) in BIDI_CONTROLS { + let raw = format!("let x{ch} = 1;"); + let out = neutralize_source_for_render(&raw); + assert_eq!( + out.len(), + raw.len(), + "U+{:04X} ({} bytes) must be replaced by a same-width substitute; got: {out:?}", + ch as u32, + ch.len_utf8() + ); + assert!( + !out.contains(ch), + "raw U+{:04X} must not survive neutralization; got: {out:?}", + ch as u32 + ); + // Positive: the width-appropriate replacement, not merely "something else". + let expected = if ch.len_utf8() == 2 { + '\u{00A0}' + } else { + '\u{FFFD}' + }; + assert!( + out.contains(expected), + "U+{:04X} must neutralize to U+{:04X}; got: {out:?}", + ch as u32, + expected as u32 + ); + // Non-vacuity: the surrounding source is untouched. + assert!( + out.contains("let x") && out.contains(" = 1;"), + "non-vacuity: surrounding source must survive; got: {out:?}" + ); + } + } + + /// T-16b: U+2028 LINE SEPARATOR and U+2029 PARAGRAPH SEPARATOR are escaped. + /// Both terminate a JS string literal, so they must never reach a consumer raw. + #[test] + fn sanitize_escapes_line_and_paragraph_separators() { + assert_eq!(&*sanitize_control_chars("a\u{2028}b"), "a\\u2028b"); + assert_eq!(&*sanitize_control_chars("a\u{2029}b"), "a\\u2029b"); + } + + /// T-16c: U+FEFF (BOM / ZWNBSP) is escaped — it is invisible in every renderer. + #[test] + fn sanitize_escapes_bom() { + assert_eq!(&*sanitize_control_chars("a\u{FEFF}b"), "a\\uFEFFb"); + } + + /// T-16d: `neutralize_source_for_render` must handle the widened class + /// symmetrically (PF-004: no wire/render parallel-path gap) while keeping the + /// byte-length invariant the T-10a anchor pins. Every new codepoint is 3-byte + /// UTF-8, and U+FFFD is also 3 bytes. + #[test] + fn neutralize_source_replaces_bidi_and_separators_byte_for_byte() { + let raw = "let x\u{202E} = 1;\u{2028}next\u{FEFF}line\u{2066}end"; + let out = neutralize_source_for_render(raw); + assert_eq!( + out.len(), + raw.len(), + "neutralize_source_for_render must preserve byte length; raw={raw:?} out={out:?}" + ); + for ch in ['\u{202E}', '\u{2028}', '\u{FEFF}', '\u{2066}'] { + assert!( + !out.contains(ch), + "raw U+{:04X} must not survive neutralization; got: {out:?}", + ch as u32 + ); + } + assert_eq!( + out.matches('\u{FFFD}').count(), + 4, + "each neutralized format char must become U+FFFD; got: {out:?}" + ); + // Non-vacuity: surrounding source text is untouched. + assert!(out.contains("let x"), "clean source must survive: {out:?}"); + assert!(out.contains("next"), "clean source must survive: {out:?}"); + } + + // ── T-NS: named_source_for_render — the shared filename/source boundary (#176) ── + + /// T-NS-1 [S14 / CWE-117 / PF-013]: a newline-bearing FILENAME is escaped, so it + /// cannot forge an extra line in miette's `[file:line:col]` frame header. + /// + /// Vector: POSIX permits `\n` inside a filename, and the user never types the name — + /// `mds lint .` discovers it by directory walk. HUMAN mode (which the filename used + /// to get) preserves newlines by design, so the forged text survived verbatim. + #[test] + fn named_source_escapes_newline_in_filename() { + let hostile = "evil.mds\nClean: real.mds"; + let ns = named_source_for_render(hostile, "body\n"); + + // Non-vacuity: the real part of the filename is still there. + assert!( + ns.name().contains("evil.mds"), + "non-vacuity: the filename must still render; got: {:?}", + ns.name() + ); + // Negative: no raw newline survives, so no second line can be forged. + assert!( + !ns.name().contains('\n'), + "a filename must be single-line after sanitization; got: {:?}", + ns.name() + ); + // Positive: escaped to the uppercase 6-char literal (WIRE mode). + assert!( + ns.name().contains("\\u000A"), + "the newline must be escaped to its \\u000A literal; got: {:?}", + ns.name() + ); + } + + /// T-NS-2 [#176]: the same helper escapes the widened hazard class in the filename, + /// including U+061C, and still escapes ESC. + #[test] + fn named_source_escapes_hazard_class_in_filename() { + let ns = named_source_for_render("a\u{1b}b\u{061C}c\u{202E}d.mds", "body\n"); + for (raw, escaped) in [ + ('\u{1b}', "\\u001B"), + ('\u{061C}', "\\u061C"), + ('\u{202E}', "\\u202E"), + ] { + assert!( + !ns.name().contains(raw), + "raw U+{:04X} must not survive in a filename; got: {:?}", + raw as u32, + ns.name() + ); + assert!( + ns.name().contains(escaped), + "U+{:04X} must escape to {escaped}; got: {:?}", + raw as u32, + ns.name() + ); + } + } + + /// T-NS-3 [PF-014 / T-10a]: the SOURCE half is neutralized, never escaped — the + /// distinction the whole helper exists to keep straight. + /// + /// If source text went through `sanitize_control_chars*` instead, each 1-byte + /// control would become a 6-byte literal and every following span offset would be + /// wrong. This asserts byte length is preserved and that no `\uXXXX` literal (the + /// signature of the wrong function) appears. + #[test] + fn named_source_neutralizes_source_without_changing_byte_length() { + let src = "let x\u{1b} = 1;\u{061C}\n"; + let ns = named_source_for_render("clean.mds", src); + let rendered = { + use miette::SourceCode as _; + let contents = ns + .read_span(&(0..src.len()).into(), 0, 0) + .expect("span must be readable"); + String::from_utf8(contents.data().to_vec()).expect("neutralized source stays UTF-8") + }; + assert_eq!( + rendered.len(), + src.len(), + "source neutralization must preserve byte length; got: {rendered:?}" + ); + assert!( + !rendered.contains("\\u001B"), + "source must be NEUTRALIZED, not escaped — a \\uXXXX literal means the \ + wrong function was used and every following span offset is now wrong; \ + got: {rendered:?}" + ); + // Positive: the width-appropriate substitutes are present. + assert!( + rendered.contains('?') && rendered.contains('\u{00A0}'), + "1-byte ESC must become '?' and 2-byte U+061C must become NBSP; got: {rendered:?}" + ); + // Non-vacuity: surrounding source survives. + assert!( + rendered.contains("let x"), + "non-vacuity: clean source must survive; got: {rendered:?}" + ); + } + + /// T-16e [PF-013]: the RLO reversal vector reaches the wire through + /// `to_canonical_json` and comes out escaped. + /// + /// Vector: a `duplicate-import` style message embedding an import path that + /// carries U+202E. Without the guard the raw RLO reaches the JSON string value + /// and any terminal/IDE renderer reverses the rest of the line. + /// + /// Non-vacuity guards: the diagnostic list is non-empty, the expected rule is + /// present, and the POSITIVE assertion requires the escaped form to be there. + #[test] + fn to_canonical_json_escapes_bidi_override() { + let hostile_path = "./fo\u{202E}gnp.mds"; + let result = LintResult { + diagnostics: vec![LintDiagnostic { + rule: "duplicate-import".to_string(), + severity: Severity::Error, + message: format!("Module '{hostile_path}' is imported more than once."), + help: Some(format!("Remove the duplicate '{hostile_path}' import.")), + span: Some(crate::error::SerializedSpan { + offset: 9, + length: 16, + line: None, + column: None, + }), + file: Some("ma\u{202E}in.mds".to_string()), + fix_removals: None, + fix_edits: None, + }], + truncated: false, + is_standalone: false, + }; + + let json = result.to_canonical_json(); + let files = json["files"].as_array().expect("files array"); + assert_eq!(files.len(), 1, "non-vacuity: exactly one file group"); + let diags = files[0]["diagnostics"].as_array().expect("diagnostics"); + assert_eq!(diags.len(), 1, "non-vacuity: exactly one diagnostic"); + assert_eq!( + diags[0]["rule"], "duplicate-import", + "non-vacuity: expected rule must be present" + ); + + for field in ["message", "help"] { + let s = diags[0][field].as_str().unwrap(); + assert!( + !s.contains('\u{202E}'), + "raw U+202E must not appear in {field}; got: {s:?}" + ); + assert!( + s.contains("\\u202E"), + "escaped \\u202E must appear in {field}; got: {s:?}" + ); + } + + // The file group key travels the same boundary. + let file_key = files[0]["file"].as_str().unwrap(); + assert!( + !file_key.contains('\u{202E}'), + "raw U+202E must not appear in the file key; got: {file_key:?}" + ); + assert!( + file_key.contains("\\u202E"), + "escaped \\u202E must appear in the file key; got: {file_key:?}" + ); + + // Span offsets stay byte-accurate against the raw source. + assert_eq!(diags[0]["span"]["offset"], 9); + assert_eq!(diags[0]["span"]["length"], 16); + } + + /// T-16f: U+2028 in a diagnostic message is escaped on the wire — a raw one + /// would terminate a JS string literal in any consumer that inlines the value. + #[test] + fn to_canonical_json_escapes_line_separator() { + let result = LintResult { + diagnostics: vec![LintDiagnostic { + rule: "unused-variable".to_string(), + severity: Severity::Warn, + message: "Variable 'a\u{2028}b' is never referenced.".to_string(), help: None, span: None, - file: Some("f.mds".to_string()), + file: Some("t.mds".to_string()), fix_removals: None, fix_edits: None, }], @@ -489,19 +1402,128 @@ mod tests { is_standalone: false, }; let json = result.to_canonical_json(); - let raw_msg = json["files"][0]["diagnostics"][0]["message"] + let msg = json["files"][0]["diagnostics"][0]["message"] .as_str() - .unwrap(); - // The raw bytes should appear in JSON (serde_json escapes them as \u00xx). - // Crucially, we must NOT have applied sanitize_control_chars in the constructor. + .expect("non-vacuity: message must be a string"); + assert!( + !msg.contains('\u{2028}'), + "raw U+2028 must not appear on the wire; got: {msg:?}" + ); assert!( - raw_msg.contains('\x1B') || raw_msg.contains("\\u001B"), - "JSON should preserve or properly escape ESC byte, got: {raw_msg:?}" + msg.contains("\\u2028"), + "escaped \\u2028 must appear on the wire; got: {msg:?}" + ); + } + + /// T-16g: wire-mode newline escaping — log/YAML-key forging guard. + /// + /// A diagnostic message carrying an embedded newline can forge an extra + /// "diagnostic" line in any line-oriented consumer of the JSON string value. + /// On the WIRE the newline (U+000A) must become its 6-char escape literal; the + /// HUMAN render path must keep it raw so multi-line miette frames stay readable. + #[test] + fn to_canonical_json_escapes_newline_but_human_mode_preserves_it() { + let forged = "a\nerror[mds::forged]: FAKE\nb"; + let result = LintResult { + diagnostics: vec![LintDiagnostic { + rule: "unused-variable".to_string(), + severity: Severity::Warn, + message: forged.to_string(), + help: Some(forged.to_string()), + span: None, + file: Some("t.mds".to_string()), + fix_removals: None, + fix_edits: None, + }], + truncated: false, + is_standalone: false, + }; + let json = result.to_canonical_json(); + let diag = &json["files"][0]["diagnostics"][0]; + + for field in ["message", "help"] { + let s = diag[field] + .as_str() + .unwrap_or_else(|| panic!("non-vacuity: {field} must be a string")); + assert!( + !s.contains('\n'), + "raw newline must not survive into the wire {field}; got: {s:?}" + ); + assert!( + s.contains("\\u000A"), + "escaped \\u000A must appear in the wire {field}; got: {s:?}" + ); + // Non-vacuity: the surrounding text is preserved, only the newline changed. + assert!( + s.contains("error[mds::forged]"), + "message body must be preserved verbatim; got: {s:?}" + ); + } + + // Human mode keeps the newline — this is the render-path contract. + let human = sanitize_control_chars(forged); + assert!( + human.contains('\n'), + "human mode must preserve raw newlines; got: {human:?}" ); assert!( - raw_msg.contains('\x00') || raw_msg.contains("\\u0000"), - "JSON should preserve or properly escape NUL byte, got: {raw_msg:?}" + !human.contains("\\u000A"), + "human mode must not escape newlines; got: {human:?}" ); + // \t is preserved in BOTH modes. + assert!( + sanitize_control_chars("a\tb").contains('\t'), + "human mode must preserve tabs" + ); + } + + /// T-16h: the two modes differ on `\n` and on nothing else. + /// + /// Guards against the two ways the split could rot: WIRE forgetting a character + /// HUMAN escapes (or vice versa), and WIRE escaping `\t` as collateral damage. + #[test] + fn wire_and_human_modes_differ_only_on_newline() { + // \n: the one intentional divergence. + assert_eq!(&*sanitize_control_chars_wire("a\nb"), "a\\u000Ab"); + assert_eq!(&*sanitize_control_chars("a\nb"), "a\nb"); + // \t: preserved by both. + assert_eq!(&*sanitize_control_chars_wire("a\tb"), "a\tb"); + assert_eq!(&*sanitize_control_chars("a\tb"), "a\tb"); + // Everything else: identical output from both modes. + for probe in [ + "a\x00b", + "a\x1Bb", + "a\u{007F}b", + "a\u{0085}b", + "a\u{200E}b", + "a\u{202E}b", + "a\u{2028}b", + "a\u{2029}b", + "a\u{2069}b", + "a\u{FEFF}b", + "plain text", + ] { + assert_eq!( + sanitize_control_chars(probe), + sanitize_control_chars_wire(probe), + "modes must agree on {probe:?}" + ); + } + } + + /// T-16i: WIRE mode keeps the [`sanitize_control_chars`] properties — borrowed + /// on clean input (zero allocation) and idempotent. + #[test] + fn wire_mode_is_borrowed_when_clean_and_idempotent() { + assert!(matches!( + sanitize_control_chars_wire("normal text"), + Cow::Borrowed(_) + )); + for input in ["\x1B", "a\nb", "a\u{202E}b", "a\u{FEFF}b", "clean", ""] { + let once = sanitize_control_chars_wire(input); + let twice = sanitize_control_chars_wire(&once); + assert_eq!(once, twice, "wire mode not idempotent for {input:?}"); + } } // ── LintResultBuilder truncation ────────────────────────────────────────── diff --git a/crates/mds-core/src/lint/fix.rs b/crates/mds-core/src/lint/fix.rs index 0e17c98a..0af1677a 100644 --- a/crates/mds-core/src/lint/fix.rs +++ b/crates/mds-core/src/lint/fix.rs @@ -88,7 +88,9 @@ //! for non-truncated results. use crate::error::MdsError; -use crate::lint::diagnostic::{FixLineSpan, LintDiagnostic, LintResult, Severity}; +use crate::lint::diagnostic::{ + sanitize_control_chars_wire, FixLineSpan, LintDiagnostic, LintResult, Severity, +}; // Tier classification lives in the leaf `tier` module to break the would-be // circular dependency (fix.rs → diagnostic.rs → fix.rs). Re-export here so @@ -122,10 +124,39 @@ pub struct ByteEdit { pub struct RejectedEdit { /// The edit that was rejected. pub edit: ByteEdit, - /// Human-readable reason for rejection. + /// Human-readable reason for rejection. Sanitized at construction — see + /// [`FixOutcome::Rejected::reason`]. pub reason: String, } +/// Render a reverify failure into a single-line, display-safe rejection reason. +/// +/// The single construction site for every rejection reason that embeds an +/// [`MdsError`]. `MdsError`'s `Display` is deliberately raw (see its "Display contract" +/// note): variants such as `syntax error: {message}`, `file not found: {path}` and +/// `circular import detected: {cycle}` interpolate untrusted template and filesystem +/// text verbatim. Interpolating that directly into `reason` would push raw control +/// bytes into a field whose consumers print it on a status line. +/// +/// WIRE mode (not HUMAN) is correct here for the same reason it is correct for a +/// filename: the CLI prints this value as `fix rejected: {reason}` — one unframed, +/// unindented status line. A raw `\n` in the reason would let a hostile template forge +/// a second line indistinguishable from genuine status output (CWE-117). Every +/// `MdsError` `Display` variant is single-line by construction, so escaping `\n` here +/// discards nothing legitimate. +/// +/// Sanitizing at construction rather than at each print site is deliberate: `reason` is +/// a public field of a public enum in a published crate, so there is no bound on the +/// number of print sites, and a per-site check is exactly the parallel-path pattern +/// that lapses (PF-004). +fn reverify_failure_reason(err: &MdsError) -> String { + format!( + "could not verify fix — the edited source did not re-parse cleanly ({}); \ + leaving the file unchanged", + sanitize_control_chars_wire(&err.to_string()) + ) +} + /// A plan of fix edits for a single file's source. #[derive(Debug, Default)] pub struct FixPlan { @@ -172,9 +203,16 @@ pub enum FixOutcome { }, /// The edit batch was rejected (overlap detected or reverify failed). Rejected { - /// The original (unchanged) source. + /// The original (unchanged) source. Raw — this is the file's bytes, not prose. source: String, /// Human-readable reason for rejection. + /// + /// **Display-safe by construction.** Every untrusted fragment interpolated into + /// this string (currently only an [`MdsError`]'s raw `Display`) is escaped with + /// WIRE-mode [`sanitize_control_chars_wire`][crate::sanitize_control_chars_wire] + /// before it is stored, so the value is always single-line and free of C0/DEL/C1 + /// control bytes and bidi controls. Callers may print it directly on a status + /// line without further escaping. See `reverify_failure_reason`. reason: String, }, /// No fixable edits were found in the lint result. @@ -610,10 +648,7 @@ where match reverify(&fixed_source) { Err(err) => FixOutcome::Rejected { source: source.to_string(), - reason: format!( - "could not verify fix — the edited source did not re-parse cleanly \ - ({err}); leaving the file unchanged" - ), + reason: reverify_failure_reason(&err), }, Ok(residual) => { // Count untargeted diagnostics in the residual, per rule. @@ -777,10 +812,7 @@ where let reverify_result = reverify(&test_source); let reject_reason: Option = match &reverify_result { - Err(err) => Some(format!( - "could not verify fix — the edited source did not re-parse cleanly \ - ({err}); leaving the file unchanged" - )), + Err(err) => Some(reverify_failure_reason(err)), Ok(residual) => { // Use the full targeted_rules set (identical to the baseline) so the // comparison is symmetric: other targeted-rule diagnostics that are @@ -1598,6 +1630,115 @@ mod tests { ); } + // ── T-REASON: rejection reasons are display-safe by construction (#176) ────── + // + // `MdsError`'s `Display` is deliberately raw (see its "Display contract" note). + // Interpolating it into `reason` pushed unescaped control bytes into a field the + // CLI prints as an unframed status line: `fix rejected: {reason}`. + + /// Build the hostile `MdsError` used by the T-REASON tests. + /// + /// `MdsError::Syntax` renders as `syntax error: {message}`, so the message text — + /// which in production comes from template source — lands verbatim in `Display`. + /// The vector carries one member of each escape sub-class: a C0 byte (ESC), a + /// 3-byte bidi control, the 2-byte bidi control that #176 added, and a newline. + fn hostile_reverify_error() -> MdsError { + MdsError::syntax(format!( + "unexpected token{}[2J at{}line{}mark{}Clean: real.mds", + '\u{1b}', '\u{202E}', '\u{061C}', '\n' + )) + } + + /// T-REASON-1 [security-11 / CWE-117 / PF-013 / #176]: the reverify failure reason + /// produced by `apply_fixes` escapes the embedded `MdsError` Display. + #[test] + fn apply_fixes_rejection_reason_escapes_embedded_error_display() { + let source = "@import \"./a.mds\" as a\n@import \"./a.mds\" as b\n"; + let diag = make_diag("duplicate-import", 23, "@import".len()); + let result = make_result(vec![diag]); + let plan = plan_fixes(&result, source); + assert!( + !plan.edits.is_empty(), + "non-vacuity: the plan must have edits or apply_fixes never reverifies" + ); + + let outcome = apply_fixes( + source, + plan, + &result, + |_fixed| Err(hostile_reverify_error()), + ); + let reason = match outcome { + FixOutcome::Rejected { reason, .. } => reason, + other => panic!("expected Rejected, got: {other:?}"), + }; + + assert_reason_is_display_safe(&reason); + } + + /// T-REASON-2 [PF-004 / #176]: the per-edit fallback in `apply_fixes_incremental` + /// builds its reason through a *second* code path. It must be covered by the same + /// choke-point, or the guarantee holds on one path and lapses on its sibling. + #[test] + fn incremental_rejection_reason_escapes_embedded_error_display() { + let source = "@import \"./a.mds\" as a\n@import \"./a.mds\" as b\n"; + let diag = make_diag("duplicate-import", 23, "@import".len()); + let result = make_result(vec![diag]); + let plan = plan_fixes(&result, source); + assert!( + !plan.edits.is_empty(), + "non-vacuity: the plan must have edits or the fallback never runs" + ); + + let outcome = + apply_fixes_incremental( + source, + plan, + &result, + |_fixed| Err(hostile_reverify_error()), + ); + let reason = match outcome { + FixOutcome::Rejected { reason, .. } => reason, + other => panic!("expected Rejected, got: {other:?}"), + }; + + assert_reason_is_display_safe(&reason); + } + + /// Shared assertions for T-REASON-1/2: negative, positive, and non-vacuity. + fn assert_reason_is_display_safe(reason: &str) { + // Non-vacuity: the real error text actually reached the reason, so the escape + // assertions cannot pass by the error being dropped instead of sanitized. + assert!( + reason.contains("syntax error") && reason.contains("unexpected token"), + "non-vacuity: the embedded error text must be present; got: {reason:?}" + ); + // Negative: no raw hostile codepoint survives. + for raw in ['\u{1b}', '\u{202E}', '\u{061C}', '\n'] { + assert!( + !reason.contains(raw), + "raw U+{:04X} must not appear in a rejection reason; got: {reason:?}", + raw as u32 + ); + } + // Positive: each is present in its escaped form. The literals are UPPERCASE + // while the hex in the source vector is lowercase, which proves the byte really + // decoded and was really escaped rather than passing through as literal text. + for escaped in ["\\u001B", "\\u202E", "\\u061C", "\\u000A"] { + assert!( + reason.contains(escaped), + "{escaped} must appear in the rejection reason; got: {reason:?}" + ); + } + // The reason is printed as one unframed status line: it must be single-line, or + // a hostile template can forge output indistinguishable from genuine status. + assert_eq!( + reason.lines().count(), + 1, + "a rejection reason must be single-line; got: {reason:?}" + ); + } + /// A single non-overlapping `duplicate-import` on line 2 must plan, apply, /// pass the (stubbed) reverify, and return `FixOutcome::Fixed`. /// diff --git a/crates/mds-core/src/lint/mod.rs b/crates/mds-core/src/lint/mod.rs index 7581a433..afb5fa8c 100644 --- a/crates/mds-core/src/lint/mod.rs +++ b/crates/mds-core/src/lint/mod.rs @@ -34,7 +34,8 @@ pub(crate) mod tier; pub use config::LintConfig; pub use diagnostic::{ - sanitize_control_chars, FixLineSpan, LintDiagnostic, LintResult, Severity, TextEdit, + named_source_for_render, neutralize_source_for_render, sanitize_control_chars, + sanitize_control_chars_wire, FixLineSpan, LintDiagnostic, LintResult, Severity, TextEdit, }; use crate::error::MdsError; diff --git a/crates/mds-core/src/resolver.rs b/crates/mds-core/src/resolver.rs index 4e92e0fe..e8a69990 100644 --- a/crates/mds-core/src/resolver.rs +++ b/crates/mds-core/src/resolver.rs @@ -1062,11 +1062,15 @@ impl ModuleCache { // silently yields a partial FragmentMap (incorrect splice attributions) with // no AC-PERF-03 warning. Mirror the top-level degradation logic exactly. let fmap = if returned.segments_dropped { + // The module filename is an untrusted identifier — WIRE, per the + // spec 7.5 per-field rule: a filename is never legitimately multi-line, + // and `emit_warnings` prints this string to stderr in HUMAN mode, where a + // raw `\n` would forge a standalone status line (CWE-117). warnings.push(format!( "source map segment cap ({} segments) exceeded in imported module '{}'; \ no source map will be generated", crate::limits::MAX_SOURCEMAP_SEGMENTS, - ctx.file_str, + crate::lint::sanitize_control_chars_wire(ctx.file_str), )); None } else { diff --git a/crates/mds-core/src/sourcemap.rs b/crates/mds-core/src/sourcemap.rs index fd12e514..2e595e64 100644 --- a/crates/mds-core/src/sourcemap.rs +++ b/crates/mds-core/src/sourcemap.rs @@ -117,6 +117,26 @@ pub(crate) fn map_source_label(name: &str) -> &str { /// sourcesContent → names → mappings); `serde` emits struct fields in /// declaration order. /// +/// # Paths here are untrusted and unescaped +/// +/// `file`, `sources` and `sources_content` carry values derived from the filesystem — +/// including names produced by a directory walk, which the user never typed. They are a +/// **named carve-out** from the sanitization rule in spec §7.5 ("Carve-out: functional +/// path references"): they are emitted **verbatim**, with no `\uXXXX` escaping, because +/// devtools, bundlers and IDEs resolve them against the filesystem and an escaped path +/// would not exist. +/// +/// A path in this type may therefore contain any byte a filesystem permits: C0 control +/// characters, `\n`, DEL, bidi controls (Trojan Source, CVE-2021-42574), U+FEFF. JSON +/// string encoding is **not** escaping — it makes the document parseable, and a decoded +/// `"\n"` is a real newline again. +/// +/// **Consumers MUST escape these paths for whatever destination they render them to** — +/// a terminal, a log line, HTML. [`crate::sanitize_control_chars_wire`] applies the same +/// escaping the diagnostic surfaces use. The MDS CLI does exactly that for its own +/// output: `Compiled to …` and `Source map written to …` print the escaped form even +/// though the sidecar they name does not. +/// /// [Source Map v3 / ECMAScript 426]: https://tc39.es/ecma426/ #[non_exhaustive] #[derive(Debug, Clone, PartialEq, Serialize)] diff --git a/crates/mds-napi/__test__/index.spec.mjs b/crates/mds-napi/__test__/index.spec.mjs index 2522681e..21230f21 100644 --- a/crates/mds-napi/__test__/index.spec.mjs +++ b/crates/mds-napi/__test__/index.spec.mjs @@ -1189,3 +1189,247 @@ describe('source maps (F-SM)', () => { ); }); }); + +// ── T-12/T-13 (E-10/E-11): ESC-injection hardening — napi direct (issue #176 / CWE-150) ──────── +// +// Four vectors (E-10..E-13): +// (E-10) error path: `@include foo` — parser rejects invalid alias, message +// embeds the raw alias. After fix: err.message has no raw C0/DEL/C1 chars. +// (E-11) lint path: lintVirtual with module name containing U+001B, imported twice — +// duplicate-import fires; diagnostic message has no raw ESC char. +// (E-12) error path with DEL (U+007F) — same as E-10 with a different control char. +// (E-13) lint path with U+0085 (NEL/C1) — passes serde_yaml_ng, provides C1 coverage. + +describe('ESC-injection hardening (issue #176 / CWE-150)', () => { + // Helper: assert no raw C0 (excl. \t \n), DEL, or C1 chars in a string. + // Uses charCodeAt (UTF-16 code units); all C0/DEL/C1 codepoints are in BMP so + // charCodeAt correctly identifies them without surrogate pair handling. + function assertNoControlChars(s, label) { + for (let i = 0; i < s.length; i++) { + const code = s.charCodeAt(i); + const isC0 = code < 0x20 && code !== 0x09 && code !== 0x0a; + const isDel = code === 0x7f; + const isC1 = code >= 0x80 && code <= 0x9f; + // Bidi controls (Trojan Source, CVE-2021-42574), U+2028/U+2029 (JS string + // literal terminators), and U+FEFF (invisible BOM) — all escaped by the + // sanitizers. `\n` is allowed here; wire-mode newline escaping is asserted + // explicitly by E-15. + const isFormatHazard = + code === 0x200e || code === 0x200f || + code === 0x2028 || code === 0x2029 || + (code >= 0x202a && code <= 0x202e) || + (code >= 0x2066 && code <= 0x2069) || + code === 0xfeff; + assert.ok( + !isC0 && !isDel && !isC1 && !isFormatHazard, + `${label}: raw hostile char U+${code.toString(16).toUpperCase().padStart(4,'0')} ` + + `at index ${i} must not appear; got: ${JSON.stringify(s)}` + ); + } + } + + test('T-12 / E-10: error path — compile error message sanitized for ESC-in-alias', () => { + const esc = String.fromCharCode(0x1b); + const source = `@include fo${esc}o\n`; + try { + compile(source); + assert.fail('expected compile to throw'); + } catch (err) { + const msg = err.message; + assert.ok(typeof msg === 'string' && msg.length > 0, 'message must be non-empty'); + assertNoControlChars(msg, 'err.message'); + assert.ok( + msg.includes('\\u001B'), + `sanitized \\u001B must appear in err.message; got: ${JSON.stringify(msg)}` + ); + } + }); + + test('T-13 / E-11: lint path — lintVirtual with ESC in module name sanitizes duplicate-import message', () => { + // Use lintVirtual with a module whose NAME contains a raw ESC byte (U+001B), + // imported twice so duplicate-import fires and embeds the raw path in its message. + // Mirrors Python E12 (test_e12_lint_virtual_esc_in_import_path_message_sanitized). + // Verifies: + // (1) No raw C0/DEL/C1 bytes in any diagnostic message. + // (2) Sanitized \u001B literal IS present (positive evidence, non-vacuous). + // (3) Result shape: version 1, duplicate-import rule present. + const esc = String.fromCharCode(0x1b); + const moduleName = `fo${esc}o.mds`; + const modules = { + [moduleName]: 'hi\n', + 'main.mds': `@import "./${moduleName}"\n@import "./${moduleName}"\n`, + }; + const result = lintVirtual(modules, 'main.mds'); + + // (3) Result shape: version 1. + assert.equal(result.version, 1, 'T-13/E-11: version must be 1'); + assert.ok(Array.isArray(result.files), 'T-13/E-11: files must be an array'); + + const allDiags = result.files.flatMap((f) => f.diagnostics); + assert.ok( + allDiags.length > 0, + 'T-13/E-11: expected at least one diagnostic (duplicate-import should fire); got: ' + + JSON.stringify(allDiags), + ); + + // (1) No raw control bytes in any diagnostic message. + for (const diag of allDiags) { + if (typeof diag.message === 'string') { + assertNoControlChars(diag.message, `T-13/E-11: diag[${diag.rule}].message`); + } + } + + // (2) At least one message carries the sanitized \u001B literal (positive evidence). + const hasSanitizedEsc = allDiags.some( + (d) => + typeof d.message === 'string' && + d.message.includes('\\u001B'), + ); + assert.ok( + hasSanitizedEsc, + 'T-13/E-11: expected \\u001B in at least one diagnostic message; got: ' + + JSON.stringify(allDiags.map((d) => d.message)), + ); + + // (3) Expected rule: duplicate-import must be among the diagnostics. + const hasDupImport = allDiags.some((d) => d.rule === 'duplicate-import'); + assert.ok( + hasDupImport, + 'T-13/E-11: expected duplicate-import diagnostic; got rules: ' + + JSON.stringify(allDiags.map((d) => d.rule)), + ); + }); + + test('E-12: error path — compile error message sanitized for DEL (U+007F) in alias', () => { + // DEL (U+007F) in @include alias; parser rejects the invalid alias and embeds + // the raw bytes in the error message. After fix, serialize() sanitizes DEL to \u007F. + const del = String.fromCharCode(0x7f); + const source = `@include fo${del}o\n`; + try { + compile(source); + assert.fail('expected compile to throw'); + } catch (err) { + const msg = err.message; + assert.ok(typeof msg === 'string' && msg.length > 0, 'E-12: message must be non-empty'); + assertNoControlChars(msg, 'E-12: err.message'); + assert.ok( + msg.includes('\\u007F'), + `E-12: sanitized \\u007F must appear in err.message; got: ${JSON.stringify(msg)}` + ); + } + }); + + test('E-13: lint path — lintVirtual with U+0085 (NEL/C1) in module name sanitizes message', () => { + // U+0085 (NEL) is a C1 control char that passes serde_yaml_ng YAML parsing + // (unlike ESC/DEL), making it a reachable C1 ESC-injection vector for lintVirtual. + // The duplicate-import rule fires and embeds the raw module name in its message; + // after sanitization the message must carry … and no raw C1 chars. + const nel = String.fromCharCode(0x85); + const moduleName = `fo${nel}o.mds`; + const modules = { + [moduleName]: 'hi\n', + 'main.mds': `@import "./${moduleName}"\n@import "./${moduleName}"\n`, + }; + const result = lintVirtual(modules, 'main.mds'); + assert.equal(result.version, 1, 'E-13: version must be 1'); + const allDiags = result.files.flatMap((f) => f.diagnostics); + assert.ok( + allDiags.length > 0, + 'E-13: expected at least one diagnostic; got: ' + JSON.stringify(allDiags), + ); + for (const diag of allDiags) { + if (typeof diag.message === 'string') { + assertNoControlChars(diag.message, `E-13: diag[${diag.rule}].message`); + } + } + const hasSanitizedNel = allDiags.some( + (d) => typeof d.message === 'string' && d.message.includes('\\u0085'), + ); + assert.ok( + hasSanitizedNel, + 'E-13: expected \\u0085 in at least one diagnostic message; got: ' + + JSON.stringify(allDiags.map((d) => d.message)), + ); + }); + + test('E-14: lint path — U+202E (RLO) in module name is escaped on the wire', () => { + // Trojan Source (CVE-2021-42574): U+202E is not a C0/DEL/C1 control char, so it + // used to travel the wire untouched and reverse the display order of everything + // after it in any bidi-aware renderer (terminal, IDE, code-review UI). + // "fognp.mds" renders as "fopng.mds". + const rlo = String.fromCharCode(0x202e); + const moduleName = `fo${rlo}gnp.mds`; + const modules = { + [moduleName]: 'hi\n', + 'main.mds': `@import "./${moduleName}"\n@import "./${moduleName}"\n`, + }; + const result = lintVirtual(modules, 'main.mds'); + assert.equal(result.version, 1, 'E-14: version must be 1'); + const allDiags = result.files.flatMap((f) => f.diagnostics); + assert.ok( + allDiags.length > 0, + 'E-14: expected at least one diagnostic; got: ' + JSON.stringify(allDiags), + ); + assert.ok( + allDiags.some((d) => d.rule === 'duplicate-import'), + 'E-14: expected duplicate-import; got rules: ' + + JSON.stringify(allDiags.map((d) => d.rule)), + ); + for (const diag of allDiags) { + if (typeof diag.message === 'string') { + assertNoControlChars(diag.message, `E-14: diag[${diag.rule}].message`); + } + } + // Cheap invariant check only — NOT coverage of the `file`-key escape. The + // hostile RLO is in the *imported* module's name, but this key is the *entry* + // filename ("main.mds"), so no hostile byte reaches it and this cannot fail via + // this vector (PF-013). Real `file`-key coverage: mds-core + // `to_canonical_json_escapes_bidi_override`. + for (const f of result.files) { + assertNoControlChars(f.file, 'E-14: files[].file'); + } + assert.ok( + allDiags.some((d) => typeof d.message === 'string' && d.message.includes('\\u202E')), + 'E-14: expected \\u202E in at least one diagnostic message; got: ' + + JSON.stringify(allDiags.map((d) => d.message)), + ); + }); + + test('E-15: lint path — newline in a frontmatter key is escaped to \\u000A on the wire', () => { + // Log/YAML-key forging: a raw newline inside a diagnostic message lets an + // attacker forge what reads as a second, independent finding in any + // line-oriented consumer of the JSON string value. WIRE mode escapes it. + // + // Reachability: a newline inside an `@import "..."` path is rejected by the + // lexer, so that route is vacuous. A YAML *double-quoted* frontmatter key is + // not: serde_yaml_ng decodes the \n escape into a real newline and + // unused-variable embeds the decoded key verbatim in its message. + const source = + '---\n"a\\nerror[mds::forged]: FAKE\\nb": 1\n---\nHello\n'; + const result = lintVirtual({ 'main.mds': source }, 'main.mds'); + assert.equal(result.version, 1, 'E-15: version must be 1'); + const allDiags = result.files.flatMap((f) => f.diagnostics); + assert.ok( + allDiags.some((d) => d.rule === 'unused-variable'), + 'E-15: expected unused-variable; got rules: ' + + JSON.stringify(allDiags.map((d) => d.rule)), + ); + for (const diag of allDiags) { + assert.ok( + !diag.message.includes('\n'), + `E-15: raw newline must not survive into the wire message; got: ${JSON.stringify(diag.message)}`, + ); + } + assert.ok( + allDiags.some((d) => d.message.includes('\\u000A')), + 'E-15: expected \\u000A in at least one diagnostic message; got: ' + + JSON.stringify(allDiags.map((d) => d.message)), + ); + // Escaped, not stripped — the payload text itself is preserved verbatim. + assert.ok( + allDiags.some((d) => d.message.includes('error[mds::forged]')), + 'E-15: message body must be preserved verbatim; got: ' + + JSON.stringify(allDiags.map((d) => d.message)), + ); + }); +}); diff --git a/crates/mds-napi/src/lib.rs b/crates/mds-napi/src/lib.rs index 199a8e66..73a1fbbf 100644 --- a/crates/mds-napi/src/lib.rs +++ b/crates/mds-napi/src/lib.rs @@ -307,6 +307,14 @@ where let err_obj = raw_create_error(raw_env, "mds::internal", "internal compiler error"); if !err_obj.is_null() { + // NOTE: `err.detail` carries the raw panic payload and is + // intentionally NOT passed through `MdsError::serialize()` or + // `sanitize_control_chars`. This is a deliberate exclusion from + // the sanitization boundary enumeration (see diagnostic.rs module + // doc). The path exists only when the `debug-panics` Cargo + // feature is enabled, which must never ship in a production build + // — see CLAUDE.md: "`debug-panics` Cargo feature must never ship + // enabled (all three binding crates)". raw_set_string_prop(raw_env, err_obj, "detail", &detail); let _ = sys::napi_throw(raw_env, err_obj); return Err(napi::Error::new(Status::PendingException, "")); diff --git a/crates/mds-python/src/lib.rs b/crates/mds-python/src/lib.rs index 65e94166..56c3dce8 100644 --- a/crates/mds-python/src/lib.rs +++ b/crates/mds-python/src/lib.rs @@ -710,11 +710,17 @@ pub struct LintResult { #[pymethods] impl LintResult { /// Reconstruct from a canonical mapping (used by unpickling). + /// + /// Calls [`sanitize_lint_value`] after deserializing so the backing store is + /// always sanitized — the same guarantee `to_canonical_json()` provides on the + /// live lint path. This closes the parallel-path gap for the + /// `LintResult(canonical)` constructor (PF-004). #[new] fn new(canonical: &Bound<'_, PyAny>) -> PyResult { - let value: serde_json::Value = depythonize(canonical).map_err(|e| { + let mut value: serde_json::Value = depythonize(canonical).map_err(|e| { options_error(canonical.py(), &format!("invalid LintResult state: {e}")) })?; + sanitize_lint_value(&mut value); Ok(LintResult { value }) } @@ -749,6 +755,9 @@ impl LintResult { arr.iter() .map(|file_val| { + // `self.value` is always sanitized: `to_canonical_json()` sanitizes at + // the lint boundary; `sanitize_lint_value()` in `new()` covers the + // `LintResult(canonical)` / pickle path. Plain read is safe (avoids PF-004). let file_key = json_str(file_val, "file"); let diagnostics: Vec = file_val .get("diagnostics") @@ -772,6 +781,8 @@ impl LintResult { LintDiagnostic { rule: json_str(d, "rule"), severity: json_str(d, "severity"), + // Plain reads: `self.value` is sanitized before reaching here + // (see file_key comment above). No allocation on clean strings. message: json_str(d, "message"), help: d .get("help") @@ -843,6 +854,60 @@ impl LintResult { } } +// ── Lint value sanitization ───────────────────────────────────────────────────── + +/// Sanitize a string field in a JSON object in-place via +/// [`mds::sanitize_control_chars_wire`]. +/// +/// WIRE mode: this backing store feeds `as_json()` / `to_dict()` as well as the typed +/// getters, and must stay byte-identical to `LintResult::to_canonical_json()` on the +/// other three surfaces — CLI, napi and WASM (PF-007) — including the `\n` escape. +/// +/// Uses `Cow` so clean strings (the common case) cause no allocation — only strings +/// that actually contain hostile characters are replaced. No-op when the field is +/// absent or not a string. +fn sanitize_json_str_field(obj: &mut serde_json::Map, key: &str) { + // Clone the field value to release the immutable borrow of `obj` before the + // subsequent `obj.insert(...)` mutable borrow. + let s_owned = match obj.get(key) { + Some(serde_json::Value::String(s)) => s.clone(), + _ => return, + }; + if let std::borrow::Cow::Owned(sanitized) = mds::sanitize_control_chars_wire(&s_owned) { + obj.insert(key.to_string(), serde_json::Value::String(sanitized)); + } +} + +/// Sanitize all message, help, and file string fields in a canonical lint result value +/// in-place. +/// +/// Called in [`LintResult::new`] so any data arriving through the +/// `LintResult(canonical)` / pickle path is sanitized before the typed getters or +/// `to_dict()` read from the backing store. Mirrors what +/// `LintResult::to_canonical_json()` does on the live lint path, closing the +/// parallel-path gap (PF-004). +fn sanitize_lint_value(value: &mut serde_json::Value) { + let Some(files) = value.get_mut("files").and_then(|v| v.as_array_mut()) else { + return; + }; + for file_val in files.iter_mut() { + let Some(obj) = file_val.as_object_mut() else { + continue; + }; + sanitize_json_str_field(obj, "file"); + let Some(diags) = obj.get_mut("diagnostics").and_then(|v| v.as_array_mut()) else { + continue; + }; + for d in diags.iter_mut() { + let Some(d_obj) = d.as_object_mut() else { + continue; + }; + sanitize_json_str_field(d_obj, "message"); + sanitize_json_str_field(d_obj, "help"); + } + } +} + // ── Error / value conversion helpers ─────────────────────────────────────────── /// Convert an [`mds::MdsError`] into a raised [`MdsError`] with typed attributes. diff --git a/crates/mds-python/tests/test_errors.py b/crates/mds-python/tests/test_errors.py index 5a9002cd..18146766 100644 --- a/crates/mds-python/tests/test_errors.py +++ b/crates/mds-python/tests/test_errors.py @@ -173,6 +173,328 @@ def test_e5_span_none_when_core_reports_none() -> None: pytest.fail("expected MdsError") +# ── T-14: ESC-injection hardening — Python surface (issue #176 / CWE-150) ────── +# +# Two sub-tests: +# E11: error path — @include alias with U+001B in mid-token; err.message must +# carry the sanitized \\u001B literal; str(e) == e.message (AC-C2) must hold. +# E12: lint path — frontmatter key with U+001B; LintDiagnostic.message clean. +# +# Naming: test_e11_* / test_e12_* — chosen so -k substrings cannot collide with +# other tests (PF-008: pytest -k matches substrings of the full node id). + + +def _assert_no_control_chars(s: str, label: str) -> None: + """Assert no raw C0 (excl. \\t \\n), DEL, C1, bidi, separator, or BOM char in `s`. + + `\\n` is permitted because this helper also runs against HUMAN-mode output; + wire-mode newline escaping is asserted explicitly by the E13 test. + """ + for i, ch in enumerate(s): + code = ord(ch) + is_c0 = code < 0x20 and code not in (0x09, 0x0A) + is_del = code == 0x7F + is_c1 = 0x80 <= code <= 0x9F + # Bidi controls (Trojan Source, CVE-2021-42574), JS line/paragraph + # separators, and the invisible BOM. + is_format_hazard = ( + code in (0x200E, 0x200F, 0x2028, 0x2029, 0xFEFF) + or 0x202A <= code <= 0x202E + or 0x2066 <= code <= 0x2069 + ) + assert not (is_c0 or is_del or is_c1 or is_format_hazard), ( + f"raw hostile char U+{code:04X} at index {i} must not appear in {label}; got: {s!r}" + ) + + +def test_e11_control_chars_in_message_are_escaped() -> None: + """T-14 / E11 [AC-F3, AC-C2]: error-path sanitization for Python surface. + + @include with a raw ESC byte (U+001B) mid-alias is rejected by the parser + with MdsError::Syntax("invalid include alias: 'foo'"). After Change #1, + serialize() sanitizes the message so e.message contains no raw control bytes + and the sanitized \\u001B literal is visible. AC-C2 (str(e) == e.message) must + still hold because both use the same sanitized string. + """ + esc = "\x1b" + source = f"@include fo{esc}o\n" + try: + m.compile(source) + except m.MdsError as e: + msg = e.message + _assert_no_control_chars(msg, "e.message") + # Sanitized literal must be present. + assert "\\u001B" in msg, ( + f"sanitized \\u001B literal must appear in e.message; got: {msg!r}" + ) + # AC-C2: str(e) == e.message. + assert str(e) == e.message, ( + f"AC-C2 violated: str(e)={str(e)!r} != e.message={e.message!r}" + ) + else: + pytest.fail("expected m.MdsError to be raised") + + +@pytest.mark.parametrize( + "ctrl_char,expected_escape", + [ + ("\x1b", "\\u001B"), # ESC (U+001B) — C0 control char + ("\x7f", "\\u007F"), # DEL (U+007F) — serde_json does not auto-escape 0x7F + # U+0085 NEL (C1) — passes serde_yaml_ng where ESC/DEL are rejected in YAML keys; + # the reachable YAML vector per KB Gotchas. Also exercised here via lint_virtual + # (module names are plain strings, not YAML, so all three chars reach the engine). + ("\x85", "\\u0085"), + # Widened escape class (#176): none of these are C0/DEL/C1, and all of them + # used to travel the wire untouched. Written as escapes, never as raw + # characters -- a literal RLO would reverse how this source file displays. + ("\u202e", "\\u202E"), # RLO - Trojan Source display reversal (CVE-2021-42574) + ("\u2066", "\\u2066"), # LRI - bidi isolate + ("\u2028", "\\u2028"), # LINE SEPARATOR - terminates a JS string literal + ("\ufeff", "\\uFEFF"), # BOM / ZWNBSP - invisible in every renderer + ], + ids=["ESC", "DEL", "NEL", "RLO", "LRI", "LS", "BOM"], +) +def test_e12_lint_virtual_ctrl_in_import_path_message_sanitized( + ctrl_char: str, expected_escape: str +) -> None: + """T-14 / E12 [AC-F4]: Python typed LintDiagnostic.message and as_json() sanitization. + + Uses the lint_virtual API with a module whose NAME contains a raw control byte + to trigger a duplicate-import rule whose message embeds the raw path — a reachable + end-to-end vector that exercises the Python typed surface without touching YAML parsing. + + Parametrized over ESC, DEL, and U+0085 NEL (PF-007 python-7). + + Verifies: + (a) LintDiagnostic.message contains no raw C0/DEL/C1 bytes (typed attribute clean) + (b) LintDiagnostic.message contains the sanitized escape literal (explicit evidence) + (c) LintDiagnostic.to_dict()["message"] is identical to .message (parity guard, PF-007) + (d) LintFileReport.file contains no raw control bytes (python-3 regression anchor) + """ + # Module whose name contains the raw control byte — import path embeds it in the message. + module_name = f"fo{ctrl_char}o.mds" + modules = { + module_name: "hi\n", + # Import the same module twice to trigger duplicate-import; message will embed module_name. + "main.mds": f'@import "./{module_name}"\n@import "./{module_name}"\n', + } + result = m.lint_virtual(modules, "main.mds") + + files = result.files + assert files, "expected at least one LintFileReport from lint_virtual" + + # (d) Cheap invariant check only -- NOT coverage of the ``file``-key escape. The + # hostile codepoint is in the *imported* module's name, but this key is the *entry* + # filename, so no hostile byte reaches it and this cannot fail via this vector + # (PF-013). Real ``file``-key coverage: ``test_par7_...``, which constructs a + # LintResult with ``"file": "fo\u202egnp.mds"`` directly. + for fr in files: + _assert_no_control_chars(fr.file, "LintFileReport.file") + + all_diags = [d for fr in files for d in fr.diagnostics] + assert all_diags, ( + "expected at least one LintDiagnostic (duplicate-import should fire for " + "the twice-imported module)" + ) + + # (a) No raw C0/DEL/C1 bytes in typed .message attribute. + for diag in all_diags: + msg = diag.message + assert isinstance(msg, str) and msg, "message must be a non-empty string" + _assert_no_control_chars(msg, "LintDiagnostic.message") + + # (b) At least one diagnostic must carry the sanitized escape literal — + # confirming the control byte in the module name was sanitized, not dropped. + # (Only the duplicate-import diagnostic embeds the path; check all.) + found_escaped = [d for d in all_diags if expected_escape in d.message] + assert found_escaped, ( + f"expected at least one diagnostic whose message carries the sanitized " + f"{expected_escape!r} literal (module path); got: " + + str([d.message for d in all_diags]) + ) + + # (c) Parity guard: to_dict()["message"] must equal .message (PF-007). + # as_json() / to_dict() must not re-introduce raw control bytes from pyclass fields. + for diag in all_diags: + d_dict = diag.to_dict() + assert isinstance(d_dict, dict), "to_dict() must return a dict" + dict_msg = d_dict.get("message", "") + assert isinstance(dict_msg, str), "to_dict()[message] must be a string" + assert dict_msg == diag.message, ( + f"to_dict()[message] must equal .message; " + f"typed={diag.message!r}, dict={dict_msg!r}" + ) + + +def test_e13_lint_virtual_newline_in_frontmatter_key_escaped_on_wire() -> None: + """T-14 / E13 [AC-F4]: WIRE-mode newline escaping on the Python surface. + + A raw newline inside a diagnostic message lets an attacker forge what reads as + a second, independent finding in any line-oriented consumer of the value (log + forging, YAML key injection). On the wire it must arrive as the six-character + ``\\u000A`` literal -- and the Python surface must emit the same bytes as the + other three surfaces: CLI, napi and WASM (PF-007). + + Reachability: a newline inside an ``@import "..."`` path is rejected by the lexer + (that route would be vacuous, PF-013). A YAML *double-quoted* frontmatter key is + not -- serde_yaml_ng decodes the ``\\n`` escape into a real newline, and + ``unused-variable`` embeds the decoded key verbatim in its message. + + Verifies: + (a) no typed ``.message`` carries a raw newline + (b) the escaped ``\\u000A`` literal IS present (positive, non-vacuous) + (c) the payload text survives verbatim -- escaped, not stripped + (d) ``to_dict()`` agrees with the typed attribute (parity guard) + """ + source = '---\n"a\\nerror[mds::forged]: FAKE\\nb": 1\n---\nHello\n' + result = m.lint_virtual({"main.mds": source}, "main.mds") + + all_diags = [d for fr in result.files for d in fr.diagnostics] + assert any(d.rule == "unused-variable" for d in all_diags), ( + "expected unused-variable to fire; got rules: " + + str([d.rule for d in all_diags]) + ) + + # (a) No raw newline survives into a wire message. + for diag in all_diags: + assert "\n" not in diag.message, ( + f"raw newline must not survive into the wire message; got: {diag.message!r}" + ) + + # (b) Positive evidence of the escape. + assert any("\\u000A" in d.message for d in all_diags), ( + "expected \\u000A in at least one diagnostic message; got: " + + str([d.message for d in all_diags]) + ) + + # (c) Escaped, not stripped. + assert any("error[mds::forged]" in d.message for d in all_diags), ( + "message body must be preserved verbatim; got: " + + str([d.message for d in all_diags]) + ) + + # (d) Parity: to_dict() must agree with the typed attribute (PF-007). + for diag in all_diags: + assert diag.to_dict()["message"] == diag.message + + +def test_par7_lint_result_constructor_wire_escapes_bidi_and_newline() -> None: + """PF-004 anchor: the ``LintResult(canonical)`` / unpickle path uses WIRE mode. + + ``sanitize_lint_value()`` in ``LintResult::new()`` is a parallel path into the same + backing store the typed getters and ``to_dict()`` read from. If it drifted to + HUMAN mode (or missed the widened class) this constructor would become a way to + smuggle a bidi override or a forged newline past the live-lint boundary. + """ + raw_canonical = { + "version": 1, + "truncated": False, + "files": [ + { + "file": "fo\u202egnp.mds", + "diagnostics": [ + { + "rule": "unused-variable", + "severity": "warn", + "message": "unused \u202ekey\nerror[mds::forged]: FAKE", + "help": "remove \u2028 or use it", + "fixable": False, + "span": None, + "fix_edits": None, + } + ], + } + ], + } + result = m.LintResult(raw_canonical) + fr = result.files[0] + diag = fr.diagnostics[0] + + _assert_no_control_chars(fr.file, "LintFileReport.file (constructor)") + assert "\\u202E" in fr.file, f"file must be escaped; got: {fr.file!r}" + + _assert_no_control_chars(diag.message, "LintDiagnostic.message (constructor)") + assert "\\u202E" in diag.message, f"message must escape RLO; got: {diag.message!r}" + assert "\\u000A" in diag.message, ( + f"message must escape the newline on the wire; got: {diag.message!r}" + ) + assert "\n" not in diag.message, ( + f"raw newline must not survive the constructor; got: {diag.message!r}" + ) + # Escaped, not stripped. + assert "error[mds::forged]" in diag.message + + assert diag.help is not None + assert "\\u2028" in diag.help, f"help must escape U+2028; got: {diag.help!r}" + + +def test_par6_lint_result_constructor_sanitizes_typed_fields() -> None: + """python-1 / python-2 regression anchor: LintResult(canonical) sanitizes via new(). + + Constructs LintResult directly via its Python constructor (the pickle/unpickle entry + point) with raw ESC bytes in message, help, and file fields. After the fix, + sanitize_lint_value() runs in new() and all typed getters must return sanitized values. + + This test FAILS if sanitize_lint_value() is removed from LintResult::new() + (avoids PF-013 — the belt-and-suspenders at the population site is vacuous without + this constructor-level guard). + """ + esc = "\x1b" + raw_canonical = { + "version": 1, + "truncated": False, + "files": [ + { + "file": f"fo{esc}o.mds", + "diagnostics": [ + { + "rule": "unused-variable", + "severity": "warn", + "message": f"unused variable {esc}key", + "help": f"remove or use the variable {esc}key", + "fixable": False, + "span": None, + "fix_edits": None, + } + ], + } + ], + } + result = m.LintResult(raw_canonical) + files = result.files + assert len(files) == 1, "expected one LintFileReport" + fr = files[0] + + # LintFileReport.file must be sanitized. + _assert_no_control_chars(fr.file, "LintFileReport.file (from constructor)") + assert "\\u001B" in fr.file, f"file must contain sanitized literal; got: {fr.file!r}" + + assert len(fr.diagnostics) == 1, "expected one LintDiagnostic" + diag = fr.diagnostics[0] + + # LintDiagnostic.message must be sanitized. + _assert_no_control_chars(diag.message, "LintDiagnostic.message (from constructor)") + assert "\\u001B" in diag.message, f"message must contain sanitized literal; got: {diag.message!r}" + + # LintDiagnostic.help must be sanitized. + assert diag.help is not None + _assert_no_control_chars(diag.help, "LintDiagnostic.help (from constructor)") + assert "\\u001B" in diag.help, f"help must contain sanitized literal; got: {diag.help!r}" + + # Parity: to_dict()["message"] must equal .message (PF-007). + d_dict = diag.to_dict() + assert isinstance(d_dict, dict) + assert d_dict["message"] == diag.message, ( + f"to_dict()[message] must equal .message; " + f"typed={diag.message!r}, dict={d_dict['message']!r}" + ) + + # LintResult.to_dict() must also expose sanitized file key. + result_dict = result.to_dict() + file_in_dict = result_dict["files"][0]["file"] + _assert_no_control_chars(file_in_dict, "to_dict() file key (from constructor)") + + # ── D2: type_mismatch_at — span present on @if cross-type comparison ───────────── diff --git a/crates/mds-wasm/tests/web.rs b/crates/mds-wasm/tests/web.rs index 65af9115..d25da474 100644 --- a/crates/mds-wasm/tests/web.rs +++ b/crates/mds-wasm/tests/web.rs @@ -776,3 +776,387 @@ fn compile_extends_undefined_var_in_base_default_carries_real_span() { .as_f64() .expect("WASM C4: span.column must be a number, not undefined"); } + +// ── T-15: ESC-injection hardening — WASM surface (issue #176 / CWE-150) ────── +// +// Two sub-tests: +// F5: error path — @include alias with U+001B mid-token; err.message must +// carry the sanitized \uXXXX literal and contain no raw control bytes. +// F6: lint path — frontmatter key with U+001B; first diagnostic message clean. + +/// Assert that a string contains no raw C0 (excl. \t \n), DEL, C1, bidi control, +/// line/paragraph separator, or BOM codepoint. +fn assert_no_control_chars(s: &str, label: &str) { + for (i, ch) in s.char_indices() { + let code = ch as u32; + let is_c0 = code < 0x20 && code != 0x09 && code != 0x0A; + let is_del = code == 0x7F; + let is_c1 = (0x80..=0x9F).contains(&code); + // Bidi controls (Trojan Source, CVE-2021-42574), JS line/paragraph + // separators, and the invisible BOM. + let is_format_hazard = matches!(ch, + '\u{200E}' | '\u{200F}' + | '\u{2028}' | '\u{2029}' + | '\u{202A}'..='\u{202E}' + | '\u{2066}'..='\u{2069}' + | '\u{FEFF}' + ); + assert!( + !is_c0 && !is_del && !is_c1 && !is_format_hazard, + "raw hostile char U+{code:04X} at byte {i} must not appear in {label}; got: {s:?}" + ); + } +} + +#[wasm_bindgen_test] +fn wasm_control_chars_in_error_message_are_escaped() { + // T-15 / F5 [AC-F3]: error-path sanitization for WASM surface. + // @include with a raw ESC byte (U+001B) mid-alias is rejected by the parser. + // After Change #1, serialize() sanitizes so err.message contains no raw + // control bytes and the sanitized \u001B literal is visible. + let esc = '\u{001B}'; + let source = format!("@include fo{esc}o\n"); + let err = mds_wasm::compile(&source, JsValue::NULL).unwrap_err(); + let msg = get_str(&err, "message"); + assert!( + !msg.is_empty(), + "T-15/F5: err.message must not be empty for an ESC-in-alias error" + ); + assert_no_control_chars(&msg, "err.message (T-15/F5)"); + assert!( + msg.contains("\\u001B"), + "T-15/F5: sanitized \\u001B literal must appear in err.message; got: {msg:?}" + ); +} + +#[wasm_bindgen_test] +fn wasm_lint_virtual_esc_in_module_name_sanitizes_duplicate_import_message() { + // T-15 / F6 [AC-F4]: lint-path sanitization via lintVirtual — WASM surface. + // Use a module whose NAME contains a raw ESC byte (U+001B), imported twice so + // duplicate-import fires and embeds the raw path in its message. + // After sanitization: message must contain no raw control bytes and must carry + // the sanitized \u001B literal (positive evidence). Mirrors Python E12 pattern. + // Verifies: + // (1) No raw C0/DEL/C1 bytes in any diagnostic message. + // (2) Sanitized \u001B literal IS present (positive evidence, non-vacuous). + // (3) Result shape: version 1, duplicate-import rule present. + let esc = '\u{001B}'; + let module_name = format!("fo{esc}o.mds"); + let main_src = format!("@import \"./{module_name}\"\n@import \"./{module_name}\"\n"); + + // Build the modules JS object with js_sys::Reflect so the key preserves the raw + // ESC byte as a JS string character (U+001B in UTF-16). + let modules_obj = js_sys::Object::new(); + js_sys::Reflect::set( + &modules_obj, + &JsValue::from_str(&module_name), + &JsValue::from_str("hi\n"), + ) + .unwrap(); + js_sys::Reflect::set( + &modules_obj, + &JsValue::from_str("main.mds"), + &JsValue::from_str(&main_src), + ) + .unwrap(); + + let result = mds_wasm::lint_virtual(modules_obj.into(), "main.mds", JsValue::NULL) + .expect("T-15/F6: lintVirtual must succeed with ESC in module name"); + + // (3) Result shape: version 1. + let version = get_prop(&result, "version") + .as_f64() + .expect("T-15/F6: result.version must be a number") as u32; + assert_eq!(version, 1, "T-15/F6: result.version must be 1"); + + let files = get_prop(&result, "files"); + let files_arr = js_sys::Array::from(&files); + assert!( + files_arr.length() > 0, + "T-15/F6: expected at least one file entry with diagnostics" + ); + + let mut all_messages: Vec = Vec::new(); + for i in 0..files_arr.length() { + let file_entry = files_arr.get(i); + let diags = get_prop(&file_entry, "diagnostics"); + let diags_arr = js_sys::Array::from(&diags); + for j in 0..diags_arr.length() { + let diag = diags_arr.get(j); + let msg = get_str(&diag, "message"); + // (1) No raw control bytes in any diagnostic message. + assert_no_control_chars( + &msg, + &format!("T-15/F6: files[{i}].diagnostics[{j}].message"), + ); + all_messages.push(msg); + } + } + + assert!( + !all_messages.is_empty(), + "T-15/F6: expected at least one diagnostic (duplicate-import should fire)" + ); + + // (2) At least one message contains the sanitized \u001B literal (positive evidence). + let has_sanitized = all_messages.iter().any(|m| m.contains("\\u001B")); + assert!( + has_sanitized, + "T-15/F6: expected sanitized \\u001B in at least one message; got: {all_messages:?}" + ); +} + +#[wasm_bindgen_test] +fn wasm_del_in_error_message_is_escaped() { + // T-15/F5-DEL: DEL (U+007F) in @include alias — same error-path pattern as F5 + // with a different control character. serde_json does not escape DEL by default, + // making this a load-bearing second vector. Verifies DEL is sanitized to \u007F. + let del = '\u{007F}'; + let source = format!("@include fo{del}o\n"); + let err = mds_wasm::compile(&source, JsValue::NULL).unwrap_err(); + let msg = get_str(&err, "message"); + assert!( + !msg.is_empty(), + "T-15/F5-DEL: err.message must not be empty for a DEL-in-alias error" + ); + assert_no_control_chars(&msg, "err.message (T-15/F5-DEL)"); + assert!( + msg.contains("\\u007F"), + "T-15/F5-DEL: sanitized \\u007F literal must appear in err.message; got: {msg:?}" + ); +} + +#[wasm_bindgen_test] +fn wasm_lint_virtual_nel_in_module_name_sanitizes_message() { + // T-15/F6-C1: U+0085 (NEL/C1) in lintVirtual module name — same lint-path pattern + // as F6 with a C1 control character. NEL passes serde_yaml_ng (unlike ESC/DEL), + // making it a reachable C1 vector. Verifies the sanitized … literal appears. + let nel = '\u{0085}'; + let module_name = format!("fo{nel}o.mds"); + let main_src = format!("@import \"./{module_name}\"\n@import \"./{module_name}\"\n"); + + let modules_obj = js_sys::Object::new(); + js_sys::Reflect::set( + &modules_obj, + &JsValue::from_str(&module_name), + &JsValue::from_str("hi\n"), + ) + .unwrap(); + js_sys::Reflect::set( + &modules_obj, + &JsValue::from_str("main.mds"), + &JsValue::from_str(&main_src), + ) + .unwrap(); + + let result = mds_wasm::lint_virtual(modules_obj.into(), "main.mds", JsValue::NULL) + .expect("T-15/F6-C1: lintVirtual must succeed with NEL in module name"); + + let version = get_prop(&result, "version") + .as_f64() + .expect("T-15/F6-C1: result.version must be a number") as u32; + assert_eq!(version, 1, "T-15/F6-C1: result.version must be 1"); + + let files = get_prop(&result, "files"); + let files_arr = js_sys::Array::from(&files); + assert!( + files_arr.length() > 0, + "T-15/F6-C1: expected at least one file entry with diagnostics" + ); + + let mut all_messages: Vec = Vec::new(); + for i in 0..files_arr.length() { + let file_entry = files_arr.get(i); + let diags = get_prop(&file_entry, "diagnostics"); + let diags_arr = js_sys::Array::from(&diags); + for j in 0..diags_arr.length() { + let diag = diags_arr.get(j); + let msg = get_str(&diag, "message"); + assert_no_control_chars( + &msg, + &format!("T-15/F6-C1: files[{i}].diagnostics[{j}].message"), + ); + all_messages.push(msg); + } + } + + assert!( + !all_messages.is_empty(), + "T-15/F6-C1: expected at least one diagnostic" + ); + + let has_sanitized_nel = all_messages.iter().any(|m| m.contains("\\u0085")); + assert!( + has_sanitized_nel, + "T-15/F6-C1: expected sanitized \\u0085 in at least one message; got: {all_messages:?}" + ); +} + +#[wasm_bindgen_test] +fn wasm_lint_virtual_bidi_override_in_module_name_is_escaped() { + // T-15/F6-BIDI: U+202E RIGHT-TO-LEFT OVERRIDE in a lintVirtual module name. + // U+202E is outside C0/DEL/C1, so it used to reach the wire untouched and + // reverse the display order of the rest of the line in any bidi-aware renderer + // (Trojan Source, CVE-2021-42574). "fognp.mds" renders as "fopng.mds". + // Verifies: + // (1) No raw hostile codepoint in any diagnostic message or file key. + // (2) The escaped \\u202E literal IS present (positive evidence, non-vacuous). + // (3) Result shape: version 1, duplicate-import rule present. + let rlo = '\u{202E}'; + let module_name = format!("fo{rlo}gnp.mds"); + let main_src = format!("@import \"./{module_name}\"\n@import \"./{module_name}\"\n"); + + let modules_obj = js_sys::Object::new(); + js_sys::Reflect::set( + &modules_obj, + &JsValue::from_str(&module_name), + &JsValue::from_str("hi\n"), + ) + .unwrap(); + js_sys::Reflect::set( + &modules_obj, + &JsValue::from_str("main.mds"), + &JsValue::from_str(&main_src), + ) + .unwrap(); + + let result = mds_wasm::lint_virtual(modules_obj.into(), "main.mds", JsValue::NULL) + .expect("T-15/F6-BIDI: lintVirtual must succeed with RLO in module name"); + + let version = get_prop(&result, "version") + .as_f64() + .expect("T-15/F6-BIDI: result.version must be a number") as u32; + assert_eq!(version, 1, "T-15/F6-BIDI: result.version must be 1"); + + let files = get_prop(&result, "files"); + let files_arr = js_sys::Array::from(&files); + assert!( + files_arr.length() > 0, + "T-15/F6-BIDI: expected at least one file entry with diagnostics" + ); + + let mut all_messages: Vec = Vec::new(); + let mut all_rules: Vec = Vec::new(); + for i in 0..files_arr.length() { + let file_entry = files_arr.get(i); + // Cheap invariant check only — NOT coverage of the `file`-key escape. + // The hostile RLO is in the *imported* module's name, but this key is the + // *entry* filename ("main.mds"), so no hostile byte ever reaches it and this + // assertion cannot fail via this vector (PF-013: it would pass even if the + // `file`-key sanitizer were deleted). Real coverage of the `file` key lives in + // mds-core `to_canonical_json_escapes_bidi_override`, which constructs a + // diagnostic with `file: Some("ma\u{202E}in.mds")` directly. + assert_no_control_chars( + &get_str(&file_entry, "file"), + &format!("T-15/F6-BIDI: files[{i}].file"), + ); + let diags = get_prop(&file_entry, "diagnostics"); + let diags_arr = js_sys::Array::from(&diags); + for j in 0..diags_arr.length() { + let diag = diags_arr.get(j); + let msg = get_str(&diag, "message"); + assert_no_control_chars( + &msg, + &format!("T-15/F6-BIDI: files[{i}].diagnostics[{j}].message"), + ); + all_messages.push(msg); + all_rules.push(get_str(&diag, "rule")); + } + } + + assert!( + !all_messages.is_empty(), + "T-15/F6-BIDI: expected at least one diagnostic" + ); + assert!( + all_rules.iter().any(|r| r == "duplicate-import"), + "T-15/F6-BIDI: expected duplicate-import; got rules: {all_rules:?}" + ); + + let has_escaped_rlo = all_messages.iter().any(|m| m.contains("\\u202E")); + assert!( + has_escaped_rlo, + "T-15/F6-BIDI: expected escaped \\u202E in at least one message; got: {all_messages:?}" + ); +} + +#[wasm_bindgen_test] +fn wasm_lint_virtual_newline_in_frontmatter_key_is_escaped_on_the_wire() { + // T-15/F6-NL [PF-007]: cross-surface parity for the WIRE-mode `\n` escape. + // + // `lint_virtual` returns `LintResult::to_canonical_json()`, which sanitizes in + // WIRE mode — so a raw newline inside a diagnostic message becomes the literal + // six-character escape (backslash-u-0-0-0-A). Without this test the WASM surface + // is the only one of the five with no assertion pinning that behaviour, which is + // exactly the per-surface blind spot PF-007 describes: each surface's own golden + // passes while the surfaces silently diverge from one another. + // + // Log/YAML-key forging: a raw newline in a message lets an attacker forge what + // reads as a second, independent finding in any line-oriented consumer. + // + // Reachability: a newline inside an `@import "..."` path is rejected by the + // lexer, so that route is vacuous. A YAML *double-quoted* frontmatter key is + // not — serde_yaml_ng decodes the `\n` escape into a real newline, and + // unused-variable embeds the decoded key verbatim in its message. + // + // Mirrors napi E-15 and universal-JS U-E14 exactly (same vector, same + // assertions) so the three surfaces are differentially comparable. + let source = "---\n\"a\\nerror[mds::forged]: FAKE\\nb\": 1\n---\nHello\n"; + + let modules_obj = js_sys::Object::new(); + js_sys::Reflect::set( + &modules_obj, + &JsValue::from_str("main.mds"), + &JsValue::from_str(source), + ) + .unwrap(); + + let result = mds_wasm::lint_virtual(modules_obj.into(), "main.mds", JsValue::NULL) + .expect("T-15/F6-NL: lintVirtual must succeed with a newline in a frontmatter key"); + + let version = get_prop(&result, "version") + .as_f64() + .expect("T-15/F6-NL: result.version must be a number") as u32; + assert_eq!(version, 1, "T-15/F6-NL: result.version must be 1"); + + let files_arr = js_sys::Array::from(&get_prop(&result, "files")); + let mut all_messages: Vec = Vec::new(); + let mut all_rules: Vec = Vec::new(); + for i in 0..files_arr.length() { + let file_entry = files_arr.get(i); + let diags_arr = js_sys::Array::from(&get_prop(&file_entry, "diagnostics")); + for j in 0..diags_arr.length() { + let diag = diags_arr.get(j); + all_messages.push(get_str(&diag, "message")); + all_rules.push(get_str(&diag, "rule")); + } + } + + // Non-vacuity guard: the vector must actually have reached the guarded path. + assert!( + all_rules.iter().any(|r| r == "unused-variable"), + "T-15/F6-NL: expected unused-variable; got rules: {all_rules:?}" + ); + + // Negative: no raw newline may survive into a wire message. + for msg in &all_messages { + assert!( + !msg.contains('\n'), + "T-15/F6-NL: raw newline must not survive into the wire message; got: {msg:?}" + ); + } + + // Positive (PF-013): the escaped form must be present. + assert!( + all_messages.iter().any(|m| m.contains("\\u000A")), + "T-15/F6-NL: expected escaped \\u000A in at least one message; got: {all_messages:?}" + ); + + // Escaped, not stripped — the payload text itself is preserved verbatim. + assert!( + all_messages + .iter() + .any(|m| m.contains("error[mds::forged]")), + "T-15/F6-NL: message body must be preserved verbatim; got: {all_messages:?}" + ); +} diff --git a/packages/mds/__test__/error.spec.mjs b/packages/mds/__test__/error.spec.mjs index c12f3e10..3f42f64b 100644 --- a/packages/mds/__test__/error.spec.mjs +++ b/packages/mds/__test__/error.spec.mjs @@ -1,10 +1,10 @@ /** * Error shape tests for @mdscript/mds universal package. - * Tests: U-E1 through U-E9 + * Tests: U-E1 through U-E10 */ import { test, describe, before } from 'node:test'; import assert from 'node:assert/strict'; -import { compile, check, isMdsError, init } from '../dist/node.js'; +import { compile, check, isMdsError, init, lintVirtual } from '../dist/node.js'; describe('error shape', () => { before(() => init()); @@ -87,4 +87,260 @@ describe('error shape', () => { assert.ok(err.message.length > 0, 'error message should not be empty'); } }); + + // T-11 / U-E10..U-E-DIFF [AC-F3, AC-F4]: ESC-injection hardening (issue #176 / CWE-150). + // `@include foo` — alias contains a raw ESC byte (U+001B) mid-token so + // trim() cannot strip it. The parser rejects the alias as an invalid identifier + // and produces a MdsError::Syntax whose message interpolates the raw alias. + // After the fix, err.message must carry the sanitized 6-char \u001B literal and + // must contain no raw C0/DEL/C1 bytes. + // Helper: assert no raw C0 (excl. \t \n), DEL, or C1 chars in a string. + // Uses charCodeAt (UTF-16 code units); all C0/DEL/C1 codepoints are in BMP so + // charCodeAt correctly identifies them without surrogate pair handling. + function assertNoControlChars(s, label) { + for (let i = 0; i < s.length; i++) { + const code = s.charCodeAt(i); + const isC0 = code < 0x20 && code !== 0x09 && code !== 0x0a; + const isDel = code === 0x7f; + const isC1 = code >= 0x80 && code <= 0x9f; + // Bidi controls (Trojan Source, CVE-2021-42574), U+2028/U+2029 (JS string + // literal terminators), and U+FEFF (invisible BOM). `\n` is allowed here; + // wire-mode newline escaping is asserted explicitly by U-E14. + const isFormatHazard = + code === 0x200e || code === 0x200f || + code === 0x2028 || code === 0x2029 || + (code >= 0x202a && code <= 0x202e) || + (code >= 0x2066 && code <= 0x2069) || + code === 0xfeff; + assert.ok( + !isC0 && !isDel && !isC1 && !isFormatHazard, + `${label}: raw hostile char U+${code.toString(16).toUpperCase().padStart(4,'0')} ` + + `at index ${i} must not appear; got: ${JSON.stringify(s)}` + ); + } + } + + test('U-E10: control chars in error message are escaped to \\uXXXX literals', () => { + // Build source string with raw ESC (0x1B) mid-alias at runtime to avoid any + // editor/tool stripping the control byte. + const esc = String.fromCharCode(0x1b); + const source = `@include fo${esc}o\n`; + try { + compile(source); + assert.fail('expected error to be thrown'); + } catch (err) { + assert.ok(isMdsError(err), `expected MdsError, got: ${err}`); + const msg = err.message; + assert.ok(typeof msg === 'string' && msg.length > 0, + 'message must be a non-empty string'); + assertNoControlChars(msg, 'U-E10: err.message'); + // Sanitized literal \u001B must be present. + assert.ok( + msg.includes('\\u001B'), + `sanitized \\u001B literal must appear in err.message; got: ${JSON.stringify(msg)}` + ); + } + }); + + test('U-E11: DEL (U+007F) in error message is escaped to \\u007F literal', () => { + // DEL (U+007F) in @include alias — serde_json does NOT escape DEL by default, + // so this is a distinct load-bearing vector from U-E10 (ESC). + const del = String.fromCharCode(0x7f); + const source = `@include fo${del}o\n`; + try { + compile(source); + assert.fail('expected error to be thrown'); + } catch (err) { + assert.ok(isMdsError(err), `U-E11: expected MdsError, got: ${err}`); + const msg = err.message; + assert.ok(typeof msg === 'string' && msg.length > 0, + 'U-E11: message must be a non-empty string'); + assertNoControlChars(msg, 'U-E11: err.message'); + assert.ok( + msg.includes('\\u007F'), + `U-E11: sanitized \\u007F literal must appear in err.message; got: ${JSON.stringify(msg)}` + ); + } + }); + + test('U-E12: U+0085 (NEL/C1) in lintVirtual module name is sanitized in diagnostic message', () => { + // U+0085 (NEL) is a C1 control char that passes serde_yaml_ng YAML parsing + // (unlike ESC/DEL), making it a reachable C1 ESC-injection vector for lintVirtual. + // The duplicate-import rule fires and embeds the raw module name in its message. + const nel = String.fromCharCode(0x85); + const moduleName = `fo${nel}o.mds`; + const modules = { + [moduleName]: 'hi\n', + 'main.mds': `@import "./${moduleName}"\n@import "./${moduleName}"\n`, + }; + const result = lintVirtual(modules, 'main.mds'); + assert.equal(result.version, 1, 'U-E12: version must be 1'); + const allDiags = result.files.flatMap((f) => f.diagnostics); + assert.ok( + allDiags.length > 0, + 'U-E12: expected at least one diagnostic; got: ' + JSON.stringify(allDiags), + ); + for (const diag of allDiags) { + if (typeof diag.message === 'string') { + assertNoControlChars(diag.message, `U-E12: diag[${diag.rule}].message`); + } + } + const hasSanitizedNel = allDiags.some( + (d) => typeof d.message === 'string' && d.message.includes('\\u0085'), + ); + assert.ok( + hasSanitizedNel, + 'U-E12: expected \\u0085 in at least one diagnostic message; got: ' + + JSON.stringify(allDiags.map((d) => d.message)), + ); + }); + + test('U-E13: U+202E (RLO) in lintVirtual module name is escaped on the wire', () => { + // Trojan Source (CVE-2021-42574). U+202E is outside C0/DEL/C1, so it used to + // reach the wire untouched and reverse how the rest of the line displays. + const rlo = String.fromCharCode(0x202e); + const moduleName = `fo${rlo}gnp.mds`; + const modules = { + [moduleName]: 'hi\n', + 'main.mds': `@import "./${moduleName}"\n@import "./${moduleName}"\n`, + }; + const result = lintVirtual(modules, 'main.mds'); + assert.equal(result.version, 1, 'U-E13: version must be 1'); + const allDiags = result.files.flatMap((f) => f.diagnostics); + assert.ok( + allDiags.some((d) => d.rule === 'duplicate-import'), + 'U-E13: expected duplicate-import; got rules: ' + + JSON.stringify(allDiags.map((d) => d.rule)), + ); + for (const diag of allDiags) { + if (typeof diag.message === 'string') { + assertNoControlChars(diag.message, `U-E13: diag[${diag.rule}].message`); + } + } + // Cheap invariant check only — NOT coverage of the `file`-key escape. The + // hostile RLO is in the *imported* module's name, but this key is the *entry* + // filename ("main.mds"), so no hostile byte reaches it and this cannot fail via + // this vector (PF-013). Real `file`-key coverage: mds-core + // `to_canonical_json_escapes_bidi_override`. + for (const f of result.files) { + assertNoControlChars(f.file, 'U-E13: files[].file'); + } + assert.ok( + allDiags.some((d) => typeof d.message === 'string' && d.message.includes('\\u202E')), + 'U-E13: expected \\u202E in at least one diagnostic message; got: ' + + JSON.stringify(allDiags.map((d) => d.message)), + ); + }); + + test('U-E14: newline in a frontmatter key is escaped to \\u000A on the wire', () => { + // Log-forging guard: a raw newline in a diagnostic message lets an attacker + // forge what reads as a second, independent finding in any line-oriented + // consumer of the JSON string value. + // + // Reachability: a newline inside an `@import "..."` path is rejected by the + // lexer (vacuous route). A YAML double-quoted frontmatter key is not — the + // \n escape decodes to a real newline that unused-variable embeds verbatim. + const source = + '---\n"a\\nerror[mds::forged]: FAKE\\nb": 1\n---\nHello\n'; + const result = lintVirtual({ 'main.mds': source }, 'main.mds'); + assert.equal(result.version, 1, 'U-E14: version must be 1'); + const allDiags = result.files.flatMap((f) => f.diagnostics); + assert.ok( + allDiags.some((d) => d.rule === 'unused-variable'), + 'U-E14: expected unused-variable; got rules: ' + + JSON.stringify(allDiags.map((d) => d.rule)), + ); + for (const diag of allDiags) { + assert.ok( + !diag.message.includes('\n'), + `U-E14: raw newline must not survive into the wire message; got: ${JSON.stringify(diag.message)}`, + ); + } + assert.ok( + allDiags.some((d) => d.message.includes('\\u000A')), + 'U-E14: expected \\u000A in at least one diagnostic message; got: ' + + JSON.stringify(allDiags.map((d) => d.message)), + ); + // Escaped, not stripped. + assert.ok( + allDiags.some((d) => d.message.includes('error[mds::forged]')), + 'U-E14: message body must be preserved verbatim; got: ' + + JSON.stringify(allDiags.map((d) => d.message)), + ); + }); + + test('U-E-DIFF: native and WASM lintVirtual produce identical results for ESC-injection input', async () => { + // Differential assertion: the same ESC-injection input run through both the + // native (napi) and WASM backends must produce deeply equal results. + // Skips gracefully when either backend is unavailable locally; must run in CI + // where both backends are built. + let native; + try { + const { createNativeBackend } = await import('../dist/backend/native.js'); + const { createRequire } = await import('node:module'); + const { fileURLToPath } = await import('node:url'); + const { join, dirname } = await import('node:path'); + const testDir = dirname(fileURLToPath(import.meta.url)); + const require = createRequire(import.meta.url); + const napiAddon = require(join(testDir, '../../../crates/mds-napi/index.js')); + native = createNativeBackend(napiAddon); + } catch { + return; // native backend not available — skip + } + + let wasm; + try { + const { initWasmNode, createWasmBackend } = await import('../dist/backend/wasm.js'); + const wasmModule = await initWasmNode(); + wasm = createWasmBackend(wasmModule); + } catch { + return; // WASM backend not available — skip + } + + // One vector covering every escape class: C0 (ESC), C1 (NEL), bidi override + // (RLO), JS line separator, BOM — carried in the module NAME — plus the + // wire-mode newline, carried in a YAML double-quoted frontmatter key (a + // newline inside an `@import "..."` path is rejected by the lexer, so the + // module-name route is unreachable for that one character). + // PF-007: a per-surface golden cannot catch cross-surface divergence, so the + // widened class has to be exercised through the differential too. + const esc = String.fromCharCode(0x1b); + const nel = String.fromCharCode(0x85); + const rlo = String.fromCharCode(0x202e); + const ls = String.fromCharCode(0x2028); + const bom = String.fromCharCode(0xfeff); + const moduleName = `fo${esc}${nel}${rlo}${ls}${bom}o.mds`; + const mainSource = + '---\n"a\\nerror[mds::forged]: FAKE\\nb": 1\n---\n' + + `@import "./${moduleName}"\n@import "./${moduleName}"\n`; + const modules = { + [moduleName]: 'hi\n', + 'main.mds': mainSource, + }; + + const nativeResult = native.lintVirtual(modules, 'main.mds'); + const wasmResult = wasm.lintVirtual(modules, 'main.mds'); + + // Non-vacuity: the differential is worthless if neither backend produced + // diagnostics carrying the escaped forms. duplicate-import carries the module + // name (ESC/NEL/RLO/LS/BOM); unused-variable carries the frontmatter key (\n). + const nativeMessages = nativeResult.files + .flatMap((f) => f.diagnostics) + .map((d) => d.message) + .filter((m) => typeof m === 'string'); + for (const escaped of ['\\u001B', '\\u0085', '\\u202E', '\\u2028', '\\uFEFF', '\\u000A']) { + assert.ok( + nativeMessages.some((m) => m.includes(escaped)), + `U-E-DIFF: expected ${escaped} in some message; got: ` + + JSON.stringify(nativeMessages), + ); + } + + // deepEqual of plain-object round-trip proves wire-format parity. + assert.deepEqual( + JSON.parse(JSON.stringify(nativeResult)), + JSON.parse(JSON.stringify(wasmResult)), + 'U-E-DIFF: native and WASM lintVirtual must produce identical results for the same input', + ); + }); }); diff --git a/spec.md b/spec.md index a74a110f..28c23237 100644 --- a/spec.md +++ b/spec.md @@ -994,6 +994,185 @@ mode, which uses a single config located from the directory argument. Keys are in alphabetical order (BTreeMap serialization). `"truncated": true` when the result set was capped by the per-file diagnostic cap of 1,000. `"span"` is absent for diagnostics that lack a source location. +#### Sanitization invariant (v1) + +Under `"version": 1`, the following guarantees are normative. The prior behavior of +passing raw control bytes through to JSON is superseded. + +The **escaped class** is: + +| Codepoints | Why | +|------------|-----| +| C0 (U+0000–U+001F) except `\t` (U+0009) | Terminal escape-sequence injection (CWE-150) | +| `\n` (U+000A) | Line forging in any consumer that prints or line-splits the value | +| DEL (U+007F) | Interpreted as a destructive backspace by some terminals | +| C1 (U+0080–U+009F) | Terminal control, incl. NEL (U+0085) | +| U+061C, U+200E, U+200F, U+202A–U+202E, U+2066–U+2069 | The complete Unicode `Bidi_Control=Yes` set (12 codepoints) — they visually reorder the line (Trojan Source, CVE-2021-42574). U+061C ARABIC LETTER MARK is the only member outside U+200E–U+2069. | +| U+2028, U+2029 | Terminate a JavaScript string literal | +| U+FEFF | Invisible BOM / ZWNBSP — hides or splits content | + +Each is replaced with its six-character `\uXXXX` literal (uppercase hex) before +serialization. `\t` (U+0009) is the sole exemption from the C0 range: it is never +escaped, in either mode. + +| Field | Invariant | +|-------|-----------| +| `message`, `help` | Every codepoint in the escaped class above is replaced with its six-character `\uXXXX` literal before serialization. | +| `file` | Sanitized on the same pass as `message`/`help`. Hostile filenames cannot inject control, bidi, or separator characters into this JSON output. A filename occupying one of the **diagnostic** `file` fields — this JSON key, a CLI status line, or a `[file:line:col]` frame header — is escaped with the **full** class including `\n` on each of those, human surfaces included, because it is always rendered on a single line and POSIX permits a newline inside a filename. Two path positions are outside that rule and are **not** escaped: a path interpolated into a diagnostic *message body*, which is prose (see "Residual" below), and a path in a source map or in `CompileResult.dependencies`, which is a functional reference (see "Carve-out" below). | +| `rule` | Fixed ASCII identifier; never contains control bytes by construction. Not sanitized. | +| `span`, `fix_edits` | **Raw byte offsets** into the unmodified source — deliberately not sanitized. These are numeric position values and must reflect the original source exactly. | + +This invariant applies across all surfaces that emit `"version": 1` JSON: CLI +(`mds lint --format json`), napi (`lintVirtual` / `lint` / `lintFile`), WASM +(`lintVirtual` / `lint`), and Python (`lint_virtual` / `lint` / `lint_file`). +All four surfaces emit byte-identical values. + +##### Mode is chosen per field, not per surface + +The escape class above is fixed. The only thing that varies is whether `\n` is escaped +with it, and that choice is **normatively a property of the field, not of the output +surface**: + +> **On the diagnostic surfaces — the `"version": 1` JSON wire, CLI status and warning +> lines, and `[file:line:col]` frame headers — untrusted identifiers, filenames, and +> error causes are escaped in WIRE mode, human terminal output included. Prose — a +> diagnostic message body or help body — is escaped in HUMAN mode on terminal surfaces, +> so that multi-line frames keep rendering.** + +The rule governs *diagnostic* output. Two categories of output are named carve-outs and +are not escaped at all, because escaping them would destroy their function rather than +protect it: the command's **product** (compiled template output) and **functional path +references** (source-map `file`/`sources`, `CompileResult.dependencies`). Both are +listed in the table below and the second is specified under "Carve-out" further down. + +The discriminator is whether the value is ever *legitimately* multi-line. A filename, a +config key, a `--format` argument, an `io::Error` cause, and a fix-rejection reason are +each displayed on exactly one line, so preserving a raw `\n` in one buys nothing and +lets it forge a standalone line that is byte-identical in form to genuine output +(CWE-117). A diagnostic body genuinely is multi-line, so escaping its newlines would +break the frame. + +This rule supersedes any per-surface reading of the earlier "human escapes the class +minus `\n`" formulation: `\n` is escaped on all machine-readable boundaries listed +above, **and** on every identifier / filename / cause field of a diagnostic, on every +surface that renders one — human terminal output included. It says nothing about the +two carve-outs, which are not diagnostics. + +Applied, that means: + +| Value | Mode | Because | +|-------|------|---------| +| `message`, `help`, warning bodies, `LabeledSpan` text | HUMAN on terminal surfaces, WIRE on the JSON wire | Prose; legitimately multi-line in a rendered frame | +| A filename or path in a diagnostic `file` **field**: the JSON `file` key, a CLI status line, a `[file:line:col]` frame header | WIRE on every surface that renders one | Single-line by construction; POSIX permits `\n` in a filename and the user never types it | +| `mds.json` rule names and config values, `--format` arguments | WIRE on every surface that renders one | Single-line identifiers read from the working tree or the command line | +| `io::Error` / `MdsError` causes interpolated into a CLI status or warning line | WIRE on every surface that renders one | Single-line, and they embed paths of their own | +| A path, identifier or cause interpolated into a diagnostic **message body** | HUMAN on terminal surfaces, WIRE on the JSON wire | Follows the message row above — it is part of prose. This is the **residual** below: it is not covered by the WIRE rows | +| Compiled template output (`mds build -o -`) | not escaped | It is the command's product, not a diagnostic; redirects must stay byte-faithful | +| Source-map `file` / `sources` / `sourcesContent`, and `CompileResult.dependencies` | not escaped | Functional references, not display text; escaping would break resolution. This is the **carve-out** below | + +Source excerpts embedded in a rendered diagnostic frame are neutralized +byte-length-preservingly instead of escaped, so span offsets and caret columns stay +exact. The substitute is chosen per UTF-8 width: 1-byte C0/DEL → `?`; 2-byte C1 and +U+061C → U+00A0; 3-byte bidi controls, separators and BOM → U+FFFD. + +On the CLI this is enforced at a single choke-point: every diagnostic printed to +stderr — compiler errors and CLI-authored errors alike — has its message, help, and +caret-label text escaped **before** the diagnostic renderer runs. The rendered frame is +never post-processed, so the renderer's own terminal styling is left intact and caret +columns stay aligned. + +##### Residual: paths and identifiers inside a message body + +The rule above is per field, and a value interpolated into a diagnostic **message body** +is part of that body. It is therefore escaped in HUMAN mode on terminal surfaces, which +preserves `\n`. Two construction sites produce such messages: + +- **CLI `miette::miette!()` messages**, which interpolate `mds.json` values and + filesystem paths. +- **`mds-core` `MdsError` message bodies**, which interpolate paths and `io::Error` + causes (`cannot read {path}: {e}`, `invalid UTF-8 in {path}: {e}` in + `crates/mds-core/src/fs.rs`) and template identifiers (`invalid import alias: + '{alias}'` in `parser_helpers.rs`). + +Both are **known residuals, not closed boundaries**, and they are the same defect at two +different construction sites. A hostile path or identifier containing `\n` survives into +the rendered frame and occupies a line of its own there. + +The residual is a *weaker* surface than a status line, and deliberately so: everything +inside a rendered frame is indented and `│`-prefixed by the renderer, and that prefix +survives `strip()`, so forged frame content cannot masquerade as a bare CLI status line +the way an unescaped filename in a `Clean: …` line could. No raw control byte reaches +the terminal from either path — HUMAN mode still escapes the whole class except `\n`. + +Closing it would mean WIRE-escaping every untrusted interpolation at every `MdsError` +and `miette!()` construction site — over a hundred in `mds-core` alone — and changing +the public `MdsError` message text seen by all three binding layers. That is a larger, +separately-specified change; until it is made, this section is the disclosure, not a +gap someone forgot. + +##### Carve-out: functional path references (source maps, `dependencies`) + +Source-map documents and `CompileResult.dependencies` are **explicitly outside** the +per-field rule. The paths they carry are emitted **verbatim** — no escaping, no +neutralization — in every one of these positions: + +- the sidecar written by `mds build --source-map` (`.map`): its `file` key, + every entry of `sources`, and every entry of `sourcesContent`; +- the `sourceMap` object embedded in `CompileResult::to_canonical_json()`, and hence in + the napi / WASM / Python compile results; +- the `dependencies` array of `CompileResult::to_canonical_json()`. + +These are **functional references, not display text**. Source Map v3 `file` and +`sources` are resolved against the filesystem by devtools, bundlers and IDEs; +`dependencies` is a watch/rebuild input for the bundler plugins. Rewriting a path to a +`\uXXXX` literal would produce a path that does not exist, breaking source-map +resolution and dependency tracking in order to defend against a pathological filename. +That is the same product-versus-display distinction that keeps compiled output +unescaped: escaping the artefact corrupts the artefact. + +Consequently, and normatively: + +> **Consumers of a source map or of `dependencies` MUST treat every path they contain +> as untrusted input.** A path may contain any byte a filesystem permits, including C0 +> control characters, `\n`, bidi controls and U+FEFF. A consumer that prints such a path +> to a terminal, writes it into a log line, or interpolates it into HTML must escape it +> for that destination itself. JSON string encoding is *not* that escaping: it makes the +> document parseable, and a decoded `"\n"` is a real newline again. + +The CLI does not rely on this contract for its own output: the `Compiled to …` and +`Source map written to …` status lines print the path through `safe_path`, so they carry +the WIRE-escaped form even though the sidecar they name does not. + +Closing this differently — rejecting control characters in filenames at the input +boundary rather than escaping them at output — is a plausible longer-term design and is +deliberately not specified here. + +##### Escaping is one-way + +The transformation is **lossy and non-injective, by design**. A template that +literally contains the six characters `\`, `u`, `0`, `0`, `1`, `B` and a template +containing an actual ESC byte both serialize to the identical six-character +string `\u001B`; +after serialization they are indistinguishable. + +Consumers **MUST NOT** un-escape `\uXXXX` sequences back into bytes. Doing so +reconstitutes exactly the injection this invariant prevents — an attacker who +controls a diagnostic message controls what a naive un-escaper writes to your +terminal. The escape exists for display, not for transport. + +Round-tripping is an explicit **non-goal**: no backslash-escaping (`\` → `\\`) +will be added to make the mapping reversible, in this or any later wire version. A +consumer that needs the original bytes must read them from the source file using the +raw `span` / `fix_edits` byte offsets, which are deliberately left unsanitized for +precisely this purpose. + +##### `--diff` preview output (`mds lint --fix --diff` and `mds fmt --diff`) + +Preview output is diff text, not a diagnostic field, and is governed separately: it +is neutralized when stdout is a TTY (where control bytes would execute), and emitted +**byte-faithful when stdout is piped or redirected** (where the diff must remain +applicable). It is not part of the `"version": 1` JSON wire format. + ### 7.6 `mds init` ```bash