Windows: _pip_install's commit destroys existing deps packages while the MCP server is running (rmtree + os.replace vs locked .pyd) — armed on every session start when the ML stack is missing

[mbe14]

Environment: Windows 11 Pro (build 26200), Python 3.13.13, cortex plugin 4.13.0 installed through the marketplace. Same box as #91–#95. Found while diagnosing the SessionStart charmap crash (#96).

Setup that arms the bug

The 4.13.0 plugin cache's active deps dir (~\.claude\plugins\cache\cortex-plugins\cortex\4.13.0\deps) came up with the base stack (~200 top-level entries) but without the ML stack (no sentence_transformers / torch / flashrank). Since session_start is the one entry point that calls _ensure_all_deps (scripts/launcher.py:303-304), every session start walks into _pip_install(deps_dir, ["sentence-transformers==5.4.1", "flashrank==0.2.10"]).

The bug

_pip_install's commit step (scripts/launcher.py:246-252) moves each top-level entry from the temp target into deps/:

for entry in os.listdir(tmp_dir):
    dest = os.path.join(deps_dir, entry)
    if os.path.isdir(dest):
        shutil.rmtree(dest, ignore_errors=True)
    elif os.path.exists(dest):
        os.remove(dest)
    os.replace(os.path.join(tmp_dir, entry), dest)

pip resolves numpy as a transitive dep of sentence-transformers, so tmp_dir contains a numpy entry whose dest already exists. Meanwhile the MCP server — spawned in parallel by Claude Code at the same session start — has numpy imported, so its .pyd files are mapped and locked. On Windows that means:

1. shutil.rmtree(dest, ignore_errors=True) deletes everything in deps/numpy except the locked .pyd files (silent, ignore_errors=True),
2. os.replace fails: PermissionError: [WinError 5] Access is denied: '...\deps.tmp-11640\numpy' -> '...\deps\numpy',
3. the exception propagates and the finally: shutil.rmtree(tmp_dir, ignore_errors=True) (lines 253-254) deletes the freshly installed copy too.

Net result on my box: deps/numpy reduced to a husk — only _core/ and linalg/ (the dirs holding the locked .pyds) survived; numpy-2.5.1.dist-info and numpy.libs only survived because the loop crashed before reaching them alphabetically.

  File "...\cortex\4.13.0\scripts\launcher.py", line 304, in main
    _ensure_all_deps(deps_dir)
  File "...\cortex\4.13.0\scripts\launcher.py", line 262, in _ensure_all_deps
    _pip_install(
  File "...\cortex\4.13.0\scripts\launcher.py", line 252, in _pip_install
    os.replace(os.path.join(tmp_dir, entry), dest)
PermissionError: [WinError 5] Access is denied: '...\deps.tmp-11640\numpy' -> '...\deps\numpy'

Aftermath / blast radius

• Every subsequent hook spawn finds numpy as a namespace husk; _importable's husk eviction can't fully delete it either (same locks), so each hook invocation re-attempts a multi-hundred-MB pip install and re-fails at the same os.replace — per prompt, for the rest of the session.
• Entries alphabetically before numpy (e.g. flashrank, huggingface_hub, …) get silently replaced with whatever pip just resolved before the crash — partial, unversioned drift of shared deps.
• Whether a given session start corrupts or succeeds is a race against the server importing numpy, so the failure is intermittent and easy to misattribute.

Suggested fixes (any subset)

1. Idempotence guard: skip an entry when dest exists and its *.dist-info already satisfies the pin — shared transitive deps like numpy should never be touched at all.
2. Non-destructive commit: copy-merge (no-clobber) instead of rmtree + os.replace, or rename dest aside and roll back on failure; and don't delete tmp_dir in finally when the commit failed — leaving it is exactly what makes the current failure unrecoverable.
3. Take the ML-stack ensure out of the SessionStart hot path: a one-time bootstrap with a success stamp + file lock can't race the server boot.
4. Cheaper presence probe: _importable("sentence_transformers", ...) imports torch inside the hook process just to answer "is it installed"; a dist-info check is enough.

Workaround used

Out-of-band pip install --target <staging> of numpy==2.5.1 sentence-transformers==5.4.1 flashrank==0.2.10, then a no-clobber copy into deps/ (locked files are same-version, so skipping them is safe). With the ML stack present, _ensure_all_deps is a no-op and the destructive path is unreachable.

[cdeust]

Fixed on main (80bf474b) — all four of your suggestions, they were the right ones:

1. Idempotence guard: the commit now skips any entry whose dist-info in deps/ already satisfies the pin — shared transitives like numpy are never touched at all (this alone closes your common case).
2. Non-destructive commit: no more blind rmtree; dest is renamed aside, replaced, and the backup removed only on success. On failure dest is restored AND tmp_dir is preserved (the unconditional finally: rmtree(tmp_dir) that made your failure permanent is gone). Verified by replaying the OLD loop shape against a simulated locked file — it reproduces your exact husk-plus-lost-tmp outcome; the new one restores dest bit-for-bit.
3. Out of the hot path: first successful bootstrap writes a version-pinned success stamp guarded by a directory lock (stdlib-only, 30s bounded wait so the hook can never hang past Claude Code's timeout); subsequent session starts check the stamp, not the packages.
4. Cheap probe: presence is now answered by dist-info inspection — no more importing torch inside the hook process.

The launcher was also split (scripts/launcher_deps.py) so this commit logic has an isolated, tested surface (21 new tests, incl. mid-commit PermissionError rollback).

Epistemic caveat as always: the WinError 5 was simulated via a monkeypatched os.replace, not a genuine Windows file lock. If your box confirms (a) no more per-prompt re-install loop and (b) numpy intact after a session start with the ML stack missing, we close. Your out-of-band no-clobber staging workaround remains safe in the meantime — the idempotence guard will simply no-op over it. Ships in the next plugin release.

[cdeust]

v4.13.1 is out with this fix — please verify on the installed release rather than main: https://github.com/cdeust/Cortex/releases/tag/v4.13.1

[mbe14]

Confirmed on the real box — Windows 11 Pro (build 26200), Python 3.13.13, updated to 4.13.1 through the marketplace, fresh version dir bootstrapped from scratch. Both of your asks verified, plus I closed your epistemic gap: every test below ran against genuine Windows locks (a live process holding numpy's image-mapped .pyds from deps/, same lock class as the running MCP server), not a monkeypatched os.replace.

(b) numpy intact, common case. Base bootstrap first (74 dists, base stamp written), then a lock-holder process importing numpy from deps/, then ensure_all_deps with the ML stack missing — the exact arming state from the report. Result: exit 0 in 54s (warm pip cache), ML committed, both stamps written, and numpy byte-identical before/after — a 1330-file inventory (relative path + size + mtime) diffs empty. The idempotence guard did exactly its job under real locks: tmp's transitive numpy matched dest, so numpy never entered the commit path.

(a) no reinstall loop. With the stamps in place (locks still held), repeated ensure_all_deps calls cost 0.2–0.3 ms — pure stamp read, no pip subprocess, no imports. The per-prompt multi-hundred-MB retry is gone.

Worst case forced (guard miss): non-destructive and self-healing. I then forced the case the guard can't cover — dest numpy 2.4.4 (locked), pip resolving 2.5.1 into tmp — so the commit entered the previously-fatal path for real. Finding: on NTFS the rename-aside succeeds — image-mapped .pyds resist deletion, not parent-directory rename. So dest → .bak worked under the lock, the new numpy landed, ML completed, stamp written, exit 0, and the lock-holder process ran undisturbed throughout (its mapped files survive inside the .bak). Notably, the genuine lock never made os.replace(src, dest) raise: 4.13.0's WinError 5 came from replacing onto the non-empty husk its own rmtree had just created — a state the rename-aside design never produces. Your simulated rollback test remains the only coverage of the restore leg, but empirically it's a rarer path than the old failure suggested.

Three residue observations from that worst-case run (minor, non-blocking):

1. *.bak-<pid> husks linger when the aside-backup holds locked files — shutil.rmtree(backup, ignore_errors=True) silently keeps them (here: numpy.bak-<pid> reduced to just the 2 mapped .pyds, numpy.libs.bak-<pid> likewise). Dead weight once the locking process exits; an opportunistic sweep of stale *.bak-* on a later run would tidy it.
2. Superseded dist-info survives a cross-version commit: numpy-2.4.4.dist-info stayed next to the new numpy-2.5.1.dist-info (the old one isn't an entry in tmp, so the loop never touches it). _dist_info_versions's dict happens to prefer the newer key, but pip/importlib.metadata now see duplicate metadata for one dist.
3. _importable probes the whole process sys.path, so a base pin already satisfied by the HOST interpreter's site-packages (numpy is installed globally on this box) is dropped from the install set, and the package enters deps/ only as an unpinned transitive (2.5.1 via pgvector, vs. the 2.4.4 pin). Deps content thus depends on the host's global site-packages, and the base-vs-ML version agreement that arms the idempotence guard becomes resolution luck. A constraints file (-c with the base pins) on the ML install — or making the install decision dist-info-based like the stamp check — would pin it down.

Bottom line: (a) and (b) both hold on real Windows, and the guard-miss case degrades to self-healing instead of destructive. My out-of-band staging workaround is retired. Good to close from my side.

[cdeust]

Closing on your real-box verification — and thank you for going beyond the ask: testing under genuine image-mapped locks closed the epistemic gap our simulated os.replace could not, and the NTFS insight (locks resist deletion, not parent rename — so 4.13.0's WinError 5 was the replace hitting the husk its own rmtree had just created, a state the rename-aside design never produces) is now part of the record.

Your three residue observations are all real and are being fixed for the next release: (1) opportunistic sweep of stale *.bak-<pid> on later runs; (2) superseded dist-info cleanup on cross-version commits; (3) the substantive one — _importable letting host site-packages satisfy base pins makes deps content host-dependent; moving the install decision to dist-info-in-deps (like the stamp check) + a constraints file on the ML install will make the bootstrap hermetic. Glad the staging workaround is retired.