fix(mcp-analytics): hand off cleanly when the SDK install is blocked#929
Draft
sarahxsanders wants to merge 8 commits into
Draft
Conversation
David's run reported "MCP analytics configured!" on a project that didn't compile: the agent made the edits, but `yarn add` was blocked by a filesystem sandbox limit (his package is a monorepo member, so the install had to write a hoisted node_modules outside the agent's writable directory). The failure was buried in the report file and the run still finished as a success. Add a `manual install required` abort case so a blocked install surfaces a clear handoff outro with the command to run, instead of a false success. Rides on resolveAbortOutcome (#920): a matched abort renders the friendly outro and is NOT reported to error tracking, since a blocked monorepo install is an expected user condition, not a bug. Requires the context-mill mcp-analytics skill to emit `[ABORT] manual install required: <command>` when STEP 7 verify fails — that change is a separate context-mill PR. Until it ships this case is inert. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
Solution A is handoff-only: we won't move package installs (and their lifecycle scripts) outside the agent sandbox to close one monorepo case. The skill, report, and edits are already left in place on the abort path (the keep-skills prompt is success-only), so tell the user they can finish either by running the install themselves or by pointing their own AI coding tool at the still-installed .claude/skills/mcp-analytics. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make it explicit why the wizard didn't install for the user: it runs in a sandbox for security and won't write outside the project folder, so a monorepo install (which writes a parent folder) is out of reach by design. Note that their own AI tool is not sandboxed, so it can finish the step.
…mmand Rework the "manual install required" handoff so it does not hardcode the install command and leads with the skill: - AbortCase.body may now be a function that receives the regex match, so a case can pull captured groups out of the abort reason. resolveAbortOutcome resolves it. - The mcp-analytics regex captures the exact command the skill detected (it travels after the colon in "[ABORT] manual install required: <command>"), so the handoff shows the right one for the user's package manager. Command is optional, so an older skill emitting a bare reason still matches. - Copy leads with the skill (the user's own unsandboxed AI tool can finish; the skill is saved at .claude/skills/mcp-analytics). Terminal fallback shows the detected command, or names the packages if none was sent. docsUrl renders the docs link. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
gewenyu99
approved these changes
Jul 20, 2026
Co-authored-by: Vincent (Wen Yu) Ge <29069505+gewenyu99@users.noreply.github.com>
The web edit left tab-indented lines in the handoff body, which failed the prettier check. Reformat to spaces and shorten the over-long comment. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cut comments that restated the code (the body-type branch, several test-case comments) and tightened the three that carry real intent down to the non-obvious why. Also drops the em dashes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
sarahxsanders
marked this pull request as draft
July 20, 2026 19:52
… left" success A blocked SDK install was modeled as [ABORT], which hard-halts the run, so the agent could bail before writing any code. Replace it with a generic, non-fatal [MANUAL-STEP] signal: the agent finishes the code changes, the run succeeds, and the outro shows the install as the one remaining step. Adds the signal + collection, threads manualSteps through both harnesses, folds it into the success outro, and removes the manual-install abort case (reverting the now-unused function-body AbortCase). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What it changes
When the mcp-analytics skill (PostHog/context-mill#252) can't install the SDK inside the sandbox, the wizard no longer treats it as an error or an abort. It is a non-fatal handoff: the agent finishes all the code changes, the run succeeds, and the outro shows the install as the one step left for the user.
Under the hood this replaces the old
[ABORT] manual install requiredpath with a new, generic[MANUAL-STEP]signal. Any program can now emit[MANUAL-STEP] <text>to flag a remaining action; it is collected like[WIZARD-REMARK], never halts the run, and is rendered in the success outro's next-steps section. The wizard stays product-agnostic (it does not know what "install" is); the skill supplies the text.Why it is a success, not an abort
[ABORT]is a hard halt: the moment the agent emits it, the run is killed. Agents usually install before they write code, so an abort on a blocked install fires before any instrumentation happens, and the handoff copy would claim the code changes were done when nothing had been. Asking the agent to instead defer the abort until after the code changes is exactly the "finish everything, then fail in a structured way" reasoning that weaker models don't do reliably.A sandbox-blocked install is not a failure. In a monorepo the install has to write a hoisted
node_modulesabove the project folder, outside the sandbox we deliberately keep installs inside for security, so it always belongs to the user. Modeling it as a non-fatal success with one step left removes the early bail entirely (a blocked install no longer halts anything) and drops the fragile reasoning. A completion guard still blocks a hollow success: a run that did no work, or left planned tasks open, is still an error, not a false "one step left."This replaces the abort-based handoff after review feedback on the stack.
User experience
The user sees a normal green success, with the install as the remaining step:
The command adapts to the user's package manager, so npm, pnpm, and bun users see their own line. And because of #920, the two genuine dead ends (no MCP server, unsupported language) still stop as expected conditions that are not logged as errors.
Merge order
These three PRs are one fix in three parts. Merge in this order:
This PR is step 2. It is stacked on #920 and needs #252 to actually fire.
Nothing breaks if they land out of order. #929 and #252 are each harmless on their own; they just do nothing until both are live.
🤖 Generated with Claude Code