fix(mcp-analytics): hand off for blocked installs - #252
Draft
sarahxsanders wants to merge 2 commits into
Draft
Conversation
…omplete A wizard run instrumented a monorepo MCP server, but `yarn add` failed (the install had to write a hoisted node_modules outside the sandboxed working dir → EPERM). The agent misread it as a network issue, ignored the resulting `tsc` "Cannot find module" failure, and reported success anyway — leaving the user a green "configured!" on a project that didn't compile. Harden the skill so it can't fake success: - STEP 3: if the install fails, don't hand-edit package.json; make the additive edits so they're ready, but the run must end at STEP 7 with the abort, never a success summary. - STEP 7: a `Cannot find module '@posthog/mcp'` / unresolved `posthog` import means the install never happened — abort, don't patch around it. - Abort cases: add `[ABORT] manual install required: <command>` carrying the exact install command. Pairs with wizard #929, which renders this abort as a clear handoff (skill, report, and edits are left in place so the user can finish in their own AI tool) and, via #920, does not report it to error tracking. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
This was referenced Jul 20, 2026
sarahxsanders
marked this pull request as ready for review
July 20, 2026 15:13
gewenyu99
approved these changes
Jul 20, 2026
gewenyu99
left a comment
gewenyu99
left a comment
Collaborator
There was a problem hiding this comment.
Good stuff, but I am mildly concerned by using the abort pattern. Maybe this should all be come form of end state instead of "abort" because I think the agent will abort as soon as it hits the problem instead of finishing the job best it can before aborting
sarahxsanders
marked this pull request as draft
July 20, 2026 19:51
…ot an abort A sandbox-blocked install is expected, not a failure. Instead of aborting, the skill now tells the agent to finish all the code changes and emit a [MANUAL-STEP] line with the exact install command, so the run ends as a success with one step left. Removes the manual-install abort case; rewrites STEP 3 and STEP 7. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
sarahxsanders
changed the title
sarahxsanders
changed the title
sarahxsanders
changed the title
sarahxsanders
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What it changes
This is the instruction sheet the AI agent follows to add PostHog to a project. Before, a blocked package install ended the run with an
[ABORT]. This PR changes that: a sandbox-blocked install is treated as expected, not a failure. The agent finishes all the code changes and emits a non-fatal[MANUAL-STEP]line carrying the exact install command, so the run ends as a success with one step left, not an error.It removes the
manual install requiredabort case and rewrites STEP 3 (install) and STEP 7 (verify) so the agent never bails on a blocked install and never fakes a green summary on a project whose packages aren't installed.Why it is a handoff, not an abort
The agent runs inside a sandbox, a locked-down space that can only write inside the project folder. That limit is deliberate, and it is a security measure. The agent is driven by an LLM, and an LLM can be steered by the text it reads, so the sandbox keeps a bad or hijacked run from touching anything outside the project you pointed it at.
In a monorepo the install has to write a shared
node_modulesthat lives above the project folder, which is outside the sandbox, so it fails. We are not loosening that boundary. Letting installs run outside the sandbox would also let the install scripts that ship inside packages run outside it, and running unknown code with full access to your machine is exactly what the sandbox exists to prevent. So the install genuinely belongs to a human.The old approach modeled that as an abort. But an abort is a hard stop, and the agent usually installs before it writes code, so it would bail before doing any work. Weaker models also can't reliably do "install fails, keep going anyway, then abort at the end." Making the blocked install a non-fatal
[MANUAL-STEP]fixes both: the agent always finishes the code changes, and the leftover install is surfaced honestly as one remaining step.User experience
The wizard still can't tell you "configured!" when it isn't. It finishes everything the agent can do, then hands off honestly: instead of a false success or a red error, the user gets a green success with the exact install command as the one step left.
Merge order
These three PRs are one fix in three parts. Merge in this order:
This PR is step 3. It is inert until #929 is live, and #929 is inert until this is, so they go together.
Nothing breaks if they land out of order. #929 and #252 are each harmless on their own; they just do nothing until both are live.
🤖 Generated with Claude Code