feat(platform): per-step error feedback in the automation test panel#1870
Merged
Merged
Conversation
Contributor
|
Warning Review limit reached
More reviews will be available in 34 minutes and 27 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more credits in the billing tab to continue. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (29)
✨ Finishing Touches🧪 Generate unit tests (beta)
Warning Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Fix run-level error persistence first: no failure path ever wrote wfExecutions.error — failExecution serialized the error into metadata and replaced the JSON wholesale (destroying componentWorkflowIds, the step journal's id list) and never set completedAt. All four failure chokepoints (failExecution, updateExecutionStatus, the stuck-recovery watchdog, user cancel) now write the top-level error, merge metadata instead of clobbering it, stamp completedAt, and record a coarse errorCode (start_failure / step_failure / timeout / canceled / invalid_input) surfaced by getExecutionStatus. On top of #1487's getExecutionStepStatuses query the tester renders a live step list: per-step status icon, an attempts counter for retried and looped steps (collapsed to the latest attempt), and the failing step's error inline — its name deep-links to that step's editor via the existing panel/step URL state. Runs that fail without a failed journal step (never started, timed out, canceled) fall back to the run-level error plus a translated reason. Test runs are additionally preflighted server-side against the start node's inputSchema (validateWorkflowInput), rejecting missing or mistyped fields with field-specific messages before an execution record exists; production trigger paths stay unenforced.
yannickmonney
force-pushed
the
feat/automation-tester-step-error-feedback
branch
from
9ed77c4 to
94408f8
Compare
yannickmonney
added a commit
that referenced
this pull request
Jun 11, 2026
…tion Two fixes to e2e/specs/automation.spec.ts: - Scope the completion assertion to the tester side panel (role=complementary, labelled "Test automation"). After PRs #1868/#1870 the canvas now renders a "viewing run" role=status banner whose "Completed" label collides with the tester result's, making a bare getByRole('status') a strict-mode violation (2 elements). The banner lives on the canvas, outside the panel, so the complementary scope is unambiguous. - Make reaching the editor idempotent across retries. The first attempt installs the `test` template, which removes it from the "From template" picker; a retry that still went through the install flow hung on a missing button. Now: if `test` is already installed, open its list row directly; otherwise install via template. Wait for the table to settle (row or empty state) before the branch so a mid-load read can't wrongly pick install.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1484. Built on #1487 (merged in #1868) — consumes its
getExecutionStepStatusesquery without redefining it. Originally stacked onfeat/workflow-node-execution-status; rebased ontomainafter #1868 merged.What / why
Run-level error persistence (bug fix). No failure path ever wrote
wfExecutions.error— every path serialized the error into themetadataJSON string, andfailExecutionreplaced that string wholesale, destroyingcomponentWorkflowIds(the step journal's primary id list) and never settingcompletedAt(failed runs had no duration). The four chokepoints now write top-levelerror, merge metadata via a sharedmergeExecutionMetadatahelper, stampcompletedAt, and record a coarseerrorCode:failExecution→ caller-supplied code (step_failurefrom the engine callback and the next-step pre-mark,start_failurefromstartWorkflowFromFileConfig)updateExecutionStatus→ top-level error + metadata merge,completedAtforfailed(not onlycompleted),canceledcode from the canceled engine outcometimeoutcancelExecution→canceledgetExecutionStatusandgetExecutionStepStatusesnow exposeerrorCode. The schema field is a plain optional string (adding codes never needs a migration); writers are typed to theExecutionErrorCodeunion and internal-mutation args validate against a literal union.Test panel per-step feedback. The tester subscribes to #1487's
getExecutionStepStatusesand renders a live step list in the result card: status icon per step (icon/color conventions shared with the canvas badges, extracted toexecution-status-icons.ts), an attempts counter for retried/looped steps, and the failing step's error inline. The failing step's name deep-links to its editor through the existingpanel/stepURL state. Runs that fail with no failed journal step (never started, timed out, canceled) fall back to the run-level error plus a translated reason line.Server-side input preflight (test runs only).
startWorkflowFromFilevalidates the tester's input against the start node'sinputSchemavia the existingvalidateWorkflowInput, throwing field-specific messages (Missing required parameter: 'x', type mismatches) before an execution record is created; the tester's existing catch surfaces them in the toast. Client-side gating from #1792 remains the first line.Decision-gate recommendations baked in
errorCodeenum at the five chokepoints (recommended) — coarse codes only; no per-node-executor structured codes.Notes for review:
invalid_inputis reserved in the code union: the preflight rejects before an execution record exists, so it is never persisted today.executions-table.tsxrenders parsed metadata verbatim and no consumer relies on replacement.execution_lifecycle.test.tsassertedcompletedAtstays unset onfailed; that was the bug.Verification
Performed:
bun run checkat repo root (format, lint, typecheck, all tests) — pass, re-run after the rebase ontomain.fail_execution.test.ts(new failExecution coverage incl. metadata merge preservingcomponentWorkflowIds), newupdate_execution_status.test.ts, newpreflight_test_run_input.test.ts,recover_stuck_executions.test.ts,get_execution_step_statuses.test.ts, fullapp/features/automationsUI suite (21 files) incl. extendedautomation-tester.test.tsx(step rows in start order, failing-row deep link, attempts counter, run-level fallback, a11y).bun run --filter @tale/docs lint+test(locale parity) — pass.Manual verification required (not performed here — needs the local dev stack):
error/duration on the failed run.Pre-PR checklist
bun run check(format, lint, typecheck, all tests).services/platform/messages/{en,de,fr}.json./docs/{en,de,fr}/(workflows.md test-panel section).bun run --filter @tale/docs lintandbun run --filter @tale/docs test.