[No QA] Synthetic target for Melvin agent-device mobile smoke test#92233
[No QA] Synthetic target for Melvin agent-device mobile smoke test#92233kacper-mikolajczak wants to merge 11 commits into
Conversation
No-op commit. This PR exists only as a stable, main-aligned target for manually dispatching the Melvin App workflow's Android mobile path, so the Rock fingerprint matches a cached build. It changes no app code and should not be merged.
|
|
|
|
|
⏱️ Ran out of time before I could finish. This usually means the task was too complex or I got stuck exploring. Please try again with a more specific request or post in #melvinbot-ai. |
|
⏱️ Ran out of time before I could finish. This usually means the task was too complex or I got stuck exploring. Please try again with a more specific request or post in #melvinbot-ai. |
|
🤖 Android smoke test blocked — the dev build won't start I booted the Android emulator and launched the Expensify dev build, but the app can't get past startup. It hits a I confirmed it's not a stale bundle: a clean force-stop + relaunch reproduced the same startup error every time. This is a build/dependency-alignment problem that can't be fixed or validated on the CI emulator — it needs the Android dev client rebuilt so the native Reanimated matches the JS bundle. I avoid native/dependency changes here since they can't be verified on CI. Recommended next step: rebuild the Android dev APK with — Melvin |
|
🤖 Android smoke test blocked — Metro bundler went down mid-run I drove the Android emulator with agent-device and got the dev build up to the SignIn screen, but couldn't complete the login + send-message flow because the JS bundler (Metro, port 8081) stopped responding partway through. What happened
This is an environment/infra issue with the Metro bundler in this CI run — not a problem with this PR (which is an empty synthetic target anyway). It's the same Metro instability seen in earlier runs on this PR. agent-device itself worked fine — open, snapshot, screenshot, fill, and device-log capture all behaved as expected.
Heads-up for the smoke-test design (not yet verified this run): a fresh throw-away-account sign-in completes via a 6-digit magic code emailed to the account — there's no hardcoded dev code in the open-source sign-in flow. So even with Metro healthy, automating this end-to-end will need either a pre-provisioned account/session or an inbox the automation can read for the code. I hit the bundle error before I could observe the backend's response, so treat this as a caveat to plan around rather than a confirmed finding. Recommended next step: re-run once Metro is confirmed healthy and kept alive for the whole session, and decide how the test account's magic code will be supplied. I'll retry the full login → send-message flow then. — Melvin |
|
I drove the Android emulator for this smoke test but could not complete the task. The app opened to the sign-in screen initially, but I hit two independent blockers and did not send the message to exfy-perf@callstack.com. Blocker 1 — Dev environment broke mid-run (infra). Blocker 2 — No way to sign in with a throw-away account (provisioning). Suggested fixes for the smoke-test harness:
Happy to retry once either blocker is addressed. |
|
🤖 Mobile smoke test (Android) — blocked before I could send the message Here's how far I got on the Android dev build and where it stopped:
❌ Blocked at the onboarding "What's your work email?" step. The step won't advance: neither Skip nor submitting a work email moves forward, and Android back is gated ("Please finish the setup questions to start using the app"). A persistent error is surfaced — "The 'navigation' object hasn't been initialized yet" — and the accessibility tree shows two navigation subtrees mounted at the same time (a corrupted navigation state). Because onboarding can't be completed, I never reached Home and couldn't start a chat with exfy-perf@callstack.com to send the message. This does not appear to be caused by this PR — the same stuck state reproduces after a JS reload and after a full app relaunch, and this PR has no changed files. It looks like an environment/dev-build issue rather than anything in the PR. If this reproduces locally, the issue is in the onboarding navigation-initialization path; otherwise it may be a transient dev-build/Metro hiccup worth a re-run. Happy to retry once the build is healthy. |
|
|
|
🪲 [debug] CHECK 1 (live reload): edited the sign-in screen's button label and ran |
|
🪲 [debug] CHECK 2 (onboarding nav): stuck on the "What's your work email?" step. The Skip button is fully visible and unobstructed, but pressing it (durable selector AND exact-center coordinate) no-ops every time. Root cause confirmed: the accessibility snapshot shows the onboarding modal and two full copies of the Home tab bar mounted at once, and a tap on the modal's email field instead collapsed the press-release cards on the Home screen behind it — so touches are passing through the modal to the Home navigator underneath. This is the known double-mount / detached-overlay issue on modal transitions, not the warning toast. Trying a Metro re-mount next. |
Android emulator — two-check reportCHECK 1 — live-reload iteration loop: does not work in this CI envMade a small visible edit to the sign-in screen's button label, ran
CHECK 2 — sign in + complete onboarding to Home: reached HomeSigned in with a throwaway test account and walked the full onboarding using the navigation ladder (durable Correction to my earlier debug note: I initially suspected taps were passing through the onboarding modal — that was a misread (the Home feed behind it was just loading async). The modal was interactive the whole time. The real obstacle: the dev-mode warning toasts (incl. a continuously re-firing "getSnapshot should be cached" warning) float at the bottom and overlap the Skip button while the keyboard is down, intercepting the tap. Reliable workaround that unblocked it: focus the email field to raise the keyboard, which lifts Skip above the toast band — the tap then registers cleanly. The accessibility snapshot does also show the documented double-mount during the modal transition (two Home tab-bar subtrees + the modal at once), but it did not block interaction here — the toast overlap was the actual blocker. How far each check got
|
|
🪲 [debug] Issued a second |
Live-reload verification (Android emulator) — ❌ edit did not propagateResult: A visible JS edit to the sign-in screen did not render after What I did
Before / After
Both read Continue — the expected Step 1 (watchman) — could not fully confirmI could not run
But I could not confirm watchman is actively watching the App root or that Metro is wired to use it — which is exactly the thing that would explain the failure below. Likely cause (medium confidence)The edit is on disk and Metro re-bundles on reload, yet the changed module's new text never reaches the served bundle. That means the file-change event for the edited source isn't reaching Metro, so it serves the cached transform for that module. This is consistent with the file watcher not delivering filesystem events (a common failure for watchman on containerized CI filesystems) or Metro not actually using watchman. I couldn't pin it down precisely without Suggested next steps
|
|
🪲 [debug] After editing the sign-in submit label and running |
Live-reload diagnosis (Android emulator)Verdict: the edit did NOT propagate. Watchman is watching the App project root, but a JS source edit + 1. Watchman stateThe App project root is in watchman's 2. Edit → reload → render
Result: the button still rendered To rule out a screenshot-timing artifact, I fetched Metro's served bundle directly: The new string is absent from Metro's own output, so this is not an app-render lag — Metro itself served a stale bundle. The Metro log shows it re-bundled Conclusion: Metro's file watcher is not observing on-disk edits in this agent environment. Even though 3. RevertEdit reverted; Side note (not the failure)A system "Pixel Launcher isn't responding" ANR dialog repeatedly overlaid the screen (emulator under load during bundling) and had to be dismissed before each capture. A LogBox notice ( |
|
🪲 [debug] Step D ( |
|
🪲 [debug] Step E (after reset-cache restart + reload): the served bundle now contains the unique string (cold recompile picked up the edit), but the app is sitting on a blank dark screen while it downloads/executes the freshly-built dev bundle over the slow emulator link. Waiting for it to finish rendering to confirm the button text visually. |
|
🪲 [debug] Step E follow-up: even after a full app relaunch (MainActivity foreground, Metro served the rebuilt bundle), the React tree never mounted — the screen stays blank and the a11y snapshot shows only the native shell with an empty content frame. The bundle-level proof for the workaround still holds (unique string present in Metro output); the on-device re-render just couldn't be confirmed on this emulator, which was already unstable earlier in the run. |
Watcher diagnosis + reset-cache workaround test — resultsRan the full A→F sequence on the Android emulator against the SignIn submit button label. Edit used the unique marker string Results
Diagnosis
Honest caveat on Step EI could not visually confirm the new text rendering on the device. After the reset-cache restart the app stayed on a blank content frame across multiple reloads and a full relaunch — the native shell was foregrounded but the JS tree did not mount. This emulator was already unstable during the run (a launcher ANR and a worklets ScreenshotsStep D — redbox Step E — blank content after reset-cache reload: Step E — still blank after full relaunch: Bottom lineThe file watcher is healthy and sees edits immediately. Live reload is broken because Metro re-serves a cached transform of the changed module. A cache-clearing restart fixes the bundle (the marker string appears), confirming the staleness is at the Metro cache layer — though a stable emulator is still needed to verify the final on-screen render end-to-end. |
… + reanimated alignment)
|
🪲 [debug] Launched the dev app on the Android emulator and waited for splash. Instead of the SignIn screen, an RN redbox appeared: |
Android validation result — ❌ app crashes on launch, never reaches SignInTL;DR: The dev app does not load past splash to SignIn. It throws a full-screen RN redbox on launch, so sign-in/onboarding could not be attempted. No LogBox warning overlay/toast was seen — but only because the JS runtime never initialized (a redbox preempts everything). The crash is caused by the LogBox-suppression injection itself. (1) Launch → SignIn — ❌ FAILEDOn launch, instead of SignIn, a hard RN redbox appears and stays: It never advances past this screen. The error persists across a JS reload, so it is deterministic, not a transient bundling hiccup. (A "Pixel Launcher isn't responding" ANR also popped up intermittently — that's emulator load, unrelated to the app.) (2) Sign in + onboarding → Home — ⛔ BLOCKEDCould not start. The app never reaches SignIn, so the sign-in macro and onboarding ladder had nothing to drive. Home screenshot: not reached. (3) LogBox overlays / warning toasts — no overlay seen, but NOT a passNo LogBox warning overlays or yellow/red warning toasts appeared at any point. However, this is not evidence that suppression works — the app crashed with a more severe RN redbox before LogBox (or any app JS) ever initialized, so suppression was never actually exercised. Root causeThe LogBox-suppression snippet is injected as a Metro polyfill. Metro polyfills run in the global scope before the module system is installed, so App's own startup polyfill works fine because it only touches globals ( So the very feature under test (LogBox suppression) is what breaks the launch. Suggested direction (not implemented — read-only run, as requested)Move the LogBox-silencing out of a Metro polyfill and into a module that runs after the runtime is initialized but before the main module (e.g. Metro's I did not edit code, commit, or open a PR. |
|
🤖 I could not complete the Concierge send-message smoke test. It was blocked by two local dev-environment staleness issues — neither is a defect in the Concierge or messaging code. 1.
I unblocked compilation by adding a 2. Dev build is below the production API's minimum version (hard blocker) Ran: dev server serves the login page; email entry + Continue work and hit the API. To actually run this smoke test: refresh the local checkout so the build version is at or above the current production minimum, run |
|
🤖 Mobile (Android emulator) — could not complete the Concierge send-message test. Verified on standalone Android NewDot ( The app crashes at boot with an What happened: on launch, the standalone NewDot build hits a full-screen red-box render error — "Upgrade Required" — thrown at Reached: app installed and launches on the emulator; splash → boot. To run this smoke test: refresh the local checkout so the build version is at or above the current production minimum, or point the dev build at a non-production API. Environment changes outside a verify-only run. Scope note: this exercises standalone NewDot RN/JS only — it does not cover HybridApp-shell (OldDot ↔ NewDot) behavior or iOS. |
|
🤖 Done with the compile fix; sign-in was partially blocked. Here's what happened. 1. Compile error — fixed ✅The Fixed 8 imports across 8 files (local working tree only — see note below):
Verified: the web build compiled successfully with no errors (webpack compiled in ~275s), which confirms the fix.
2. Sign in + message Concierge — partially blocked
|
|
⏱️ Ran out of time before I could finish. This usually means the task was too complex or I got stuck exploring. Please try again with a more specific request or post in #melvinbot-ai. |
Results1. Compile fix — done (applied locally, not pushed). Dropped the
2. Sign in + message Concierge — partial. The SignIn screen loaded and was captured, but sign-in could not be completed:
Because sign-in never completed, no Concierge message was sent and no Concierge chat screenshot exists. Screenshots / recordings ▶ Web drive recording 1 (sign-in attempt) To get the Concierge screenshot, re-run me — the tester needs to clear the email field before |
Android emulator — test run for #94938's stepsShort answer: yes — the flow still requires a second session/device to complete. 2 of the 4 scripted steps can't be driven by a single account, and both for the same reason: they depend on incoming messages from another sender, which one session can't produce.
So the two-session requirement is real and inherent to the incoming-message behaviors (Tests 1 & 2). Tests 3 & 4 need extra setup (deep-link/cache-clear; a second report) but not a second device. This matches the earlier Android run linked in the request.
To actually execute Tests 1 & 2 end-to-end, you'll need a second session/device (or a helper account) to send messages into the chat — a single automated account can't do it. And the emulator's "Update required" gate needs clearing (bundle/version bump) before any of the steps can be driven here. |
Results1. Compile fix — done (applied locally, not pushed). Dropped the
2. Two-session Concierge flow — FAILED (both sessions). Environment blocker, unrelated to the fix.
Root cause (server-side, confirmed via network dump): every API call to the dev server returns HTTP 426 (Upgrade Required) — This is not caused by the On the multi-session recording ask: recordings were produced under three distinct literal Evidence (all Session A — sign-in screens, no Concierge reached) Representative recordings across the three session names (14 segments total were produced, all Session A): ▶ melvin-web-session-a (segment 1) To unblock: the dev environment needs a bundle version the API accepts (the |
🤖 Android smoke test: ❌ blocked before sign-inThe smoke test could not be completed. The app never got past startup — the installed This is an environment/build issue, not a regression in this PR (it's an empty no-op commit). The dispatched build is simply below the current server-enforced minimum version, and the "Update required" gate is a JS Result: sign-in ❌ · home screen (LHN) ❌ · green + FAB ❌ — all blocked by the update gate. What happened, step by step
Console errors observed:
Note on session recording: the drive session did produce watchable |
🤖 Android smoke test: ✅ PASSRan the smoke test on the Android emulator (standalone NewDot, Result: app launch ✅ · sign-in ✅ · home screen (LHN) ✅ · green + FAB ✅ The home screen shows two FABs bottom-right, both confirmed present via accessibility-tree assertions:
Session recording validated: the drive produced watchable Steps performed
No console errors observed. |
|
🤖 Web flow tested: sign in + message Concierge — ✅ core flow works. Signed in to NewDot, opened the Concierge chat, and sent a message successfully. Concierge received it and entered a "thinking…" state, but didn't produce a final reply within the ~5 min observation window (likely dev-env backend latency for a fresh account, not a UI issue — the send itself succeeded).
No console errors observed. Evidence Test details
|
|
|
✅ Android smoke test — PASSRan on the Android emulator (standalone NewDot). Signed in with a fresh throwaway account, completed onboarding, and the Home screen (LHN) loaded with both the green Scan receipt FAB and the + Open actions menu FAB visible. All 10 session recordings were produced and validated as watchable
ScreenshotsSession recordings (validated)All 10 clips are valid ISO Media MP4 v2 files (2–30 MB each, with recorder gesture-telemetry sidecars):
Notes & caveats
|
|
|
✅ Android — distance-expense split in a 1:1 DM completes without an errorVerified on the Android emulator (standalone NewDot,
ScreenshotsSession recordings (validated)All 9 clips exist on disk and are valid ISO Media MP4 files (0.3–35 MB). The
Notes & caveats
|
|
|
|
|
|
🤖 Web verification could not be run — dev environment is broken, and I lack the permissions to repair it. Result: ❌ Blocked. I could not confirm whether a long email/link wraps left-aligned, because the web dev server never came up and no browser session could be opened. Root cause: the App's
Why I couldn't self-heal it: every repair path is denied by this run's permissions — What's needed to unblock: run a clean dependency install against the checked-out branch before dispatching the web path — e.g. Flagging this as the validation outcome for the review-fix run (Expensify/melvin#258 / #92668): the mobile/Android path may be provisioned, but the web path currently has no working dev environment. |
|
⏱️ Ran out of time before I could finish. This usually means the task was too complex or I got stuck exploring. Please try again with a more specific request or post in #melvinbot-ai. |
|
🤖 Android verification: ✅ PASS — splitting a distance expense in a 1:1 DM completes without an error. I opened a 1:1 DM, created a Distance expense, split it with the other participant, and submitted. The split expense was created and posted to the chat with no error message (no red error text, no failure banner). What was verified:
Screenshots ▶ Session recordings (2)Validation outcome for the review-fix run (Expensify/melvin#258 / #93238): the Android path works end-to-end. |
|
🤖 Android verification: ✅ PASS — a long email/link wraps left-aligned; the wrapped lines are not centered. I posted an 80-char email ( Evidence: the wrapped-text element renders as a multi-line block (accessibility node rect ▶ Session recordings (1)Validation outcome for the review-fix run (Expensify/melvin#258 / #92668): the Android path is working end-to-end — sign-in, onboarding, message send, screenshots, and session recording all captured successfully. |
|
🤖 Web verification could not be run — the dev environment is still broken (same blocker as the previous validation run on this PR). Result: ❌ Blocked. I could not open the private personal details settings page ( Root cause (unchanged): the App's
Why it's not self-healable here: every repair path is permission-denied in this single-shot run — What's needed to unblock: provision the environment before dispatching the web path — Flagging as the validation outcome for the review-fix run (Expensify/melvin#258 / #91646): the web path currently has no working dev environment. This matches the earlier finding on this PR — the fix belongs in how the web workflow provisions dependencies, not in this synthetic target. |
|
🤖 I couldn't complete this verification — the test environment failed to come up, so I have no result on the popover behavior. What happened: The web tester ran twice. In both runs the dev server was healthy and the app URL opened, but the sign-in replay failed immediately with Verification status: ❌ Not verified (blocked before reaching Settings › Profile). Next step: Please retry with |
|
🤖 Could not verify — blocked by a session-tooling failure, not App behavior. I ran the web verification twice using the So the left-aligned-wrap check is not verified — neither pass nor fail. The blocker is in the test harness, not the App. Evidence
▶ Session recordings (1)Re-run once the |
|
🤖 Could not verify — web test harness failed to sign in. I was unable to test splitting a distance expense in a 1:1 DM on web. The Both attempts (3 sign-in retries each) failed identically at bring-up: Session recording (login page only — sign-in never completed): ▶ Session recordings (1)Failure detail
None of the actual verification steps (open DM → create Distance expense → enter waypoints → Split → submit) could be exercised. Once the web |
|
🤖 Verified on web — the Categories filter list does not scroll or jump on selection. ✅ Drove NewDot on web, opened Search → Filters → Category, scrolled the list, and selected three items in a row (Car, Equipment, Fees). After each selection the visible items stayed in the exact same positions — the selected row just gained a checkmark in place. No scroll, no jump, no reordering across all three selections.
Notes
|
|
🤖 Verification status: ❌ Blocked — I couldn't confirm the popover-stays-visible behavior, because the web test driver can't perform a right-click. What I got through: Signed in → Settings › Profile, and confirmed the "What's new" row is visible in the sidebar (under General, with an external-link icon). I also resized the viewport (1280×900 → 800×600) and the sidebar collapsed as expected. Where it stopped: Opening the right-click "Copy URL to clipboard" popover requires a secondary-button click. agent-device on the web platform returns ▶ Session recordingBottom line: This flow (right-click → context popover → resize) isn't automatable in the current web tooling — right-click needs to be verified manually until agent-device gains secondary-click support on web. I also hit a sign-in selector mismatch ( |
|
🤖 Verified on web via the Opened Search → Filters → Category, viewed the list at rest, then selected two items in a row (Fees, then Home Office). After each selection the visible items (No category → Insurance) stayed in the exact same positions — the selected row just gained a checkmark in place. No scroll, no jump. The filter chip updated to
Notes
|












































































Explanation of Change
Synthetic, no-op PR. Do not merge.
This PR contains a single empty commit and changes no app code. It exists only as a stable,
main-aligned target for manually dispatching the Melvin App workflow's Android mobile path (agent-device PoC, Expensify/melvin#172).Because it sits exactly on top of
mainwith an empty tree diff, its Rock native fingerprint matches a cacheddevelopmentDebugbuild, so the workflow'srock remote-cache downloadstep hits the cache (instead of fast-bailing on a miss). Pointing the manual dispatch at this PR gives the run a real target for its result comment without polluting any real PR.Fixed Issues
$
PROPOSAL:
Tests
N/A - empty commit, no app code changed. This PR is a workflow dispatch target only.
Offline tests
N/A - no app code changed.
QA Steps
N/A -
[No QA]synthetic target; no app code changed and this PR must not be merged.PR Author Checklist
### Fixed Issuessection aboveTestssectionOffline stepssectionQA stepssectiontoggleReportand notonIconClick)src/languages/*files and using the translation methodSTYLE.md) were followedAvatar, I verified the components usingAvatarare working as expected)StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))npm run compress-svg)Avataris modified, I verified thatAvataris working as expected in all cases)Designlabel and/or tagged@Expensify/designso the design team can review the changes.ScrollViewcomponent to make it scrollable when more elements are added to the page.mainbranch was merged into this PR after a review, I tested again and verified the outcome was still expected according to theTeststeps.Screenshots/Videos
Android: Native
N/A - no app code changed.
Android: mWeb Chrome
N/A
iOS: Native
N/A
iOS: mWeb Safari
N/A
MacOS: Chrome / Safari
N/A