OUT-3782: prevent Dropbox↔Assembly ping-pong on file create#107
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR closes the Dropbox↔Assembly ping-pong race by deploying three coordinated fixes: an early
Confidence Score: 4/5Safe to merge; all three previously raised issues are addressed and the dedupe logic correctly covers both the post-stamp and pre-stamp windows. The three-layered race fix (early stamp + path-based lookup + delete-before-create ordering) is sound and all previous thread concerns have been resolved. The only new gap is observability: when webhook.controller.ts — the suppression branch at line 71 has no log emit; worth a one-liner before shipping to production. Important Files Changed
Sequence DiagramsequenceDiagram
participant DBX as Dropbox
participant Sync as SyncService
participant DB as Database
participant ASM as Assembly API
participant WH as Webhook Controller
DBX->>Sync: file change detected
Sync->>DB: "pre-insert row (pendingAction=CREATE, assemblyFileId=NULL)"
Sync->>ASM: copilotApi.createFile()
ASM-->>Sync: "fileCreateResponse (id=X)"
Sync->>DB: "updateFileMap(assemblyFileId=X) ← early stamp"
ASM->>WH: "file.created echo (data.id=X) ~100ms later"
WH->>WH: sleep(800ms)
WH->>DB: "existingFile lookup (assemblyFileId=X)"
DB-->>WH: row found → skip handleFileCreated ✓
Sync->>ASM: uploadFileInAssembly (S3, seconds)
Sync->>DB: "markUpdated(assemblyFileId=X, pendingAction=null)"
note over WH,DB: Residual window: stamp not yet committed
ASM->>WH: file.created echo arrives before stamp
WH->>DB: existingFile lookup → null
WH->>DB: findInFlightAssemblyCreate(channelId, path)
DB-->>WH: pre-insert row matched (assemblyFileId IS NULL) → skip ✓
Reviews (3): Last reviewed commit: "fix(OUT-3782): scope in-flight dedupe to..." | Re-trigger Greptile |
ce7a9e1 to
1d1345d
Compare
1d1345d to
81f0c7d
Compare
b42b714 to
4abcc14
Compare
- Stamp assemblyFileId via updateFileMap immediately after copilotApi.createFile in completePendingAssemblyCreate, before uploadFileInAssembly. Shrinks the race window from upload-duration to one UPDATE so the controller's existingFile lookup dedupes the Assembly file.created echo. - Add a path-based dedupe in the Assembly webhook controller: lookup file_folder_sync by (portalId, normalized itemPath, pendingAction=CREATE, pendingActionTarget=ASSEMBLY, deletedAt IS NULL) and skip handleFileCreated/FolderCreated when a pre-inserted row matches. Stored itemPath has a leading "/"; Copilot's data.path does not, so we normalize. - Swap order in handleChannelFileChanges: process deletes before creates so the (portal, channel, dbx_file_id) partial unique index from OUT-3778 isn't violated during rename flows. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…rvice
Move the channel lookup, path normalization, and pre-insert row match
out of webhook.controller.ts and into a single method:
AssemblyWebhookService.findInFlightAssemblyCreate(
assemblyChannelId,
filePath,
)
- Dedupe is now scoped to (portalId, channelSyncId, itemPath) rather
than (portalId, itemPath). Path uniqueness lives at per-channel
granularity (matches the partial unique indexes from OUT-3778), so
the previous query could silently drop legitimate file.created
events for a different channel that shared a path.
- Normalization (leading "/") moved inside the service method. It now
only runs for create events instead of for every webhook, addressing
the latent crash on non-create events that omit data.path.
- Controller becomes one call: lighter to read, easier to test.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…FileId The pendingCreate lookup in findInFlightAssemblyCreate matched any row with pendingAction=CREATE, target=ASSEMBLY, and a matching itemPath, ignoring whether assemblyFileId was set. The JSDoc already documented the intent as "row exists with assemblyFileId IS NULL", but the query omitted the filter. Defensive correctness fix: if completePendingAssemblyCreate gets past the early updateFileMap (stamping assemblyFileId) but uploadFileInAssembly then throws, the row stays at (pendingAction=CREATE, assemblyFileId=B). The normal delete-webhook cleanup chain usually soft-deletes such rows when the orphaned Assembly file is removed — but if that chain breaks (lost webhook, errored handler, sweeper exhausted at MAX_ATTEMPTS), a later legitimate file.created event for the same path would falsely match the stuck row and silently skip handleFileCreated. Adding isNull(t.assemblyFileId) to the query restricts the match to the actual in-flight window (post-insertCreatePending, pre-updateFileMap) that the dedupe was designed for. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
81f0c7d to
4474ab7
Compare
Summary
Closes a race where a Dropbox→Assembly create can echo back via Assembly's
file.createdwebhook and produce duplicate files on both sides. Stacks on top of OUT-3645 (#106).What's in this PR
1. Early-stamp
assemblyFileId—Sync.service.ts#completePendingAssemblyCreatePersists
assemblyFileIdviaupdateFileMapimmediately aftercopilotApi.createFile()returns, before the (potentially seconds-long) S3 upload. Shrinks the race window from the upload duration to a single UPDATE roundtrip, so the Assembly webhook controller's 800ms sleep +existingFilelookup actually catches the echo.2. itemPath-based dedupe in the Assembly webhook controller —
webhook.controller.tsAdds a second lookup on
file_folder_syncmatching(portalId, normalized itemPath, pendingAction = CREATE, pendingActionTarget = ASSEMBLY, deletedAt IS NULL). Catches the residual race where Assembly firesfile.createdbefore ourcreateFileHTTP response returns — at that point we haven't stampedassemblyFileId, but the pre-inserted tombstone row already has theitemPathpopulated, so we can still match. StoreditemPathhas a leading/; Copilot'sdata.pathdoesn't, so the lookup normalizes.3. Handler ordering swap —
processFileSync.ts#handleChannelFileChangesProcess deletes before creates. With the partial unique indexes from OUT-3778, a rename flow can otherwise hit
there is no unique or exclusion constraint matching the ON CONFLICT specificationif the new path's create races the old path's delete.Why the ping-pong happens
If the Assembly echo arrives between (1) and (3), our existing dedupe (which keys off
assemblyFileId) misses → we triggersyncAssemblyFileToDropbox→ duplicate appears in Dropbox → loop.Test plan
pnpm typecheckcleanpnpm lintcleanpnpm testpasses🤖 Generated with Claude Code