Skip to content

fix(engine): warm-up seek after session init to prevent parallel worker flash frames#633

Open
miguel-heygen wants to merge 1 commit intomainfrom
feat/engine-warmup-render
Open

fix(engine): warm-up seek after session init to prevent parallel worker flash frames#633
miguel-heygen wants to merge 1 commit intomainfrom
feat/engine-warmup-render

Conversation

@miguel-heygen
Copy link
Copy Markdown
Collaborator

Problem

When rendering with parallel workers (-w 5 or auto), compositions using async GLTF model loading + Three.js MeshPhysicalMaterial with transmission: 1 produce a single flash frame at worker boundaries. The flash shows the wrong 3D object orientation (e.g., iPhone front when it should show back) for exactly one frame.

Root cause: Each parallel worker loads the page in a separate Chrome tab. After the page is ready (window.__hf.duration > 0), the first frame captured by a worker uses an uninitialized WebGL transmission render target. Three.js's transmission pipeline (MeshPhysicalMaterial.transmission) requires at least one prior render to populate its internal render targets — without it, the first render produces incorrect output.

Reproduction: Render any composition with async-loaded GLTF models + glass/transmissive materials using 2+ workers. The flash appears at worker boundary frames (e.g., frame 90, 180, 270, 360 with 5 workers at 30fps).

Works with 1 worker because all frames are captured sequentially from a single page — the first frame at t=0 primes the pipeline, and subsequent frames reuse the initialized state.

Fix

After session initialization in frameCapture.ts (initializeSession), perform three warm-up seeks via the __hf protocol:

await page.evaluate(`window.__hf?.seek?.(0)`);
await page.evaluate(`window.__hf?.seek?.(window.__hf?.duration * 0.5 || 0)`);
await page.evaluate(`window.__hf?.seek?.(0)`);

This primes the page's render pipeline at t=0, t=midpoint, and t=0 again — ensuring WebGL buffers, transmission render targets, and any lazy-initialized GPU resources are fully populated before real frame capture begins.

Applied to both capture modes (screenshot and beginFrame).

Changes

File Change
packages/engine/src/services/frameCapture.ts Add 3 warm-up seeks after session init in both screenshot and beginFrame paths (+16 lines)

Test plan

  • All 513 engine tests pass (2 pre-existing failures from missing generated file, unrelated)
  • Render GLTF composition with -w 5 — verify no flash frames at worker boundaries
  • Render standard (non-GLTF) composition with -w 5 — verify no regression
  • Render with -w 1 — verify still works (warm-up is harmless for single worker)

…er flash frames

When parallel workers each load a page independently, the first frame
captured by a worker could show an uninitialized WebGL render state —
typically a single flash frame with wrong 3D object orientation or
missing MeshPhysicalMaterial transmission effects. This happened
because the Three.js transmission pipeline needs at least one prior
render to populate its internal render targets.

Fix: after session initialization (page ready, fonts loaded, videos
ready), perform three warm-up seeks via the __hf protocol before
marking the session as initialized. The seeks prime the page's
render pipeline at t=0, t=duration/2, and t=0 again, ensuring WebGL
buffers and transmission render targets are fully initialized before
real frame capture begins.

Affects both screenshot and beginFrame capture modes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant