fix(engine): warm-up seek after session init to prevent parallel worker flash frames#633
Open
miguel-heygen wants to merge 1 commit intomainfrom
Open
fix(engine): warm-up seek after session init to prevent parallel worker flash frames#633miguel-heygen wants to merge 1 commit intomainfrom
miguel-heygen wants to merge 1 commit intomainfrom
Conversation
…er flash frames When parallel workers each load a page independently, the first frame captured by a worker could show an uninitialized WebGL render state — typically a single flash frame with wrong 3D object orientation or missing MeshPhysicalMaterial transmission effects. This happened because the Three.js transmission pipeline needs at least one prior render to populate its internal render targets. Fix: after session initialization (page ready, fonts loaded, videos ready), perform three warm-up seeks via the __hf protocol before marking the session as initialized. The seeks prime the page's render pipeline at t=0, t=duration/2, and t=0 again, ensuring WebGL buffers and transmission render targets are fully initialized before real frame capture begins. Affects both screenshot and beginFrame capture modes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When rendering with parallel workers (
-w 5or auto), compositions using async GLTF model loading + Three.jsMeshPhysicalMaterialwithtransmission: 1produce a single flash frame at worker boundaries. The flash shows the wrong 3D object orientation (e.g., iPhone front when it should show back) for exactly one frame.Root cause: Each parallel worker loads the page in a separate Chrome tab. After the page is ready (
window.__hf.duration > 0), the first frame captured by a worker uses an uninitialized WebGL transmission render target. Three.js's transmission pipeline (MeshPhysicalMaterial.transmission) requires at least one prior render to populate its internal render targets — without it, the first render produces incorrect output.Reproduction: Render any composition with async-loaded GLTF models + glass/transmissive materials using 2+ workers. The flash appears at worker boundary frames (e.g., frame 90, 180, 270, 360 with 5 workers at 30fps).
Works with 1 worker because all frames are captured sequentially from a single page — the first frame at t=0 primes the pipeline, and subsequent frames reuse the initialized state.
Fix
After session initialization in
frameCapture.ts(initializeSession), perform three warm-up seeks via the__hfprotocol:This primes the page's render pipeline at t=0, t=midpoint, and t=0 again — ensuring WebGL buffers, transmission render targets, and any lazy-initialized GPU resources are fully populated before real frame capture begins.
Applied to both capture modes (screenshot and beginFrame).
Changes
packages/engine/src/services/frameCapture.tsTest plan
-w 5— verify no flash frames at worker boundaries-w 5— verify no regression-w 1— verify still works (warm-up is harmless for single worker)