feat(registry): add experimental HTML-in-canvas launch blocks#611
feat(registry): add experimental HTML-in-canvas launch blocks#611miguel-heygen wants to merge 7 commits intomainfrom
Conversation
Not as part of this PR. The current Docker/test paths install Debian This PR is intentionally fallback-safe: the renderer passes the flag, then runtime feature detection only uses native capture if For actually exercising/benchmarking the native path in Docker, we should do a follow-up that pins an opt-in Chrome for Testing/Canary build that has CanvasDrawElement, updates the internal render image the same way, and compares it against the current beginFrame path before making it the default. |
|
@miguel-heygen we may want to update the internal version's docker image as well then we can test HTML to canvas screenshotting maybe for renders? to see if it's any faster than beginFrame as well |
93323f6 to
201e4ce
Compare
|
On the internal image: yes, but I would keep it as a follow-up rather than coupling it to this PR. This PR keeps stable/Docker renders fallback-safe. For native-path perf testing we should pin the same Chrome for Testing/Canary build in the external + internal render images, then benchmark HTML-in-canvas screenshotting against beginFrame before changing defaults. |
Cinematic 3D spatial UI showcase using the HTML-in-Canvas API. Live HTML panels rendered as WebGL textures via drawElementImage(), floating in a Three.js scene with post-processing (bloom, DoF, chromatic aberration, volumetric light, film grain, vignette), glass edges, reflection plane, and GSAP-driven camera autopilot derived from composition timing. Interactive orbit mode via spacebar. Requires Chrome/Brave with CanvasDrawElement flag enabled. Depends on PR #611 for engine --enable-features=CanvasDrawElement.
Cinematic 3D spatial UI showcase — live HTML panels as WebGL textures via drawElementImage(), Three.js scene with bloom, DoF, chromatic aberration, volumetric light, glass edges, reflection plane, and camera autopilot derived from composition timing.
Cherry-picked from feat/html-in-canvas-launch (PR #611): - Enable --enable-features=CanvasDrawElement in Chrome browser args so HTML-in-canvas compositions render correctly - Add native drawElementImage() capture path for shader transitions with existing fallback preserved - Reuse renderer Chrome args in hyperframes validate for consistent WebGL/CanvasDrawElement environment - Add capture.test.ts for the new shader transition capture path
Problem
Chrome/Brave now expose the experimental CanvasDrawElement / HTML-in-canvas API, but HyperFrames did not have a launch-ready catalog story for it. The raw capability is easy to dismiss as a tech demo unless creators can install a block that turns one editable product UI into useful launch/tutorial shots.
hyperframes validatealso needed to use the same browser args as the renderer so WebGL and CanvasDrawElement compositions are validated against the right environment.What this fixes
drawElementImage()capture support for@hyperframes/shader-transitions, with the existing fallback path preserved.CanvasDrawElementin renderer Chrome args and reuses renderer Chrome args fromhyperframes validate.stability: "experimental"metadata, schema/type/catalog/CLI surfacing, and generated docs warnings with agent setup instructions.canvas-dom-refractoras experimental.experimental-screen-callout, the simple workflow proof for one DOM screen -> zoom lens, spotlight, cursor trail, annotations, and crop strip.experimental-product-launch-portal, the flagship creator block: one editable product UI becomes shadered portal warps, device panels, crop callouts, overload glitch beats, and a final launch CTA from the same HTML-in-canvas source.Root cause
The project had shader and catalog infrastructure, but not a native browser capture path or a creator-facing component that made the API useful. CanvasDrawElement needs a layout-subtree canvas,
requestPaint()/paint lifecycle, anddrawElementImage()working together; the catalog blocks now keep that native path gated and preserve DOM fallbacks for browsers without the experimental API.Verification
Local checks
bun run --filter @hyperframes/shader-transitions testbun run --filter @hyperframes/shader-transitions typecheckbun run --filter @hyperframes/shader-transitions buildbun run --filter @hyperframes/engine test src/services/browserManager.test.tsbun run --filter @hyperframes/engine typecheckbun run --filter @hyperframes/core test src/registry/types.test.tsbun run --filter @hyperframes/core typecheckbun run --filter @hyperframes/cli typecheckbunx oxlint packages/cli/src/commands/catalog.ts packages/core/src/registry/types.ts packages/core/src/registry/index.ts packages/core/src/index.ts packages/core/src/registry/types.test.ts scripts/generate-catalog-pages.tsbun run scripts/sync-schemas.ts --checkbun run scripts/generate-catalog-pages.tsbunx oxfmt --check registry/blocks/experimental-product-launch-portal/registry-item.json registry/blocks/experimental-product-launch-portal/experimental-product-launch-portal.html registry/registry.json docs/docs.json docs/public/catalog-index.json docs/catalog/blocks/experimental-product-launch-portal.mdxgit diff --checkformatandcommitlint.Catalog validation
bun run --filter @hyperframes/cli dev -- lint /Users/miguel07code/.codex/worktrees/html-in-canvas-launch/hyperframes-oss/.codex-artifacts/catalog-experimental-screen-callout-> 0 errors, 1 expected large-composition warning.bun run --filter @hyperframes/cli dev -- validate /Users/miguel07code/.codex/worktrees/html-in-canvas-launch/hyperframes-oss/.codex-artifacts/catalog-experimental-screen-callout --no-contrast --timeout 6000-> no console errors.bun run --filter @hyperframes/cli dev -- lint /Users/miguel07code/.codex/worktrees/html-in-canvas-launch/hyperframes-oss/.codex-artifacts/catalog-experimental-product-launch-portal-> 0 errors, 1 expected large-composition warning.bun run --filter @hyperframes/cli dev -- validate /Users/miguel07code/.codex/worktrees/html-in-canvas-launch/hyperframes-oss/.codex-artifacts/catalog-experimental-product-launch-portal --no-contrast --timeout 8000 --json->ok: true.CanvasDrawElement+ renderer GPU mode ->ok: true.Date.now, unseededMath.random,requestAnimationFrame,setInterval, orfetch(.Browser verification
@hyperframes/shader-transitions:isHtmlInCanvasCaptureSupported() === true,drawElementImageCalls === 2, WebGL transition canvas active at1920x1080.agent-browserforcanvas-dom-refractorfallback/native screenshots and recordings.agent-browserforexperimental-screen-calloutfallback:native: false,drawElementCalls: 0,hasDrawElementImage: false,hasLayoutSubtree: false.agent-browserforexperimental-screen-calloutnative:native: true,drawElementCalls: 2,hasDrawElementImage: true,hasLayoutSubtree: true.agent-browserforexperimental-product-launch-portalfallback:native: false,ready: true,drawElementCalls: 0.qa-artifacts/experimental-product-launch-portal/fallback-9.2.png.qa-artifacts/experimental-product-launch-portal/fallback-flow.webm.agent-browserwith Brave 147 +--enable-features=CanvasDrawElementfor product portal native:native: true,ready: true,drawElementCalls: 7,hasDrawElementImage: true,hasLayoutSubtree: true,data-native-canvas="true".qa-artifacts/experimental-product-launch-portal/native-4.6-oriented.png,native-9.2-overload.png,native-14.2-oriented.png,native-17.2.png.qa-artifacts/experimental-product-launch-portal/native-flow.webm.Render validation
900/900frames,1920x1080,30fps,30.000000s.canvas-dom-refractor:360/360frames,1920x1080,30fps,12.000000s.experimental-screen-callout:360/360frames,1920x1080,30fps,12.000000s.experimental-product-launch-portal:540/540frames,1920x1080,30fps,18.000000s.qa-artifacts/experimental-product-launch-portal/experimental-product-launch-portal.mp4.qa-artifacts/experimental-product-launch-portal/frames/frame-4.6.png,frame-9.2.png,frame-14.2.png,frame-17.2.png.Notes
chrome://flags/#canvas-draw-element, restart the browser, and preserve fallback behavior.