Skip to content

feat(studio): add animation inspector alpha#610

Open
miguel-heygen wants to merge 11 commits intonextfrom
feat/studio-animation-inspector
Open

feat(studio): add animation inspector alpha#610
miguel-heygen wants to merge 11 commits intonextfrom
feat/studio-animation-inspector

Conversation

@miguel-heygen
Copy link
Copy Markdown
Collaborator

@miguel-heygen miguel-heygen commented May 3, 2026 •
edited
Loading

Problem

Studio did not have a deterministic, source-backed way to author motion or a practical inspector workflow for manual DOM edits. Users could scrub and preview projects, but selecting real DOM layers, persisting visual edits, inspecting nested composition layers, and keeping preview/history/render behavior aligned were split across separate paths.

What this fixes

  • Adds the data-hf-motion runtime adapter and tests for deterministic seekable Studio-owned motion.
  • Adds Studio motion editing helpers, source patching support for attribute removal, motion ownership/curve plumbing, and owned-motion preview behavior.
  • Adds shared edit-history API routes and Studio undo/redo wiring so UI and external edits can flow through one history boundary.
  • Adds timeline inspector affordances: per-clip inspector buttons, nested layer counts, draggable clip-layer panel, LOC copy, and composition-view child selection.
  • Seeks to the first visible sampled moment before opening the Design panel when selecting a nested clip layer, so layer selection lands on the frame where that exact DOM node is actually on screen.
  • Adds thumbnail infrastructure with memory/IndexedDB caches, scheduling, media-aware thumbnail/waveform toggles, and composition thumbnail hover playback.
  • Restores seamless Studio preview refresh behavior without reloading the whole editor and keeps the player load path resilient for warm iframe loads.
  • Updates Studio favicon and sidebar/timeline UI polish needed for the alpha demo flow.

Performance

  • Keeps timeline thumbnails opt-in per clip instead of globally generating every thumbnail by default.
  • Uses a bounded thumbnail scheduler so thumbnail work is queued and deduped instead of stampeding the preview/player.
  • Splits audio clips onto waveform rendering instead of pretending audio has visual thumbnails.
  • Adds memory and IndexedDB thumbnail caches keyed by source, time, and mode so repeated composition/timeline views reuse prior captures.
  • Uses Mediabunny/WebCodecs-backed video thumbnail extraction when available, with a browser-video fallback for compatibility.
  • Narrows Studio preview refreshes to the player/document path after source edits instead of forcing broader editor reload churn.
  • Preserves the deferred @hyperframes/player load path while adding a readiness probe for warm iframe loads, avoiding missed load events without reintroducing a static player import.
  • Optimizes composition path handling and timeline layer discovery so nested layer inspection works from composition views without repeatedly rebuilding unrelated timeline state.

Performance benchmark

Measured locally on hf-manual-editing-demo with Chrome headless, 5 runs each, comparing this PR head 795f278f against origin/next b07b888b. The benchmark opened the same project on isolated Studio dev servers, waited for the player/timeline to become usable, and captured CDP performance counters plus network request counts.

Metric PR head origin/next Delta
Player duration ready 412.8 ms 734.3 ms 43.8% faster
Studio interactive 1,456.3 ms 1,772.1 ms 17.8% faster
Initial thumbnail requests 0 48 eliminated on warm median
Initial API requests 32 117 72.6% fewer
Failed requests 0 8 eliminated
Hidden preview iframes 0 8 eliminated
DOM nodes after load 3,877 5,502 29.5% fewer
Transferred bytes 26.9 MB 29.4 MB 8.5% lower
JS heap after load 47.5 MB 38.3 MB 24.0% higher

The important win is that timeline/composition thumbnails no longer stampede hidden preview iframes and thumbnail/API requests during initial Studio load. JS heap is higher because this alpha branch adds substantial editor surface area: manual DOM selection, inspector controls, thumbnail services/caches, shared history, and motion UI.

Root cause

Studio’s existing editing model was centered on composition/timeline structure, not real DOM-layer ownership. Preview selection, source persistence, render-time runtime adapters, and history updates each had their own assumptions, so manual edits could work visually without having a deterministic owner or shared transaction model. The composition-view inspector also had to reason about iframe DOM nodes, where parent-window HTMLElement checks are unreliable.

Verification

Local

  • bun run --filter @hyperframes/core test -- src/runtime/adapters/hfMotion.test.ts src/runtime/init.test.ts src/runtime/player.test.ts
  • bun run --filter @hyperframes/studio test -- src/components/nle/NLELayout.test.ts src/components/nle/NLEPreview.test.ts src/player/components/Timeline.test.ts src/player/components/TimelineThumbnailContent.test.ts src/components/sidebar/CompositionsTab.test.ts src/utils/timelineInspector.test.ts src/utils/compositionPaths.test.ts src/thumbnails/sourceHash.test.ts src/thumbnails/thumbnailKey.test.ts src/thumbnails/thumbnailMode.test.ts src/thumbnails/memoryThumbnailCache.test.ts src/thumbnails/indexedDbThumbnailCache.test.ts src/thumbnails/studioThumbnailService.test.ts src/thumbnails/thumbnailScheduler.test.ts
  • bun run --filter @hyperframes/studio test -- src/components/nle/NLELayout.test.ts src/components/nle/NLEPreview.test.ts src/player/components/Timeline.test.ts src/utils/timelineInspector.test.ts src/utils/compositionPaths.test.ts
  • bun run --filter @hyperframes/core build
  • bun run --filter @hyperframes/studio build
  • bunx oxlint packages/studio/src/App.tsx packages/studio/src/components/editor/TimelineLayerPanel.tsx
  • bunx oxfmt packages/studio/src/App.tsx packages/studio/src/components/editor/TimelineLayerPanel.tsx
  • Pre-commit also passed oxlint, oxfmt --check, and typecheck after the amended commit.

Browser

  • Used agent-browser on http://127.0.0.1:5196/?v=pr-recording#project/hf-manual-editing-demo.
  • Verified the Studio preview loads after the player conflict cleanup.
  • Verified the Hook timeline clip shows a 23 nested-layer button.
  • Opened Clip layers, selected DI Topline, and confirmed the Design panel opens for the nested element with layout controls.
  • Used agent-browser again on http://127.0.0.1:5196/?v=draggable-layer-seek#project/hf-manual-editing-demo.
  • Dragged the Clip layers panel away from the preview content and confirmed the panel stays repositioned with viewport clamping.
  • Selected H1 Title from the nested layer list and confirmed Studio seeks from the initial frame to the first visible sampled moment before opening the Design panel for that exact title layer.
  • Local proof artifacts were captured but not committed:
    • qa-artifacts/pr-studio-animation-inspector/rebased-composition-layer-panel.png
    • qa-artifacts/pr-studio-animation-inspector/rebased-layer-flow.webm
    • qa-artifacts/pr-studio-animation-inspector/draggable-layer-panel.png

Notes

  • This is intentionally an alpha/demo slice and is large because it ties together runtime ownership, Studio UI, source persistence, preview refresh, thumbnails, and nested layer inspection.
  • Benchmark result files, performance writeups, and skill/spec docs were removed from the PR; the diff now keeps the product code and tests.
  • qa-artifacts/ is local-only and was not committed.
  • Studio build exits successfully; Vite still reports the existing large chunk warning.

@miguel-heygen
feat(studio): add manual DOM editing inspector (#466)
4707329 
* fix: stabilize studio preview and runtime sync

* fix: pass selector through timeline thumbnails

* feat: add studio timeline editing

* fix: disambiguate timeline edit targets

* fix: stop timeline auto-scroll in fit mode

* feat: use percentage-based timeline zoom

* fix: sync timeline playhead on zoom changes

* fix: reset timeline scroll when returning to fit

* feat(studio): add manual DOM editing inspector

* docs: update studio manual dom editing guide

* feat(studio): add image asset picker for fills

* feat(studio): add inline image uploads for fills

* fix(studio): use real file input for image fill uploads

* fix(studio): restore toast plumbing after rebase

* fix(studio): explain in-app upload limitation

* fix(studio): reuse asset-tab upload pattern in fills

* feat(studio): refine manual design inspector

* fix(studio): polish manual design inspector

* fix(studio): keep color picker in viewport

* fix(studio): clarify color picker selection

* docs: update manual DOM editing guide

* fix(studio): keep gradient color picker open

* fix(studio): scope text color to text layers

* fix(studio): add agent fallback for immovable layers

* fix(studio): address manual editing review feedback

* fix(studio): make local font selection reliable
@miguel-heygen
fix(studio): improve dom picking and thumbnails
adbf707
@miguel-heygen
fix(studio): copy absolute paths in agent prompts
900550b
@miguel-heygen
fix(studio): prevent timeline track cutoff
b395475
@miguel-heygen
fix: copy Studio agent prompts in Safari
d5f1afe
@miguel-heygen
fix(studio): hold canvas movement from inspector
2c22276
@miguel-heygen
feat(studio): add persistent undo redo (#537)
2da3419 
Studio manual editing and timeline editing mutate project files directly, but those edits had no reliable undo/redo path. Before releasing manual editing, users need a way to recover from visual property changes, source-editor saves, timeline moves/resizes/deletes, and timeline asset drops.

The history also needs to survive a page refresh. A refresh should not erase the only way back from a bad manual edit.

- Adds a persistent per-project edit-history model for file snapshots.
- Stores undo/redo stacks in IndexedDB so history survives Studio refreshes.
- Records source editor saves, manual DOM edits, and timeline mutations.
- Adds toolbar undo/redo buttons with standard keyboard shortcuts: `Cmd/Ctrl+Z`, `Cmd/Ctrl+Shift+Z`, and `Ctrl+Y`.
- Validates current file hashes before applying undo/redo so external file changes do not silently overwrite newer content.
- Keeps history available in memory if IndexedDB persistence fails during a session.
- Adds focused unit coverage for the pure history model, storage adapter, controller/hook behavior, and project-file save helper.

Studio previously treated every editor mutation as an immediate file write. Manual DOM editing, timeline updates, and source-editor saves each had separate write paths, so there was no common transaction boundary where Studio could capture the file contents before and after an edit.

Undo/redo needed to sit above those write paths as a file-level transaction system: capture changed files before saving, write the new contents, persist the history entry by project, then apply undo/redo only when the current file content still matches the expected snapshot.

- `bun --filter @hyperframes/studio test src/utils/editHistory.test.ts src/utils/editHistoryStorage.test.ts src/hooks/usePersistentEditHistory.test.ts src/utils/studioFileHistory.test.ts` -> 4 files pass, 15 tests pass
- `bun --filter @hyperframes/studio test` -> 26 files pass, 289 tests pass
- `bun --filter @hyperframes/studio typecheck`
- `bunx oxlint packages/studio/src/App.tsx packages/studio/src/icons/SystemIcons.tsx packages/studio/src/hooks/usePersistentEditHistory.ts packages/studio/src/hooks/usePersistentEditHistory.test.ts packages/studio/src/utils/editHistory.ts packages/studio/src/utils/editHistory.test.ts packages/studio/src/utils/editHistoryStorage.ts packages/studio/src/utils/editHistoryStorage.test.ts packages/studio/src/utils/studioFileHistory.ts packages/studio/src/utils/studioFileHistory.test.ts` -> 0 warnings, 0 errors
- `bunx oxfmt --check packages/studio/src/App.tsx packages/studio/src/icons/SystemIcons.tsx packages/studio/src/hooks/usePersistentEditHistory.ts packages/studio/src/hooks/usePersistentEditHistory.test.ts packages/studio/src/utils/editHistory.ts packages/studio/src/utils/editHistory.test.ts packages/studio/src/utils/editHistoryStorage.ts packages/studio/src/utils/editHistoryStorage.test.ts packages/studio/src/utils/studioFileHistory.ts packages/studio/src/utils/studioFileHistory.test.ts`
- `git diff --check`
- `bun run --filter @hyperframes/core build:hyperframes-runtime` before commit hook, because the clean worktree needed the ignored runtime-inline artifact for typecheck
- Lefthook pre-commit -> lint, format, typecheck pass
- Lefthook commit-msg -> commitlint pass

- Started Studio locally at `http://127.0.0.1:5190/#project/undo-redo-sample`.
- Used `agent-browser` to select a preview element in the Inspector and change `#hero-card` from `left: 220px` to `left: 260px`.
- Refreshed Studio and verified Undo stayed enabled.
- Clicked Undo and verified the project file returned to `left: 220px`; clicked Redo and verified the inline `left: 260px` returned.
- Used `agent-browser` to drag the `side-card` timeline clip, refreshed Studio, then verified Undo restored the previous timeline attributes and Redo reapplied the timeline move.
- Recorded the tested undo/redo flow with `agent-browser`: `qa-artifacts/studio-undo-redo-2026-04-28/studio-undo-redo-flow.webm`.

- Local screenshots and recordings are kept under `qa-artifacts/studio-undo-redo-2026-04-28/` and are intentionally not committed.
- The scratch Studio project used for browser proof is local-only under `packages/studio/data/projects/undo-redo-sample/` and is intentionally not committed.
- The PR intentionally excludes the earlier PRD/TDD planning notes under `docs/superpowers/`; those remain local-only per request.
@miguel-heygen
chore: release v0.5.0-alpha.14
b07b888
@miguel-heygen miguel-heygen force-pushed the feat/studio-animation-inspector branch 2 times, most recently from 8b50532 to 8beb451 Compare May 3, 2026 21:41
@miguel-heygen miguel-heygen force-pushed the feat/studio-animation-inspector branch from 8beb451 to 795f278 Compare May 3, 2026 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@miguel-heygen