feat(ai-openrouter): video generation adapter (/api/v1/videos) + image activity follow-ups by tombeckenham · Pull Request #740 · TanStack/ai

tombeckenham · 2026-06-10T08:35:18Z

Important

Blocked by #624 (implements #618) — this PR is stacked on 618-image-to-image-and-image-to-video-support and stays draft until #624 merges, then gets rebased onto main and marked ready.

🎯 Changes

Implements #707 (follow-up to #618 / #624). Stacked on #624 — based on 618-image-to-image-and-image-to-video-support; rebase onto main once that merges.

`openRouterVideo` adapter

New tree-shakeable adapter for OpenRouter's dedicated async video API (POST /api/v1/videos → poll GET /api/v1/videos/{jobId} → download) — Seedance 2.0, Veo 3.1, Wan, Kling, Sora 2 Pro through one key, on the same jobs/polling architecture as the Sora adapter.
imageInputs role mapping per the issue table: start_frame/end_frame → frame_images[] (first_frame/last_frame), reference/character → input_references[], mask/control → throw, unroled image → start frame. Frame roles are validated against each model's supported_frame_images. videoInputs/audioInputs throw (unsupported by the API).
Per-model size/duration/resolution/aspectRatio types and runtime validation are generated from GET /api/v1/videos/models (OPENROUTER_VIDEO_MODEL_META); seed/generateAudio are dropped from the type for models whose metadata reports them unsupported. The sync scripts (fetch-openrouter-models.ts / convert-openrouter-models.ts) now fetch and convert the second endpoint.
Completed videos are returned as data: URLs — unsigned_urls 401 without the API key header (verified live), so they can't go straight into a <video> tag. Downloads >10 MiB log the same OOM warning as the OpenAI adapter. Gateway-reported cost is surfaced as usage.cost.
⚠️ Bypasses the SDK's getVideoContent: its response matcher (as of @openrouter/sdk 0.12.35) only accepts application/octet-stream while the live endpoint serves video/mp4 — worth reporting upstream. Download uses a config-injectable fetch seam.

Image activity follow-ups (#624 review)

Unmapped size now throws with the supported list. Root cause found: the OpenRouterImageModelSizeByName union used the Unicode × (U+00D7) while the lookup table used ASCII x, so every typed size except 1024x1024 silently dropped its aspect ratio. Union fixed to ASCII; × still normalized at runtime.
numberOfImages > 1 now throws. Live-verified: the chat-completions pathway ignores every count key (numberOfImages, num_images, count) and always returns one image. Also confirmed image_config casing live: snake_case aspect_ratio changes output dimensions, camelCase is silently ignored — comment added at the request site.
image_config.strength (0.0–1.0 i2i influence) exposed via modelOptions.strength.

Tests / E2E / Docs

23 new unit tests (request shapes incl. role mapping, polling lifecycle, download/error paths) + image-fix tests; full submit→poll→download lifecycle and image-to-video verified live against OpenRouter (grok-imagine-video, ~$0.15 total).
E2E: matrix exclusion documented in feature-support.ts — aimock 1.29 only mocks the OpenAI-shaped /v1/videos, not OpenRouter's polling_url/unsigned_urls job shape (same constraint noted in feat: multimodal prompt for generateImage/generateVideo (image-to-image, image-to-video) #624).
Docs: media/video-generation.md, media/image-generation.md, adapters/openrouter.md (+ config.json dates), media-generation SKILL.md, changeset (minor).

✅ Checklist

I have followed the steps in the Contributing guide.
I have tested this code locally with pnpm run test:pr.

🚀 Release Impact

This change affects published code, and I have generated a changeset.
This change is docs/CI/dev-only (no release).

🤖 Generated with Claude Code

coderabbitai · 2026-06-10T08:35:27Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: fe19e3c0-8f14-466f-9b2a-8da68eb021a6

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch 707-featai-openrouter-video-generation-adapter-apiv1videos-+-image-activity-follow-ups

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

socket-security · 2026-06-10T08:36:05Z

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	@openrouter/sdk@0.12.79

View full report

github-actions · 2026-06-10T08:36:11Z

🚀 Changeset Version Preview

8 package(s) bumped directly, 23 bumped as dependents.

🟥 Major bumps

Package	Version	Reason
`@tanstack/ai-event-client`	0.5.4 → 1.0.0	Changeset
`@tanstack/ai-fal`	0.7.23 → 1.0.0	Changeset
`@tanstack/ai-gemini`	0.15.1 → 1.0.0	Changeset
`@tanstack/ai-grok`	0.11.2 → 1.0.0	Changeset
`@tanstack/ai-openai`	0.14.1 → 1.0.0	Changeset
`@tanstack/ai-openrouter`	0.13.1 → 1.0.0	Changeset
`@tanstack/ai-anthropic`	0.15.1 → 1.0.0	Dependent
`@tanstack/ai-code-mode`	0.2.5 → 1.0.0	Dependent
`@tanstack/ai-code-mode-skills`	0.2.5 → 1.0.0	Dependent
`@tanstack/ai-elevenlabs`	0.2.20 → 1.0.0	Dependent
`@tanstack/ai-groq`	0.4.2 → 1.0.0	Dependent
`@tanstack/ai-isolate-node`	0.1.30 → 1.0.0	Dependent
`@tanstack/ai-isolate-quickjs`	0.1.30 → 1.0.0	Dependent
`@tanstack/ai-ollama`	0.8.1 → 1.0.0	Dependent
`@tanstack/ai-preact`	0.9.4 → 1.0.0	Dependent
`@tanstack/ai-react`	0.15.4 → 1.0.0	Dependent
`@tanstack/ai-react-ui`	0.8.6 → 1.0.0	Dependent
`@tanstack/ai-solid`	0.13.4 → 1.0.0	Dependent
`@tanstack/ai-solid-ui`	0.7.6 → 1.0.0	Dependent
`@tanstack/ai-svelte`	0.13.4 → 1.0.0	Dependent
`@tanstack/ai-vue`	0.13.4 → 1.0.0	Dependent
`@tanstack/openai-base`	0.8.1 → 1.0.0	Dependent

🟨 Minor bumps

Package	Version	Reason
`@tanstack/ai`	0.28.0 → 0.29.0	Changeset
`@tanstack/ai-client`	0.16.3 → 0.17.0	Changeset

🟩 Patch bumps

Package	Version	Reason
`@tanstack/ai-devtools-core`	0.4.8 → 0.4.9	Dependent
`@tanstack/ai-isolate-cloudflare`	0.2.21 → 0.2.22	Dependent
`@tanstack/ai-mcp`	0.1.0 → 0.1.1	Dependent
`@tanstack/ai-vue-ui`	0.2.16 → 0.2.17	Dependent
`@tanstack/preact-ai-devtools`	0.1.51 → 0.1.52	Dependent
`@tanstack/react-ai-devtools`	0.2.51 → 0.2.52	Dependent
`@tanstack/solid-ai-devtools`	0.2.51 → 0.2.52	Dependent

nx-cloud · 2026-06-10T08:37:14Z

View your CI Pipeline Execution ↗ for commit 7bb9066

Command	Status	Duration	Result
`nx run-many --targets=build --exclude=examples/...`	✅ Succeeded	1s	View ↗

☁️ Nx Cloud last updated this comment at 2026-06-11 00:42:48 UTC

pkg-pr-new · 2026-06-10T08:39:37Z

Open in StackBlitz

@tanstack/ai

npm i https://pkg.pr.new/@tanstack/ai@740

@tanstack/ai-anthropic

npm i https://pkg.pr.new/@tanstack/ai-anthropic@740

@tanstack/ai-client

npm i https://pkg.pr.new/@tanstack/ai-client@740

@tanstack/ai-code-mode

npm i https://pkg.pr.new/@tanstack/ai-code-mode@740

@tanstack/ai-code-mode-skills

npm i https://pkg.pr.new/@tanstack/ai-code-mode-skills@740

@tanstack/ai-devtools-core

npm i https://pkg.pr.new/@tanstack/ai-devtools-core@740

@tanstack/ai-elevenlabs

npm i https://pkg.pr.new/@tanstack/ai-elevenlabs@740

@tanstack/ai-event-client

npm i https://pkg.pr.new/@tanstack/ai-event-client@740

@tanstack/ai-fal

npm i https://pkg.pr.new/@tanstack/ai-fal@740

@tanstack/ai-gemini

npm i https://pkg.pr.new/@tanstack/ai-gemini@740

@tanstack/ai-grok

npm i https://pkg.pr.new/@tanstack/ai-grok@740

@tanstack/ai-groq

npm i https://pkg.pr.new/@tanstack/ai-groq@740

@tanstack/ai-isolate-cloudflare

npm i https://pkg.pr.new/@tanstack/ai-isolate-cloudflare@740

@tanstack/ai-isolate-node

npm i https://pkg.pr.new/@tanstack/ai-isolate-node@740

@tanstack/ai-isolate-quickjs

npm i https://pkg.pr.new/@tanstack/ai-isolate-quickjs@740

@tanstack/ai-mcp

npm i https://pkg.pr.new/@tanstack/ai-mcp@740

@tanstack/ai-ollama

npm i https://pkg.pr.new/@tanstack/ai-ollama@740

@tanstack/ai-openai

npm i https://pkg.pr.new/@tanstack/ai-openai@740

@tanstack/ai-openrouter

npm i https://pkg.pr.new/@tanstack/ai-openrouter@740

@tanstack/ai-preact

npm i https://pkg.pr.new/@tanstack/ai-preact@740

@tanstack/ai-react

npm i https://pkg.pr.new/@tanstack/ai-react@740

@tanstack/ai-react-ui

npm i https://pkg.pr.new/@tanstack/ai-react-ui@740

@tanstack/ai-solid

npm i https://pkg.pr.new/@tanstack/ai-solid@740

@tanstack/ai-solid-ui

npm i https://pkg.pr.new/@tanstack/ai-solid-ui@740

@tanstack/ai-svelte

npm i https://pkg.pr.new/@tanstack/ai-svelte@740

@tanstack/ai-utils

npm i https://pkg.pr.new/@tanstack/ai-utils@740

@tanstack/ai-vue

npm i https://pkg.pr.new/@tanstack/ai-vue@740

@tanstack/ai-vue-ui

npm i https://pkg.pr.new/@tanstack/ai-vue-ui@740

@tanstack/openai-base

npm i https://pkg.pr.new/@tanstack/openai-base@740

@tanstack/preact-ai-devtools

npm i https://pkg.pr.new/@tanstack/preact-ai-devtools@740

@tanstack/react-ai-devtools

npm i https://pkg.pr.new/@tanstack/react-ai-devtools@740

@tanstack/solid-ai-devtools

npm i https://pkg.pr.new/@tanstack/solid-ai-devtools@740

commit: 7bb9066

…e activity follow-ups Closes #707. - Add openRouterVideo: async jobs adapter for OpenRouter's dedicated video API (submit -> poll -> download). Per-model size/duration/option types are generated from GET /api/v1/videos/models; frame roles map onto frame_images[] / input_references[] per the MediaInputRole taxonomy. - Teach the model-meta sync scripts the videos/models endpoint (openrouter.video-models.json + OPENROUTER_VIDEO_MODEL_META). - Image adapter follow-ups from the #624 review: throw on unmapped sizes (the size union used a Unicode multiplication sign so every non-square size silently dropped its aspect ratio), throw on numberOfImages > 1 (live-verified: the gateway ignores all count keys), expose image_config.strength. - Completed videos are returned as data: URLs (unsigned_urls 401 without the API key header) with gateway-reported cost on usage.cost. The SDK's getVideoContent is bypassed: its matcher only accepts application/octet-stream while the endpoint serves video/mp4. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The getVideoContent response-matcher bug is still present in 0.12.79 (the stream matcher only accepts application/octet-stream while the endpoint serves video/mp4), so the direct unsigned-URL download stays. Link the aimock feature request (CopilotKit/aimock#261) from the e2e matrix exclusion. Submit/poll/download lifecycle re-verified live on the new SDK. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

tombeckenham requested a review from a team as a code owner June 10, 2026 08:35

tombeckenham mentioned this pull request Jun 10, 2026

Support OpenRouter's async video generation endpoints (/api/v1/videos job lifecycle) CopilotKit/aimock#261

Closed

tombeckenham force-pushed the 618-image-to-image-and-image-to-video-support branch from 0c65cc7 to acd7319 Compare June 11, 2026 00:18

tombeckenham and others added 3 commits June 11, 2026 10:37

ci: apply automated fixes

2c3c463

tombeckenham force-pushed the 707-featai-openrouter-video-generation-adapter-apiv1videos-+-image-activity-follow-ups branch from 413f0a7 to 7bb9066 Compare June 11, 2026 00:41

tombeckenham marked this pull request as draft June 11, 2026 00:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(ai-openrouter): video generation adapter (/api/v1/videos) + image activity follow-ups#740

feat(ai-openrouter): video generation adapter (/api/v1/videos) + image activity follow-ups#740
tombeckenham wants to merge 3 commits into
618-image-to-image-and-image-to-video-supportfrom
707-featai-openrouter-video-generation-adapter-apiv1videos-+-image-activity-follow-ups

tombeckenham commented Jun 10, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Jun 10, 2026 •

edited

Loading

Review skipped

Uh oh!

socket-security Bot commented Jun 10, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 10, 2026 •

edited

Loading

Uh oh!

nx-cloud Bot commented Jun 10, 2026 •

edited

Loading

Uh oh!

pkg-pr-new Bot commented Jun 10, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

tombeckenham commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🎯 Changes

openRouterVideo adapter

Image activity follow-ups (#624 review)

Tests / E2E / Docs

✅ Checklist

🚀 Release Impact

Uh oh!

coderabbitai Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

socket-security Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Changeset Version Preview

🟥 Major bumps

🟨 Minor bumps

🟩 Patch bumps

Uh oh!

nx-cloud Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pkg-pr-new Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

tombeckenham commented Jun 10, 2026 •

edited

Loading

`openRouterVideo` adapter

coderabbitai Bot commented Jun 10, 2026 •

edited

Loading

socket-security Bot commented Jun 10, 2026 •

edited

Loading

github-actions Bot commented Jun 10, 2026 •

edited

Loading

nx-cloud Bot commented Jun 10, 2026 •

edited

Loading

pkg-pr-new Bot commented Jun 10, 2026 •

edited

Loading