feat: add prerender command — slideshow video with real speech + images#187
Open
SecurityQQ wants to merge 1 commit intomainfrom
Open
feat: add prerender command — slideshow video with real speech + images#187SecurityQQ wants to merge 1 commit intomainfrom
SecurityQQ wants to merge 1 commit intomainfrom
Conversation
…es, no video gen Adds a new 'varg prerender' command that generates a slideshow video for visual-audio sync review before expensive video generation. Prerender mode: - Real speech generation (ElevenLabs) - Real image generation for <Image> elements - Still-frame images for <Video> elements (t2v uses configurable image model, i2v uses input image directly) - Real music generation - Real captions (Groq Whisper) - Exact clip durations preserved - ffmpeg assembly with transitions The --image-model flag controls which model replaces t2v video generation (default: nano-banana-2). Example: --image-model flux-schnell New files: - src/ai-sdk/middleware/prerender.ts — video middleware that intercepts video generation and creates still-frame videos from images - src/cli/commands/prerender.tsx — CLI command with --image-model flag Modified files: - RenderMode type extended with 'prerender' - DefaultModels extended with optional prerenderImage - renderRoot() handles prerender mode (bypasses video cache) - render.tsx exports shared utilities for reuse - CLI index registers prerender command Cost savings: ~$0.50 prerender vs ~$3-5 full render for typical workflow.
Contributor
📝 Walkthroughwalkthroughintroduces a new "prerender" render mode that generates still-frame mp4 videos from images as a fallback mechanism. adds middleware, cli command, type definitions, and render pipeline updates to support this mode with optional image model configuration. changes
sequence diagramsequenceDiagram
actor user as user/cli
participant cli as prerender cmd
participant loader as component loader
participant models as model resolver
participant render as renderroot
participant middleware as prerenderFallbackMiddleware
participant imggen as image generator
participant imgvideo as imageToStillVideo
participant ffmpeg as ffmpeg
user->>cli: run `varg prerender`
cli->>loader: loadComponent(file)
loader-->>cli: varg element
cli->>models: detectDefaultModels()
cli->>models: resolvePrerenderImageModel()
models-->>cli: image model
cli->>render: render(mode="prerender", defaults.prerenderImage)
render->>middleware: wrapVideoModel(prerenderFallbackMiddleware)
middleware->>middleware: extract/find image from input
alt has input image (i2v)
middleware-->>imgvideo: use input image
else no input (t2v)
middleware->>imggen: generateImage(prompt)
imggen-->>middleware: image bytes
middleware-->>imgvideo: use generated image
end
imgvideo->>imgvideo: write png to temp dir
imgvideo->>ffmpeg: run ffmpeg (loop, scale, pad)
ffmpeg-->>imgvideo: mp4 bytes
imgvideo->>imgvideo: cleanup temp files
imgvideo-->>middleware: still-frame mp4
middleware-->>render: video response + warning
render-->>user: output mp4 file
estimated code review effort🎯 4 (complex) | ⏱️ ~50 minutes possibly related prs
poem
meow 🐱 🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/cli/commands/render.tsx (1)
84-97:⚠️ Potential issue | 🟠 Majorthe import detection here is too narrow
hasRelativeImportonly catchesfrom "./", so../fooand side-effect relative imports still fall into the temp-copy path, where those specifiers now resolve relative to.cache/varg-render. and because this branch returns early, files that mix local imports withvargai/*skip the rewrite block entirely. that's going to break a bunch of normal component layouts.Also applies to: 99-123
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/cli/commands/render.tsx` around lines 84 - 97, The current hasRelativeImport check is too narrow (only looks for "from './") and the early return causes files that mix local relative imports with "vargai/*" to skip the rewrite; update the detection to match any relative import specifier (both "./" and "../") including side-effect imports (e.g. use a regex like /from\s+['"](?:\.{1,2}\/)|import\s+['"](?:\.{1,2}\/)/m or equivalent) so hasRelativeImport becomes true for all "./" or "../" specifiers, and remove the early return that exits before the rewrite block — instead, when hasRelativeImport is true still import via import(resolvedPath) where needed but allow the subsequent rewrite logic to run (references: hasRelativeImport, resolvedPath, tmpDir, resolveDefaultExport).
🧹 Nitpick comments (3)
src/cli/commands/render.tsx (1)
17-19: add jsdoc now that these helpers are public
detectDefaultModels()andloadComponent()just became reusable exports, but they still ship without api docs.As per coding guidelines, "Ensure all public functions and classes have JSDoc comments".
Also applies to: 73-73
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/cli/commands/render.tsx` around lines 17 - 19, Add JSDoc comments for the newly exported helpers detectDefaultModels and loadComponent (and any other exported symbols like the DefaultModels type) describing their purpose, inputs, and return values; place a short summary line, `@returns` with the Promise<DefaultModels | undefined> for detectDefaultModels, and for loadComponent include parameter descriptions and the return type. Ensure the JSDoc appears immediately above each exported function declaration (detectDefaultModels, loadComponent) and follows project style (brief description, `@param` for each arg if any, `@returns`, and `@example` if useful).src/react/types.ts (1)
338-338: tighten the newprerendertype surface
RenderModenow lives here and again insrc/ai-sdk/middleware/wrap-video-model.ts, whileprerenderImageis only enforced by a late runtime throw. pulling this into one shared discriminated type would keep the modes in sync and catch bad callers beforerenderRoot()explodes.Also applies to: 346-347
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/react/types.ts` at line 338, Consolidate the RenderMode union into a shared discriminated type (e.g., export type RenderMode = { mode: "strict" } | { mode: "preview" } | { mode: "prerender"; prerenderImage: true }) in a common types module and import that type wherever RenderMode currently appears (including src/react/types.ts and src/ai-sdk/middleware/wrap-video-model.ts); update usages in renderRoot, prerenderImage checks, and any callers to use the discriminant (.mode) so TypeScript enforces that only the "prerender" variant can carry prerenderImage, and remove the late runtime throw by making the compiler require the prerenderImage property on the prerender variant.src/cli/commands/prerender.tsx (1)
226-228: add jsdoc for the exported helper
showPrerenderHelpis public (exported) and currently undocumented. add a short jsdoc block to match repo standards.proposed fix
+/** + * render the static help view for the `varg prerender` command. + */ export function showPrerenderHelp() { renderStatic(<PrerenderHelpView />); }as per coding guidelines, "
**/*.{js,jsx,ts,tsx}: Ensure all public functions and classes have JSDoc comments".🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/cli/commands/prerender.tsx` around lines 226 - 228, Add a JSDoc block above the exported function showPrerenderHelp describing its purpose (renders the prerender help view to static output), noting it takes no parameters and returns void; reference the function name showPrerenderHelp and the fact it calls renderStatic(<PrerenderHelpView />) so the doc aligns with the implementation and repo JSDoc standards for public functions.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/ai-sdk/middleware/prerender.ts`:
- Around line 100-129: extractFirstImage currently only preserves image files
and silently falls through for audio/video inputs; update extractFirstImage (and
the same logic at the other occurrence) to explicitly handle non-image media:
detect mediaType starting with "video/" and "audio/"; for "video/" try to
extract a poster or first frame (e.g., call a new helper like
extractFirstFrameFromVideo(file.data) or use file.metadata?.poster if present
and return its Uint8Array), and for "audio/" (or when video frame extraction
fails) fail fast by throwing a clear error (or return a distinct failure) so we
do not silently synthesize a T2V frame—add/implement
extractFirstFrameFromVideo(data: Uint8Array | string): Promise<Uint8Array |
undefined> and use it inside extractFirstImage for file.type === "file" and
mediaType.startsWith("video/"), otherwise throw an error describing the
unsupported non-image input.
In `@src/cli/commands/prerender.tsx`:
- Around line 45-50: The code that derives basename using
file.replace(...).split("/").pop() is not path-safe and breaks on Windows paths;
update the logic in the prerender command to use Node's path utilities (import {
basename as pathBasename } from "node:path" or similar) to compute
basename(file) after stripping the .tsx? extension, then use that safe basename
when building outputPath (the variables to change are basename and outputPath in
prerender.tsx).
---
Outside diff comments:
In `@src/cli/commands/render.tsx`:
- Around line 84-97: The current hasRelativeImport check is too narrow (only
looks for "from './") and the early return causes files that mix local relative
imports with "vargai/*" to skip the rewrite; update the detection to match any
relative import specifier (both "./" and "../") including side-effect imports
(e.g. use a regex like /from\s+['"](?:\.{1,2}\/)|import\s+['"](?:\.{1,2}\/)/m or
equivalent) so hasRelativeImport becomes true for all "./" or "../" specifiers,
and remove the early return that exits before the rewrite block — instead, when
hasRelativeImport is true still import via import(resolvedPath) where needed but
allow the subsequent rewrite logic to run (references: hasRelativeImport,
resolvedPath, tmpDir, resolveDefaultExport).
---
Nitpick comments:
In `@src/cli/commands/prerender.tsx`:
- Around line 226-228: Add a JSDoc block above the exported function
showPrerenderHelp describing its purpose (renders the prerender help view to
static output), noting it takes no parameters and returns void; reference the
function name showPrerenderHelp and the fact it calls
renderStatic(<PrerenderHelpView />) so the doc aligns with the implementation
and repo JSDoc standards for public functions.
In `@src/cli/commands/render.tsx`:
- Around line 17-19: Add JSDoc comments for the newly exported helpers
detectDefaultModels and loadComponent (and any other exported symbols like the
DefaultModels type) describing their purpose, inputs, and return values; place a
short summary line, `@returns` with the Promise<DefaultModels | undefined> for
detectDefaultModels, and for loadComponent include parameter descriptions and
the return type. Ensure the JSDoc appears immediately above each exported
function declaration (detectDefaultModels, loadComponent) and follows project
style (brief description, `@param` for each arg if any, `@returns`, and `@example` if
useful).
In `@src/react/types.ts`:
- Line 338: Consolidate the RenderMode union into a shared discriminated type
(e.g., export type RenderMode = { mode: "strict" } | { mode: "preview" } | {
mode: "prerender"; prerenderImage: true }) in a common types module and import
that type wherever RenderMode currently appears (including src/react/types.ts
and src/ai-sdk/middleware/wrap-video-model.ts); update usages in renderRoot,
prerenderImage checks, and any callers to use the discriminant (.mode) so
TypeScript enforces that only the "prerender" variant can carry prerenderImage,
and remove the late runtime throw by making the compiler require the
prerenderImage property on the prerender variant.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 53a4e020-cba0-4684-afec-201c5083c654
📒 Files selected for processing (9)
src/ai-sdk/middleware/index.tssrc/ai-sdk/middleware/prerender.tssrc/ai-sdk/middleware/wrap-video-model.tssrc/cli/commands/index.tssrc/cli/commands/prerender.tsxsrc/cli/commands/render.tsxsrc/cli/index.tssrc/react/renderers/render.tssrc/react/types.ts
Comment on lines
+100
to
+129
| async function extractFirstImage( | ||
| params: VideoModelV3CallOptions, | ||
| ): Promise<Uint8Array | undefined> { | ||
| if (!params.files) return undefined; | ||
|
|
||
| for (const file of params.files) { | ||
| if (file.type === "file" && file.mediaType?.startsWith("image/")) { | ||
| if (file.data instanceof Uint8Array) { | ||
| return file.data; | ||
| } | ||
| if (typeof file.data === "string") { | ||
| // base64 | ||
| return Uint8Array.from(atob(file.data), (c) => c.charCodeAt(0)); | ||
| } | ||
| } | ||
| if (file.type === "url") { | ||
| // Fetch the URL to get binary data | ||
| try { | ||
| const response = await fetch(file.url); | ||
| const contentType = response.headers.get("content-type") ?? ""; | ||
| if (contentType.startsWith("image/")) { | ||
| return new Uint8Array(await response.arrayBuffer()); | ||
| } | ||
| } catch { | ||
| // Skip URLs that can't be fetched | ||
| } | ||
| } | ||
| } | ||
|
|
||
| return undefined; |
Contributor
There was a problem hiding this comment.
non-image source inputs get dropped here
the public VideoPrompt already allows audio and video sources, but this path only preserves image files. when params.files contains a video/audio input, extractFirstImage() falls through and we synthesize a fresh t2v frame from the prompt instead, which changes the scene you're trying to review. at minimum, fail fast for unsupported params.files; ideally extract a poster or first frame for video inputs.
Also applies to: 152-179
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/ai-sdk/middleware/prerender.ts` around lines 100 - 129, extractFirstImage
currently only preserves image files and silently falls through for audio/video
inputs; update extractFirstImage (and the same logic at the other occurrence) to
explicitly handle non-image media: detect mediaType starting with "video/" and
"audio/"; for "video/" try to extract a poster or first frame (e.g., call a new
helper like extractFirstFrameFromVideo(file.data) or use file.metadata?.poster
if present and return its Uint8Array), and for "audio/" (or when video frame
extraction fails) fail fast by throwing a clear error (or return a distinct
failure) so we do not silently synthesize a T2V frame—add/implement
extractFirstFrameFromVideo(data: Uint8Array | string): Promise<Uint8Array |
undefined> and use it inside extractFirstImage for file.type === "file" and
mediaType.startsWith("video/"), otherwise throw an error describing the
unsupported non-image input.
Comment on lines
+45
to
+50
| const basename = file | ||
| .replace(/\.tsx?$/, "") | ||
| .split("/") | ||
| .pop(); | ||
| const outputPath = | ||
| (args.output as string) ?? `output/${basename}-prerender.mp4`; |
Contributor
There was a problem hiding this comment.
use path-safe filename extraction for default output path
line 45-line 48 uses split("/"), which is unix-only. with windows-style paths, the generated default output can be malformed (for example includes drive/path fragments). use node:path basename extraction instead.
proposed fix
-import { dirname } from "node:path";
+import { basename, dirname } from "node:path";
@@
- const basename = file
- .replace(/\.tsx?$/, "")
- .split("/")
- .pop();
+ const fileStem = basename(file).replace(/\.tsx?$/, "");
const outputPath =
- (args.output as string) ?? `output/${basename}-prerender.mp4`;
+ (args.output as string) ?? `output/${fileStem}-prerender.mp4`;📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| const basename = file | |
| .replace(/\.tsx?$/, "") | |
| .split("/") | |
| .pop(); | |
| const outputPath = | |
| (args.output as string) ?? `output/${basename}-prerender.mp4`; | |
| import { basename, dirname } from "node:path"; | |
| ... | |
| const fileStem = basename(file).replace(/\.tsx?$/, ""); | |
| const outputPath = | |
| (args.output as string) ?? `output/${fileStem}-prerender.mp4`; |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/cli/commands/prerender.tsx` around lines 45 - 50, The code that derives
basename using file.replace(...).split("/").pop() is not path-safe and breaks on
Windows paths; update the logic in the prerender command to use Node's path
utilities (import { basename as pathBasename } from "node:path" or similar) to
compute basename(file) after stripping the .tsx? extension, then use that safe
basename when building outputPath (the variables to change are basename and
outputPath in prerender.tsx).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
varg prerendercommand that generates a slideshow video for visual-audio sync review before expensive video generation--image-modelflag (default:nano-banana-2) controls which model generates images for text-to-video clipsNew files
src/ai-sdk/middleware/prerender.ts— video middleware that intercepts video generation, extracts/generates images, creates still-frame videos via ffmpegsrc/cli/commands/prerender.tsx— CLI command with--image-modelflag and full help viewModified files
src/react/types.ts—RenderModeextended with"prerender",DefaultModelsextended with optionalprerenderImagesrc/ai-sdk/middleware/wrap-video-model.ts—RenderModetype updatedsrc/react/renderers/render.ts—renderRoot()handles prerender mode (bypasses video cache, wraps model with prerender middleware)src/cli/commands/render.tsx— exportsloadComponent,detectDefaultModels,sharedArgsfor reusesrc/ai-sdk/middleware/index.ts— exports prerender middlewaresrc/cli/commands/index.ts— exports prerender commandsrc/cli/index.ts— registers prerender in subCommands and help routingUsage
Cost comparison
previewprerenderrenderTested