Add AI video generation using Wan2.1/2.2 models#11
Open
Asad-Ismail wants to merge 4 commits intomainfrom
Open
Conversation
Two model options: - wan2.1-1.3b: 480p/15fps, 8GB VRAM (RTX 3060+) - wan2.2-5b: 720p/24fps, 24GB VRAM (RTX 4090) Both Apache 2.0, full HuggingFace diffusers integration. Generates per-segment clips from the script text instead of searching stock footage sites.
- Add ai_generated branch in get_video_materials() - Skip search term generation for AI source (uses script directly) - Pass video_script to get_video_materials for prompt splitting
- Add "AI Generated (Wan2.1/2.2)" to video source dropdown - Show model selector (1.3B vs 5B) when AI source selected - Warn if CUDA/diffusers not available - Add video_gen_model config to config.example.toml - Add diffusers + accelerate to requirements.txt
Avoids merge conflict with other PRs that add deps at the end of the file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces stock footage dependency with AI-generated video clips. Instead of searching Pexels/Pixabay for generic clips, this generates custom video that actually matches the script content.
Uses Wan2.1/2.2 text-to-video models via HuggingFace diffusers. Both are Apache 2.0 licensed.
Two model options:
How it works
Falls back gracefully — if no CUDA GPU or diffusers not installed, the UI warns the user. Existing Pexels/Pixabay sources work exactly as before.
Changes
app/services/video_gen.py— generation service with model loading, prompt enhancement, clip generationapp/services/task.py—ai_generatedbranch in get_video_materials()webui/Main.py— new video source option + model selectorconfig.example.toml— video_gen_model settingrequirements.txt— diffusers + accelerateTest plan
pip install diffusers accelerate