feat(api): add Gemini video metadata controls#161
Merged
Conversation
added 4 commits
March 20, 2026 23:39
Introduce ProviderHint as a generic, provider-scoped extension on Source so provider-specific metadata stays behind a uniform wall rather than leaking Gemini-isms into the core model. Add Source.with_gemini_video_settings() for validated clip windows and FPS controls on video sources. Make content_hash() private since callers should use cache_identity_hash() which incorporates provider context.
build_shared_parts now accepts a provider parameter and attaches provider hints to transport dicts. Upload substitution in both the execute and cache paths preserves hints through the file-to-URI conversion. Cache identity uses cache_identity_hash so Gemini video settings fragment Gemini cache keys without affecting other providers.
Translate provider hints into types.VideoMetadata in _convert_parts, create_cache, and deferred request resolution. Source-level validation is trusted; the adapter only does type narrowing and lets the SDK try/except catch any remaining mismatches.
…pers Add Gemini video clipping example to sending-content, note cache identity behavior in caching docs, and reference the feature in the portability guide, provider capabilities, API reference, and README.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add
Source.with_gemini_video_settings(start_offset, end_offset, fps)for Gemini-specific video clipping and frame sampling. Settings are stored as generic provider hints so Gemini-isms stay behind a uniform wall in the core model, threaded through the pipeline, and materialized only in the Gemini adapter.Related issue
None
Test plan
Three test boundaries exercised, all passing via
just check:test_source.py): validation (non-video rejection, invalid fps, blank offsets, at-least-one-value), provider scoping (Gemini returns settings, OpenAI/None return None), immutability, hashability, and payload isolationtest_pipeline.py): video settings survive upload substitution end-to-end; cache identity varies with Gemini video settings but stays stable for non-Gemini providerstest_providers.py): Gemini adapter attachesVideoMetadatato generate, create_cache, and deferred submission pathsDocs build verified clean (
just docs-build).Notes
ProviderHintis intentionally generic rather than a typedGeminiVideoSettingsfield on Source. This keeps provider-specific concerns behind a uniform wall and prevents Gemini-isms from drifting into the core model over time.content_hash()was made private (_content_hash()) since callers should usecache_identity_hash()which incorporates provider context.