feat(gemini): add GeminiDenseEmbedder text embedding provider

[chethanuk]

Description

Phase 1 of #566: Adds GeminiDenseEmbedder — a production-ready, opt-in text-only dense embedding provider backed by Google's google-genai SDK. Volcengine remains the default; Gemini is enabled via provider: "gemini" in ov.conf. No existing behaviour changes.

This PR implements Phase 1 (text-only). Phase 2 (multimodal embedding) is tracked in #566 — the issue remains open.

Phased Implementation

Phase	Scope	Status
Phase 1 (this PR)	Text-only dense embedding: 8 task types, MRL 1-3072, SDK retry, async batching, non-symmetric routing via #702 pattern	✅
Phase 2 (future)	Multimodal embedding: image/audio/video via Parts API, cross-modal retrieval, align with #718 approach	Planned

Key capabilities

• Text-only — supports_multimodal = False; multimodal is Phase 2
• Non-symmetric routing — get_query_embedder() / get_document_embedder() route query_param/document_param to Gemini task types via factory (follows feat(embedding): combine document embedder and query embedder to avoi… #702 pattern)
• Per-call task_type + title — override at call time; all 8 Gemini task types
• Case-insensitive config — task_type, query_param, document_param auto-uppercased in config normalization
• SDK-native retry — HttpRetryOptions(attempts=3, exp_base=2, max_delay=30s) with import guard for SDK < 0.8
• Async concurrent batching — async_embed_batch() dispatches 100-text chunks in parallel via anyio
• Empty-text guard — returns zero-vectors with warning log; preserves index alignment

flowchart LR
    subgraph Config["ov.conf / EmbeddingModelConfig"]
        CFG["provider: gemini\napi_key: sk-…\ntask_type: RETRIEVAL_DOCUMENT\ndimension: 1536"]
    end

    subgraph Embedder["GeminiDenseEmbedder(DenseEmbedderBase)"]
        direction TB
        BC["_build_config(task_type?, title?)"]
        E["embed(text, task_type?, title?)"]
        EB["embed_batch(texts, titles?)"]
        AEB["async_embed_batch(texts)"]
        E --> BC
        EB --> BC
        AEB --> BC
    end

    subgraph SDK["google-genai SDK"]
        direction TB
        HTTP["HttpOptions\n(retry: 3× exp backoff)"]
        SYNC["models.embed_content()"]
        ASYNC["aio.models.embed_content()\n(semaphore-bounded)"]
        HTTP --> SYNC
        HTTP --> ASYNC
    end

    Config --> Embedder
    BC --> SYNC
    AEB --> ASYNC

Loading

Related Issue

Partial implementation of #566 (Phase 1: text-only dense embedding; Phase 2: multimodal — issue stays open)

Type of Change

• Bug fix
• New feature (non-breaking, opt-in via config)
• Breaking change
• Documentation update
• Refactoring
• Performance improvement
• Test update

Changes Made

• openviking/models/embedder/gemini_embedders.py — GeminiDenseEmbedder(DenseEmbedderBase): _build_config(), per-call task_type/title, SDK retry, __repr__, empty-text guard with warning log
• openviking_cli/utils/config/embedding_config.py — registers "gemini" provider; validates + auto-uppercases task_type/query_param/document_param; factory routes context for non-symmetric mode
• openviking/models/embedder/__init__.py — exports GeminiDenseEmbedder; annotation: "text-only; multimodal planned"
• examples/ov.conf.example — Gemini config snippet
• pyproject.toml — google-genai>=0.8 dep; optional gemini-async extra for anyio
• tests/unit/test_gemini_embedder.py — 41 unit tests (mock-only, no API key needed)
• tests/unit/test_embedding_config_gemini.py — 18 config validation tests (incl. case-insensitive task_type)
• tests/integration/ — 49 integration tests (auto-skip without GOOGLE_API_KEY)

Testing

Suite	Tests	Live key required
test_gemini_embedder.py	41	No
test_embedding_config_gemini.py	18	No
test_gemini_e2e.py	7	Yes
test_gemini_embedding_it.py	22	Yes
test_gemini_openviking_it.py	20	Yes

108 passed, 0 skipped  ← with GOOGLE_API_KEY
 59 passed, 49 skipped ← without GOOGLE_API_KEY (CI)

Checklist

• Code follows project coding style (ruff + black formatted)
• Self-review completed
• Tests added and passing
• Tested on Linux and macOS

[chethanuk]

@qin-ctx Please review and merge this

[ZaynJarvis]

#702 is megred for query and document embedding support.

get_query_embedder() is not the expected usage.

btw check if you can improve based on #718

[qin-ptr]

Review Summary

This PR adds Google Gemini as a text embedding provider with excellent test coverage (107 tests) and solid error handling. However, there are three blocking issues that need to be addressed:

1. Design mismatch: Issue #566 explicitly requests multimodal retrieval support, but this PR only implements text-only embedding (supports_multimodal = False). The PR description should clarify this is a phased implementation with a roadmap for multimodal support, or change the issue reference to indicate partial completion.

2. Documentation inconsistency: Code comment claims "multimodal" but implementation is text-only.

3. CI failure: lint job failed due to formatting issues in test_gemini_embedder.py.

Once these are resolved, the PR will be ready to merge. The implementation quality is strong, with comprehensive testing, graceful error handling, and proper SDK retry integration.

🤖 I am a bot owned by @qin-ctx.

[qin-ptr]

[Bug] (blocking)

Comment says "Google Gemini: Dense only (multimodal)" but the actual implementation has GeminiDenseEmbedder.supports_multimodal = False. This is contradictory.

Should either:

• Change to "Google Gemini: Dense only (text-only; multimodal planned)"
• Or remove the "(multimodal)" annotation entirely until multimodal support is actually implemented

[qin-ptr]

[Design] (blocking)

This PR claims to close issue #566, which explicitly requests "first-class Gemini Embedding 2 support for multimodal retrieval". The issue emphasizes:

• "it should not reduce everything to plain text before embedding"
• "multimodal inputs handled as multimodal, not flattened by default"

However, this implementation sets supports_multimodal = False and only handles text. This is a fundamental mismatch between the issue requirement and the PR implementation.

Recommendation: Either:

1. Update the PR description to clarify this is phase 1 (text-only) with a roadmap for phase 2 (multimodal), and change the issue reference to "Partial implementation of [Feature]: Add first-class Gemini Embedding 2 support for multimodal retrieval #566", keeping the issue open
2. Or implement multimodal support in this PR as originally requested by the issue

The current "Closes: #566" statement is misleading and will cause users to expect multimodal functionality that doesn't exist.

[qin-ptr]

[Bug] (blocking)

CI lint job failed because this file needs formatting:

Would reformat: tests/unit/test_gemini_embedder.py

Please run black tests/unit/test_gemini_embedder.py and commit the formatted version.

[qin-ptr]

[Suggestion] (non-blocking)

The for key in ("input_type",): loop normalizes input_type to lowercase, but the new Gemini provider uses task_type instead of input_type. Consider also normalizing task_type here, or document that task_type must be uppercase (which is currently enforced in the Gemini validator at line 176).

[qin-ptr]

[Suggestion] (non-blocking)

Returning a zero vector for empty text is a reasonable fallback, but it happens silently. Consider logging a warning so callers know their empty input was handled specially:

if not text or not text.strip():
    logger.warning("Empty text passed to embed(), returning zero vector")
    return EmbedResult(dense_vector=[0.0] * self._dimension)

This helps with debugging when unexpected empty strings appear in production.

[chethanuk]

Thanks for the thorough review! All issues addressed in commit bcfd612:

@qin-ptr — blocking fixes

• B1 __init__.py:15: Fixed — annotation now reads "Dense only (text-only; multimodal planned)"
• B2 Design mismatch: Reframed PR as Phase 1 (text-only). Changed "Closes: [Feature]: Add first-class Gemini Embedding 2 support for multimodal retrieval #566" → "Partial implementation of [Feature]: Add first-class Gemini Embedding 2 support for multimodal retrieval #566". Issue stays open for Phase 2 (multimodal).
• B3 Lint: Ran black on test_gemini_embedder.py

@qin-ptr — non-blocking (adopted both)

• NB1 task_type normalization: Added auto-uppercase in sync_provider_backend for task_type, query_param, document_param. Users can now write task_type: "retrieval_document" (any case) in config. Added test test_gemini_task_type_case_insensitive.
• NB2 Empty text warning: Added logger.warning("Empty text passed to embed(), returning zero vector")

@ZaynJarvis — #702 pattern + #718

• feat(embedding): combine document embedder and query embedder to avoi… #702 pattern: Already implemented ✅ — factory at embedding_config.py:370-384 does context-based task_type routing via get_query_embedder() / get_document_embedder() / _get_contextual_embedder(). Non-symmetric mode activates when query_param or document_param is set in config.
• feat: add native Google/Gemini Embedding 2 support with Parts API #718 alignment: Reviewed. Our SDK-based approach (google-genai) vs feat: add native Google/Gemini Embedding 2 support with Parts API #718's raw REST approach are complementary. Phase 2 multimodal work will incorporate Parts API structure from feat: add native Google/Gemini Embedding 2 support with Parts API #718. If feat: add native Google/Gemini Embedding 2 support with Parts API #718 merges first, we'll consolidate file naming (google_embedders.py vs gemini_embedders.py).

Verification

ruff check: All checks passed!
pytest: 59 passed (41 embedder + 18 config, incl. 1 new normalization test)
black --check: 1 file would be left unchanged