Skip to content

Collections: making this module llm provider agnostic #489

@nishika26

Description

@nishika26

Is your feature request related to a problem? Please describe.
Kaapi is currently tightly coupled to OpenAI's services, specifically in the collections module where we create OpenAI vector stores for file search. As we expand to support multiple LLM providers (Anthropic, gemini, etc.), this hard dependency creates critical issues:

  • Cannot use non-OpenAI providers for file search (e.g., Anthropic vector stores)
  • Risk of duplicating file search logic for each provider
  • Existing collections locked to OpenAI, unable to switch providers

Describe the solution you'd like

  1. Abstract Provider Interface
  • Base classes for vector store operations (VectorStoreProvider) with common methods
  1. Provider Implementations
  • OpenAIVectorStoreProvider, AnthropicVectorStoreProvider, etc.
  • Each handles provider-specific API calls and authentication
  1. Provider Factory
  • Registry to manage available providers
  1. Collection Model Updates
  • Add provider: str field (default: "openai") to request model
  1. Backward Compatibility
  • Existing OpenAI collections work without changes
  • Migration path for updating collections

Sub-issues

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

Projects

Status

In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions