feat: extended /v1/models endpoint, now it returns with fallbacks on demand

[murad-khafizov]

Type

🆕 New Feature

Changes

Summary

Enhanced the existing /v1/models endpoint to include complete
fallback information for models while maintaining full backward
compatibility. This enables clients to discover all available
fallbacks for each model based on the router's fallback
configuration.

Key Features

• New Query Parameters:
  • include_metadata: bool = False - Include additional metadata
    in response
  • fallback_type: Optional[str] = None - Specify fallback type
    (defaults to "general")
• Complete Fallback Lists: Returns all configured fallbacks for
  each model, not just the next one
• Three Fallback Types: Supports general, context_window,
  and content_policy fallbacks
• Backward Compatible: Existing /v1/models calls work
  unchanged
• Generic Metadata Structure: Extensible for future
  enhancements

Implementation Details

Core Function (litellm/proxy/auth/model_checks.py)

def get_all_fallbacks(
    model: str,
    llm_router: Optional[Router] = None,
    fallback_type: str = "general",
) -> List[str]:
- Leverages existing get_fallback_model_group() function
- Handles all three fallback types from router configuration
- Returns complete fallback list for specified model
- Graceful error handling with empty list fallback

Enhanced Endpoint (litellm/proxy/proxy_server.py)

- Added new optional query parameters to model_list() function
- Defaults to "general" fallback type when include_metadata=true
- Proper parameter validation with HTTP 400 for invalid types
- Maintains OpenAI-compatible response format

API Usage Examples

# Standard response (backward compatible)
GET /models

# Include metadata with general fallbacks (default)
GET /models?include_metadata=true

# Include specific fallback types
GET /models?include_metadata=true&fallback_type=context_window
GET /models?include_metadata=true&fallback_type=content_policy

Response Format

Standard Response (unchanged):
{
  "data": [
    {
      "id": "claude-4-sonnet",
      "object": "model",
      "created": 1234567890,
      "owned_by": "openai"
    }
  ],
  "object": "list"
}

With Fallback Metadata:
{
  "data": [
    {
      "id": "claude-4-sonnet",
      "object": "model",
      "created": 1234567890,
      "owned_by": "openai",
      "metadata": {
        "fallbacks": ["bedrock-claude-sonnet-4",
"google-claude-sonnet-4"]
      }
    }
  ],
  "object": "list"
}

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 21, 2025 0:20am

[krrishdholakia]

Nice feature! @murad-khafizov

[krrishdholakia]

would be great to document this somewhere. Maybe under model access? https://docs.litellm.ai/docs/proxy/model_access

open to your thoughts on this.