Skip to content
2 changes: 2 additions & 0 deletions docs/api/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@

::: pydantic_ai.providers.vercel.VercelProvider

::: pydantic_ai.providers.cloudflare.CloudflareProvider

::: pydantic_ai.providers.huggingface.HuggingFaceProvider

::: pydantic_ai.providers.moonshotai.MoonshotAIProvider
Expand Down
61 changes: 61 additions & 0 deletions docs/models/openai.md
Original file line number Diff line number Diff line change
Expand Up @@ -441,6 +441,67 @@ agent = Agent(model)
...
```

### Cloudflare AI Gateway

To use [Cloudflare AI Gateway](https://developers.cloudflare.com/ai-gateway/), first set up a gateway in your [Cloudflare dashboard](https://dash.cloudflare.com/?to=/:account/ai/ai-gateway) and obtain your account ID and gateway ID.

!!! note
This provider uses Cloudflare's [unified API endpoint](https://developers.cloudflare.com/ai-gateway/usage/chat-completion/) for routing requests to multiple AI providers. For the full list of supported providers, see [Cloudflare's documentation](https://developers.cloudflare.com/ai-gateway/usage/chat-completion/#supported-providers).

You can set the `CLOUDFLARE_ACCOUNT_ID`, `CLOUDFLARE_GATEWAY_ID`, and optionally `CLOUDFLARE_AI_GATEWAY_AUTH` environment variables and use the `cloudflare:` model name prefix:

```python test="skip - requires actual API keys"
from pydantic_ai import Agent

# Set via environment or in code:
# CLOUDFLARE_ACCOUNT_ID='your-account-id'
# CLOUDFLARE_GATEWAY_ID='your-gateway-id'
# OPENAI_API_KEY='your-openai-api-key'
Comment on lines +455 to +459
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Set via environment or in code:
# CLOUDFLARE_ACCOUNT_ID='your-account-id'
# CLOUDFLARE_GATEWAY_ID='your-gateway-id'
# OPENAI_API_KEY='your-openai-api-key'


agent = Agent('cloudflare:openai/gpt-4o')
...
```

Or use [`CloudflareProvider`][pydantic_ai.providers.cloudflare.CloudflareProvider] directly:

```python
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.cloudflare import CloudflareProvider

model = OpenAIChatModel(
'openai/gpt-4o',
provider=CloudflareProvider(
account_id='your-account-id',
gateway_id='your-gateway-id',
api_key='your-openai-api-key',
),
)
agent = Agent(model)
...
```

For authenticated gateways with stored API keys in Cloudflare's dashboard:

```python
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.cloudflare import CloudflareProvider

model = OpenAIChatModel(
'anthropic/claude-3-5-sonnet',
provider=CloudflareProvider(
account_id='your-account-id',
gateway_id='your-gateway-id',
gateway_auth_token='your-gateway-token',
),
)
agent = Agent(model)
...
```

See [`CloudflareProvider`][pydantic_ai.providers.cloudflare.CloudflareProvider] for additional configuration options including BYOK modes and authenticated gateways.

### Grok (xAI)

Go to [xAI API Console](https://console.x.ai/) and create an API key.
Expand Down
1 change: 1 addition & 0 deletions docs/models/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ In addition, many providers are compatible with the OpenAI API, and can be used
- [Ollama](openai.md#ollama)
- [OpenRouter](openai.md#openrouter)
- [Vercel AI Gateway](openai.md#vercel-ai-gateway)
- [Cloudflare AI Gateway](openai.md#cloudflare-ai-gateway)
- [Perplexity](openai.md#perplexity)
- [Fireworks AI](openai.md#fireworks-ai)
- [Together AI](openai.md#together-ai)
Expand Down
11 changes: 11 additions & 0 deletions pydantic_ai_slim/pydantic_ai/models/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,16 @@
'cerebras:qwen-3-32b',
'cerebras:qwen-3-coder-480b',
'cerebras:qwen-3-235b-a22b-thinking-2507',
'cloudflare:anthropic/claude-3-5-sonnet',
'cloudflare:cohere/command-r-plus',
'cloudflare:deepseek/deepseek-chat',
'cloudflare:google/gemini-2.0-flash',
'cloudflare:groq/llama-3.3-70b-versatile',
'cloudflare:mistral/mistral-large-latest',
'cloudflare:openai/gpt-4o',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are more recent models missing?

'cloudflare:perplexity/llama-3.1-sonar-small-128k-online',
'cloudflare:workers-ai/@cf/meta/llama-3.1-8b-instruct',
'cloudflare:xai/grok-2-1212',
'cohere:c4ai-aya-expanse-32b',
'cohere:c4ai-aya-expanse-8b',
'cohere:command-nightly',
Expand Down Expand Up @@ -675,6 +685,7 @@ def infer_model(model: Model | KnownModelName | str) -> Model: # noqa: C901
'azure',
'deepseek',
'cerebras',
'cloudflare',
'fireworks',
'github',
'grok',
Expand Down
2 changes: 2 additions & 0 deletions pydantic_ai_slim/pydantic_ai/models/openai.py
Original file line number Diff line number Diff line change
Expand Up @@ -272,6 +272,7 @@ def __init__(
'azure',
'deepseek',
'cerebras',
'cloudflare',
'fireworks',
'github',
'grok',
Expand Down Expand Up @@ -329,6 +330,7 @@ def __init__(
'azure',
'deepseek',
'cerebras',
'cloudflare',
'fireworks',
'github',
'grok',
Expand Down
8 changes: 8 additions & 0 deletions pydantic_ai_slim/pydantic_ai/profiles/perplexity.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
from __future__ import annotations as _annotations

from . import ModelProfile


def perplexity_model_profile(model_name: str) -> ModelProfile | None:
"""Get the model profile for a Perplexity model."""
return None
4 changes: 4 additions & 0 deletions pydantic_ai_slim/pydantic_ai/providers/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,10 @@ def infer_provider_class(provider: str) -> type[Provider[Any]]: # noqa: C901
from .vercel import VercelProvider

return VercelProvider
elif provider == 'cloudflare':
from .cloudflare import CloudflareProvider

return CloudflareProvider
elif provider == 'azure':
from .azure import AzureProvider

Expand Down
51 changes: 30 additions & 21 deletions pydantic_ai_slim/pydantic_ai/providers/cerebras.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,35 @@
) from _import_error


def cerebras_provider_model_profile(model_name: str) -> ModelProfile | None:
"""Get the model profile for a model routed through Cerebras provider.

This function handles model profiling for models that use Cerebras's API,
and applies Cerebras-specific settings like unsupported model parameters.
"""
prefix_to_profile = {'llama': meta_model_profile, 'qwen': qwen_model_profile, 'gpt-oss': harmony_model_profile}

profile = None
for prefix, profile_func in prefix_to_profile.items():
model_name = model_name.lower()
if model_name.startswith(prefix):
profile = profile_func(model_name)

# According to https://inference-docs.cerebras.ai/resources/openai#currently-unsupported-openai-features,
# Cerebras doesn't support some model settings.
unsupported_model_settings = (
'frequency_penalty',
'logit_bias',
'presence_penalty',
'parallel_tool_calls',
'service_tier',
)
return OpenAIModelProfile(
json_schema_transformer=OpenAIJsonSchemaTransformer,
openai_unsupported_model_settings=unsupported_model_settings,
).update(profile)


class CerebrasProvider(Provider[AsyncOpenAI]):
"""Provider for Cerebras API."""

Expand All @@ -39,27 +68,7 @@ def client(self) -> AsyncOpenAI:
return self._client

def model_profile(self, model_name: str) -> ModelProfile | None:
prefix_to_profile = {'llama': meta_model_profile, 'qwen': qwen_model_profile, 'gpt-oss': harmony_model_profile}

profile = None
for prefix, profile_func in prefix_to_profile.items():
model_name = model_name.lower()
if model_name.startswith(prefix):
profile = profile_func(model_name)

# According to https://inference-docs.cerebras.ai/resources/openai#currently-unsupported-openai-features,
# Cerebras doesn't support some model settings.
unsupported_model_settings = (
'frequency_penalty',
'logit_bias',
'presence_penalty',
'parallel_tool_calls',
'service_tier',
)
return OpenAIModelProfile(
json_schema_transformer=OpenAIJsonSchemaTransformer,
openai_unsupported_model_settings=unsupported_model_settings,
).update(profile)
return cerebras_provider_model_profile(model_name)

@overload
def __init__(self) -> None: ...
Expand Down
Loading