Skip to content

Add sambanova provider #2313

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
luisfucros wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Add sambanova provider #2313

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c3e48f6
add sambanova provider
snova-luiss Apr 13, 2026
21901fc
update file
luisfucros Apr 14, 2026
ba4048f
update file
luisfucros Apr 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions docs/docs/usage-guide/changing_a_model.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ You can give parameters via a configuration file, or from environment variables.
Failing to set the needed keys of a specific model will usually result in litellm not identifying the model type, and failing to utilize it.

### OpenAI like API

To use an OpenAI like API, set the following in your `.secrets.toml` file:

```toml
Expand Down Expand Up @@ -111,7 +112,7 @@ Please note that the `custom_model_max_tokens` setting should be configured in a
Commercial models such as GPT-5, Claude Sonnet, and Gemini have demonstrated robust capabilities in generating structured output for code analysis tasks with large input. In contrast, most open-source models currently available (as of January 2025) face challenges with these complex tasks.

Based on our testing, local open-source models are suitable for experimentation and learning purposes (mainly for the `ask` command), but they are not suitable for production-level code analysis tasks.

Hence, for production workflows and real-world usage, we recommend using commercial models.

### Hugging Face
Expand Down Expand Up @@ -161,6 +162,20 @@ key = ... # your Groq api key

(you can obtain a Groq key from [here](https://console.groq.com/keys))

### SambaNova

To use MiniMax-M2.5 model with SambaNova, for example, set:

```toml
[config] # in configuration.toml
model = "sambanova/MiniMax-M2.5"
fallback_models = ["sambanova/MiniMax-M2.5"]
[sambanova] # in .secrets.toml
key = ... # your SambaNova api key
```

(you can obtain a SambaNova key from [here](https://cloud.sambanova.ai/apis))

### xAI

To use xAI's models with PR-Agent, set:
Expand Down Expand Up @@ -339,7 +354,7 @@ key = "..." # your Codestral api key
To use model from Openrouter, for example, set:

```toml
[config] # in configuration.toml
[config] # in configuration.toml
model="openrouter/anthropic/claude-3.7-sonnet"
fallback_models=["openrouter/deepseek/deepseek-chat"]
custom_model_max_tokens=20000
Expand Down Expand Up @@ -372,7 +387,7 @@ custom_model_max_tokens= ...
3. Go to [litellm documentation](https://litellm.vercel.app/docs/proxy/quick_start#supported-llms), find the model you want to use, and set the relevant environment variables.

4. Most reasoning models do not support chat-style inputs (`system` and `user` messages) or temperature settings.
To bypass chat templates and temperature controls, set `config.custom_reasoning_model = true` in your configuration file.
To bypass chat templates and temperature controls, set `config.custom_reasoning_model = true` in your configuration file.

## Dedicated parameters

Expand Down
4 changes: 4 additions & 0 deletions pr_agent/algo/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,10 @@
'groq/meta-llama/llama-4-scout-17b-16e-instruct': 131072,
'groq/llama-3.3-70b-versatile': 128000,
'groq/llama-3.1-8b-instant': 128000,
'sambanova/MiniMax-M2.5': 128000,
'sambanova/Meta-Llama-3.3-70B-Instruct': 128000,
'sambanova/gpt-oss-120b': 128000,
'sambanova/DeepSeek-V3.1': 128000,
'xai/grok-2': 131072,
'xai/grok-2-1212': 131072,
'xai/grok-2-latest': 131072,
Expand Down
2 changes: 2 additions & 0 deletions pr_agent/algo/ai_handlers/litellm_ai_handler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ def __init__(self):
litellm.cohere_key = get_settings().cohere.key
if get_settings().get("GROQ.KEY", None):
litellm.api_key = get_settings().groq.key
if get_settings().get("SAMBANOVA.KEY", None):
litellm.api_key = get_settings().sambanova.key
Comment on lines +74 to +75
Copy link
Copy Markdown
Contributor

@qodo-free-for-open-source-projects qodo-free-for-open-source-projects bot Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

3. Wrong provider key injected 🐞 Bug ☼ Reliability

LiteLLMAIHandler.__init__ now overwrites litellm.api_key with the SambaNova key, and
chat_completion injects this single global key into every LiteLLM call, so mixed-provider
fallback_models can be invoked with the wrong API key and fail authentication.
Agent Prompt 
### Issue description
`LiteLLMAIHandler` uses `litellm.api_key` as a single global, but PR-Agent can call multiple providers via `fallback_models`. With the new SambaNova assignment, whichever provider runs last in `__init__` wins, and `chat_completion` forwards that key for *all* models.

### Issue Context
- `__init__` sets `litellm.api_key` for multiple providers (Groq, SambaNova, xAI, Ollama, OpenRouter, Azure AD, etc.).
- `chat_completion` injects `kwargs["api_key"] = litellm.api_key` for every call.
- `retry_with_fallback_models()` may invoke different providers in a single run.

### Fix Focus Areas
- pr_agent/algo/ai_handlers/litellm_ai_handler.py[68-90]
- pr_agent/algo/ai_handlers/litellm_ai_handler.py[412-416]
- pr_agent/algo/pr_processing.py[320-354]

### Suggested approach
1. Stop relying on a single global `litellm.api_key` for multiple providers.
2. In `chat_completion`, derive provider from `model` (e.g., prefix before `/`) and set `kwargs["api_key"]` from the corresponding settings (e.g., `get_settings().groq.key`, `get_settings().sambanova.key`, `get_settings().xai.key`, etc.).
3. Only inject `api_key` when you have a provider-specific match; otherwise let LiteLLM handle OpenAI/Azure/Anthropic keys via their dedicated fields/env vars.
4. Add/extend a unit test covering Groq + SambaNova configured simultaneously, then calling both a `groq/...` and `sambanova/...` model and verifying the forwarded `api_key` differs appropriately.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

if get_settings().get("REPLICATE.KEY", None):
litellm.replicate_key = get_settings().replicate.key
if get_settings().get("XAI.KEY", None):
Expand Down
3 changes: 3 additions & 0 deletions pr_agent/settings/.secrets_template.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,9 @@ key = "" # Optional, uncomment if you want to use Replicate. Acquire through htt
[groq]
key = "" # Acquire through https://console.groq.com/keys

[sambanova]
key = "" # Acquire through https://cloud.sambanova.ai/apis

[xai]
key = "" # Optional, uncomment if you want to use xAI. Acquire through https://console.x.ai/

Expand Down