Skip to content

Modernized and streamlined default LLMs #1151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

Modernized and streamlined default LLMs #1151

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
f434228
Removed extra model defaults sprinkled throughout the repo
jamesbraza Oct 20, 2025
4d81c31
Modernized LLMs inside settings
jamesbraza Oct 20, 2025
8310d60
Refreshing cassettes as needed
jamesbraza Oct 20, 2025
f9bc11a
Loosened test_images_corrupt to allow for Anthropic
jamesbraza Oct 27, 2025
5a8e421
Updated default temperature to 1 for GPT-5
jamesbraza Oct 27, 2025
dd5c9c7
Updated default models to GPT-5
jamesbraza Oct 27, 2025
8daa4f9
Backdated test_pre_prompt to use gpt-4o since gpt-5 is too smart for …
jamesbraza Oct 27, 2025
c672891
Refreshing cassettes as needed
jamesbraza Oct 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -451,7 +451,7 @@ asyncio.run(main())

### Choosing Model

By default, PaperQA2 uses OpenAI's `gpt-4o-2024-11-20` model for the
By default, PaperQA2 uses OpenAI's `gpt-5-2025-08-07` model for the
`summary_llm`, `llm`, and `agent_llm`.
Please see the [Settings Cheatsheet](#settings-cheatsheet)
for more information on these settings.
Expand Down Expand Up @@ -895,13 +895,13 @@ will return much faster than the first query and we'll be certain the authors ma

| Setting | Default | Description |
| -------------------------------------------- | -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| `llm` | `"gpt-4o-2024-11-20"` | LLM for general use including metadata inference (see Docs.aadd) and answer generation (see Docs.aquery and gen_answer tool). |
| `llm` | `"gpt-5-2025-08-07"` | LLM for general use including metadata inference (see Docs.aadd) and answer generation (see Docs.aquery and gen_answer tool). |
| `llm_config` | `None` | Optional configuration for `llm`. |
| `summary_llm` | `"gpt-4o-2024-11-20"` | LLM for creating contextual summaries (see Docs.aget_evidence and gather_evidence tool). |
| `summary_llm` | `"gpt-5-2025-08-07"` | LLM for creating contextual summaries (see Docs.aget_evidence and gather_evidence tool). |
| `summary_llm_config` | `None` | Optional configuration for `summary_llm`. |
| `embedding` | `"text-embedding-3-small"` | Embedding model for embedding text chunks when adding papers. |
| `embedding_config` | `None` | Optional configuration for `embedding`. |
| `temperature` | `0.0` | Temperature for LLMs. |
| `temperature` | `1.0` | Temperature for LLMs. |
| `batch_size` | `1` | Batch size for calling LLMs. |
| `texts_index_mmr_lambda` | `1.0` | Lambda for MMR in text index. |
| `verbosity` | `0` | Integer verbosity level for logging (0-3). 3 = all LLM/Embeddings calls logged. |
Expand Down Expand Up @@ -944,7 +944,7 @@ will return much faster than the first query and we'll be certain the authors ma
| `prompt.summary_json_system` | `summary_json_system_prompt` | System prompt for JSON summaries. |
| `prompt.context_outer` | `CONTEXT_OUTER_PROMPT` | Prompt for how to format all contexts in generate answer. |
| `prompt.context_inner` | `CONTEXT_INNER_PROMPT` | Prompt for how to format a single context in generate answer. Must contain 'name' and 'text' variables. |
| `agent.agent_llm` | `"gpt-4o-2024-11-20"` | LLM inside the agent making tool selections. |
| `agent.agent_llm` | `"gpt-5-2025-08-07"` | LLM inside the agent making tool selections. |
| `agent.agent_llm_config` | `None` | Optional configuration for `agent_llm`. |
| `agent.agent_type` | `"ToolSelector"` | Type of agent to use. |
| `agent.agent_config` | `None` | Optional kwarg for AGENT constructor. |
Expand Down
2 changes: 1 addition & 1 deletion src/paperqa/agents/helpers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ def get_year(ts: datetime | None = None) -> str:

async def litellm_get_search_query(
question: str,
llm: LLMModel | str,
count: int,
template: str | None = None,
llm: LLMModel | str = "gpt-4o-mini",
temperature: float = 1.0,
) -> list[str]:
search_prompt = ""
Expand Down
2 changes: 1 addition & 1 deletion src/paperqa/agents/models.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ def strip_answer(
v.filter_content_for_user()
return v

async def get_summary(self, llm_model: LLMModel | str = "gpt-4o") -> str:
async def get_summary(self, llm_model: LLMModel | str) -> str:
sys_prompt = (
"Revise the answer to a question to be a concise SMS message. "
"Use abbreviations or emojis if necessary."
Expand Down
18 changes: 8 additions & 10 deletions src/paperqa/settings.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,7 @@

import anyio
from aviary.core import Tool, ToolSelector
from lmi import (
CommonLLMNames,
EmbeddingModel,
LiteLLMModel,
embedding_model_factory,
)
from lmi import CommonLLMNames, EmbeddingModel, LiteLLMModel, embedding_model_factory
from pydantic import (
BaseModel,
ConfigDict,
Expand Down Expand Up @@ -525,7 +520,7 @@ class AgentSettings(BaseModel):
model_config = ConfigDict(extra="forbid")

agent_llm: str = Field(
default=CommonLLMNames.GPT_4O.value,
default=CommonLLMNames.GPT_5.value,
description="LLM inside the agent making tool selections.",
)

Expand Down Expand Up @@ -693,7 +688,7 @@ class Settings(BaseSettings):
model_config = SettingsConfigDict(extra="ignore")

llm: str = Field(
default=CommonLLMNames.GPT_4O.value,
default=CommonLLMNames.GPT_5.value,
description=(
"LLM for general use including metadata inference (see Docs.aadd)"
" and answer generation (see Docs.aquery and gen_answer tool)."
Expand All @@ -717,7 +712,7 @@ class Settings(BaseSettings):
),
)
summary_llm: str = Field(
default=CommonLLMNames.GPT_4O.value,
default=CommonLLMNames.GPT_5.value,
description=(
"LLM for creating contextual summaries"
" (see Docs.aget_evidence and gather_evidence tool)."
Expand All @@ -741,7 +736,10 @@ class Settings(BaseSettings):
default=None,
description="Optional configuration for the embedding model.",
)
temperature: float = Field(default=0.0, description="Temperature for LLMs.")
temperature: float = Field(
default=1.0,
description="Temperature for LLMs, default is 1 for compatibility with OpenAI's GPT-5.",
)
batch_size: int = Field(default=1, description="Batch size for calling LLMs.")
texts_index_mmr_lambda: float = Field(
default=1.0, description="Lambda for MMR in text index."
Expand Down
Loading
Loading