Skip to content

Commit 1b90b4b

Browse files
chenmoneygithubchenmoneygithub
authored
add guide on adapters (#8671)
1 parent e8c0df7 commit 1b90b4b

File tree

2 files changed

+308
-0
lines changed

2 files changed

+308
-0
lines changed

‎docs/docs/learn/programming/adapters.md‎

Lines changed: 307 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,307 @@
1+
# Understanding DSPy Adapters
2+
3+
## What are Adapters?
4+
5+
Adapters are the bridge between `dspy.Predict` and the actual Language Model (LM). When you call a DSPy module, the
6+
adapter takes your signature, user inputs, and other attributes like `demos` (few-shot examples) and converts them
7+
into multi-turn messages that get sent to the LM.
8+
9+
The adapter system is responsible for:
10+
11+
- Translating DSPy signatures into system messages that define the task and request/response structure.
12+
- Formatting input data according to the request structure outlined in DSPy signatures.
13+
- Parsing LM responses back into structured DSPy outputs, such as `dspy.Prediction` instances.
14+
- Managing conversation history and function calls.
15+
- Converting pre-built DSPy types into LM prompt messages, e.g., `dspy.Tool`, `dspy.Image`, etc.
16+
17+
## Configure Adapters
18+
19+
You can use `dspy.configure(adapter=...)` to choose the adapter for the entire Python process, or
20+
`with dspy.context(adapter=...):` to only affect a certain namespace.
21+
22+
If no adapter is specified in the DSPy workflow, each `dspy.Predict.__call__` defaults to using the `dspy.ChatAdapter`. Thus, the two code snippets below are equivalent:
23+
24+
```python
25+
import dspy
26+
27+
dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))
28+
29+
predict = dspy.Predict("question -> answer")
30+
result = predict(question="What is the capital of France?")
31+
```
32+
33+
```python
34+
import dspy
35+
36+
dspy.configure(
37+
lm=dspy.LM("openai/gpt-4o-mini"),
38+
adapter=dspy.ChatAdapter(), # This is the default value
39+
)
40+
41+
predict = dspy.Predict("question -> answer")
42+
result = predict(question="What is the capital of France?")
43+
```
44+
45+
## Where Adapters Fit in the System
46+
47+
The flow works as follows:
48+
49+
1. The user calls their DSPy agent, typically a `dspy.Module` with inputs.
50+
2. The inner `dspy.Predict` is invoked to obtain the LM response.
51+
3. `dspy.Predict` calls **Adapter.format()**, which converts its signature, inputs, and demos into multi-turn messages sent to the `dspy.LM`. `dspy.LM` is a thin wrapper around `litellm`, which communicates with the LM endpoint.
52+
4. The LM receives the messages and generates a response.
53+
5. **Adapter.parse()** converts the LM response into structured DSPy outputs, as specified in the signature.
54+
6. The caller of `dspy.Predict` receives the parsed outputs.
55+
56+
You can explicitly call `Adapter.format()` to view the messages sent to the LM.
57+
58+
```python
59+
# Simplified flow example
60+
signature = dspy.Signature("question -> answer")
61+
inputs = {"question": "What is 2+2?"}
62+
demos = [{"question": "What is 1+1?", "answer": "2"}]
63+
64+
adapter = dspy.ChatAdapter()
65+
print(adapter.format(signature, demos, inputs))
66+
```
67+
68+
The output should resemble:
69+
70+
```
71+
{'role': 'system', 'content': 'Your input fields are:\n1. `question` (str):\nYour output fields are:\n1. `answer` (str):\nAll interactions will be structured in the following way, with the appropriate values filled in.\n\n[[ ## question ## ]]\n{question}\n\n[[ ## answer ## ]]\n{answer}\n\n[[ ## completed ## ]]\nIn adhering to this structure, your objective is: \n Given the fields `question`, produce the fields `answer`.'}
72+
{'role': 'user', 'content': '[[ ## question ## ]]\nWhat is 1+1?'}
73+
{'role': 'assistant', 'content': '[[ ## answer ## ]]\n2\n\n[[ ## completed ## ]]\n'}
74+
{'role': 'user', 'content': '[[ ## question ## ]]\nWhat is 2+2?\n\nRespond with the corresponding output fields, starting with the field `[[ ## answer ## ]]`, and then ending with the marker for `[[ ## completed ## ]]`.'}
75+
```
76+
77+
## Types of Adapters
78+
79+
DSPy offers several adapter types, each tailored for specific use cases:
80+
81+
### ChatAdapter
82+
83+
**ChatAdapter** is the default adapter and works with all language models. It uses a field-based format with special markers.
84+
85+
#### Format Structure
86+
87+
ChatAdapter uses `[[ ## field_name ## ]]` markers to delineate fields. For fields of non-primitive Python types, it includes the JSON schema of the type. Below, we use `dspy.inspect_history()` to display the formatted messages by `dspy.ChatAdapter` clearly.
88+
89+
```python
90+
import dspy
91+
import pydantic
92+
93+
dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"), adapter=dspy.ChatAdapter())
94+
95+
96+
class ScienceNews(pydantic.BaseModel):
97+
text: str
98+
scientists_involved: list[str]
99+
100+
101+
class NewsQA(dspy.Signature):
102+
"""Get news about the given science field"""
103+
104+
science_field: str = dspy.InputField()
105+
year: int = dspy.InputField()
106+
num_of_outputs: int = dspy.InputField()
107+
news: list[ScienceNews] = dspy.OutputField(desc="science news")
108+
109+
110+
predict = dspy.Predict(NewsQA)
111+
predict(science_field="Computer Theory", year=2022, num_of_outputs=1)
112+
dspy.inspect_history()
113+
```
114+
115+
The output looks like:
116+
117+
```
118+
[2025-08-15T22:24:29.378666]
119+
120+
System message:
121+
122+
Your input fields are:
123+
1. `science_field` (str):
124+
2. `year` (int):
125+
3. `num_of_outputs` (int):
126+
Your output fields are:
127+
1. `news` (list[ScienceNews]): science news
128+
All interactions will be structured in the following way, with the appropriate values filled in.
129+
130+
[[ ## science_field ## ]]
131+
{science_field}
132+
133+
[[ ## year ## ]]
134+
{year}
135+
136+
[[ ## num_of_outputs ## ]]
137+
{num_of_outputs}
138+
139+
[[ ## news ## ]]
140+
{news} # note: the value you produce must adhere to the JSON schema: {"type": "array", "$defs": {"ScienceNews": {"type": "object", "properties": {"scientists_involved": {"type": "array", "items": {"type": "string"}, "title": "Scientists Involved"}, "text": {"type": "string", "title": "Text"}}, "required": ["text", "scientists_involved"], "title": "ScienceNews"}}, "items": {"$ref": "#/$defs/ScienceNews"}}
141+
142+
[[ ## completed ## ]]
143+
In adhering to this structure, your objective is:
144+
Get news about the given science field
145+
146+
147+
User message:
148+
149+
[[ ## science_field ## ]]
150+
Computer Theory
151+
152+
[[ ## year ## ]]
153+
2022
154+
155+
[[ ## num_of_outputs ## ]]
156+
1
157+
158+
Respond with the corresponding output fields, starting with the field `[[ ## news ## ]]` (must be formatted as a valid Python list[ScienceNews]), and then ending with the marker for `[[ ## completed ## ]]`.
159+
160+
161+
Response:
162+
163+
[[ ## news ## ]]
164+
[
165+
{
166+
"scientists_involved": ["John Doe", "Jane Smith"],
167+
"text": "In 2022, researchers made significant advancements in quantum computing algorithms, demonstrating their potential to solve complex problems faster than classical computers. This breakthrough could revolutionize fields such as cryptography and optimization."
168+
}
169+
]
170+
171+
[[ ## completed ## ]]
172+
```
173+
174+
!!! info "Practice: locate Signature information in the printed LM history"
175+
176+
Try adjusting the signature, and observe how the changes are reflected in the printed LM message.
177+
178+
179+
Each field is preceded by a marker `[[ ## field_name ## ]]`. If an output field has non-primitive types, the instruction includes the type's JSON schema, and the output is formatted accordingly. Because the output field is structured as defined by ChatAdapter, it can be automatically parsed into structured data.
180+
181+
#### When to Use ChatAdapter
182+
183+
`ChatAdapter` offers the following advantages:
184+
185+
- **Universal compatibility**: Works with all language models, though smaller models may generate responses that do not match the required format.
186+
- **Fallback protection**: If `ChatAdapter` fails, it automatically retries with `JSONAdapter`.
187+
188+
In general, `ChatAdapter` is a reliable choice if you don't have specific requirements.
189+
190+
#### When Not to Use ChatAdapter
191+
192+
Avoid using `ChatAdapter` if you are:
193+
194+
- **Latency sensitive**: `ChatAdapter` includes more boilerplate output tokens compared to other adapters, so if you're building a system sensitive to latency, consider using a different adapter.
195+
196+
### JSONAdapter
197+
198+
**JSONAdapter** prompts the LM to return JSON data containing all output fields as specified in the signature. It is effective for models that support structured output via the `response_format` parameter, leveraging native JSON generation capabilities for more reliable parsing.
199+
200+
#### Format Structure
201+
202+
The input part of the prompt formatted by `JSONAdapter` is similar to `ChatAdapter`, but the output part differs, as shown below:
203+
204+
```python
205+
import dspy
206+
import pydantic
207+
208+
dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"), adapter=dspy.JSONAdapter())
209+
210+
211+
class ScienceNews(pydantic.BaseModel):
212+
text: str
213+
scientists_involved: list[str]
214+
215+
216+
class NewsQA(dspy.Signature):
217+
"""Get news about the given science field"""
218+
219+
science_field: str = dspy.InputField()
220+
year: int = dspy.InputField()
221+
num_of_outputs: int = dspy.InputField()
222+
news: list[ScienceNews] = dspy.OutputField(desc="science news")
223+
224+
225+
predict = dspy.Predict(NewsQA)
226+
predict(science_field="Computer Theory", year=2022, num_of_outputs=1)
227+
dspy.inspect_history()
228+
```
229+
230+
```
231+
System message:
232+
233+
Your input fields are:
234+
1. `science_field` (str):
235+
2. `year` (int):
236+
3. `num_of_outputs` (int):
237+
Your output fields are:
238+
1. `news` (list[ScienceNews]): science news
239+
All interactions will be structured in the following way, with the appropriate values filled in.
240+
241+
Inputs will have the following structure:
242+
243+
[[ ## science_field ## ]]
244+
{science_field}
245+
246+
[[ ## year ## ]]
247+
{year}
248+
249+
[[ ## num_of_outputs ## ]]
250+
{num_of_outputs}
251+
252+
Outputs will be a JSON object with the following fields.
253+
254+
{
255+
"news": "{news} # note: the value you produce must adhere to the JSON schema: {\"type\": \"array\", \"$defs\": {\"ScienceNews\": {\"type\": \"object\", \"properties\": {\"scientists_involved\": {\"type\": \"array\", \"items\": {\"type\": \"string\"}, \"title\": \"Scientists Involved\"}, \"text\": {\"type\": \"string\", \"title\": \"Text\"}}, \"required\": [\"text\", \"scientists_involved\"], \"title\": \"ScienceNews\"}}, \"items\": {\"$ref\": \"#/$defs/ScienceNews\"}}"
256+
}
257+
In adhering to this structure, your objective is:
258+
Get news about the given science field
259+
260+
261+
User message:
262+
263+
[[ ## science_field ## ]]
264+
Computer Theory
265+
266+
[[ ## year ## ]]
267+
2022
268+
269+
[[ ## num_of_outputs ## ]]
270+
1
271+
272+
Respond with a JSON object in the following order of fields: `news` (must be formatted as a valid Python list[ScienceNews]).
273+
274+
275+
Response:
276+
277+
{
278+
"news": [
279+
{
280+
"text": "In 2022, researchers made significant advancements in quantum computing algorithms, demonstrating that quantum systems can outperform classical computers in specific tasks. This breakthrough could revolutionize fields such as cryptography and complex system simulations.",
281+
"scientists_involved": [
282+
"Dr. Alice Smith",
283+
"Dr. Bob Johnson",
284+
"Dr. Carol Lee"
285+
]
286+
}
287+
]
288+
}
289+
```
290+
291+
#### When to Use JSONAdapter
292+
293+
`JSONAdapter` is good at:
294+
295+
- **Structured output support**: When the model supports the `response_format` parameter.
296+
- **Low latency**: Minimal boilerplate in the LM response results in faster responses.
297+
298+
#### When Not to Use JSONAdapter
299+
300+
Avoid using `JSONAdapter` if you are:
301+
302+
- Using a model that does not natively support structured output, such as a small open-source model hosted on Ollama.
303+
304+
## Summary
305+
306+
Adapters are a crucial component of DSPy that bridge the gap between structured DSPy signatures and language model APIs.
307+
Understanding when and how to use different adapters will help you build more reliable and efficient DSPy programs.

‎docs/mkdocs.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ nav:
1717
- Language Models: learn/programming/language_models.md
1818
- Signatures: learn/programming/signatures.md
1919
- Modules: learn/programming/modules.md
20+
- Adapters: learn/programming/adapters.md
2021
- DSPy Evaluation:
2122
- Evaluation Overview: learn/evaluation/overview.md
2223
- Data Handling: learn/evaluation/data.md

0 commit comments

Comments
 (0)