We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 61a0bcb commit 6b3ae28Copy full SHA for 6b3ae28
hub/public-endpoint-reference.mdx
@@ -220,14 +220,10 @@ response = client.chat.completions.create(
220
model=model_name,
221
messages=messages,
222
max_tokens=525,
223
- stream_options={"include_usage": True},
224
stream=True
225
)
226
```
227
228
-<Tip>
229
-`stream_options={"include_usage": True}` is required for streaming to work with vLLM Public Endpoints.
230
-</Tip>
231
</Accordion>
232
233
<Accordion title="Response format">
0 commit comments