Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 127 additions & 0 deletions src/content/docs/agents/api-reference/agents-api.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -933,6 +933,37 @@ const { messages } = useAgentChat({

### Chat Agent React API

#### filesToParts

Helper function to convert File objects to FileUIPart format for use with `sendMessage`.

```ts
import { filesToParts } from "agents/ai-react";
import type { FileUIPart } from "ai";

// Convert File[] to FileUIPart[] for use with sendMessage
async function filesToParts(files: File[]): Promise<FileUIPart[]>;
```

This utility is useful when you have an array of Files from component state. If you have a FileList directly from an input element, use the AI SDK's `convertFileListToFileUIParts` instead.

<TypeScriptExample>

```tsx
import { filesToParts } from "agents/ai-react";

// Convert files from state to parts for sending
const fileArray = [file1, file2];
const fileParts = await filesToParts(fileArray);

await sendMessage({
text: "Please analyze these files",
files: fileParts,
});
```

</TypeScriptExample>

#### useAgentChat

React hook for building AI chat interfaces using an Agent.
Expand Down Expand Up @@ -1005,6 +1036,11 @@ function useAgentChat(options: UseAgentChatOptions): {
}) => void;
// Current error if any
error: Error | undefined;
// Send a message with optional file attachments
sendMessage: (options: {
text: string;
files?: FileUIPart[];
}) => Promise<void>;
};
```

Expand Down Expand Up @@ -1074,6 +1110,97 @@ function ChatInterface() {

</TypeScriptExample>

#### Sending messages with file attachments

You can send files (images, documents) along with text messages using the `sendMessage` method and `filesToParts` helper:

<TypeScriptExample>

```tsx
import { useAgentChat, filesToParts } from "agents/ai-react";
import { useAgent } from "agents/react";
import { useState, useRef } from "react";

function ChatWithFiles() {
const [files, setFiles] = useState<File[]>([]);
const [input, setInput] = useState("");
const fileInputRef = useRef<HTMLInputElement>(null);

const agent = useAgent({
agent: "chat-agent",
name: "user-123",
});

const { messages, sendMessage, status } = useAgentChat({
agent,
});

const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault();
if (!input.trim() && files.length === 0) return;

const message = input;
const fileArray = files;

// Clear input and files
setInput("");
setFiles([]);

// Send message with files
await sendMessage({
text: message || "Please analyze these files",
files: fileArray.length > 0 ? await filesToParts(fileArray) : undefined,
});
};

const handleFileChange = (e: React.ChangeEvent<HTMLInputElement>) => {
if (e.target.files) {
setFiles(Array.from(e.target.files));
}
};

return (
<div>
<div>
{messages.map((message, i) => (
<div key={i}>
<strong>{message.role}:</strong>
{message.parts
.filter((part) => part.type === "text")
.map((part, j) => (
<span key={j}>{(part as any).text}</span>
))}
</div>
))}
</div>

<form onSubmit={handleSubmit}>
<input
type="file"
ref={fileInputRef}
onChange={handleFileChange}
multiple
accept="image/*"
/>

<input
type="text"
value={input}
onChange={(e) => setInput(e.target.value)}
placeholder="Type a message..."
/>

<button type="submit" disabled={status === "streaming"}>
Send
</button>
</form>
</div>
);
}
```

</TypeScriptExample>

### Next steps

- [Build a chat Agent](/agents/getting-started/build-a-chat-agent/) using the Agents SDK and deploy it to Workers.
Expand Down
92 changes: 92 additions & 0 deletions src/content/docs/agents/api-reference/using-ai-models.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,98 @@ export class MyAgent extends Agent<Env> {

</TypeScriptExample>

#### File attachments in AI chat

When building AI chat applications with `AIChatAgent`, you can send file attachments (such as images or documents) along with text messages. This is useful for vision-capable models or when you need to provide context from documents.

The `filesToParts()` function converts JavaScript `File` objects into the format expected by the AI SDK. You can use this in your React client with the `useAgentChat` hook.

<TypeScriptExample filename="src/client.tsx">

```ts
import { useAgentChat, filesToParts } from "agents/ai-react";
import { useAgent } from "agents/react";
import { useState } from "react";

function ChatComponent() {
const [files, setFiles] = useState<File[]>([]);
const [input, setInput] = useState("");

const agent = useAgent({
agent: "MyAgent",
name: "chat-session",
});

const { messages, sendMessage } = useAgentChat({ agent });

const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault();

await sendMessage({
text: input || "Please describe this image",
files: files.length > 0 ? await filesToParts(files) : undefined,
});

setInput("");
setFiles([]);
};

return (
<form onSubmit={handleSubmit}>
<input
type="file"
multiple
onChange={(e) => setFiles(Array.from(e.target.files || []))}
/>
<textarea
value={input}
onChange={(e) => setInput(e.target.value)}
placeholder="Message..."
/>
<button type="submit">Send</button>
</form>
);
}
```

</TypeScriptExample>

On the server side, `AIChatAgent` automatically handles file attachments. The files are included in the `this.messages` array and can be passed directly to vision-capable models.

<TypeScriptExample filename="src/server.ts">

```ts
import { AIChatAgent } from "agents/ai-chat-agent";
import { openai } from "@ai-sdk/openai";
import {
streamText,
convertToModelMessages,
createUIMessageStream,
createUIMessageStreamResponse,
} from "ai";

export class MyAgent extends AIChatAgent<Env> {
async onChatMessage() {
const stream = createUIMessageStream({
execute: async ({ writer }) => {
const result = streamText({
model: openai("gpt-4o"), // Use a vision-capable model
messages: convertToModelMessages(this.messages),
});

writer.merge(result.toUIMessageStream());
},
});

return createUIMessageStreamResponse({ stream });
}
}
```

</TypeScriptExample>

File attachments are automatically stripped from storage to avoid SQLite size limits. When a user message contains files, the files are sent to the model but only metadata (filename and type) is persisted in the Agent state. This prevents large image data URLs from exceeding storage limits while maintaining conversation context.

### OpenAI compatible endpoints

Agents can call models across any service, including those that support the OpenAI API. For example, you can use the OpenAI SDK to use one of [Google's Gemini models](https://ai.google.dev/gemini-api/docs/openai#node.js) directly from your Agent.
Expand Down
Loading