diff --git a/.gitignore b/.gitignore index f679552..282c651 100644 --- a/.gitignore +++ b/.gitignore @@ -51,7 +51,9 @@ coverage .github/skills/ .trae/ .windsurf/ -skills/ +examples/*/skills/ # Private projects (not part of SDK) examples/yourgpt-chatbot/ +examples/yourgpt-server-demo/ +research/ diff --git a/.husky/pre-commit b/.husky/pre-commit index 5c6a0d8..e2fc3f1 100755 --- a/.husky/pre-commit +++ b/.husky/pre-commit @@ -39,4 +39,4 @@ echo "✅ Gitleaks scan passed!" echo "" # Run lint-staged -pnpm exec lint-staged \ No newline at end of file +pnpm exec lint-staged diff --git a/WEB_SEARCH_IMPLEMENTATION.md b/WEB_SEARCH_IMPLEMENTATION.md deleted file mode 100644 index eb27ae8..0000000 --- a/WEB_SEARCH_IMPLEMENTATION.md +++ /dev/null @@ -1,385 +0,0 @@ -# Web Search Implementation - Technical Documentation - -> Temporary documentation for the native web search feature implementation. - ---- - -## Current Implementation Status - -### Completed - -- [x] Native web search for all 3 LLM providers (OpenAI, Google, Anthropic) -- [x] Single API call (was 2 calls before - LLM + search provider) -- [x] Citations displayed as chips below messages (like Perplexity/ChatGPT) -- [x] Tree-shakeable subpath exports (~3KB per provider vs ~50KB for all) -- [x] Unified Citation format across all providers -- [x] HoverCard preview for citations with favicon and domain -- [x] Hidden "Web search" tool step when native citations exist -- [x] Debug logs cleaned up -- [x] Simplified naming (removed "-native" suffix) - ---- - -## Architecture - -### Package Structure - -``` -@yourgpt/copilot-sdk -├── /core # Main exports -├── /react # React hooks -├── /ui # UI components -└── /tools # Tree-shakeable tool exports - ├── /web-search # Shared types + utilities - ├── /openai # openaiSearch() - ├── /google # googleSearch() - ├── /anthropic # anthropicSearch() - ├── /tavily # tavilySearch() - ├── /serper # serperSearch() - ├── /brave # braveSearch() - ├── /exa # exaSearch() - └── /searxng # searxngSearch() -``` - -### Provider Implementation Files - -``` -packages/copilot-sdk/src/core/tools/webSearch/providers/ -├── openai.ts # OpenAI Responses API with web_search tool -├── google.ts # Gemini API with google_search grounding -├── anthropic.ts # Anthropic Messages API with web_search_20250305 -├── tavily.ts # Tavily API -├── serper.ts # Serper (Google) API -├── brave.ts # Brave Search API -├── exa.ts # Exa (semantic) API -└── searxng.ts # Self-hosted SearXNG -``` - -### LLM Adapters with Native Web Search - -``` -packages/llm-sdk/src/adapters/ -├── openai.ts # webSearch config → web_search_preview tool -├── google.ts # webSearch config → google_search grounding -└── anthropic.ts # webSearch config → web_search_20260209 tool -``` - ---- - -## Unified Citation Format - -All providers normalize to this format: - -```typescript -interface Citation { - index: number; // 1-based index - url: string; // Source URL - title: string; // Page title or domain - domain?: string; // Extracted domain (e.g., "example.com") - favicon?: string; // Google favicon URL - citedText?: string; // Relevant excerpt (Anthropic only) -} -``` - -### Stream Event - -```typescript -yield { type: "citation", citations: Citation[] }; -``` - -### Message Metadata - -Citations are stored in message metadata: - -```typescript -message.metadata.citations: Citation[] -``` - ---- - -## Provider-Specific Details - -### OpenAI - -**Tool Type:** `web_search_preview` -**API:** Chat Completions (streaming) -**Citations:** `delta.annotations[]` with `type: "url_citation"` - -```typescript -// Adapter config -webSearch: true | WebSearchConfig; - -// Emits during stream -if (annotation.type === "url_citation") { - collectedCitations.push({ - url: annotation.url_citation.url, - title: annotation.url_citation.title, - }); -} -``` - -### Google (Gemini) - -**Tool Type:** `{ google_search: {} }` -**API:** generateContent (streaming) -**Citations:** `candidate.groundingMetadata.groundingChunks[]` - -```typescript -// Grounding metadata -groundingMetadata: { - groundingChunks: [ - { web: { uri: string, title?: string } } - ] -} -``` - -### Anthropic - -**Tool Type:** `web_search_20260209` (streaming adapter) / `web_search_20250305` (standalone) -**API:** Messages (streaming) -**Citations:** `content[].citations[]` with `type: "web_search_result_location"` - -```typescript -// Citation format -{ - type: "web_search_result_location", - url: string, - title: string, - cited_text?: string, // Unique to Anthropic -} -``` - -**Note:** Anthropic provides `cited_text` - the actual text from the page that was cited. - ---- - -## UI Components - -### SourceGroup (`source.tsx`) - -Displays citations as chips with hover preview. - -```tsx - -``` - -### Source (individual chip) - -```tsx - -``` - -### HoverCard - -Uses `@radix-ui/react-hover-card` for preview on hover. -Animation requires `tw-animate-css` (Tailwind v4) in user's project. - ---- - -## Known Issues & Fixes Applied - -### 1. Citations Lost After Stream Ends - -**Problem:** `useInternalThreadManager` was calling `setMessages()` even without persistence adapter, overwriting metadata. - -**Fix:** Added `!adapter` check: - -```typescript -useEffect(() => { - if (!adapter) return; // Skip sync when no persistence - // ... -}, [adapter, messages]); -``` - -**File:** `packages/copilot-sdk/src/ui/hooks/useInternalThreadManager.ts` - -### 2. Tool Step Showing During Native Search - -**Problem:** "Web search" tool step was showing during streaming for native web search. - -**Fix:** Don't emit `action:start`/`action:end` for `web_search` tool: - -```typescript -if (currentToolUse.name !== "web_search") { - yield { type: "action:start", ... }; -} -``` - -**File:** `packages/llm-sdk/src/adapters/anthropic.ts` - -### 3. Citations Layout - -**Problem:** SourceGroup was rendering to the right of message content. - -**Fix:** Moved SourceGroup inside the content div in `default-message.tsx`. - -### 4. HoverCard Animations - -**Problem:** No transition on hover card. - -**Solution:** Users need to add `tw-animate-css` package (Tailwind v4): - -```bash -pnpm add tw-animate-css -``` - -```css -@import "tailwindcss"; -@import "tw-animate-css"; -``` - ---- - -## Suggestions for Future Improvements - -### 1. Extract Duplicate Utilities - -The `extractDomain` function is duplicated in: - -- `adapters/openai.ts` -- `adapters/google.ts` -- `adapters/anthropic.ts` -- `ui/components/ui/source.tsx` - -**Suggestion:** Create shared `packages/llm-sdk/src/utils/url.ts` - -### 2. Add Anthropic to Documentation Tabs - -The `web-search.mdx` docs are missing Anthropic tab in provider examples. - -### 3. Citation Loading State - -Currently citations appear after stream ends. Consider showing a subtle "Searching..." indicator during streaming. - -### 4. Consolidate Citation Components - -Both `citations.tsx` and `source.tsx` exist. Consider: - -- Deprecating one, or -- Clearly documenting when to use each - -### 5. Error Boundary for Citations - -Add graceful fallback if favicon fails to load (currently just hides). - -### 6. Version Consistency - -Ensure Anthropic web search version is consistent: - -- Adapter: `web_search_20260209` -- Standalone: `web_search_20250305` - -Pick one version and use consistently. - ---- - -## Usage Examples - -### Native Web Search (Recommended) - -```typescript -// In adapter config - single API call -const adapter = createAnthropicAdapter({ - apiKey: process.env.ANTHROPIC_API_KEY, - model: "claude-sonnet-4-20250514", - webSearch: true, // Enable native web search -}); -``` - -### Tree-Shakeable Tool Import - -```typescript -import { openaiSearch } from "@yourgpt/copilot-sdk/tools/openai"; - -const webSearch = openaiSearch({ - apiKey: process.env.OPENAI_API_KEY, - maxResults: 5, -}); - -const runtime = createRuntime({ - provider: openai, - model: "gpt-4o", - tools: [webSearch], -}); -``` - -### Legacy Import (All Providers) - -```typescript -import { createWebSearchTool } from "@yourgpt/copilot-sdk/core"; - -const webSearch = createWebSearchTool({ - provider: "anthropic", - apiKey: process.env.ANTHROPIC_API_KEY, -}); -``` - ---- - -## Bundle Size - -| Import Pattern | Size | -| -------------------------------------- | ------ | -| `@yourgpt/copilot-sdk/tools/openai` | ~2.5KB | -| `@yourgpt/copilot-sdk/tools/google` | ~2.5KB | -| `@yourgpt/copilot-sdk/tools/anthropic` | ~3KB | -| `@yourgpt/copilot-sdk/tools/tavily` | ~3KB | -| `@yourgpt/copilot-sdk/core` (all) | ~50KB | - -**~85% reduction** when using single provider import. - ---- - -## Testing - -### Demo App - -```bash -cd examples/web-search-demo -pnpm dev -# Open http://localhost:3009 -``` - -### Test Queries - -- "What are the latest AI news?" -- "What's the weather in New York?" -- "Who won the most recent Super Bowl?" -- "What's the current price of Bitcoin?" - ---- - -## Files Modified in This Feature - -### New Files - -- `packages/copilot-sdk/src/tools/*/index.ts` (8 tool exports) -- `packages/copilot-sdk/src/core/tools/webSearch/providers/*.ts` (8 providers) -- `packages/copilot-sdk/src/ui/components/ui/source.tsx` -- `packages/copilot-sdk/src/ui/components/ui/citations.tsx` -- `examples/web-search-demo/` (entire demo app) -- `apps/docs/content/docs/tools/built-in/web-search.mdx` - -### Modified Files - -- `packages/llm-sdk/src/adapters/openai.ts` (webSearch support) -- `packages/llm-sdk/src/adapters/google.ts` (webSearch support) -- `packages/llm-sdk/src/adapters/anthropic.ts` (webSearch support) -- `packages/copilot-sdk/src/ui/components/composed/chat/default-message.tsx` -- `packages/copilot-sdk/src/ui/hooks/useInternalThreadManager.ts` -- `packages/copilot-sdk/package.json` (subpath exports) -- `packages/copilot-sdk/tsup.config.ts` (entry points) - ---- - -_Last updated: 2026-02-23_ diff --git a/apps/docs/alpha-docs/BRANCHING.md b/apps/docs/alpha-docs/BRANCHING.md new file mode 100644 index 0000000..efd266a --- /dev/null +++ b/apps/docs/alpha-docs/BRANCHING.md @@ -0,0 +1,450 @@ +# Conversation Branching + +> Branch `feat/branching` — implements the same UX pattern as ChatGPT, Claude.ai, and Gemini: +> editing a user message creates a parallel conversation path, preserving the original, +> with `← N/M →` navigation between variants. + +--- + +## Table of Contents + +1. [Live Demo](#live-demo) +2. [What Was Built](#what-was-built) +3. [Breaking Changes](#breaking-changes) +4. [New APIs](#new-apis) +5. [Database / Persistence Changes](#database--persistence-changes) +6. [User Adoption](#user-adoption) +7. [Framework-Agnostic Usage](#framework-agnostic-usage) +8. [How It Works Internally](#how-it-works-internally) + +--- + +## Live Demo + +A full working demo is in the **experimental** examples project. + +**Location:** `examples/experimental/` +**Route:** `/branching` + +```bash +cd examples/experimental +pnpm dev +# → http://localhost:3000/branching +``` + +### What the demo shows + +Two-panel layout inside a single `CopilotProvider`: + +``` +┌─────────────────────────────────────────────────────────────┐ +│ ← Back Conversation Branching Demo [feat/branching] │ +├──────────────────────────┬──────────────────────────────────┤ +│ Branch Tree │ CopilotChat │ +│ │ │ +│ Branch Tree │ [user: Hello] ← 1/2 → │ +│ 4 total · 3 visible │ [assistant: Hi there] │ +│ branched ✦ │ │ +│ │ [user: Tell me more] ✏ │ +│ ● U Hello │ [assistant: Sure…] │ +│ ├── ● A Hi there ×2 │ │ +│ └── · A Hey │ ────────────────────────────── │ +│ └── ● U Tell me… │ [input field] │ +│ └── ● A Sure… │ │ +└──────────────────────────┴──────────────────────────────────┘ +``` + +- **Left panel** (`BranchTreePanel`) — reads `getAllMessages()` live. Green dot = on active path, grey = inactive branch. `×N` badge = sibling count. Click any node to call `switchBranch()`. +- **Right panel** — standard `CopilotChat`. Edit ✏ button appears on hover over user messages. `← N/M →` navigator appears below user messages when variants exist. + +### Demo source files + +| File | Purpose | +| ---------------------------------------------------------------- | ------------------------------------------ | +| `examples/experimental/app/branching/page.tsx` | Page: `CopilotProvider` + two-panel layout | +| `examples/experimental/components/branching/BranchTreePanel.tsx` | Live tree visualization component | +| `examples/experimental/app/api/chat/branching/route.ts` | Anthropic API route (haiku, short replies) | + +### Key code pattern in the demo + +```tsx +// page.tsx — both panels share one CopilotProvider + + {/* reads getAllMessages(), calls switchBranch() */} + {/* edit button + BranchNavigator built-in */} + + +// BranchTreePanel.tsx — the core hook usage +const { messages, getAllMessages, getBranchInfo, switchBranch, hasBranches } = useCopilot(); +const allMessages = getAllMessages(); // all branches +const visibleIds = new Set(messages.map(m => m.id)); // active path +``` + +--- + +## What Was Built + +### Core Data Layer + +| File | What Changed | +| -------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `src/chat/branching/MessageTree.ts` | **New.** Pure TypeScript tree utility. Bidirectional flat-map: `parentId` + `childrenIds[]` + `activeChildMap`. No React dependency. | +| `src/chat/branching/index.ts` | **New.** Barrel export. | +| `src/chat/types/message.ts` | Added `parentId?: string \| null` and `childrenIds?: string[]` to `UIMessage`. | +| `src/core/types/message.ts` | Added `parent_id?: string \| null` and `children_ids?: string[]` to `Message` (persistence layer). | +| `src/chat/interfaces/ChatState.ts` | Added 5 optional branching methods: `setCurrentLeaf`, `getAllMessages`, `getBranchInfo`, `switchBranch`, `hasBranches`. | +| `src/react/internal/ReactChatState.ts` | Replaced `_messages: T[]` array with `MessageTree`. `messages` getter = visible path only. | +| `src/chat/classes/AbstractChat.ts` | `regenerate()` rewritten to be branch-aware (creates sibling instead of destroying). `sendMessage()` extended with `options.editMessageId`. `onMessagesChange` callback now passes all branches via `_allMessages()`. | +| `src/chat/ChatWithTools.ts` | `sendMessage()` passes through `options.editMessageId`. | + +### React Layer + +| File | What Changed | +| ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------- | +| `src/react/internal/ReactChat.ts` | Added `switchBranch`, `getBranchInfo`, `getAllMessages`, `hasBranches` pass-throughs. | +| `src/react/internal/ReactChatWithTools.ts` | Same pass-throughs. | +| `src/react/internal/useChat.ts` | Added `switchBranch`, `getBranchInfo`, `editMessage`, `hasBranches` to `UseChatReturn`. | +| `src/react/context/CopilotContext.tsx` | Added branching methods to `ChatActions`. | +| `src/react/provider/CopilotProvider.tsx` | Wired branching methods into context. `onMessagesChange` effect uses `getAllMessages()`. Added `getAllMessages` to `CopilotContextValue`. | +| `src/react/index.ts` | Re-exports `MessageTree`, `BranchInfo`. | +| `src/chat/index.ts` | Re-exports `MessageTree`, `BranchInfo`. | + +### UI Layer + +| File | What Changed | +| ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | +| `src/ui/components/ui/branch-navigator.tsx` | **New.** `← N/M →` purely presentational component. | +| `src/ui/components/composed/chat/types.ts` | Added `getBranchInfo`, `onSwitchBranch`, `onEditMessage` to `ChatProps`. | +| `src/ui/components/composed/chat/default-message.tsx` | User messages: pencil edit button on hover, inline textarea edit, `BranchNavigator` shown when siblings exist. | +| `src/ui/components/composed/chat/chat.tsx` | Passes branch props through to each message. | +| `src/ui/components/composed/connected-chat.tsx` | Pulls `switchBranch`, `getBranchInfo`, `editMessage` from `useCopilot()` and passes to ``. | +| `src/ui/hooks/useInternalThreadManager.ts` | Save path uses `getAllMessages()`. Load paths restore `parentId`/`childrenIds`. `convertToCore` includes `parent_id`/`children_ids`. | +| `src/ui/index.ts` | Exports `BranchNavigator`, `BranchNavigatorProps`. | + +--- + +## Breaking Changes + +**None.** + +All new fields and methods are optional. Every existing usage continues to work without modification: + +| Scenario | Behavior | +| --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | +| Messages with no `parentId` | `getVisibleMessages()` falls back to insertion order (legacy linear) | +| `regenerate()` called without arguments | Finds last assistant on visible path — identical to before | +| `sendMessage()` with no third argument | Identical to before | +| `useChat()` / `useCopilot()` consumers | All branching fields available but optional — no existing destructuring breaks | +| `onMessagesChange` callback consumers | Now receives all branches instead of visible path only — **payload size may increase** if branches exist, but shape is identical (`Message[]`) | +| DB rows with no `parent_id` column | Auto-migrated via `fromFlatArray()` on load — no manual migration script needed for existing data | + +> **Note on `onMessagesChange` payload:** If a user has branched the conversation, the callback now receives all messages across all branches (not just the active path). The shape is the same `Message[]` type. If your persistence layer deduplicates by message ID, no change is needed. If it blindly appends, you may want to upsert by ID instead. + +--- + +## New APIs + +### `useCopilot()` / `CopilotProvider` + +```typescript +const { + switchBranch, // (messageId: string) => void + getBranchInfo, // (messageId: string) => BranchInfo | null + editMessage, // (messageId: string, newContent: string) => Promise + hasBranches, // boolean — true if any fork exists + getAllMessages, // () => UIMessage[] — all branches, not just visible path +} = useCopilot(); +``` + +### `useChat()` + +```typescript +const { + switchBranch, // (messageId: string) => void + getBranchInfo, // (messageId: string) => BranchInfo | null + editMessage, // (messageId: string, newContent: string) => Promise + hasBranches, // boolean +} = useChat({ ... }); +``` + +### `` props + +```typescript + BranchInfo | null} + onSwitchBranch={(messageId) => void} + onEditMessage={(messageId, newContent) => void} +/> +``` + +### `BranchInfo` type + +```typescript +interface BranchInfo { + siblingIndex: number; // 0-based — which variant this is + totalSiblings: number; // how many variants exist at this fork + siblingIds: string[]; // ordered oldest-first + hasPrevious: boolean; + hasNext: boolean; +} +``` + +### `BranchNavigator` component (UI primitives) + +```tsx +import { BranchNavigator } from "@yourgpt/copilot-sdk-ui"; + + switchBranch(info.siblingIds[info.siblingIndex - 1])} + onNext={() => switchBranch(info.siblingIds[info.siblingIndex + 1])} +/>; +``` + +### `MessageTree` (framework-agnostic) + +```typescript +import { MessageTree, type BranchInfo } from "@yourgpt/copilot-sdk"; + +const tree = new MessageTree(messages); +tree.getVisibleMessages(); // active path only +tree.getAllMessages(); // all branches +tree.getBranchInfo(messageId); // BranchInfo | null +tree.switchBranch(messageId); +tree.hasBranches; // boolean +``` + +--- + +## Database / Persistence Changes + +### New columns needed + +Two new optional columns on your messages table: + +```sql +ALTER TABLE messages + ADD COLUMN parent_id TEXT REFERENCES messages(id), + ADD COLUMN children_ids JSONB DEFAULT '[]'; +``` + +| Column | Type | Nullable | Description | +| -------------- | ----------------------- | -------- | ------------------------------------------------------------- | +| `parent_id` | `TEXT` / `VARCHAR` | YES | ID of parent message. `NULL` = root. Missing = legacy linear. | +| `children_ids` | `JSON` array of strings | YES | Ordered child IDs for O(1) sibling lookup. | + +> **These columns are optional.** Existing rows without them are auto-migrated to a linear tree on load via `fromFlatArray()`. No data loss. No required migration for existing rows. + +### What gets saved now + +When `onMessagesChange` fires (or the thread manager auto-saves), the payload contains **all messages across all branches**, not just the visible path. Each message carries: + +```json +{ + "id": "msg-abc", + "role": "assistant", + "content": "...", + "parent_id": "msg-xyz", + "children_ids": [] +} +``` + +### What gets loaded + +When a thread is loaded (auto-restore or `switchThread`), the SDK maps: + +``` +DB row.parent_id → UIMessage.parentId +DB row.children_ids → UIMessage.childrenIds +``` + +The `MessageTree` is rebuilt from these fields. The last child at each fork becomes the active path (matches what was active when saved). + +### localStorage (built-in persistence) + +No changes needed. The SDK's `localStorageAdapter` serializes the full `Thread` object including messages. The new fields are automatically included when present. + +### Server persistence (`serverAdapter`) + +Your API endpoints that receive `PUT /threads/:id` payloads will now see `parent_id` and `children_ids` on each message object. Store them as-is. If your schema doesn't have these columns yet, the fields are simply ignored — no error. + +### Upsert strategy (recommended) + +Since branched conversations can have multiple messages with the same `parent_id`, always **upsert by message ID** rather than replacing the array: + +```typescript +// ✅ Safe for branching +await db.messages.upsert({ id: msg.id, ...msg }); + +// ⚠️ Loses inactive branches +await db.threads.update({ messages: visibleMessages }); +``` + +--- + +## User Adoption + +### Zero-config (CopilotChat users) + +If you use ``, branching is **already active**. No code changes needed. + +- Edit button appears on hover over any user message +- `← 1/2 →` navigator appears below user messages when variants exist +- Regenerate creates a branch instead of overwriting + +### Manual wiring (`` users) + +Wire the three props from `useCopilot()`: + +```tsx +function MyChat() { + const { switchBranch, getBranchInfo, editMessage } = useCopilot(); + + return ( + + ); +} +``` + +### Custom message renderers + +If you render messages manually, use `getBranchInfo` + `BranchNavigator`: + +```tsx +function MyMessage({ message }) { + const { switchBranch, getBranchInfo } = useCopilot(); + const info = message.role === "user" ? getBranchInfo(message.id) : null; + + return ( +
+

{message.content}

+ {info && ( + + switchBranch(info.siblingIds[info.siblingIndex - 1]) + } + onNext={() => switchBranch(info.siblingIds[info.siblingIndex + 1])} + /> + )} +
+ ); +} +``` + +### Programmatic branching + +```typescript +// Edit a message (creates new branch from same parent) +await editMessage("msg-abc", "Updated question text"); + +// Navigate between variants +switchBranch("msg-xyz"); + +// Check if branches exist +if (hasBranches) { + const info = getBranchInfo("msg-abc"); + // info.totalSiblings, info.siblingIndex, etc. +} + +// Persist all branches (not just visible path) +const allMessages = getAllMessages(); +await saveToServer(allMessages); +``` + +--- + +## Framework-Agnostic Usage + +All branching primitives are exported from the core package (no React required): + +```typescript +import { MessageTree, type BranchInfo } from "@yourgpt/copilot-sdk"; + +// Build a tree from saved messages +const tree = new MessageTree(savedMessages); + +// Get what to send to the AI (active path only) +const apiMessages = tree.getVisibleMessages(); + +// Get everything to persist +const allMessages = tree.getAllMessages(); + +// Navigate +tree.switchBranch(messageId); +const info = tree.getBranchInfo(messageId); // BranchInfo | null + +// Migrate legacy flat arrays +const linked = MessageTree.fromFlatArray(legacyMessages); +``` + +--- + +## How It Works Internally + +### Data structure + +Each message carries two optional fields: + +``` +parentId: string | null | undefined + null = root message (first in conversation) + undefined = legacy linear message (pre-branching) + string = ID of parent message + +childrenIds: string[] + Ordered list of direct child IDs (oldest-first) +``` + +The `MessageTree` maintains three maps: + +| Map | Key | Value | Purpose | +| ---------------- | ------------------------ | --------------- | --------------------------------- | +| `nodeMap` | messageId | Message | O(1) message lookup | +| `childrenOf` | parentId (or `__root__`) | `string[]` | All children at a fork | +| `activeChildMap` | parentId | active child ID | Which branch is currently visible | + +### Regenerate flow + +``` +Before: user → assistant-A + ↑ currentLeaf + +1. setCurrentLeaf(user.id) → rewind to user +2. processRequest() → AI generates assistant-B +3. addMessage(assistant-B) → becomes active child of user + +After: user → assistant-A (inactive, navigable via ←) + ↘ assistant-B (active) +``` + +### Edit flow + +``` +Before: user-A → assistant-A + +1. sendMessage("new text", { editMessageId: "user-A" }) +2. newParentId = user-A.parentId (= null, root) +3. setCurrentLeaf(null) → rewind to before user-A +4. create user-B with parentId=null +5. processRequest() → AI generates assistant-B + +After: user-A → assistant-A (inactive) + user-B → assistant-B (active) +``` + +### Visible path vs all messages + +``` +getAllMessages() → every message across every branch (for persistence) +getVisibleMessages() → root → currentLeaf along activeChildMap (for UI + API) +``` + +The API always receives `getVisibleMessages()`. Inactive branches are never sent to the model. diff --git a/apps/docs/alpha-docs/CHAT-PRIMITIVES.md b/apps/docs/alpha-docs/CHAT-PRIMITIVES.md new file mode 100644 index 0000000..d63ac02 --- /dev/null +++ b/apps/docs/alpha-docs/CHAT-PRIMITIVES.md @@ -0,0 +1,219 @@ +# Chat Primitives + +> `release/alpha` — ships two complementary APIs for headless chat customization: the `ChatPrimitives` namespace (low-level building blocks) and compound components on `CopilotChat.*` (MessageActions, MessageList, DefaultMessage, etc.). Both are non-breaking additive exports. + +--- + +## Table of Contents + +1. [What Was Built](#what-was-built) +2. [Breaking Changes](#breaking-changes) +3. [ChatPrimitives Namespace](#chatprimitives-namespace) +4. [CopilotChat Compound Components](#copilotchat-compound-components) +5. [Usage Examples](#usage-examples) +6. [How It Works Internally](#how-it-works-internally) +7. [Relation to `messageView`](#relation-to-messageview) + +--- + +## What Was Built + +Two exports that let you compose custom chat UIs at any level of abstraction while the SDK handles all state, streaming, and context internally. + +**`ChatPrimitives`** — a named export of individual low-level components. Useful when you import under an alias and want to pick specific pieces. + +**`CopilotChat.*` compound extensions** — the same primitives accessible directly on the `CopilotChat` component for inline composition without extra imports. + +--- + +## Breaking Changes + +**None.** Both are purely additive. Existing `` usage is untouched. + +--- + +## ChatPrimitives Namespace + +```tsx +import { ChatPrimitives as Chat } from "@yourgpt/copilot-sdk/ui"; +``` + +### All Primitives + +| Primitive | Description | +| --------------------- | --------------------------------------------------------- | +| `Chat.MessageList` | Render-prop message list — reads `messages` from context | +| `Chat.DefaultMessage` | Full SDK message bubble — use as fallback in custom lists | +| `Chat.Header` | Chat header bar | +| `Chat.Welcome` | Welcome screen shown when there are no messages | +| `Chat.Input` | Composer / input box | +| `Chat.ScrollAnchor` | Auto-scroll anchor, place at end of message list | +| `Chat.Message` | Low-level message row wrapper | +| `Chat.MessageAvatar` | Avatar with fallback initials | +| `Chat.MessageContent` | Content bubble — renders markdown, supports streaming | +| `Chat.MessageActions` | Action bar layout primitive (wraps action buttons) | +| `Chat.MessageAction` | Single action icon button with tooltip | +| `Chat.Loader` | Streaming / thinking indicator | + +### `Chat.MessageList` props + +```ts +interface MessageListProps { + children?: (message: ChatMessage, index: number) => React.ReactNode; + className?: string; +} +``` + +When `children` is provided, called once per message — return your custom component or fall back to `Chat.DefaultMessage`. When omitted, renders all messages with `DefaultMessage`. + +--- + +## CopilotChat Compound Components + +The `ChatPrimitives` are also mounted on the `CopilotChat` export: + +```tsx +import { CopilotChat } from "@yourgpt/copilot-sdk/ui"; + +CopilotChat.MessageActions; // compound action registrar (see MESSAGE-ACTIONS.md) +CopilotChat.CopyAction; // built-in copy button +CopilotChat.EditAction; // built-in inline edit button +CopilotChat.FeedbackAction; // built-in thumbs up/down +CopilotChat.Action; // custom action button +``` + +These are the action-registration compound components — see [MESSAGE-ACTIONS.md](./MESSAGE-ACTIONS.md) for full docs. + +--- + +## Usage Examples + +### Custom message type with fallback + +```tsx +import { ChatPrimitives as Chat } from "@yourgpt/copilot-sdk/ui"; + + + + {(message) => + message.metadata?.type === "plan" ? ( + + ) : ( + + ) + } + +; +``` + +--- + +### Fully custom layout — compose from scratch + +```tsx +import { ChatPrimitives as Chat } from "@yourgpt/copilot-sdk/ui"; + + +
+ + + +
+ + {(message) => ( + + + + + )} + + + +
+ + +
+
; +``` + +--- + +### Mix primitives with message actions + +```tsx +import { ChatPrimitives as Chat } from "@yourgpt/copilot-sdk/ui"; + + + {/* Register floating action buttons */} + + + log(msg.id, type)} /> + + + {/* Custom message list */} + + {(message) => + message.metadata?.type === "approval" ? ( + + ) : ( + + ) + } + +; +``` + +--- + +### Per-message action buttons (using primitives directly) + +```tsx +import { ChatPrimitives as Chat } from "@yourgpt/copilot-sdk/ui"; + + + {(message) => ( + + +
+ + + } + tooltip="Copy" + onClick={() => navigator.clipboard.writeText(message.content ?? "")} + /> + +
+
+ )} +
; +``` + +--- + +## How It Works Internally + +**State access:** `Chat.MessageList` reads `messages` and `registeredTools` from `CopilotChatInternalContext` — the same context `chat.tsx` already provides. No extra wiring needed. + +**`messages` + `registeredTools` in context:** Added to `CopilotChatInternalContext` so primitives can access them without prop drilling. `connected-chat.tsx` was unchanged — values flow through the existing context setup in `chat.tsx`. + +**Files created/modified:** + +- `message-list.tsx` _(new)_ — `Chat.MessageList` component +- `chat.tsx` — added `messages` + `registeredTools` to `CopilotChatInternalContext`; extended `Chat` compound object with `MessageActions`, `CopyAction`, `EditAction`, `FeedbackAction`, `Action` +- `ui/index.ts` — added `ChatPrimitives` export +- `chat/index.ts` — added `MessageList`, all action compound types + +--- + +## Relation to `messageView` + +`messageView` prop (see [CUSTOM-MESSAGE-VIEW.md](./CUSTOM-MESSAGE-VIEW.md)) and `Chat.MessageList` solve the same use case — custom message rendering — at different abstraction levels: + +| | `messageView` | `Chat.MessageList` | +| ----------- | -------------------------------------------------------- | --------------------------------------------- | +| Style | Prop on `` | Child component inside `` | +| Access | `messages[]` + pre-rendered `messageElements[]` | `messages[]` via render-prop | +| When to use | Quick overrides, inject extra UI around existing renders | Full layout control, building from primitives | + +Both are non-breaking and can coexist. `messageView` remains the simpler option for most cases. diff --git a/apps/docs/alpha-docs/CONTEXT-MANAGEMENT.md b/apps/docs/alpha-docs/CONTEXT-MANAGEMENT.md new file mode 100644 index 0000000..78bb8a4 --- /dev/null +++ b/apps/docs/alpha-docs/CONTEXT-MANAGEMENT.md @@ -0,0 +1,733 @@ +# Context Management + +Advanced context window management for the YourGPT Copilot SDK. These features give you full control over what the AI sees, how long conversations stay alive, and how tokens are tracked and budgeted. + +--- + +## Table of Contents + +1. [Dual-Layer Message Store](#1-dual-layer-message-store) +2. [Message History & Compaction](#2-message-history--compaction) + - [Compaction Strategies](#compaction-strategies) + - [Config Reference](#config-reference) +3. [Token Counting](#3-token-counting) +4. [Session Persistence](#4-session-persistence) +5. [useContextStats](#5-usecontextstats) +6. [AgentLoop API](#6-agentloop-api) +7. [Tools — useTool / useTools / ToolDefinition](#7-tools--usetool--usetools--tooldefinition) + - [Deferred Tools](#deferred-tools) + - [Hidden Tools](#hidden-tools) + - [Fallback Tool Renderer](#fallback-tool-renderer) +8. [Message Grouping](#8-message-grouping) +9. [Server: compactSession](#9-server-compactsession) + +--- + +## 1. Dual-Layer Message Store + +Every conversation maintains two parallel views of the message history. + +| Layer | Type | Purpose | +| --------------------- | ------------------ | --------------------------------------------------------------------------------- | +| **Display layer** | `DisplayMessage[]` | Full immutable history. Rendered in the UI. Never shrinks. | +| **LLM context layer** | `LLMMessage[]` | Compacted/pruned form sent to the model on each request. Rebuilt on every render. | + +### Types + +```typescript +// Display layer — extends UIMessage for full backward-compat +interface DisplayMessage extends UIMessage { + timestamp: number; // Unix ms +} + +// Injected into displayMessages when compaction fires +interface CompactionMarker extends DisplayMessage { + role: "system"; + type: "compaction-marker"; + content: string; // Human-readable summary + summarizedMessageIds: string[]; + tokensSaved: number; +} + +// LLM context layer — what the model actually sees +interface LLMMessage { + role: "system" | "user" | "assistant" | "tool"; + content: string; + tool_calls?: ToolCall[]; + tool_call_id?: string; +} + +// Replaces a full tool result when old enough to prune +interface CompactedToolResult { + type: "compacted-tool-result"; + toolName: string; + toolCallId: string; + args: Record; + executedAt: number; + status: "success" | "error"; + originalSize: number; + summary: string; + extract?: string; // First 200 chars if no LLM summary +} +``` + +### Conversion helpers + +```typescript +import { + toDisplayMessage, + toLLMMessage, + toLLMMessages, + keepToolPairsAtomic, +} from "@yourgpt/copilot-sdk-react"; +``` + +`keepToolPairsAtomic` ensures that when you slice a window, an `assistant` message with `tool_calls` is never separated from its corresponding tool-result messages. + +--- + +## 2. Message History & Compaction + +### useMessageHistory + +```typescript +import { useMessageHistory } from "@yourgpt/copilot-sdk-react"; + +function MyChat() { + const { + displayMessages, // Full UI history + llmMessages, // Compacted LLM context + tokenUsage, // Live token estimate + isCompacting, // true while auto-compaction runs + compactionState, // Metadata & rolling summary + compactSession, // Manual trigger + addToWorkingMemory, + clearWorkingMemory, + resetSession, + } = useMessageHistory({ + strategy: "summary-buffer", + maxContextTokens: 128000, + compactionThreshold: 0.75, + compactionUrl: "/api/compact", + persistSession: true, + }); +} +``` + +#### Return type + +```typescript +interface UseMessageHistoryReturn { + displayMessages: DisplayMessage[]; + llmMessages: LLMMessage[]; + tokenUsage: TokenUsage; + isCompacting: boolean; + compactionState: SessionCompactionState; + compactSession: (instructions?: string) => Promise; + addToWorkingMemory: (fact: string) => void; + clearWorkingMemory: () => void; + resetSession: () => void; +} +``` + +### Compaction Strategies + +Four strategies are available via the `strategy` config field. + +#### `"none"` (default) + +No compaction. Zero-config, 100% backward-compatible. All messages sent verbatim. + +```typescript +useMessageHistory({ strategy: "none" }); +``` + +#### `"sliding-window"` + +Keeps only the most recent N tokens of history. Oldest messages are dropped when the token budget is exceeded. + +```typescript +useMessageHistory({ + strategy: "sliding-window", + maxContextTokens: 128000, + reserveForResponse: 4096, + recentBuffer: 10, // Always keep at least 10 recent messages + toolResultMaxChars: 10000, // Truncate large tool results +}); +``` + +#### `"selective-prune"` + +Removes tool-result messages that are older than `recentBuffer`, keeping the conversation skeleton (user/assistant turns) intact. Lighter than sliding-window — no token counting required. + +```typescript +useMessageHistory({ + strategy: "selective-prune", + recentBuffer: 10, +}); +``` + +#### `"summary-buffer"` + +Summarizes old messages into a rolling summary when usage exceeds `compactionThreshold`. The summary is injected into the LLM context as a system message. Requires a `/api/compact` endpoint (or custom `summarizer`). + +```typescript +useMessageHistory({ + strategy: "summary-buffer", + compactionThreshold: 0.75, // Compact at 75% of maxContextTokens + compactionUrl: "/api/compact", + recentBuffer: 10, + onCompaction: (event) => { + console.log( + `Compacted ${event.messagesSummarized} messages, saved ~${event.tokensSaved} tokens`, + ); + }, +}); +``` + +Custom summarizer (skip the HTTP round-trip): + +```typescript +useMessageHistory({ + strategy: "summary-buffer", + summarizer: async (messages) => { + const res = await myLLM.summarize(messages); + return res.text; + }, +}); +``` + +### Config Reference + +```typescript +interface MessageHistoryConfig { + strategy?: "none" | "sliding-window" | "summary-buffer" | "selective-prune"; + maxContextTokens?: number; // default: 128000 + reserveForResponse?: number; // default: 4096 + compactionThreshold?: number; // default: 0.75 + recentBuffer?: number; // default: 10 + toolResultMaxChars?: number; // default: 10000 (0 = no cap) + compactionUrl?: string; // required for summary-buffer + persistSession?: boolean; // default: false + storageKey?: string; // default: "copilot-session" + onCompaction?: (event: CompactionEvent) => void; + onTokenUsage?: (usage: TokenUsage) => void; +} +``` + +#### Per-call options + +```typescript +interface UseMessageHistoryOptions extends MessageHistoryConfig { + skipCompaction?: boolean; + tokenEstimation?: "fast" | "accurate" | "off"; // default: "fast" + summarizer?: (messages: LLMMessage[]) => Promise; +} +``` + +### Provider-level config + +Set defaults once in `` instead of each `useMessageHistory` call: + +```tsx + + + +``` + +### Working Memory + +Pin facts that survive all future compactions: + +```typescript +const { addToWorkingMemory, clearWorkingMemory } = useMessageHistory({ ... }); + +// Survives compaction +addToWorkingMemory("User is on the Pro plan. Account ID: acct_123"); + +// Remove all pinned facts +clearWorkingMemory(); +``` + +### Compaction event & token usage types + +```typescript +interface CompactionEvent { + type: "auto" | "manual"; + compactionCount: number; + messagesSummarized: number; + tokensSaved: number; + timestamp: number; +} + +interface TokenUsage { + current: number; // Estimated tokens in LLM context + max: number; // maxContextTokens + percentage: number; // current / max (0–1) + isApproaching: boolean; // percentage >= compactionThreshold +} + +interface SessionCompactionState { + rollingSummary: string | null; + lastCompactionAt: number | null; + compactionCount: number; + totalTokensSaved: number; + workingMemory: string[]; + displayMessageCount: number; + llmMessageCount: number; +} +``` + +--- + +## 3. Token Counting + +Two-tier estimation — pick the right trade-off between speed and accuracy. + +### Tier 1: Fast (zero dependencies) + +Uses a `chars / 3.5` heuristic. ~85–90% accurate for English. Always available, no bundle cost. + +```typescript +import { + estimateTokensFast, + estimateMessageTokens, + estimateMessagesTokens, +} from "@yourgpt/copilot-sdk-react"; + +const tokens = estimateTokensFast("Hello world"); // fast, synchronous +const msgTokens = estimateMessagesTokens(llmMessages); +``` + +### Tier 2: Accurate (lazy-loaded) + +Uses `gpt-tokenizer` with the `o200k_base` encoding. Lazy-loaded only when called — no upfront bundle cost. Falls back to Tier 1 if `gpt-tokenizer` is not installed. + +```typescript +import { + countTokensAccurate, + countMessagesTokensAccurate, +} from "@yourgpt/copilot-sdk-react"; + +// Only loads gpt-tokenizer on first call +const tokens = await countTokensAccurate("Hello world"); +const msgTokens = await countMessagesTokensAccurate(llmMessages); +``` + +### Dispatcher + +```typescript +import { estimateTokens } from "@yourgpt/copilot-sdk-react"; +import type { TokenEstimationMode } from "@yourgpt/copilot-sdk-react"; + +// mode: "fast" | "accurate" | "off" +const tokens = estimateTokens(llmMessages, "fast"); +``` + +Set via `tokenEstimation` in `useMessageHistory`: + +```typescript +useMessageHistory({ tokenEstimation: "accurate" }); +``` + +--- + +## 4. Session Persistence + +Survive page reloads with zero extra code. + +```typescript +useMessageHistory({ + persistSession: true, + storageKey: "my-app-chat", // default: "copilot-session" +}); +``` + +| What is persisted | Where | +| ---------------------------------- | ---------------------------------------------------------- | +| `compactionState` (small metadata) | `localStorage` — sync, available immediately on cold start | +| `displayMessages` (can be large) | `IndexedDB` — async, avoids localStorage quota issues | + +Both are keyed by `storageKey`. Multiple chat instances can coexist with different keys. + +Clear everything (including storage) with: + +```typescript +const { resetSession } = useMessageHistory({ persistSession: true }); +await resetSession(); +``` + +--- + +## 5. useContextStats + +Live snapshot of context window usage. Updates reactively on every message send. + +```typescript +import { useContextStats } from "@yourgpt/copilot-sdk-react"; + +function ContextMonitor() { + const { + contextUsage, // Full breakdown by bucket (richest field) + totalTokens, // Convenience: total estimated tokens + usagePercent, // Convenience: window fill 0–1 + contextChars, // Characters contributed by AI context injections + toolCount, // Number of currently registered tools + messageCount, // Visible (non-system) messages + lastResponseUsage, // Token usage from last assistant message + } = useContextStats(); + + // Breakdown by bucket + const historyTokens = contextUsage?.breakdown.history.tokens; + const systemPercent = contextUsage?.breakdown.systemPrompt.percent; + + return ( +
+

{Math.round(usagePercent * 100)}% of context used

+

{totalTokens} tokens / {toolCount} tools

+ {lastResponseUsage && ( +

Last turn: {lastResponseUsage.total_tokens} tokens

+ )} +
+ ); +} +``` + +### Return type + +```typescript +interface ContextStats { + contextUsage: ContextUsage | null; // null until first message + totalTokens: number; + usagePercent: number; // 0 until first message + contextChars: number; + toolCount: number; + messageCount: number; + lastResponseUsage: MessageTokenUsage | null; +} + +interface MessageTokenUsage { + prompt_tokens: number; + completion_tokens: number; + total_tokens: number; +} +``` + +--- + +## 6. AgentLoop API + +`AbstractAgentLoop` is the framework-agnostic core that manages the tool execution loop, approvals, and cancellation. + +```typescript +import { AbstractAgentLoop } from "@yourgpt/copilot-sdk"; + +const loop = new AbstractAgentLoop( + { + maxIterations: 20, + tools: [myTool], + }, + { + onToolExecutionsChange: (executions) => setExecutions(executions), + onToolApprovalRequired: (execution) => showApprovalModal(execution), + }, +); + +// Register/unregister tools at runtime +loop.registerTool(weatherTool); +loop.unregisterTool("old_tool"); + +// Execute tool calls returned by the LLM +const results = await loop.executeToolCalls(toolCallsFromLLM); + +// Cancel in-flight execution +loop.cancel(); +``` + +### Config + +```typescript +interface AgentLoopConfig { + maxIterations?: number; // default: 20 + maxExecutionHistory?: number; // default: 100 + tools?: ToolDefinition[]; +} +``` + +Tools use reference counting so React StrictMode double-invocations don't leave orphaned registrations. + +--- + +## 7. Tools — useTool / useTools / ToolDefinition + +### useTool + +Register a single client-side tool from a React component. Accepts both Zod schemas and JSON Schema. + +```typescript +import { useTool } from "@yourgpt/copilot-sdk-react"; +import { z } from "zod"; + +function MyComponent() { + useTool({ + name: "navigate_to_page", + description: "Navigate to a page in the app", + inputSchema: z.object({ + path: z.string().describe("Route path to navigate to"), + }), + handler: async ({ path }) => { + router.push(path); + return { success: true }; + }, + // Optional UI rendering + render: ({ args, result }) => , + }); +} +``` + +### useTools (ToolSet pattern) + +Register multiple tools at once using the Vercel AI SDK `ToolSet` pattern: + +```typescript +import { useTools, tool } from "@yourgpt/copilot-sdk-react"; + +function MyApp() { + useTools({ + get_weather: tool({ + description: "Get weather for a location", + inputSchema: { + type: "object", + properties: { location: { type: "string" } }, + required: ["location"], + }, + handler: async ({ location }) => fetchWeather(location), + }), + open_modal: tool({ + description: "Open a UI modal", + inputSchema: z.object({ id: z.string() }), + handler: async ({ id }) => { + openModal(id); + return { success: true }; + }, + }), + }); +} +``` + +### UseToolConfig reference + +```typescript +interface UseToolConfig { + name: string; + description: string; + inputSchema: ZodSchema | JSONSchema; // Both accepted + handler: ( + params: TParams, + context?: ToolContext, + ) => Promise | ToolResponse; + + // UI + render?: (props: ToolRenderProps) => React.ReactNode; + title?: string | ((args: TParams) => string); + executingTitle?: string | ((args: TParams) => string); + completedTitle?: string | ((args: TParams) => string); + + // Behaviour + available?: boolean; // default: true + needsApproval?: boolean; + approvalMessage?: string | ((params: TParams) => string); + hidden?: boolean; // default: false — see Hidden Tools + aiResponseMode?: AIResponseMode; + aiContext?: string | ((result, args) => string); + resultConfig?: ToolResultConfig; + + // Loading strategy + deferLoading?: boolean; // see Deferred Tools + profiles?: string[]; + searchKeywords?: string[]; + group?: string; + category?: string; +} +``` + +### Deferred Tools + +Large tool registries can bloat the LLM request payload. Mark tools with `deferLoading: true` to keep them out of the default request — they are auto-detected and injected only when the user's query semantically matches the tool. + +```typescript +useTool({ + name: "run_sql_query", + description: "Execute a SQL query against the database", + deferLoading: true, // Not sent on every request + searchKeywords: ["sql", "query", "database", "table"], + inputSchema: z.object({ query: z.string() }), + handler: async ({ query }) => db.execute(query), +}); +``` + +Auto-detection uses `description` + `searchKeywords` to score relevance against the current message. No configuration required. + +### Hidden Tools + +Register tools that execute silently — they run when called by the AI but are never shown in the tool execution UI. + +```typescript +useTool({ + name: "log_analytics_event", + description: "Log a UI analytics event", + hidden: true, // Never rendered in chat UI + inputSchema: z.object({ event: z.string(), data: z.record(z.unknown()) }), + handler: async ({ event, data }) => { + analytics.track(event, data); + return {}; + }, +}); +``` + +### Fallback Tool Renderer + +The `` component resolves a renderer for each tool execution using this priority chain: + +1. **`toolRenderers[toolName]`** — per-tool renderer map passed to `` +2. **`tool.render`** — render function attached to the `ToolDefinition` via `useTool` +3. **`mcpToolRenderer`** — catch-all for tools with `source: "mcp"` +4. **`fallbackToolRenderer`** — catch-all for any tool not matched above +5. **Built-in default** — generic tool execution card + +```tsx + , + }} + // MCP catch-all + mcpToolRenderer={({ toolName, args, result }) => } + // Universal catch-all + fallbackToolRenderer={({ toolName, args, result }) => ( +
{JSON.stringify(result, null, 2)}
+ )} +/> +``` + +--- + +## 8. Message Grouping + +`groupConsecutiveMessages` groups consecutive messages of the same role into visual clusters. Useful for building custom chat UIs where adjacent user or assistant messages should appear as one block. + +Available from the message-utils module: + +```typescript +import { + toLLMMessages, + toLLMMessage, + keepToolPairsAtomic, +} from "@yourgpt/copilot-sdk-react"; +``` + +Core invariant: **tool-call pairs are always atomic.** An assistant message with `tool_calls` is never separated from its corresponding tool-result messages during any windowing or pruning operation. + +--- + +## 9. Server: compactSession + +The `compactSession` utility powers the `/api/compact` endpoint for `summary-buffer` compaction. It calls Claude (defaults to `claude-haiku-4-5`) to produce a structured summary that preserves: + +- User goals and requests +- Technical decisions and chosen approaches +- Tool call outcomes (name, key args, result status) +- Errors and resolutions +- Pending tasks and current work state + +```typescript +// app/api/compact/route.ts +import { compactSession } from "@yourgpt/copilot-sdk/server"; + +export async function POST(req: Request) { + const { messages, existingSummary, workingMemory } = await req.json(); + + const { summary } = await compactSession({ + messages, + existingSummary, // Passed in subsequent compactions for rolling summaries + workingMemory, // User-pinned facts (addToWorkingMemory) + model: "claude-haiku-4-5", // default + maxSummaryTokens: 1024, // default + apiKey: process.env.ANTHROPIC_API_KEY, + }); + + return Response.json({ summary }); +} +``` + +### CompactSessionOptions + +```typescript +interface CompactSessionOptions { + messages: Array<{ role: string; content?: string | null }>; + existingSummary?: string | null; + workingMemory?: string[]; + model?: string; // default: "claude-haiku-4-5" + maxSummaryTokens?: number; // default: 1024 + apiKey?: string; // fallback: process.env.ANTHROPIC_API_KEY + apiBaseUrl?: string; // default: "https://api.anthropic.com" + fetchImpl?: typeof fetch; +} +``` + +--- + +## Quick-start: Full Setup + +```tsx +// app/layout.tsx +import { CopilotProvider } from "@yourgpt/copilot-sdk-react"; + +export default function RootLayout({ children }) { + return ( + console.log("Compacted:", e), + }} + > + {children} + + ); +} +``` + +```tsx +// components/ChatPanel.tsx +import { useMessageHistory, useContextStats } from "@yourgpt/copilot-sdk-react"; + +export function ChatPanel() { + const { tokenUsage, isCompacting, compactSession } = useMessageHistory(); + const { usagePercent, toolCount } = useContextStats(); + + return ( +
+

+ {Math.round(usagePercent * 100)}% context used · {toolCount} tools +

+ {tokenUsage.isApproaching && ( + + )} + {isCompacting && Summarizing history…} +
+ ); +} +``` diff --git a/apps/docs/alpha-docs/CUSTOM-MESSAGE-VIEW.md b/apps/docs/alpha-docs/CUSTOM-MESSAGE-VIEW.md new file mode 100644 index 0000000..824804d --- /dev/null +++ b/apps/docs/alpha-docs/CUSTOM-MESSAGE-VIEW.md @@ -0,0 +1,173 @@ +# Custom Message View + +> `release/alpha` — adds a `messageView` prop to `CopilotChat` / `Chat` that gives full control over how the message list is rendered. Inject custom UI, conditionally replace messages based on `metadata.type`, or build entirely custom layouts — without touching roles or message history. + +--- + +## Table of Contents + +1. [What Was Built](#what-was-built) +2. [Breaking Changes](#breaking-changes) +3. [New API](#new-api) +4. [Usage Examples](#usage-examples) +5. [How It Works Internally](#how-it-works-internally) +6. [Roadmap — Chat.\* Primitives](#roadmap--chat-primitives) + +--- + +## What Was Built + +A `messageView` prop on `` / `` that intercepts message list rendering. + +You receive: + +- **`messageElements`** — pre-rendered default SDK elements (one per message, may include `null` for filtered messages) +- **`messages`** — raw `ChatMessage[]` for conditional logic + +This closes the use case from **issue #74** (custom message types with dedicated renderers) without touching the `role` union or message history format. + +--- + +## Breaking Changes + +**None.** Fully additive. Existing `renderMessage`, `toolRenderers`, and all other props are unchanged. + +--- + +## New API + +### `messageView` prop + +Added to `ChatProps` (and flows through to `CopilotChat` via `...chatProps`). + +```ts +messageView?: { + children?: (props: { + /** Raw messages array */ + messages: ChatMessage[]; + /** Pre-rendered default SDK elements, one per message */ + messageElements: React.ReactNode[]; + }) => React.ReactNode; +}; +``` + +--- + +## Usage Examples + +### Inject custom UI below messages + +```tsx + ( + <> + {messageElements} +
+ Powered by YourGPT +
+ + ), + }} +/> +``` + +### Custom message types via `metadata.type` + +Inject a custom message into the chat (e.g. from a tool handler or agent state), then render it with your own component: + +```tsx + ( + <> + {messages.map((message, i) => { + if (message.metadata?.type === "plan") { + return ; + } + if (message.metadata?.type === "approval") { + return ; + } + return messageElements[i]; + })} + + ), + }} +/> +``` + +### Combine with agent state + +```tsx +function Chat() { + const agentState = useMyAgentState(); + + return ( + ( +
+ {messageElements} + {agentState?.steps && } +
+ ), + }} + /> + ); +} +``` + +--- + +## How It Works Internally + +**Files changed:** `types.ts`, `chat.tsx` (2 files, ~30 lines total) + +In `chat.tsx`, the `messages.map(...)` loop is wrapped in an IIFE that collects rendered elements into a `messageElements` array first, then either: + +- Passes them to `messageView.children({ messages, messageElements })` if provided +- Or renders them directly (existing behaviour) + +```tsx +{ + (() => { + const messageElements = messages.map((message, index) => { + // ...existing render logic unchanged... + }); + + return messageView?.children + ? messageView.children({ messages, messageElements }) + : messageElements; + })(); +} +``` + +The loading placeholder and scroll anchor remain outside this block and are unaffected. + +`connected-chat.tsx` required no changes — `messageView` flows through automatically via `...chatProps`. + +--- + +## `Chat.*` Primitives — Now Shipped + +The headless primitive API described here as a roadmap item has shipped in this same alpha. You can use it today: + +```tsx +import { ChatPrimitives as Chat } from "@yourgpt/copilot-sdk/ui"; + + + + {(message) => + message.metadata?.type === "plan" ? ( + + ) : ( + + ) + } + +; +``` + +`messageView` remains the simpler option for quick overrides. `Chat.MessageList` is the lower-level primitive when you need full layout control. Both work — no migration needed between them. + +→ Full primitives docs: [CHAT-PRIMITIVES.md](./CHAT-PRIMITIVES.md) diff --git a/apps/docs/alpha-docs/MESSAGE-ACTIONS.md b/apps/docs/alpha-docs/MESSAGE-ACTIONS.md new file mode 100644 index 0000000..7759157 --- /dev/null +++ b/apps/docs/alpha-docs/MESSAGE-ACTIONS.md @@ -0,0 +1,247 @@ +# Message Actions + +> `release/alpha` — adds a compound component API for registering floating action buttons on chat messages. Declarative, role-based, fully composable — same pattern as shadcn/Radix. + +--- + +## Table of Contents + +1. [What Was Built](#what-was-built) +2. [Breaking Changes](#breaking-changes) +3. [New APIs](#new-apis) +4. [Usage Examples](#usage-examples) +5. [How It Works Internally](#how-it-works-internally) +6. [Also Shipped — ChatPrimitives Namespace](#also-shipped--chatprimitives-namespace) + +--- + +## What Was Built + +A compound component API for adding floating action buttons to chat messages — copy, edit, feedback, or fully custom actions — declared as children of ``. + +Actions appear on hover, floating below the message bubble. Role-based — configure `assistant` and `user` separately. + +--- + +## Breaking Changes + +**None.** If no `` children are declared, nothing changes. Existing chat UI looks and behaves identically. + +--- + +## New APIs + +### Compound components + +``` +CopilotChat.MessageActions — registers actions for a role +CopilotChat.CopyAction — built-in copy to clipboard (with check feedback) +CopilotChat.EditAction — built-in edit (user messages, wired to inline edit) +CopilotChat.FeedbackAction — built-in thumbs up/down +CopilotChat.Action — fully custom action +``` + +### Props + +```tsx +// MessageActions +role: "user" | "assistant" + +// CopyAction +tooltip?: string +className?: string + +// EditAction +tooltip?: string +className?: string + +// FeedbackAction +onFeedback?: (message: ChatMessage, type: "helpful" | "not-helpful") => void +tooltip?: string +className?: string + +// Action +id?: string +icon: ReactNode +tooltip: string +onClick: (props: { message: ChatMessage }) => void +hidden?: boolean | ((props: { message: ChatMessage }) => boolean) +className?: string +``` + +--- + +## Usage Examples + +### Zero config — no actions (default) + +```tsx + +// No action buttons shown — clean slate +``` + +--- + +### Copy on assistant, Edit on user + +```tsx + + + + + + + + + +``` + +--- + +### Copy + Feedback on assistant + +```tsx + + + + { + sendFeedback({ messageId: message.id, type }); + }} + /> + + +``` + +--- + +### Custom action + +```tsx + + + + } + tooltip="Share" + onClick={({ message }) => share(message.content)} + /> + + +``` + +--- + +### Conditional action (hide based on message) + +```tsx + + + + } + tooltip="Report" + hidden={({ message }) => !message.content} + onClick={({ message }) => report(message.id)} + /> + + +``` + +--- + +### Disable all actions for a role + +```tsx + + + {/* empty — no actions for assistant */} + + +``` + +--- + +### Full setup — both roles + +```tsx + + + + log(msg.id, type)} /> + } + tooltip="Save" + onClick={({ message }) => save(message)} + /> + + + + + } + tooltip="Delete" + onClick={({ message }) => deleteMessage(message.id)} + /> + + +``` + +--- + +## How It Works Internally + +**Files created/modified:** + +- `message-actions-context.tsx` _(new)_ — React context storing registered actions per role +- `message-actions-compound.tsx` _(new)_ — compound components (`MessageActions`, `CopyAction`, `EditAction`, `FeedbackAction`, `Action`) +- `chat.tsx` — wrapped with `MessageActionsProvider`, compound components added to `Chat.*` namespace +- `default-message.tsx` — `FloatingActions` helper reads from context, renders on `group-hover/message` + +**Flow:** + +1. `` scans its children's props via `React.Children.forEach`, builds a `RegisteredAction[]` +2. `useLayoutEffect` registers them into `MessageActionsContext` +3. `DefaultMessage` renders `` for each message +4. `FloatingActions` calls `ctx.getActions(role)` — if empty, renders nothing + +**Copy action** has local state (`copiedId`) — switches icon to ✓ for 1.5s then reverts. + +**Edit action** routes to the existing `startEdit()` function already in `DefaultMessage` — no duplication. + +--- + +## Also Shipped — `ChatPrimitives` Namespace + +A `ChatPrimitives` export was also added for headless composition: + +```tsx +import { ChatPrimitives as Chat } from "@yourgpt/copilot-sdk-ui"; + + + + {(message) => + message.metadata?.type === "plan" ? ( + + ) : ( + + ) + } + +; +``` + +| Primitive | Description | +| --------------------- | -------------------------------------------- | +| `Chat.MessageList` | Render-prop message list, reads from context | +| `Chat.DefaultMessage` | Full SDK message bubble, use as fallback | +| `Chat.Header` | Chat header bar | +| `Chat.Welcome` | Welcome screen (no messages) | +| `Chat.Input` | Composer / input box | +| `Chat.ScrollAnchor` | Auto-scroll anchor | +| `Chat.Message` | Low-level row wrapper | +| `Chat.MessageAvatar` | Avatar with fallback | +| `Chat.MessageContent` | Content bubble, supports markdown | +| `Chat.MessageActions` | Action bar layout primitive | +| `Chat.MessageAction` | Single action button with tooltip | +| `Chat.Loader` | Streaming indicator | diff --git a/apps/docs/alpha-docs/SKILLS.md b/apps/docs/alpha-docs/SKILLS.md new file mode 100644 index 0000000..d2e02aa --- /dev/null +++ b/apps/docs/alpha-docs/SKILLS.md @@ -0,0 +1,518 @@ +# Skills System + +Skills are instruction playbooks the AI loads on demand. They shape the model's **behavior** — separate from Tools, which perform actions. + +A skill is a Markdown file (or inline string) containing instructions. Skills can be: + +- **eager** — always injected into the system prompt +- **auto** — listed in a catalog; the AI calls `load_skill` to retrieve them when relevant +- **manual** — available via `load_skill` but not advertised in the catalog + +--- + +## Table of Contents + +1. [Concepts](#1-concepts) +2. [Client-side: SkillProvider + useSkill](#2-client-side-skillprovider--useskill) +3. [Server-side: loadSkills](#3-server-side-loadskills) +4. [Skill File Format](#4-skill-file-format) +5. [defineSkill helper](#5-defineskill-helper) +6. [useSkillStatus](#6-useskillstatus) +7. [Source precedence & collision detection](#7-source-precedence--collision-detection) +8. [Type Reference](#8-type-reference) +9. [Full Example](#9-full-example) + +--- + +## 1. Concepts + +| Strategy | Behavior | +| -------- | --------------------------------------------------------------------------------------------------- | +| `eager` | Content prepended to system prompt on every request. Always active. | +| `auto` | Listed in the skill catalog appended to the system prompt. AI calls `load_skill({ name })` to load. | +| `manual` | Accessible via `load_skill` but not advertised — for internal/conditional skills. | + +The `load_skill` tool is automatically registered when a `` is present (client) or when `loadSkills()` builds the tools object (server). No manual wiring required. + +--- + +## 2. Client-side: SkillProvider + useSkill + +### SkillProvider + +Wrap your app (inside ``) to enable client-side skills: + +```tsx +import { SkillProvider, defineSkill } from "@yourgpt/copilot-sdk-react"; + +const brandVoice = defineSkill({ + name: "brand-voice", + description: "Ensures responses match our brand tone and terminology", + strategy: "eager", + source: { + type: "inline", + content: + "Always respond in a friendly, concise tone. Use 'we' not 'I'. Avoid jargon.", + }, +}); + +const codeReview = defineSkill({ + name: "code-review", + description: "Performs structured code reviews with actionable feedback", + strategy: "auto", // AI loads this on demand + source: { + type: "inline", + content: "When reviewing code: 1) Check for bugs first...", + }, +}); + +export default function App() { + return ( + + + + + + ); +} +``` + +> **Note:** `` only supports `inline` source skills client-side. For `file` or `url` sources, use `loadSkills()` on the server. + +### useSkill + +Register a skill from deep inside the component tree — it activates on mount and cleans up on unmount. + +```tsx +import { useSkill } from "@yourgpt/copilot-sdk-react"; + +function CheckoutPage() { + useSkill({ + name: "checkout-flow", + description: "Guides the user through the checkout process step by step", + strategy: "auto", + source: { + type: "inline", + content: ` +## Checkout Assistant + +When the user asks about checkout: +1. Confirm their cart items +2. Check for applicable promo codes +3. Walk through shipping options +4. Confirm payment method before submitting + `, + }, + }); + + return ; +} +``` + +The skill is automatically unregistered when `CheckoutPage` unmounts. + +**Dev warning:** If an inline skill exceeds 2000 characters in development, a console warning is shown. Large inline skills are sent on every request — consider using a server-side file skill instead. + +--- + +## 3. Server-side: loadSkills + +For `file` and `url` sources, or when you want server-controlled skill loading: + +```typescript +// app/api/chat/route.ts +import path from "path"; +import { loadSkills } from "@yourgpt/copilot-sdk/server"; + +export async function POST(req: Request) { + const { messages, __skills } = await req.json(); + + const { skills, buildSystemPrompt, tools, diagnostics } = await loadSkills({ + // Source 1: .md files from a local directory (highest precedence) + dir: path.join(process.cwd(), "skills"), + + // Source 2: Remote .md URLs + remoteUrls: ["https://cdn.myapp.com/skills/support-policy.md"], + + // Source 3: Inline skills forwarded from client (lowest precedence) + clientSkills: __skills ?? [], + }); + + // Log any name collisions + if (diagnostics.length) { + console.warn("Skill collisions:", diagnostics); + } + + const systemPrompt = buildSystemPrompt( + "You are a helpful assistant for Acme Corp.", + ); + + // Pass tools.load_skill to your AI provider + return streamText({ + model: anthropic("claude-sonnet-4-6"), + system: systemPrompt, + messages, + tools: { + ...tools, // includes load_skill + ...myOtherTools, + }, + }); +} +``` + +### loadSkills options + +```typescript +interface LoadSkillsOptions { + dir?: string; // Path to /skills directory (Node.js only) + remoteUrls?: string[]; // Remote .md URLs to fetch + clientSkills?: ClientInlineSkill[]; // Forwarded from useSkill() hooks +} +``` + +### loadSkills result + +```typescript +interface LoadSkillsResult { + skills: ResolvedSkill[]; + diagnostics: SkillDiagnostic[]; + + // Build system prompt: prepends eager content, appends auto catalog + buildSystemPrompt(basePrompt?: string): string; + + // Ready-to-use load_skill tool definition + tools: { + load_skill: { + description: string; + parameters: { ... }; + execute: (args: { name: string }) => Promise; + }; + }; +} +``` + +### Forwarding client skills to the server + +`` automatically syncs inline skills to `CopilotProvider`, which includes them in every API request as `__skills`. Read them in your route handler: + +```typescript +const { messages, __skills } = await req.json(); + +const { buildSystemPrompt, tools } = await loadSkills({ + dir: path.join(process.cwd(), "skills"), + clientSkills: __skills ?? [], // Inline skills from useSkill() hooks +}); +``` + +--- + +## 4. Skill File Format + +Skill files are Markdown with an optional YAML frontmatter block. + +```markdown +--- +name: code-review +description: Performs structured code reviews with actionable feedback +strategy: auto +version: 1.2.0 +--- + +## Code Review Instructions + +When asked to review code, follow this structure: + +1. **Correctness** — Check for logic errors and edge cases +2. **Security** — Flag injection risks, exposed secrets, insecure defaults +3. **Performance** — Note O(n²) loops, unnecessary re-renders, missing indexes +4. **Style** — Suggest naming and structure improvements (non-blocking) + +Always include a summary section with an overall assessment. +``` + +### Frontmatter fields + +| Field | Required | Description | +| ------------- | ----------- | ------------------------------------------------------------------------------------- | +| `name` | Recommended | Skill name. Derived from filename if omitted (e.g. `code-review.md` → `code-review`). | +| `description` | Recommended | One-line description shown in the AI's skill catalog. | +| `strategy` | No | `eager`, `auto`, or `manual`. Default: `auto`. | +| `version` | No | Informational version string. | + +### Directory layout + +``` +skills/ +├── brand-voice.md # Flat .md file +├── code-review.md +└── sql-expert/ + └── SKILL.md # Folder-based skill (use for multi-file skills) +``` + +For folder-based skills, place the main skill file at `/SKILL.md`. The folder name is used as the skill name unless overridden by frontmatter. + +--- + +## 5. defineSkill helper + +Type-safe factory for creating skill definitions. An identity function with TypeScript inference — same pattern as `useTool`. + +```typescript +import { defineSkill } from "@yourgpt/copilot-sdk-react"; +// or from server: +import { defineSkill } from "@yourgpt/copilot-sdk/server"; + +const mySkill = defineSkill({ + name: "api-docs-helper", + description: "Helps users understand and use the Acme API", + strategy: "auto", + version: "2.0.0", + source: { + type: "inline", + content: "When explaining API endpoints, always include example requests...", + }, +}); + +// Reuse in multiple providers + +``` + +--- + +## 6. useSkillStatus + +Observe the live skill registry state from any component inside ``: + +```tsx +import { useSkillStatus } from "@yourgpt/copilot-sdk-react"; + +function DebugPanel() { + const { skills, count, has } = useSkillStatus(); + + return ( +
+

{count} skill(s) active

+ {has("code-review") && Code Review} +
    + {skills.map((s) => ( +
  • + {s.name} ({s.strategy ?? "auto"}) +
  • + ))} +
+
+ ); +} +``` + +### Return type + +```typescript +interface UseSkillStatusReturn { + skills: ResolvedSkill[]; // All currently registered skills + count: number; // Number of registered skills + has: (name: string) => boolean; // Check if a named skill is active +} +``` + +--- + +## 7. Source Precedence & Collision Detection + +When the same skill name appears in multiple sources, the higher-precedence source wins and a diagnostic is recorded. + +``` +server-dir > remote-url > client-inline +``` + +```typescript +const { diagnostics } = await loadSkills({ ... }); + +// diagnostics: SkillDiagnostic[] +// [{ +// type: "collision", +// name: "code-review", +// winner: "server-dir", +// loser: "client-inline", +// }] +``` + +This lets you safely override client-provided skills with authoritative server versions — for example, preventing users from injecting their own `brand-voice` skill that conflicts with your official one. + +--- + +## 8. Type Reference + +```typescript +type SkillStrategy = "eager" | "auto" | "manual"; + +type SkillSource = + | { type: "inline"; content: string } + | { type: "url"; url: string } + | { type: "file"; path: string }; + +interface SkillDefinition { + name: string; + description: string; + source: SkillSource; + strategy?: SkillStrategy; // default: "auto" + version?: string; +} + +interface ResolvedSkill extends SkillDefinition { + content: string; // Fully resolved content string +} + +interface ClientInlineSkill { + name: string; + description: string; + content: string; + strategy?: SkillStrategy; +} + +interface SkillDiagnostic { + type: "collision"; + name: string; + winner: "server-dir" | "remote-url" | "client-inline"; + loser: "server-dir" | "remote-url" | "client-inline"; +} + +interface LoadSkillResult { + name: string; + description: string; + strategy: SkillStrategy; + content: string; + source: "server-dir" | "remote-url" | "client-inline"; +} + +interface LoadSkillError { + error: string; +} +``` + +--- + +## 9. Full Example + +### Project structure + +``` +skills/ +├── brand-voice.md # eager — always active +└── sql-expert.md # auto — loaded on demand +``` + +```markdown +## + +name: brand-voice +description: Acme Corp tone and style guide +strategy: eager + +--- + +Always respond in a friendly, professional tone. +Refer to the product as "Acme" (not "the platform"). +Use metric units. Avoid passive voice. +``` + +```markdown +## + +name: sql-expert +description: Writes and explains SQL queries for our PostgreSQL schema +strategy: auto + +--- + +## SQL Expert + +Our database uses PostgreSQL 15. Key tables: + +- users(id, email, plan, created_at) +- orders(id, user_id, total, status, created_at) +- products(id, name, price, stock) + +When writing queries: + +1. Always use parameterized queries ($1, $2...) +2. Add LIMIT clauses to SELECT queries +3. Explain the query in plain English after writing it +``` + +### API route + +```typescript +// app/api/chat/route.ts +import path from "path"; +import { loadSkills } from "@yourgpt/copilot-sdk/server"; +import { streamText } from "ai"; +import { anthropic } from "@ai-sdk/anthropic"; + +export async function POST(req: Request) { + const { messages, __skills } = await req.json(); + + const { buildSystemPrompt, tools } = await loadSkills({ + dir: path.join(process.cwd(), "skills"), + clientSkills: __skills ?? [], + }); + + return streamText({ + model: anthropic("claude-sonnet-4-6"), + system: buildSystemPrompt("You are a helpful assistant for Acme Corp."), + messages, + tools, + }).toDataStreamResponse(); +} +``` + +### React app + +```tsx +// app/layout.tsx +import { CopilotProvider } from "@yourgpt/copilot-sdk-react"; +import { SkillProvider, defineSkill } from "@yourgpt/copilot-sdk-react"; + +// Extra client-only skill (e.g. page-specific context) +const checkoutSkill = defineSkill({ + name: "checkout-helper", + description: "Helps with the checkout flow", + strategy: "auto", + source: { type: "inline", content: "When helping with checkout..." }, +}); + +export default function Layout({ children }) { + return ( + + {children} + + ); +} +``` + +```tsx +// app/dashboard/page.tsx — add a page-scoped skill +import { useSkill, useSkillStatus } from "@yourgpt/copilot-sdk-react"; + +export default function DashboardPage() { + useSkill({ + name: "dashboard-context", + description: "Knows about the current dashboard state", + strategy: "eager", + source: { + type: "inline", + content: + "The user is viewing the analytics dashboard. Current date range: last 30 days.", + }, + }); + + const { count } = useSkillStatus(); + + return ( +
+

{count} skills active

+ +
+ ); +} +``` diff --git a/apps/docs/alpha-docs/STORAGE-ADAPTER.md b/apps/docs/alpha-docs/STORAGE-ADAPTER.md new file mode 100644 index 0000000..d17b9e5 --- /dev/null +++ b/apps/docs/alpha-docs/STORAGE-ADAPTER.md @@ -0,0 +1,166 @@ +# Storage Adapter (Alpha) + +> **Status**: Alpha — API may change. Available since `@yourgpt/llm-sdk@1.5.0-alpha`. + +## Quick Start + +```ts +import { createRuntime } from "@yourgpt/llm-sdk"; +import { createAnthropic } from "@yourgpt/llm-sdk/anthropic"; +import { createYourGPT } from "@yourgpt/llm-sdk/yourgpt"; + +// 1. Create adapter (server-side only) +const yourgpt = createYourGPT({ + apiKey: process.env.YOURGPT_API_KEY, + widgetUid: process.env.YOURGPT_WIDGET_UID, + // endpoint defaults to https://api.yourgpt.ai + // Override for dev: endpoint: 'http://localhost:3000' +}); + +// 2. Plug into runtime +const runtime = createRuntime({ + provider: createAnthropic({ apiKey: process.env.ANTHROPIC_API_KEY }), + model: "claude-haiku-4-5", + storage: yourgpt, // ← enables automatic persistence +}); + +// 3. Endpoints are one-liners +app.post("/api/copilot/chat", async (req, res) => { + const result = await runtime.chat(req.body); + res.json(result); // includes threadId +}); + +app.post("/api/copilot/stream", async (req, res) => { + await runtime.stream(req.body).pipeToResponse(res); +}); + +// 4. Optional: file upload +app.post("/api/copilot/upload", async (req, res) => { + const result = await yourgpt.uploadFile(req.body); + res.json(result); +}); +``` + +## What Happens Automatically + +| Event | Without storage | With storage | +| --------------------------- | ----------------------- | ---------------------------------------------- | +| First message (no threadId) | Uses local thread ID | Creates session via API, returns real threadId | +| User sends message | Just forwarded to LLM | Saved to session, then forwarded | +| LLM responds | Just returned to client | Saved to session, then returned | +| Tool calls + results | Not persisted | Saved as tool messages | +| File attachment | Base64 in payload | Uploaded to storage, URL in payload | +| Session creation fails | N/A | Fallback local ID, chat continues | + +## Configuration + +### `createYourGPT(config)` + +| Option | Required | Default | Description | +| ----------- | -------- | ------------------------ | -------------------------- | +| `apiKey` | Yes | — | YourGPT API key | +| `widgetUid` | Yes | — | Widget UID (project scope) | +| `endpoint` | No | `https://api.yourgpt.ai` | API base URL | + +### `createRuntime({ storage })` + +The `storage` option accepts any `StorageAdapter`. The runtime calls: + +- `storage.createSession()` — when request has no threadId +- `storage.saveMessages()` — before + after LLM call +- `storage.uploadFile()` — not called by runtime (used via upload endpoint) + +### Environment Variables (Server) + +```env +# Required +YOURGPT_API_KEY=apk-your-key-here +YOURGPT_WIDGET_UID=your-widget-uid-here + +# Optional (defaults to production) +YOURGPT_API_ENDPOINT=https://api.yourgpt.ai + +# LLM provider +ANTHROPIC_API_KEY=sk-ant-... +``` + +## Client Setup + +No special client configuration needed for sessions. The client SDK automatically: + +1. Reads `threadId` from server response +2. Uses it for subsequent requests +3. Uses it as the local thread ID (single ID system) + +### File uploads (client) + +The `upload` prop handles all upload modes — string, object, or function: + +```tsx +// Simple — just a URL: + + +// With auth headers: + ({ Authorization: `Bearer ${token}` }), +}} /> + +// Full custom: + { + const url = await myS3Upload(file); + return { type: 'image', url, mimeType: file.type, filename: file.name }; +}} /> +``` + +## Custom StorageAdapter + +Implement the interface for any backend: + +```ts +import type { StorageAdapter } from "@yourgpt/llm-sdk"; + +const myStorage: StorageAdapter = { + async createSession(data) { + // Your DB call + return { id: "session-123" }; + }, + async saveMessages(sessionId, messages) { + // Your DB call + }, + // Optional: + async uploadFile(file) { + // Your storage call + return { url: "https://..." }; + }, +}; + +const runtime = createRuntime({ provider, model, storage: myStorage }); +``` + +## Error Handling + +- `createSession` failure → Fallback local ID, storage skipped, chat works +- `saveMessages` failure → Logged, chat continues (fire-and-forget) +- `uploadFile` failure → Error returned to client (4xx/5xx) +- All errors are logged with `[Runtime]` prefix + +### `onError` callback + +```ts +const yourgpt = createYourGPT({ + apiKey, + widgetUid, + onError: (error, operation, params) => { + // operation: "createSession" | "saveMessages" | "uploadFile" + // params: { sessionId, messageCount, roles, filename, mimeType, ... } + logger.error(`[YourGPT:${operation}]`, error.message, params); + }, +}); +``` + +## Alpha Notes + +- The `endpoint` option in `createYourGPT` will become internal in GA (defaults to production API) +- `getSessions()` and `getMessages()` on StorageAdapter are reserved for future thread sync +- File upload uses pre-signed URLs via `/copilot-sdk/getSignedUrl` — contract may change diff --git a/apps/docs/alpha-docs/message-history-compaction.md b/apps/docs/alpha-docs/message-history-compaction.md new file mode 100644 index 0000000..bd38e99 --- /dev/null +++ b/apps/docs/alpha-docs/message-history-compaction.md @@ -0,0 +1,72 @@ +# Message History & Compaction + +Automatic context window management. Keeps long conversations within token limits without losing important history. + +## Strategies + +| Strategy | What it does | +| ----------------- | -------------------------------------------------------- | +| `none` (default) | No compaction — current behavior, zero breaking changes | +| `sliding-window` | Drop oldest messages when over token budget | +| `selective-prune` | Drop tool results from old turns, keep summaries | +| `summary-buffer` | Summarize old turns into a rolling summary (recommended) | + +## Usage + +```tsx + console.log("Compacted", e), + onTokenUsage: (u) => console.log(`${u.percentage * 100}% full`), + }} +> +``` + +## How It Works + +**Architecture**: `MessageHistoryBridge` (mounted inside `CopilotProvider`) wires `useMessageHistory` into `AbstractChat.buildRequest()` via `setRequestMessageTransform`. + +``` +User sends message + → AbstractChat.buildRequest() calls requestMessageTransform(allMessages) + → Transform splits: historyMessages (before last user msg) + currentTurn (from last user msg) + → buildSummaryBufferContext() compacts historyMessages only + → currentTurn always kept verbatim (no broken tool call/result pairs) + → Compacted history + currentTurn sent to API + → In-memory store unchanged (full history kept for display) +``` + +**Auto-compaction**: When `tokenUsage.isApproaching = true` (threshold crossed), `runCompaction` summarizes old messages and updates `compactionState.rollingSummary`. The transform picks up the new summary automatically on next request. + +**UI indicators**: When compaction triggers, a system message (`type: "compaction-marker"`) is added to chat: + +- Loading: `"Compacting conversation…"` (while summarizing) +- Done: `"Conversation compacted — context window refreshed"` (permanent divider) + +## Token Counting + +Token usage is computed from the **full display history** (`toLLMMessages(displayMessages)`), not the already-pruned output. This ensures the threshold reflects actual accumulation. + +```tsx +// Access token usage directly +const { tokenUsage, compactionState } = useMessageHistory(); +// tokenUsage.current, .max, .percentage, .isApproaching +// compactionState.compactionCount, .rollingSummary, .totalTokensSaved +``` + +## Manual Compaction + +```tsx +const { compactSession } = useMessageHistory(); + +// Trigger manually with optional instructions +await compactSession("Focus on user preferences and key decisions"); +``` diff --git a/apps/docs/alpha-docs/skills-system.md b/apps/docs/alpha-docs/skills-system.md new file mode 100644 index 0000000..033e918 --- /dev/null +++ b/apps/docs/alpha-docs/skills-system.md @@ -0,0 +1,63 @@ +# Skills System + +On-demand instruction sets the AI can load at runtime — keeps the system prompt lean. + +## Two Strategies + +| Strategy | Behavior | +| -------- | ------------------------------------------------------- | +| `eager` | Content injected into AI context immediately on mount | +| `auto` | Listed in catalog; AI calls `load_skill(name)` to fetch | + +## API + +```tsx +import { defineSkill, SkillProvider, useSkill } from "@yourgpt/copilot-sdk/react"; + +// 1. Define a skill +const diagnosticSkill = defineSkill({ + name: "diagnostic", + description: "Troubleshoot chatbot issues: errors, limits, integrations", + strategy: "eager", // always in context + source: { type: "inline", content: "..." }, +}); + +const trainingSkill = defineSkill({ + name: "training", + description: "Manage knowledge base: add FAQs, URLs, files", + strategy: "auto", // AI loads on demand + source: { type: "inline", content: "..." }, +}); + +// 2. Provide at app level + + + {children} + + + +// 3. Register per-route (auto skills only active on that route) +function TrainingLayout() { + useSkill(trainingSkill); // registers on mount, unregisters on unmount + return ; +} +``` + +## How It Works + +- **Eager**: `SkillProvider` renders an `EagerSkillInjector` which calls `useAIContext` with the skill content. Appears in the AI context as `__skill_eager__:`. +- **Auto**: A `load_skill` tool is registered. The catalog context lists available auto skills. AI calls `load_skill({ name })` → receives full content in tool result. +- **Ref counting**: Multiple `useSkill` calls for the same skill are safe — the registry tracks ref counts and only unregisters when count hits 0. + +## Runtime Behavior + +``` +User navigates to /training + → useSkill(trainingSkill) mounts + → Catalog updates: "Available skills:\n- training: Manage knowledge base..." + → AI can now call load_skill({ name: "training" }) + +User navigates away + → useSkill cleanup fires + → training removed from catalog +``` diff --git a/apps/docs/content/docs/api-reference/react/components.mdx b/apps/docs/content/docs/api-reference/react/components.mdx index 02e3f82..84bcb1e 100644 --- a/apps/docs/content/docs/api-reference/react/components.mdx +++ b/apps/docs/content/docs/api-reference/react/components.mdx @@ -42,6 +42,7 @@ function App() { | `initialMessages` | `Message[]` | No | Initial messages to populate the chat | | `onMessagesChange` | `(messages: Message[]) => void` | No | Callback when messages change | | `onError` | `(error: Error) => void` | No | Callback when an error occurs | +| `parseError` | `(status: number, body: unknown) => string \| null` | No | Extract message from custom server error formats | | `streaming` | `boolean` | No | Enable/disable streaming (default: true) | | `debug` | `boolean` | No | Enable debug logging | @@ -136,3 +137,4 @@ function ChatApp() { ``` + diff --git a/apps/docs/content/docs/multimodal.mdx b/apps/docs/content/docs/attachments.mdx similarity index 80% rename from apps/docs/content/docs/multimodal.mdx rename to apps/docs/content/docs/attachments.mdx index c038646..4edd12d 100644 --- a/apps/docs/content/docs/multimodal.mdx +++ b/apps/docs/content/docs/attachments.mdx @@ -1,7 +1,7 @@ --- -title: Multimodal -description: Send images, files, and documents with messages -icon: Image +title: Attachments +description: Send images, PDFs, and files alongside chat messages +icon: Paperclip --- import { Callout } from 'fumadocs-ui/components/callout'; @@ -103,19 +103,93 @@ interface MessageAttachment { | `attachmentsEnabled` | `boolean` | `false` | Enable attachment button | | `maxFileSize` | `number` | `5MB` | Maximum file size in bytes | | `allowedFileTypes` | `string[]` | All | Allowed MIME types or extensions | -| `processAttachment` | `function` | - | Custom upload handler | +| `upload` | `string \| object \| function` | - | Upload handler (see below) | --- -## Custom Upload Handler +## The `upload` Prop -Upload to your own storage: +One prop, three modes — from simple to full control: + +```tsx +// 1. URL string — server handles the upload + + +// 2. Object — URL + custom headers/body + ({ Authorization: `Bearer ${token}` }), + body: { projectId: "abc" }, +}} /> + +// 3. Function — full custom logic + { + const url = await myS3Upload(file); + return { type: 'image', url, mimeType: file.type, filename: file.name }; +}} /> +``` + +Without `upload`, files are embedded as base64 in the message body (works but heavier payload). + +--- + +## Server Upload (Recommended) + +Point `upload` to your server endpoint. Files are uploaded there, and the returned URL is sent with the message. ```tsx { - // Upload to S3, Cloudinary, etc. + upload="/api/copilot/upload" +/> +``` + +### With YourGPT + +If your server uses the YourGPT adapter, the upload endpoint uses the same credentials: + +```ts title="server.ts" +import { createYourGPT } from '@yourgpt/llm-sdk/yourgpt'; + +const yourgpt = createYourGPT({ apiKey, widgetUid }); + +app.post('/api/copilot/upload', async (req, res) => { + const result = await yourgpt.uploadFile(req.body); + res.json(result); +}); +``` + +### With Custom Storage + +```ts title="server.ts" +app.post('/api/copilot/upload', async (req, res) => { + const { data, mimeType, filename } = req.body; + const url = await myStorage.upload(Buffer.from(data, 'base64'), mimeType); + res.json({ url }); +}); +``` + +### With Auth Headers + +```tsx + ({ Authorization: `Bearer ${getToken()}` }), + }} +/> +``` + +--- + +## Custom Upload Function + +For complete control — handle the file however you want: + +```tsx + { const formData = new FormData(); formData.append('file', file); @@ -123,12 +197,11 @@ Upload to your own storage: method: 'POST', body: formData, }); - const { url } = await response.json(); return { type: 'image', - url, // Cloud URL instead of base64 + url, mimeType: file.type, filename: file.name, }; @@ -137,7 +210,7 @@ Upload to your own storage: ``` -Cloud storage is recommended for production. Base64 embedding works but increases message size. +Cloud storage is recommended for production. Base64 embedding works but increases message size significantly. --- diff --git a/apps/docs/content/docs/chat-history.mdx b/apps/docs/content/docs/chat-history.mdx index 9588b84..9061439 100644 --- a/apps/docs/content/docs/chat-history.mdx +++ b/apps/docs/content/docs/chat-history.mdx @@ -25,6 +25,25 @@ For simple browser-level persistence without server setup: Data is stored in localStorage (~5MB limit, single device only). +### Storage quota and auto-eviction + +When the browser's localStorage quota is exceeded (typically after many long conversations), the SDK automatically evicts the oldest threads to make room for new ones. The most recent threads are always preserved. + +This eviction happens silently — users won't see an error. If you need to monitor or control this behavior, you can listen to the `onStorageEviction` callback: + +```tsx + { + console.log('Evicted old threads:', evictedThreadIds); + }} +/> +``` + + + Auto-eviction only applies to browser localStorage. Server persistence has no such limit — use it when conversation history must be retained long-term. + + --- ## Server Persistence diff --git a/apps/docs/content/docs/chat/branching.mdx b/apps/docs/content/docs/chat/branching.mdx new file mode 100644 index 0000000..9c0f3d5 --- /dev/null +++ b/apps/docs/content/docs/chat/branching.mdx @@ -0,0 +1,210 @@ +--- +title: Conversation Branching +description: Edit messages to create parallel conversation paths, just like ChatGPT and Claude.ai +icon: GitBranch +--- + +import { Callout } from 'fumadocs-ui/components/callout'; +import { Tab, Tabs } from 'fumadocs-ui/components/tabs'; + + +**Beta** — This feature is in **alpha**. APIs may change before stable release. + + +Edit any user message to create a parallel conversation path, preserving the original. Navigate between variants with `← N/M →` — the same UX as ChatGPT, Claude.ai, and Gemini. + +--- + +## Zero-Config Usage + +If you use ``, branching is **already active**. No code changes needed. + +- Edit button (✏) appears on hover over any user message +- `← 1/2 →` navigator appears below user messages when variants exist +- Regenerate creates a new branch instead of overwriting + +```tsx +// Nothing to add — branching works out of the box + +``` + +--- + +## New APIs + +### `useCopilot()` / `useCopilotProvider` + +```typescript +const { + switchBranch, // (messageId: string) => void + getBranchInfo, // (messageId: string) => BranchInfo | null + editMessage, // (messageId: string, newContent: string) => Promise + hasBranches, // boolean — true if any fork exists + getAllMessages, // () => UIMessage[] — all branches, not just visible path +} = useCopilot(); +``` + +### `BranchInfo` type + +```typescript +interface BranchInfo { + siblingIndex: number; // 0-based — which variant this is + totalSiblings: number; // how many variants exist at this fork + siblingIds: string[]; // ordered oldest-first + hasPrevious: boolean; + hasNext: boolean; +} +``` + +### `BranchNavigator` component + +Standalone navigator for custom message renderers: + +```tsx +import { BranchNavigator } from "@yourgpt/copilot-sdk/ui"; + + switchBranch(info.siblingIds[info.siblingIndex - 1])} + onNext={() => switchBranch(info.siblingIds[info.siblingIndex + 1])} +/> +``` + +### `MessageTree` (framework-agnostic) + +```typescript +import { MessageTree, type BranchInfo } from "@yourgpt/copilot-sdk"; + +const tree = new MessageTree(messages); +tree.getVisibleMessages(); // active path only (sent to AI) +tree.getAllMessages(); // all branches (for persistence) +tree.getBranchInfo(messageId); // BranchInfo | null +tree.switchBranch(messageId); +tree.hasBranches; // boolean +``` + +--- + +## Manual Wiring (`` users) + +Wire the three props from `useCopilot()`: + +```tsx +function MyChat() { + const { switchBranch, getBranchInfo, editMessage } = useCopilot(); + + return ( + + ); +} +``` + +--- + +## Custom Message Renderers + +Use `getBranchInfo` + `BranchNavigator` in your own message components: + +```tsx +function MyMessage({ message }) { + const { switchBranch, getBranchInfo } = useCopilot(); + const info = message.role === "user" ? getBranchInfo(message.id) : null; + + return ( +
+

{message.content}

+ {info && ( + switchBranch(info.siblingIds[info.siblingIndex - 1])} + onNext={() => switchBranch(info.siblingIds[info.siblingIndex + 1])} + /> + )} +
+ ); +} +``` + +--- + +## Programmatic Branching + +```typescript +// Edit a message (creates new branch from same parent) +await editMessage("msg-abc", "Updated question text"); + +// Navigate between variants +switchBranch("msg-xyz"); + +// Check if branches exist +if (hasBranches) { + const info = getBranchInfo("msg-abc"); + console.log(info.totalSiblings, info.siblingIndex); +} + +// Persist all branches (not just visible path) +const allMessages = getAllMessages(); +await saveToServer(allMessages); +``` + +--- + +## Persistence + +### New DB columns (optional) + +Two new optional columns on your messages table: + +```sql +ALTER TABLE messages + ADD COLUMN parent_id TEXT REFERENCES messages(id), + ADD COLUMN children_ids JSONB DEFAULT '[]'; +``` + + +These columns are **optional**. Existing rows without them are auto-migrated to a linear tree on load — no data loss, no required migration script. + + +### What gets saved + +When `onMessagesChange` fires, the payload now contains **all messages across all branches**. Each message carries: + +```json +{ + "id": "msg-abc", + "role": "assistant", + "content": "...", + "parent_id": "msg-xyz", + "children_ids": [] +} +``` + +### Upsert strategy (recommended) + +```typescript +// ✅ Safe for branching — upsert by ID +await db.messages.upsert({ id: msg.id, ...msg }); + +// ⚠️ Loses inactive branches +await db.threads.update({ messages: visibleMessages }); +``` + +--- + +## Breaking Changes + +**None.** All new fields and methods are optional. Existing usage is untouched. + +| Scenario | Behavior | +|----------|----------| +| Messages with no `parentId` | Falls back to insertion order (legacy linear) | +| `regenerate()` with no args | Identical to before | +| `sendMessage()` with no options | Identical to before | +| `onMessagesChange` consumers | Payload now includes all branches — shape unchanged | diff --git a/apps/docs/content/docs/chat.mdx b/apps/docs/content/docs/chat/index.mdx similarity index 82% rename from apps/docs/content/docs/chat.mdx rename to apps/docs/content/docs/chat/index.mdx index d244973..2d68742 100644 --- a/apps/docs/content/docs/chat.mdx +++ b/apps/docs/content/docs/chat/index.mdx @@ -207,6 +207,35 @@ function CustomChat() { --- +## Error Handling + +When the server returns a non-2xx response, the SDK removes the loading placeholder and surfaces the error via `state.error`. The built-in `CopilotChat` shows a dismissible banner — errors are **never written into message history**. + +Handle errors via `CopilotProvider` props: + +```tsx + toast.error(error.message)} + parseError={(status, body: any) => { + // Return null to fall back to SDK default (body.message or body.error) + return body?.errors?.[0]?.message ?? body?.detail ?? null; + }} +> +``` + +Or read error state directly from `useCopilot()`: + +```tsx +const { error, status } = useCopilot(); +// error → null when idle, Error object on failure +// status → 'ready' | 'streaming' | 'submitted' | 'error' +``` + +The error clears automatically when the user sends the next message. + +--- + ## Styling The chat component uses Tailwind CSS. Override styles with className: @@ -228,4 +257,5 @@ The chat component uses Tailwind CSS. Override styles with className: ## Next Steps - [Tools](/docs/tools) - Add AI capabilities -- [Custom Tools](/docs/custom-tools) - Build your own tools +- [Branching](/docs/chat/branching) - Conversation branching like ChatGPT +- [Message Actions](/docs/chat/message-actions) - Add copy, edit, feedback buttons diff --git a/apps/docs/content/docs/chat/message-actions.mdx b/apps/docs/content/docs/chat/message-actions.mdx new file mode 100644 index 0000000..2ee635e --- /dev/null +++ b/apps/docs/content/docs/chat/message-actions.mdx @@ -0,0 +1,158 @@ +--- +title: Message Actions +description: Add floating copy, edit, feedback, and custom action buttons to chat messages +icon: MousePointerClick +--- + +import { Callout } from 'fumadocs-ui/components/callout'; + + +**Beta** — This feature is in **alpha**. APIs may change before stable release. + + +A compound component API for registering floating action buttons on chat messages — copy, edit, feedback, or fully custom actions. Actions appear on hover, floating below the message bubble. Declarative, role-based, fully composable. + +--- + +## Quick Start + +```tsx + + + + sendFeedback({ messageId: message.id, type })} + /> + + + + + + +``` + + +If no `` children are declared, nothing changes — existing chat UI looks and behaves identically. + + +--- + +## Compound Components + +| Component | Description | +|-----------|-------------| +| `CopilotChat.MessageActions` | Registers actions for a role (`user` or `assistant`) | +| `CopilotChat.CopyAction` | Copy message to clipboard (with ✓ feedback) | +| `CopilotChat.EditAction` | Inline edit for user messages (wired to branching) | +| `CopilotChat.FeedbackAction` | Thumbs up / down | +| `CopilotChat.Action` | Fully custom action button | + +--- + +## Props Reference + +```tsx +// MessageActions +role: "user" | "assistant" + +// CopyAction +tooltip?: string +className?: string + +// EditAction +tooltip?: string +className?: string + +// FeedbackAction +onFeedback?: (message: ChatMessage, type: "helpful" | "not-helpful") => void +tooltip?: string +className?: string + +// Action (custom) +id?: string +icon: ReactNode +tooltip: string +onClick: (props: { message: ChatMessage }) => void +hidden?: boolean | ((props: { message: ChatMessage }) => boolean) +className?: string +``` + +--- + +## Examples + +### Copy + Feedback on assistant + +```tsx + + + + { + sendFeedback({ messageId: message.id, type }); + }} + /> + + +``` + +### Custom action + +```tsx + + + + } + tooltip="Share" + onClick={({ message }) => share(message.content)} + /> + + +``` + +### Conditional action (hide based on message content) + +```tsx + + + } + tooltip="Report" + hidden={({ message }) => !message.content} + onClick={({ message }) => report(message.id)} + /> + + +``` + +### Full setup — both roles + +```tsx + + + + log(msg.id, type)} /> + } + tooltip="Save" + onClick={({ message }) => save(message)} + /> + + + + + } + tooltip="Delete" + onClick={({ message }) => deleteMessage(message.id)} + /> + + +``` + +--- + +## Breaking Changes + +**None.** Purely additive. If no `MessageActions` children are declared, the chat UI is identical to before. diff --git a/apps/docs/content/docs/chat/meta.json b/apps/docs/content/docs/chat/meta.json new file mode 100644 index 0000000..edc9c09 --- /dev/null +++ b/apps/docs/content/docs/chat/meta.json @@ -0,0 +1,5 @@ +{ + "title": "Chat", + "icon": "MessageSquare", + "pages": ["index", "branching", "message-actions"] +} diff --git a/apps/docs/content/docs/context/compaction.mdx b/apps/docs/content/docs/context/compaction.mdx new file mode 100644 index 0000000..e548058 --- /dev/null +++ b/apps/docs/content/docs/context/compaction.mdx @@ -0,0 +1,204 @@ +--- +title: Message History Compaction +description: Auto-summarize old messages to keep long conversations within the AI's context window +--- + +import { Callout } from 'fumadocs-ui/components/callout'; +import { Tab, Tabs } from 'fumadocs-ui/components/tabs'; + + +**Experimental** — This feature is under active development. APIs may change before stable release. + + +Keep long conversations alive without hitting token limits. The SDK maintains two parallel views of message history — a full display layer for the UI, and a compacted layer sent to the model. + +--- + +## How It Works + +Every conversation maintains two parallel views: + +| Layer | Type | Purpose | +|-------|------|---------| +| **Display layer** | `DisplayMessage[]` | Full immutable history. Rendered in the UI. Never shrinks. | +| **LLM context layer** | `LLMMessage[]` | Compacted/pruned form sent to the model on each request. | + +When compaction fires, a `CompactionMarker` is injected into the display layer so users can see where summarization happened — but the full history is never deleted. + +--- + +## useMessageHistory + +```typescript +import { useMessageHistory } from "@yourgpt/copilot-sdk/react"; + +function MyChat() { + const { + displayMessages, // Full UI history + llmMessages, // Compacted LLM context + tokenUsage, // Live token estimate + isCompacting, // true while auto-compaction runs + compactionState, // Metadata & rolling summary + compactSession, // Manual trigger + addToWorkingMemory, + clearWorkingMemory, + resetSession, + } = useMessageHistory({ + strategy: "summary-buffer", + maxContextTokens: 128000, + compactionThreshold: 0.75, + compactionUrl: "/api/compact", + persistSession: true, + }); +} +``` + +--- + +## Compaction Strategies + +### `"none"` (default) + +No compaction. Zero-config, 100% backward-compatible. + +```typescript +useMessageHistory({ strategy: "none" }); +``` + +### `"sliding-window"` + +Keeps only the most recent N tokens of history. Oldest messages are dropped when the token budget is exceeded. + +```typescript +useMessageHistory({ + strategy: "sliding-window", + maxContextTokens: 128000, + reserveForResponse: 4096, + recentBuffer: 10, // Always keep at least 10 recent messages + toolResultMaxChars: 10000, // Truncate large tool results +}); +``` + +### `"selective-prune"` + +Removes old tool-result messages while keeping the user/assistant conversation skeleton. Lighter than sliding-window — no token counting required. + +```typescript +useMessageHistory({ + strategy: "selective-prune", + recentBuffer: 10, +}); +``` + +### `"summary-buffer"` + +Summarizes old messages into a rolling summary when usage exceeds `compactionThreshold`. The summary is injected as a system message. Requires a `/api/compact` endpoint. + +```typescript +useMessageHistory({ + strategy: "summary-buffer", + compactionThreshold: 0.75, // Compact at 75% of maxContextTokens + compactionUrl: "/api/compact", + recentBuffer: 10, + onCompaction: (event) => { + console.log(`Compacted ${event.messagesSummarized} messages, saved ~${event.tokensSaved} tokens`); + }, +}); +``` + +Custom summarizer (skip the HTTP round-trip): + +```typescript +useMessageHistory({ + strategy: "summary-buffer", + summarizer: async (messages) => { + const res = await myLLM.summarize(messages); + return res.text; + }, +}); +``` + +--- + +## Provider-level Config + +Set defaults once in ``: + +```tsx + + + +``` + +--- + +## Working Memory + +Pin facts that survive all future compactions: + +```typescript +const { addToWorkingMemory, clearWorkingMemory } = useMessageHistory({ ... }); + +// Survives compaction +addToWorkingMemory("User is on the Pro plan. Account ID: acct_123"); + +// Remove all pinned facts +clearWorkingMemory(); +``` + +--- + +## Config Reference + +```typescript +interface MessageHistoryConfig { + strategy?: "none" | "sliding-window" | "summary-buffer" | "selective-prune"; + maxContextTokens?: number; // default: 128000 + reserveForResponse?: number; // default: 4096 + compactionThreshold?: number; // default: 0.75 + recentBuffer?: number; // default: 10 + toolResultMaxChars?: number; // default: 10000 (0 = no cap) + compactionUrl?: string; // required for summary-buffer + persistSession?: boolean; // default: false + storageKey?: string; // default: "copilot-session" + onCompaction?: (event: CompactionEvent) => void; + onTokenUsage?: (usage: TokenUsage) => void; +} +``` + +--- + +## Server: `/api/compact` Endpoint + +The `compactSession` utility powers the compaction endpoint. It calls Claude (defaults to `claude-haiku-4-5`) to produce a structured summary. + +```typescript +// app/api/compact/route.ts +import { compactSession } from "@yourgpt/copilot-sdk/server"; + +export async function POST(req: Request) { + const { messages, existingSummary, workingMemory } = await req.json(); + + const { summary } = await compactSession({ + messages, + existingSummary, // For rolling summaries on subsequent compactions + workingMemory, // User-pinned facts (addToWorkingMemory) + model: "claude-haiku-4-5", + maxSummaryTokens: 1024, + apiKey: process.env.ANTHROPIC_API_KEY, + }); + + return Response.json({ summary }); +} +``` + + +The summary preserves: user goals, technical decisions, tool call outcomes, errors and resolutions, pending tasks. + diff --git a/apps/docs/content/docs/smart-ai-context.mdx b/apps/docs/content/docs/context/index.mdx similarity index 51% rename from apps/docs/content/docs/smart-ai-context.mdx rename to apps/docs/content/docs/context/index.mdx index 9329fd4..2e5df51 100644 --- a/apps/docs/content/docs/smart-ai-context.mdx +++ b/apps/docs/content/docs/context/index.mdx @@ -1,25 +1,27 @@ --- -title: Smart AI Context -description: Make AI aware of your application state +title: Context Management +description: Make AI aware of your application state and manage the context window icon: Lightbulb --- import { Callout } from 'fumadocs-ui/components/callout'; -Give the AI awareness of your application state so it can provide relevant, contextual responses. +Give the AI awareness of your application state so it can provide relevant, contextual responses. The SDK also provides advanced context window management for long conversations. --- -## Overview +## Application Context -Without context, AI doesn't know what's happening in your app: +Inject your app state into the AI's context so it can answer questions about what's happening in your app. + +### Without context ``` User: "Is this in stock?" AI: "I don't know what product you're referring to." ``` -With context: +### With context ```tsx useAIContext({ @@ -125,13 +127,9 @@ Create parent-child relationships for complex data: ```tsx function TeamDashboard({ team }) { - // Parent context const teamContextId = useAIContext({ key: 'team', - data: { - name: team.name, - memberCount: team.members.length, - }, + data: { name: team.name, memberCount: team.members.length }, description: 'The team being viewed', }); @@ -139,27 +137,18 @@ function TeamDashboard({ team }) {

{team.name}

{team.members.map(member => ( - + ))}
); } function MemberCard({ member, parentId }) { - // Child context - linked to team useAIContext({ key: `member-${member.id}`, - data: { - name: member.name, - role: member.role, - tasks: member.tasks, - }, + data: { name: member.name, role: member.role, tasks: member.tasks }, description: `Team member: ${member.name}`, - parentId, // Links to team context + parentId, }); return
{member.name}
; @@ -167,7 +156,7 @@ function MemberCard({ member, parentId }) { ``` -Hierarchical contexts help AI understand relationships. When user asks about "John's tasks", AI knows John is part of the team context. +Hierarchical contexts help AI understand relationships. When a user asks about "John's tasks", AI knows John is part of the team context. --- @@ -207,7 +196,6 @@ Contexts are automatically removed when components unmount: ```tsx function ConditionalContext({ showDetails }) { - // This context only exists when showDetails is true if (showDetails) { return ; } @@ -228,66 +216,8 @@ function DetailedContext() { ## What to Include in Context -### Good Context Data - ```tsx -// User state -{ - name: 'John', - plan: 'pro', - permissions: ['read', 'write'], -} - -// Current view -{ - page: 'checkout', - step: 2, - items: cart.items, -} - -// Form state -{ - formType: 'contact', - fields: Object.keys(formData), - hasErrors: errors.length > 0, -} -``` - -### Avoid Including - -```tsx -// Too much data (token waste) -{ - allProducts: products, // Could be thousands - fullUserHistory: history, // Unnecessary detail -} - -// Sensitive data -{ - password: user.password, // Never! - creditCard: payment.card, // Never! - apiKeys: config.keys, // Never! -} - -// Irrelevant data -{ - buildHash: process.env.BUILD_ID, // AI doesn't need this - internalIds: uuids, // No user value -} -``` - - -Never include sensitive data in AI context. The context is sent to the LLM provider. - - ---- - -## Best Practices - -### 1. Keep Context Focused - -```tsx -// ✅ Good - relevant, scoped data +// ✅ Good — relevant, scoped data useAIContext({ key: 'order-details', data: { @@ -297,113 +227,25 @@ useAIContext({ total: order.total, }, }); - -// ❌ Bad - too much unnecessary detail -useAIContext({ - key: 'order', - data: order, // Full order object with internal fields -}); -``` - -### 2. Use Descriptive Keys - -```tsx -// ✅ Good -useAIContext({ key: 'current-product', data }); -useAIContext({ key: 'user-preferences', data }); -useAIContext({ key: 'checkout-cart', data }); - -// ❌ Bad -useAIContext({ key: 'data1', data }); -useAIContext({ key: 'ctx', data }); -``` - -### 3. Add Helpful Descriptions - -```tsx -useAIContext({ - key: 'filters', - data: { category: 'electronics', priceRange: [0, 500] }, - description: 'Active product filters the user has applied', -}); ``` -### 4. Structure for AI Understanding - -```tsx -// ✅ Good - clear structure -useAIContext({ - key: 'selected-item', - data: { - name: 'Widget Pro', - canPurchase: true, - reasonIfNot: null, - }, -}); - -// ❌ Bad - ambiguous -useAIContext({ - key: 'item', - data: { - n: 'Widget Pro', - p: true, - }, -}); -``` + +Never include sensitive data (passwords, API keys, credit cards) in AI context. The context is sent to the LLM provider. + --- -## Full Example +## Advanced Context Window Management -```tsx -function EcommercePage({ product, user, cart }) { - // Global contexts (in layout/provider) - useAIContexts([ - { - key: 'user', - data: { - name: user.name, - memberSince: user.createdAt, - tier: user.loyaltyTier, - }, - description: 'Logged-in user', - }, - { - key: 'cart', - data: { - itemCount: cart.items.length, - subtotal: cart.subtotal, - hasDiscount: cart.discount > 0, - }, - description: 'Shopping cart summary', - }, - ]); - - // Page-specific context - useAIContext({ - key: 'viewing-product', - data: { - name: product.name, - price: product.price, - rating: product.rating, - inStock: product.stock > 0, - stockLevel: product.stock > 10 ? 'high' : product.stock > 0 ? 'low' : 'none', - }, - description: 'Product the user is currently viewing', - }); - - return ; -} -``` +For long conversations, the SDK provides tools to control what the AI sees and how history is managed. -Now AI can answer: -- "Is this in stock?" → Uses viewing-product context -- "What's in my cart?" → Uses cart context -- "Am I eligible for member discounts?" → Uses user context +- **[Compaction](/docs/context/compaction)** — auto-summarize old messages to stay within token limits +- **[Token Tracking](/docs/context/token-tracking)** — monitor context window usage with `useContextStats` +- **[Session Persistence](/docs/context/session)** — survive page reloads and compact on the server --- ## Next Steps -- [Custom Tools](/docs/custom-tools) - Build tools that use context -- [Agentic Loop](/docs/agentic-loop) - Multi-step AI reasoning +- [Custom Tools](/docs/tools) - Build tools that use context +- [Compaction](/docs/context/compaction) - Manage long conversation history diff --git a/apps/docs/content/docs/context/meta.json b/apps/docs/content/docs/context/meta.json new file mode 100644 index 0000000..00cb426 --- /dev/null +++ b/apps/docs/content/docs/context/meta.json @@ -0,0 +1,5 @@ +{ + "title": "Context Management", + "icon": "Lightbulb", + "pages": ["compaction", "token-tracking", "session"] +} diff --git a/apps/docs/content/docs/context/session.mdx b/apps/docs/content/docs/context/session.mdx new file mode 100644 index 0000000..9cb70c8 --- /dev/null +++ b/apps/docs/content/docs/context/session.mdx @@ -0,0 +1,98 @@ +--- +title: Session Persistence +description: Survive page reloads and persist conversation state across sessions +--- + +import { Callout } from 'fumadocs-ui/components/callout'; + + +**Beta** — This feature is in **alpha**. APIs may change before stable release. + + +Persist the full conversation state — including compaction metadata and message history — across page reloads with zero extra code. + +--- + +## Basic Setup + +```typescript +useMessageHistory({ + persistSession: true, + storageKey: "my-app-chat", // default: "copilot-session" +}); +``` + +### What gets persisted + +| Data | Where | Notes | +|------|-------|-------| +| `compactionState` (small metadata) | `localStorage` | Sync, available immediately on cold start | +| `displayMessages` (can be large) | `IndexedDB` | Async, avoids localStorage quota issues | + +Both are keyed by `storageKey`. Multiple chat instances can coexist with different keys. + +--- + +## Clear Everything + +```typescript +const { resetSession } = useMessageHistory({ persistSession: true }); + +// Clears state AND storage +await resetSession(); +``` + +--- + +## Multiple Chat Instances + +```tsx +// Support chat — separate session + + + {/* useMessageHistory({ storageKey: "support-chat" }) */} + + + +// Sales chat — separate session + + + {/* useMessageHistory({ storageKey: "sales-chat" }) */} + + +``` + +--- + +## Full Setup Example + +```tsx +// app/layout.tsx +import { CopilotProvider } from "@yourgpt/copilot-sdk/react"; + +export default function RootLayout({ children }) { + return ( + console.log("Compacted:", e), + }} + > + {children} + + ); +} +``` + +--- + +## Related + +- [Compaction](/docs/context/compaction) — how the session is summarized +- [Token Tracking](/docs/context/token-tracking) — monitor context window usage diff --git a/apps/docs/content/docs/context/token-tracking.mdx b/apps/docs/content/docs/context/token-tracking.mdx new file mode 100644 index 0000000..18a54b1 --- /dev/null +++ b/apps/docs/content/docs/context/token-tracking.mdx @@ -0,0 +1,208 @@ +--- +title: Token Tracking +description: Monitor context window usage in real time with useContextStats +--- + +import { Callout } from 'fumadocs-ui/components/callout'; + + +**Experimental** — This feature is under active development. APIs may change before stable release. + + +Monitor how much of the AI's context window is being used — broken down by message history, system prompt, tools, and injected context. + +--- + +## useContextStats + +```typescript +import { useContextStats } from "@yourgpt/copilot-sdk/react"; + +function ContextMonitor() { + const { + contextUsage, // Full breakdown by bucket + totalTokens, // Convenience: total estimated tokens + usagePercent, // Convenience: window fill 0–1 + contextChars, // Characters contributed by AI context injections + toolCount, // Number of currently registered tools + messageCount, // Visible (non-system) messages + lastResponseUsage, // Token usage from last assistant message + } = useContextStats(); + + return ( +
+

{Math.round(usagePercent * 100)}% of context used

+

{totalTokens} tokens / {toolCount} tools

+ {lastResponseUsage && ( +

Last turn: {lastResponseUsage.total_tokens} tokens

+ )} +
+ ); +} +``` + +### Return type + +```typescript +interface ContextStats { + contextUsage: ContextUsage | null; // null until first message + totalTokens: number; + usagePercent: number; // 0 until first message + contextChars: number; + toolCount: number; + messageCount: number; + lastResponseUsage: MessageTokenUsage | null; +} + +interface MessageTokenUsage { + prompt_tokens: number; + completion_tokens: number; + total_tokens: number; +} +``` + +--- + +## Token Counting Utilities + +Two tiers — pick the right trade-off between speed and accuracy. + +### Tier 1: Fast (zero dependencies) + +Uses a `chars / 3.5` heuristic. ~85–90% accurate for English. Always available, no bundle cost. + +```typescript +import { + estimateTokensFast, + estimateMessageTokens, + estimateMessagesTokens, +} from "@yourgpt/copilot-sdk/react"; + +const tokens = estimateTokensFast("Hello world"); // fast, synchronous +const msgTokens = estimateMessagesTokens(llmMessages); +``` + +### Tier 2: Accurate (lazy-loaded) + +Uses `gpt-tokenizer` with the `o200k_base` encoding. Lazy-loaded only when called — no upfront bundle cost. + +```typescript +import { + countTokensAccurate, + countMessagesTokensAccurate, +} from "@yourgpt/copilot-sdk/react"; + +// Only loads gpt-tokenizer on first call +const tokens = await countTokensAccurate("Hello world"); +const msgTokens = await countMessagesTokensAccurate(llmMessages); +``` + +### Set estimation mode in `useMessageHistory` + +```typescript +useMessageHistory({ tokenEstimation: "accurate" }); // "fast" | "accurate" | "off" +``` + +--- + +## Context Budget Enforcement + + +**Experimental** — APIs may change before stable release. + + +Automatically enforce per-bucket token limits so the prompt never overflows. Configured via `optimization.contextBudget` on ``. + +```tsx + { + console.warn("Budget exceeded", usage.total.percent); + }, + }, + monitoring: { + enabled: true, + onUsageUpdate: (usage) => trackMetrics(usage), + }, + }, + }} +> +``` + +| `enforcement.mode` | Behaviour | +|--------------------|-----------| +| `"warn"` | Logs a warning, sends full payload | +| `"truncate"` | Trims content to fit — history first, then tool results | +| `"error"` | Throws before sending if budget is exceeded | + +--- + +## Tool Result Truncation + + +**Experimental** — APIs may change before stable release. + + +Prevent a single large tool response from consuming the entire context. Configure under `optimization.toolResultConfig`: + +```tsx + +``` + +| `strategy` | Behaviour | +|------------|-----------| +| `"head-tail"` | Keeps the beginning and end of the result; drops the middle | +| `"head"` | Keeps only the beginning | +| `"tail"` | Keeps only the end | + +A truncation notice is appended so the model knows the result was trimmed. + +--- + +## Example: Context Usage Indicator + +```tsx +import { useContextStats, useMessageHistory } from "@yourgpt/copilot-sdk/react"; + +export function ChatPanel() { + const { tokenUsage, isCompacting, compactSession } = useMessageHistory(); + const { usagePercent, toolCount } = useContextStats(); + + return ( +
+

+ {Math.round(usagePercent * 100)}% context used · {toolCount} tools +

+ {tokenUsage.isApproaching && ( + + )} + {isCompacting && Summarizing history…} +
+ ); +} +``` diff --git a/apps/docs/content/docs/customizations/chat-primitives.mdx b/apps/docs/content/docs/customizations/chat-primitives.mdx new file mode 100644 index 0000000..e8ba49e --- /dev/null +++ b/apps/docs/content/docs/customizations/chat-primitives.mdx @@ -0,0 +1,173 @@ +--- +title: UI Primitives +description: Headless building blocks for composing fully custom chat UIs +--- + +import { Callout } from 'fumadocs-ui/components/callout'; + + +**Beta** — This feature is in **alpha**. APIs may change before stable release. + + +Low-level building blocks for composing custom chat UIs. The SDK handles all state, streaming, and context — you control the layout. + +Two complementary APIs: +- **`ChatPrimitives`** — named export of individual components, import under any alias +- **`CopilotChat.*`** — same primitives mounted directly on `CopilotChat` for inline composition + +--- + +## All Primitives + +```tsx +import { ChatPrimitives as Chat } from "@yourgpt/copilot-sdk/ui"; +``` + +| Primitive | Description | +|-----------|-------------| +| `Chat.MessageList` | Render-prop message list — reads `messages` from context | +| `Chat.DefaultMessage` | Full SDK message bubble — use as fallback in custom lists | +| `Chat.Header` | Chat header bar | +| `Chat.Welcome` | Welcome screen shown when there are no messages | +| `Chat.Input` | Composer / input box | +| `Chat.ScrollAnchor` | Auto-scroll anchor, place at end of message list | +| `Chat.Message` | Low-level message row wrapper | +| `Chat.MessageAvatar` | Avatar with fallback initials | +| `Chat.MessageContent` | Content bubble — renders markdown, supports streaming | +| `Chat.MessageActions` | Action bar layout primitive (wraps action buttons) | +| `Chat.MessageAction` | Single action icon button with tooltip | +| `Chat.Loader` | Streaming / thinking indicator | + +--- + +## `Chat.MessageList` props + +```ts +interface MessageListProps { + children?: (message: ChatMessage, index: number) => React.ReactNode; + className?: string; +} +``` + +When `children` is provided, called once per message — return your custom component or fall back to `Chat.DefaultMessage`. When omitted, renders all messages with `DefaultMessage`. + +--- + +## Examples + +### Custom message type with fallback + +```tsx +import { ChatPrimitives as Chat } from "@yourgpt/copilot-sdk/ui"; + + + + {(message) => + message.metadata?.type === "plan" ? ( + + ) : ( + + ) + } + + +``` + +### Fully custom layout — compose from scratch + +```tsx +import { ChatPrimitives as Chat } from "@yourgpt/copilot-sdk/ui"; + + +
+ + + +
+ + {(message) => ( + + + + + )} + + + +
+ + +
+
+``` + +### Mix primitives with message actions + +```tsx +import { ChatPrimitives as Chat } from "@yourgpt/copilot-sdk/ui"; + + + {/* Register floating action buttons */} + + + log(msg.id, type)} /> + + + {/* Custom message list */} + + {(message) => + message.metadata?.type === "approval" ? ( + + ) : ( + + ) + } + + +``` + +### Per-message action buttons using primitives directly + +```tsx +import { ChatPrimitives as Chat } from "@yourgpt/copilot-sdk/ui"; + + + {(message) => ( + + +
+ + + } + tooltip="Copy" + onClick={() => navigator.clipboard.writeText(message.content ?? "")} + /> + +
+
+ )} +
+``` + +--- + +## `messageView` vs `Chat.MessageList` + +Two ways to customize message rendering at different abstraction levels: + +| | `messageView` prop | `Chat.MessageList` | +|--|--------------------|--------------------| +| Style | Prop on `` | Child component inside `` | +| Access | `messages[]` + pre-rendered `messageElements[]` | `messages[]` via render-prop | +| When to use | Quick overrides, inject extra UI around existing renders | Full layout control, building from primitives | + +Both are non-breaking and can coexist. + +→ See [Custom Message View](/docs/customizations/custom-message-view) for the `messageView` API. + +--- + +## Breaking Changes + +**None.** Both are purely additive. Existing `` usage is untouched. diff --git a/apps/docs/content/docs/customizations/css-classes.mdx b/apps/docs/content/docs/customizations/css-classes.mdx new file mode 100644 index 0000000..06f9cda --- /dev/null +++ b/apps/docs/content/docs/customizations/css-classes.mdx @@ -0,0 +1,306 @@ +--- +title: CSS Class Reference +description: Complete list of all csdk-* CSS classes exposed by the SDK for targeting and customization +--- + +import { Callout } from 'fumadocs-ui/components/callout'; + +Every structural element in the SDK has a `csdk-*` class. Target any of them in your own CSS to override colors, spacing, borders, animations, or layout — no specificity hacks needed. + + +All `csdk-*` classes are **additive only** — they never carry layout-breaking styles on their own. The SDK's visual defaults come from Tailwind utility classes alongside them, which you can override freely. + + +--- + +## Messages + +| Class | Element | Example override | +|---|---|---| +| `csdk-message` | Every message row (user + assistant + tool) | Entry animation, row spacing | +| `csdk-user-message` | User message row wrapper | Row-level alignment | +| `csdk-assistant-message` | Assistant message row wrapper | Row-level alignment | +| `csdk-message-user` | User message bubble | Background, border radius, padding | +| `csdk-message-assistant` | Assistant message bubble | Background, border, padding | +| `csdk-message-content` | Text content inside a bubble | Font size, line height, color | +| `csdk-message-actions` | Hover-reveal action button bar | Positioning, visibility timing | + +```css +/* Example: custom user bubble */ +.csdk-message-user { + background: linear-gradient(135deg, #6366f1, #8b5cf6); + border-radius: 1rem 1rem 0.25rem 1rem; +} + +/* Example: tighten row spacing */ +.csdk-message { + margin-bottom: 0.25rem; +} +``` + +--- + +## Avatars + +| Class | Element | Example override | +|---|---|---| +| `csdk-avatar` | Avatar container (user & assistant) | Size, border, shadow | +| `csdk-avatar-fallback` | Fallback initials shown when no image | Color, font | + +```css +.csdk-avatar { + width: 2rem; + height: 2rem; + border: 2px solid var(--primary); +} +``` + +--- + +## Input + +| Class | Element | Example override | +|---|---|---| +| `csdk-input` | Input container (the whole bar) | Border, background, border-radius | +| `csdk-input-textarea` | The `