Feature: Tool Result Truncation & Large Payload Optimization

[Sahil5963]

Summary

Implement intelligent, configurable truncation for tool results to prevent context overflow when tools return large payloads.

Problem

Currently, tool results are sent to the LLM in full, which can:

• Overflow the context window with a single large response
• Waste tokens on verbose output
• Cause failures when payloads exceed model limits

Proposed Solution

Configuration Options

interface ToolResultConfig {
  truncation?: {
    enabled?: boolean;                // Default: true
    maxContextShare?: number;         // Default: 0.3 (30% of context per result)
    hardMaxChars?: number;            // Default: 400_000 (~100K tokens)
    minKeepChars?: number;            // Default: 2_000
    strategy?: 'head' | 'head-tail' | 'smart';  // Default: 'head-tail'
    preserveJson?: boolean;           // Default: true
    preserveErrors?: boolean;         // Default: true
  };
}

// Usage
const chat = createChatWithTools({
  toolResultConfig: {
    truncation: {
      enabled: true,
      maxContextShare: 0.25,
      strategy: 'head-tail',
    }
  }
});

// Or disable entirely
const chat = createChatWithTools({
  toolResultConfig: {
    truncation: { enabled: false }
  }
});

Per-Tool Override

const myTool: ToolDefinition = {
  name: 'read_file',
  // ... other fields
  resultConfig: {
    truncation: {
      maxChars: 50_000,        // Override global limit
      strategy: 'head',        // Different strategy for this tool
    }
  }
};

Truncation Strategies

Strategy	Description	Use Case
head	Keep beginning only	Structured data with headers
head-tail	70% head + 30% tail	Logs, errors at end
smart	Auto-detect important sections	General purpose

Use Cases

• File reading tools with large files
• API responses with verbose JSON
• Database queries with many rows
• Log analysis (errors usually at end)

Benefits

• ✅ Fully optional - disable if not needed
• ✅ Configurable globally or per-tool
• ✅ Prevents context overflow
• ✅ Preserves important content (errors, results)
• ✅ Reduces token costs

[Sahil5963]

Consolidated into #72 (Intelligent Tool Management System) - see Result Truncation & Budget section

[Sahil5963]

Reopening - this is a distinct feature from #72. Tool Result Truncation focuses on handling large OUTPUT from tool calls, while #72 focuses on tool discovery. These are complementary but separate concerns.