Skip to content

feat: implement retry_count for agent completions #8

@rosspeili

Description

@rosspeili

Problem Statement

Transient network errors or local LLM hiccups (e.g., temporary disconnection from Ollama) can kill a long-running session. Currently, if an inference call fails once, the entire session crashes or hangs, leading to a poor user experience.

Proposed Solution

  1. Add a retry_count: int = Field(default=3) to SessionConfig in config.py.
  2. Implement a retry loop in Agent.generate_response() (or within the litellm call wrapper).
  3. If an inference call fails, the system should catch the error and retry the completion up to the specified limit before giving up.

Alternatives Considered

Manual retries by the user, which is disruptive to the dynamic conversation flow.

Priority

Medium 🟡

Additional Context

This works in tandem with the 'timeout' feature to provide a much more stable and robust experience for users with varying hardware or network quality.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions