Skip to content

Add colony-agent test-llm command#18

Merged
jackparnell merged 1 commit intomainfrom
feature/test-llm-command
Apr 7, 2026
Merged

Add colony-agent test-llm command#18
jackparnell merged 1 commit intomainfrom
feature/test-llm-command

Conversation

@ColonistOne
Copy link
Copy Markdown
Collaborator

Summary

New colony-agent test-llm command that verifies the LLM connection before running the agent.

$ colony-agent test-llm
LLM:     openai-compatible
Model:   qwen3:8b
URL:     http://localhost:11434/v1
Agent:   MyAgent

Sending test prompt...
Response (0.8s):

  Hello! I am MyAgent, an AI agent exploring topics in AI and distributed systems.

LLM connection is working.

On failure:

No response from LLM. Check your configuration:
  - Is the LLM running at http://localhost:11434/v1?
  - Is the model 'qwen3:8b' available?
  - Does the provider require an API key? (llm.api_key is empty)

Supports --prompt "custom question" for testing specific behaviors. Uses the agent's system prompt so the response reflects the configured personality.

Test plan

  • Successful connection shows response and "working" message
  • Failed connection shows troubleshooting tips and exits non-zero
  • Custom prompt sent correctly
  • LLM config displayed (provider, model, URL)
  • Response time shown
  • Missing API key warning in troubleshooting
  • 146 tests pass, ruff + mypy clean

🤖 Generated with Claude Code

New command that sends a test prompt to the configured LLM and
shows the response. Catches misconfiguration (wrong URL, bad API
key, model not found) before the agent starts its loop.

Usage:
  colony-agent test-llm                    # default test prompt
  colony-agent test-llm --prompt "Hello"   # custom prompt

Shows LLM config (provider, model, URL), response time, and
actionable troubleshooting tips on failure.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@jackparnell jackparnell merged commit c94fd41 into main Apr 7, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants