Bug: Intention tool returns empty tool field causing 'no tool selected' errors

[teslashibe]

Summary

When using WithForceReasoning(), the pickTool function frequently fails with "no tool selected" errors even when the LLM reasoning clearly indicates which tool should be used.

Reproduction

1. Use ExecuteTools() with WithForceReasoning() option
2. Observe that sometimes tool selection fails with "no tool selected"
3. Check logs - the reasoning text contains explicit tool choice (e.g., "Action: wait")

Root Cause

In tools.go line 443-445:

case "":
    xlog.Debug("[pickTool] No tool selected")
    return nil, reasoning, fmt.Errorf("no tool selected")

The LLM calls pick_tool but returns {"tool": "", "reasoning": "..."} - an empty tool field despite:

• Schema having "required": ["tool"]
• Schema having "enum": [available tools]

Some LLMs (especially smaller/faster ones via OpenRouter) do not strictly follow schema constraints.

Evidence from Production Logs

time=2025-12-11T05:54:12.114Z level=DEBUG msg="[pickTool] No tool selected"
time=2025-12-11T05:54:12.969Z level=DEBUG msg="[pickTool] No tool selected"
time=2025-12-11T05:54:13.261Z level=DEBUG msg="[pickTool] No tool selected"
time=2025-12-11T05:54:13.932Z level=DEBUG msg="[pickTool] No tool selected"
time=2025-12-11T05:54:14.166Z level=DEBUG msg="[pickTool] Tool selected via intention" tool=wait ✅
time=2025-12-11T05:54:15.595Z level=DEBUG msg="[pickTool] Tool selected via intention" tool=wait ✅

4 failures followed by 2 successes - the behavior is flaky.

Meanwhile, the reasoning text clearly shows the LLM chose "wait":

"**Action: wait**"
"Tool Selection: wait"
"The only appropriate action is to **wait**"

Proposed Fix

When intentionResponse.Tool == "", extract the tool name from the reasoning text as a fallback:

case "":
    // Fallback: Extract tool from reasoning text
    extractedTool := extractToolFromReasoning(reasoning, toolNames)
    if extractedTool != "" {
        xlog.Debug("[pickTool] Extracted tool from reasoning", "tool", extractedTool)
        chosenTool := tools.Find(extractedTool)
        if chosenTool != nil {
            return &ToolChoice{Name: extractedTool, Arguments: make(map[string]any), Reasoning: reasoning}, reasoning, nil
        }
    }
    xlog.Debug("[pickTool] No tool selected")
    return nil, reasoning, fmt.Errorf("no tool selected")

With extraction function:

func extractToolFromReasoning(reasoning string, toolNames []string) string {
    lowerReasoning := strings.ToLower(reasoning)
    
    for _, tool := range toolNames {
        lowerTool := strings.ToLower(tool)
        patterns := []string{
            "action: " + lowerTool,
            "tool: " + lowerTool,
            "decision: " + lowerTool,
            "use the " + lowerTool + " tool",
            "**" + lowerTool + "**",
        }
        for _, pattern := range patterns {
            if strings.Contains(lowerReasoning, pattern) {
                return tool
            }
        }
    }
    return ""
}

Environment

• cogito version: v0.7.0
• LLM providers: Various models via OpenRouter
• Use case: Trading agents with ~15 tools including "wait"

Impact

• High - Causes valid decisions to fail silently
• Agents that should be waiting are instead returning errors
• Production systems are affected (~30% failure rate on some models)

Workaround

Currently no workaround other than retrying failed evaluations.

[mudler]

@teslashibe another thought just crossed my mind here - was this caused after a certain number of interactions? Note that cogito defaults to a max of tools calls in a row, and has also some other controls that you might want to tweak:

cogito.WithIterations(10), // Number of allowed max tools to call 
cogito.WithMaxAttempts(10), // Number of maximum attempts before erroring out and failing  the whole chain
cogito.WithMaxRetries(10), // Number of attempts to retry a tool call that failed