How to use ollama rather than onlin API?

[JouJman]

Hello, I tried to change self.client fonction on agent-q/core/agent/base.py as:
self.client = openai.OpenAI( base_url="http://localhost:11434", api_key="ollama", )
And the model name into qwen2.5:14b

Whene I enter a command I got:
`Executing command test

Current State: agentq_base
Agent: agentq
Current Thought:
Plan: none
Completed Tasks: none

An error occurred: RetryError[<Future at 0x73858ddb74d0 state=finished raised NotFoundError>]
Retrying in 2 seconds...
An error occurred: RetryError[<Future at 0x73858ddc1910 state=finished raised NotFoundError>]
Retrying in 2 seconds...
An error occurred: RetryError[<Future at 0x73858d3039e0 state=finished raised NotFoundError>]
Max retries reached. Unable to complete the action.
Error executing the command test: RetryError[<Future at 0x73858d3039e0 state=finished raised NotFoundError>]`

How can I solve this?

[Hackinside]

It seems like you might be missing /v1 at the end of your URL. Try changing it to http://localhost:11434/v1 to make sure you're accessing the model correctly.

To check if your model is running, open Windows Powershell and type ollama ps. This will show you a list of active models.

In my case, even after adding /v1 to the end of your URL, I'm still encountering the RetryError with a ValidationError. This suggests there might be something else going on.

Error message:

An error occurred: RetryError[<Future at 0x293a74c3d40 state=finished raised ValidationError>]
Retrying in 2 seconds...
An error occurred: RetryError[<Future at 0x293a74d0f20 state=finished raised ValidationError>]
Retrying in 2 seconds...

[vaibhavkr-dev]

i am also getting same error with Ollama. Please suggest if there is any support available for ollama to work with this agent-q

[homexkai]

I'm getting the same error when trying to use groq

[bangnguyen123]

Getting the validation error same as the one above when trying with Ollama.

[menghuazou]

I used traceback, which seems to be the problem that the model(Llama3.1) reply does not correspond to the data related to instructor and pydantic.

[sheepying]

I got the same error when I use the requested model "gpt-4o-2024-08-06".