ollama (self-hosted) support for AI Flow

[thefoxman88]

Can I request a self-hosted option for AI requests?
Hopefully others would like to avoid requesting book info to gemini/openAi about a questionable spicey books?

I have a simple setup with Ollama and Open-Webui and would love to share that resource with library-manager. Lets just hope the models required can be used with less then 12GB (RTX 3060)

[deucebucket]

Hey @thefoxman88! This is definitely on the roadmap.

Current state:

• We already support OpenRouter which gives you access to various models
• The AI is only used as a fallback when our 50M book database does not find a match
• Most lookups never hit AI at all (BookDB handles ~90% of queries)

Ollama support plan:
Adding Ollama would be straightforward since we already have the provider abstraction:

• New ai_provider: "ollama" option
• Configure ollama_url (default: http://localhost:11434)
• Model selection (llama3.2, mistral, etc.)

For a 12GB RTX 3060, you could run:

• llama3.2:3b (~2GB) - fast, good enough for metadata parsing
• mistral:7b (~4GB) - better accuracy
• llama3.1:8b (~5GB) - best balance

The task is pretty simple (parsing author/title from folder names), so even smaller models work well.

I will add this in an upcoming release. Would you prefer:

1. Just Ollama support, or
2. Generic "OpenAI-compatible API" support (works with Ollama, LM Studio, text-generation-webui, etc.)?

[deucebucket]

This has been implemented! 🎉

Ollama support is now available on the develop branch (PR #25 merged). Here's what's included:

Features:

• Ollama added as a third AI provider option in Settings → AI Setup
• Server URL configuration (default: http://localhost:11434)
• Model selector that fetches available models directly from your Ollama server
• Test Connection button to verify connectivity
• Works completely offline - no API keys or external calls needed

Recommended models for your RTX 3060 (12GB):

• llama3.2:3b - Fast, works great with 8GB+ VRAM
• llama3.2:1b - Even faster, works on 4GB+ VRAM
• mistral:7b - More capable, fits comfortably in 12GB
• phi3:mini - Microsoft's efficient small model

To try it now:
Pull the develop branch or use the Docker image tagged develop:

docker pull ghcr.io/deucebucket/library-manager:develop

Then in Settings → AI Setup, select "Ollama (Self-hosted - No API costs)" and configure your server URL.

Your spicy books stay private! 🌶️

This will be in the next stable release after some testing on develop.

[Jdiesel87]

All ollama models are showing up as undefined in library-manager. Tested with mistral:7b

[Jdiesel87]

Seeing this error in my logs

ERROR - Ollama: Model 'undefined' not found. Run: ollama pull undefined