Local OpenAI LLM (Custom Integration for Home Assistant)

Allows use of generic OpenAI-compatible LLM services, such as (but not limited to):

• llama.cpp
• vLLM
• LM Studio
• Ollama
• OpenRouter
• Scaleway

This integration has been forked from Home Assistants OpenRouter integration, with the following changes:

• Added server URL to the initial server configuration
• Made the API Key optional during initial server configuration: can be left blank if your local server does not require one
• Uses streamed LLM responses
• Conversation Agents support TTS streaming
• Automatically strips <think> tags from responses
• Added support for image inputs for AI Task Agents
• Added support for reconfiguring Conversation Agents
• Added option to trim conversation history to help stay within your context window
• Added temperature control
• Added option to strip emojis from responses
• Added option to take full manual control of the prompt
  • This will remove ALL content that Home Assistant normally inserts when compiling the system prompt that's sent to the LLM
  • Additional variables are exposed to the prompt jinja template for tools, entities, voice-satellite area, etc
  • For advanced use only: not recommended for most users, and not yet documented here
• Added support for parallel tool calling

---

Installation

Install via HACS (recommended)

Have HACS installed, this will allow you to update easily.

Adding Tools for Assist to HACS can be using this button:

Note

If the button above doesn't work, add https://github.com/skye-harris/hass_local_openai_llm as a custom repository of type Integration in HACS.

• Click install on the Local OpenAI LLM integration.
• Restart Home Assistant.

Manual Install

• Copy the local_openai folder from latest release to the custom_components folder in your config directory.
• Restart the Home Assistant.

Integration Configuration

After installation, configure the integration through Home Assistant's UI:

1. Go to Settings → Devices & Services.
2. Click Add Integration.
3. Search for Local OpenAI LLM.
4. Follow the setup wizard to configure your desired services.

Configuration Notes

• The Server URL must be a fully qualified URL pointing to an OpenAI-compatible API.
  • This typically ends with /v1 but may differ depending on your server configuration.
• If you have the Extended OpenAI Conversation integration installed, this has a dependency of an older version of the OpenAI client library.
  • It is strongly recommended this be uninstalled to ensure that HACS installs the correct OpenAI client library.
• Assist requires a fairly lengthy context for tooling and entity definitions.
  • It is strongly recommended to use at least 8k context size and to limit history length to avoid context overflow issues.
  • This is not configurable through OpenAI-compatible APIs, and needs to be configured with the inference server directly.
• Parallel tool calling requires support from both your model and inference server.
  • In some cases, control of this is handled by the server directly, in which case toggling this will not have any result.

Additional

Looking to add some more functionality to your Home Assistant conversation agent, such as web and localised business/location search? Check out my Tools for Assist integration here!

Acknowledgements

• This integration is forked from the OpenRouter integration for Home Assistant by @joostlek
• This integration uses code from the Local LLMs integration for Home Assistant by @acon96