Custom OpenAI-Compatible Provider Endpoints (BYO Endpoint for Local & Remote LLMs)

[tbitcs]

Pre-submit Checks

• I have searched Warp feature requests and there are no duplicates
• I have searched Warp docs and my feature is not there

Describe the solution you'd like?

Add support for configuring custom OpenAI-compatible API endpoints, allowing Warp to connect to local or self-hosted LLM providers.

This would enable users to specify:

• A base URL (e.g. http://localhost:8000/v1)
• A model name
• An optional API key

The configured endpoint would be used for Warp AI features such as command suggestions, agent interactions, and inline completions.

The implementation should support standard OpenAI-compatible APIs, including chat completions and streaming responses, so that it works with tools like vLLM, Ollama (via compatibility layer), and other OpenAI-compatible servers.

From a user perspective, this would extend the existing “Bring Your Own Key” model into a “Bring Your Own Endpoint” workflow, where users can choose between built-in providers and a custom endpoint.

Is your feature request related to a problem? Please describe.

Yes.

Currently, Warp AI only supports a fixed set of providers (e.g. OpenAI, Anthropic, Google) via API keys. There is no way to connect Warp to local or self-hosted models.

This creates several limitations:

• Users cannot run Warp AI in privacy-sensitive or offline environments
• Teams with existing LLM infrastructure cannot integrate it
• Developers cannot use open-source models running locally
• All prompts must be sent to external providers, even when a local option is available

Many modern LLM tools (such as vLLM and Ollama) expose OpenAI-compatible APIs, but Warp cannot currently take advantage of this without native support for custom endpoints.

Additional context

This request consolidates several related feature requests:

• Suggestion for Adding Support for Custom AI Model Configuration #6026 – Custom AI model configuration
• Custom AI API (Eg. Azure Open AI) or LiteLLM #8708 – Custom AI API endpoints
• Make Warp work with Local Language Models (like Ollama models) #4339 – Local LLM support
• Local AI #5735 – Privacy / local AI usage
• Allow users to specify the LLM endpoint (APIs) for other models #4687 – Custom endpoint (closed as duplicate)
• Using LLMs other than OpenAI for AI command search #3779 – Provider routing limitations

The proposed approach is to extend Warp’s existing provider model to allow any OpenAI-compatible endpoint, rather than introducing provider-specific integrations for each tool.

This would allow immediate compatibility with:

• vLLM (OpenAI-compatible server)
• Ollama (local runtime with compatibility layer)
• LiteLLM and similar proxy systems

Example configuration:

{
"provider": "custom",
"base_url": "http://localhost:8000/v1",
"model": "meta-llama/Llama-3-8b-instruct",
"api_key": "optional"
}

This is a provider-level feature and is expected to be cross-platform across all Warp-supported operating systems.

Operating system (OS)

Windows

How important is this feature to you?

5 (Can't work without it!)

Warp Internal (ignore) - linear-label:39cc6478-1249-4ee7-950b-c428edfeecd1

None

[oz-for-oss]

@tbitcs

You can view the triage session on Warp.

This issue appears to overlap with existing issues:

• Allow users to specify the LLM endpoint (APIs) for other models #4687 — Allow users to specify the LLM endpoint (APIs) for other models
• Make Warp work with Local Language Models (like Ollama models) #4339 — Make Warp work with Local Language Models (like Ollama models)
• Custom AI API (Eg. Azure Open AI) or LiteLLM #8708 — Custom AI API (Eg. Azure Open AI) or LiteLLM
• Suggestion for Adding Support for Custom AI Model Configuration #6026 — Suggestion for Adding Support for Custom AI Model Configuration

If this report is meaningfully different, please comment with the additional context or distinguishing behavior so a maintainer can review it. Otherwise, a maintainer may close it as a duplicate after review.

Maintainer details

I concluded that request to configure custom OpenAI-compatible LLM endpoints for Warp AI is a duplicate of several existing custom/local LLM provider feature requests..

Duplicate reasoning

• Allow users to specify the LLM endpoint (APIs) for other models #4687: Requests the same core capability: user-configurable LLM endpoints for local or alternative online models.
• Make Warp work with Local Language Models (like Ollama models) #4339: Requests local LLM/Ollama support, which the new issue proposes to satisfy through OpenAI-compatible custom endpoints.
• Custom AI API (Eg. Azure Open AI) or LiteLLM #8708: Requests custom AI API endpoints such as Azure OpenAI or LiteLLM for private/enterprise model infrastructure.
• Suggestion for Adding Support for Custom AI Model Configuration #6026: Requests custom model configuration, and discussion on the issue specifically calls for OpenAI-compatible endpoint support for vLLM/Ollama-style backends.

This is my automated analysis and may be incorrect. A maintainer will verify the details.

Powered by Oz

[tbitcs]

This is intended to go beyond the existing requests by focusing on how OpenAI-compatible endpoints integrate with Warp’s full agent system, not just basic chat completion.

From the discussion here, it looks like current approaches (e.g. local adapters) can return text responses but do not support:

• Warp tool calls
• command execution loops
• structured agent workflows

The goal of this proposal is to define a provider abstraction that allows OpenAI-compatible endpoints to participate in the full agentic loop, rather than acting as a limited chat backend.

This would unify:

• custom endpoints (Allow users to specify the LLM endpoint (APIs) for other models #4687)
• local models (Make Warp work with Local Language Models (like Ollama models) #4339)
• custom APIs (Custom AI API (Eg. Azure Open AI) or LiteLLM #8708)

into a single capability: first-class provider support for OpenAI-compatible backends within Warp’s agent architecture.

Happy to reflect this explicitly in the spec if this direction makes sense.

[shivaylamba]

I have implemented a similar design to add support for Nebius AI Token Factory models

https://github.com/shivaylamba/warp/tree/nebius-ai-provider

[shivaylamba]

@tbitcs
The current Nebius support is a local OpenAI-compatible chat adapter, not a full Warp model backend.

Main limitations:

• It can answer text in Agent, but it cannot call Warp tools yet.
• It does not run the full terminal automation loop: command execution, file edits, structured
  tool calls, approvals, etc.

Need to get to support the full agentic terminal loop next -

Do you know if OpenAI-compatible endpoints would automatically get support for the full agentic experience or do we need make it manually?

[Aeromix]

I already forked Warp and adding LM Studio local model support, but I am vibecoding it, and cannot imagine it becoming official code

[tbitcs]

This is intended to go beyond the existing requests by focusing on how OpenAI-compatible endpoints integrate with Warp’s full agent system, not just basic chat completion.

From the discussion here, it looks like current approaches (e.g. local adapters) can return text responses but do not support:

• Warp tool calls
• command execution loops
• structured agent workflows

The goal of this proposal is to define a provider abstraction that allows OpenAI-compatible endpoints to participate in the full agentic loop, rather than acting as a limited chat backend.

This would unify:

• custom endpoints (Allow users to specify the LLM endpoint (APIs) for other models #4687)
• local models (Make Warp work with Local Language Models (like Ollama models) #4339)
• custom APIs (Custom AI API (Eg. Azure Open AI) or LiteLLM #8708)

into a single capability: first-class provider support for OpenAI-compatible backends within Warp’s agent architecture

Happy to reflect this explicitly in the spec if this direction makes sense.

[Aeromix]

I am wiring up full Warp use, as that's it's value proposition, not to just use it as a chat terminal. I will report if it works