Add optional OpenRouter provider for embeddings/search pipeline

[jamiequint]

Summary

• Added provider selection via QMD_LLM_PROVIDER with local as the default.
• Added an OpenRouterLLM backend for embeddings, query expansion, and reranking.
• Kept privacy-safe defaults: no remote calls unless explicitly opted in.
• Added a one-time remote-inference notice per process when OpenRouter mode is active.
• Added API key loading from:
  • QMD_OPENROUTER_API_KEY (preferred)
  • OPENROUTER_API_KEY
  • QMD_OPENROUTER_API_KEY_FILE (defaults to ~/.config/qmd/openrouter.key)
• Made store/CLI LLM calls provider-agnostic.
• Added char-based chunking fallback for remote providers (no local tokenizer dependency).
• Added OpenRouter-focused tests and concise README docs for setup and key storage.

Testing

~/.bun/bin/bun test src/llm.openrouter.test.ts
~/.bun/bin/bun test src/llm.test.ts --test-name-pattern "Default LlamaCpp Singleton|LlamaCpp.modelExists"
~/.bun/bin/bun src/qmd.ts --help

[alexleach]

I feel like making this specific for OpenRouter reduces the usefulness of this PR. OpenRouter is itself OpenAI-compatible, so it would make better sense to map this PR around the OpenAI-spec, not OpenRouter's implementation of it. A rename of accepted environment variables is all that is required, really:-

DEFAULT_OPENROUTER_BASE_URL ⟶ QMD_OPENAI_BASE_URL
QMD_OPENROUTER_API_KEY ⟶ QMD_OPENAI_API_KEY
QMD_OPENROUTER_API_KEY_FILE ⟶ QMD_OPENAI_API_KEY_FILE

For QMD_LLM_PROVIDER, a selector between local and remote or openai would be more generic.

I see you have other environment variables, too:-

QMD_OPENROUTER_EMBED_MODEL
QMD_OPENROUTER_GENERATE_MODEL 
QMD_OPENROUTER_RERANK_MODEL 
QMD_OPENROUTER_APP_NAME
QMD_OPENROUTER_APP_URL

The first three would easily map to QMD_OPENAI_*_MODEL variables.

I'm not sure the purpose of the *_APP_NAME or _APP_URL variables, though. They look a bit unnecessary, at first glance?

[jamiequint]

@alexleach Thanks for the feedback. These are OpenRouter-specific metadata fields and aren’t required for inference, so removing them seems reasonable:

QMD_OPENROUTER_APP_NAME
QMD_OPENROUTER_APP_URL

I went ahead and did that.

On the broader point about generalizing this away from OpenRouter to OpenAI: there isn’t anything in the code that requires OpenAI embeddings specifically. This implementation allows users to choose non-OpenAI embedding models via QMD_OPENROUTER_EMBED_MODEL if they want.

Because of that, names like QMD_OPENAI_BASE_URL, QMD_OPENAI_API_KEY, and QMD_OPENAI_API_KEY_FILE would imply an OpenAI-specific coupling that does not actually exist here.

[alexleach]

Thanks for the response. What I meant was the protocol itself is basically OpenAI's. From https://openrouter.ai/docs/api/reference/overview:-

OpenRouter’s request and response schemas are very similar to the OpenAI Chat API, with a few small differences. At a high level, OpenRouter normalizes the schema across models and providers so you only need to learn one.

So, while not identical, they are "very similar", and one PR could satisfy both OpenAI and OpenRouter. Being OpenAI who designed the API, I would have thought naming it that way would be more applicable.

Personally, I run qmd's default models in Docker Model Runner, so that openclaw running in docker can use qmd with GPU acceleration on my Mac. Docker Model Runner also conforms to the OpenAI Chat API. So, I myself use neither OpenAI or OpenRouter, but your PR would presumably cover my use-case too.

[alexleach]

There are at least a couple of other implementations of this as well:-

• feat: add remote embedding providers (Voyage AI, OpenAI-compatible) #65
• feat: Add OpenAI embedding and query expansion support #116

I'm hoping one of them gets merged so I don't have to rely on a fork of a fork, which is what I'm doing now...

[gregorycollins-usmobile]

I needed a patch to get this branch working 5964419, but it works --- second one I tried to get going

[gregorycollins-usmobile]

I also had to kludge in a better rerank because OpenRouter doesn't currently support it