Support for Locally Served Models (Ollama) & Open API Compatible Models

[TharunSivamani]

What

Is there an idea / WIP for replacing OpenAI Models with local models or cloud providers key like Azure / Bedrock

Why

For someone who wants private data to be experimented with the project & use Graph's for that

Implementation guidance

Replacing OpenAI calls with using openai client by having base_url & api_key as env arguments

[lidaling]

+1

[Magke]

+1

[dayday2019]

+1

[tomsmoker]

Hey all, thanks for the request. We'd love to include this in the future (I actually have a very basic implementation started). Our team is currently heads down on other priorities at the moment, but we definitely welcome community contributions. If you're interested in building it out, please open a PR and tag us - we're happy to review. If you have any questions about the code, feel free to ping us here or on Discord, where we're pretty active. Thanks!