OpenAI API

[chigkim]

It would be great if we could use mlx-vlm with ml-explore/mlx-examples/llms/mlx_lm.server that has OpenAI API!
Thanks!

[Blaizzy]

Working on it :)

This is related to #241

[cubist38]

hi @chigkim

I’ve developed a lightweight mlx-server-OAI-compat that supports multimodal capabilities while maintaining compatibility with OpenAI APIs. Feel free to try it out ahead of the official release!

[chigkim]

Hi @Blaizzy, just curious if there’s been any progress on OpenAI API support? Ideally, it’d be great to integrate it directly into MLX-LM.server, so users wouldn’t need to run two separate servers.
It’d be amazing if users could handle audio, text, and vision all through one server, similar to how OpenAI’s API lets you access all model types through a single endpoint.
Then it would be super easy to migrate existing projects/tutorial based on OpenAI to MLX!
Thanks for all your work!

[Blaizzy]

Hey @chigkim

The OpenAI API support is done for mlx-vlm in #321

However, I understand the need for a more complete and cohesive solution. That's why I had built FastMLX :). But it got sold off to my previous employer and "died".

Need not to worry, I have been working on something better.

[chigkim]

Oh cool! Since #321 with OpenAI endpoint is merged, should we close this?