LFM2-2.6B-Exp tool-calling output doesn’t plug cleanly into Osaurus tool system

[EphraimElgrabli]

Problem statement

I’m trying to use LFM2-2.6B-Exp (mlx-community) inside Osaurus, and the model itself is honestly wild. For a 2.6B model, the instruction following and agent-style reasoning are on another level. It genuinely feels like one of the best small “agent brains” out there, and the IFBench results vs models hundreds of times larger totally match real usage.
The problem I’m hitting is purely around tool calling. LFM2 emits tool calls as custom textual tokens inside the assistant output (e.g. <|tool_call_start|> … <|tool_call_end|>). Because Osaurus exposes tools via OpenAI-compatible APIs and MCP, these tool calls aren’t automatically picked up as executable actions, which creates friction when running the model inside the Osaurus ecosystem.
This isn’t a complaint about the model at all — it’s excellent. I’m opening this because LFM2-2.6B-Exp feels like a perfect fit for Osaurus (local, fast, agentic), and this output mismatch is currently the only thing stopping it from working smoothly with Osaurus’ tool + MCP system.

Proposed solution

No response

Alternatives considered

No response

Additional context

No response

[tpae]

Thanks for the suggestion, I will take a look at this

[EphraimElgrabli]

to replicate the error you should use the model mentioned, and not the foundation model which is pretty good at tool calling but have a poor ability with general knowledge and qa

[tpae]

Hey @EphraimElgrabli , i was able to look deeper into this issue. it seems like LFM2 tool calling isn't fully supported yet by mlx-swift-lm library. i've filed an issue here: ml-explore/mlx-swift-lm#47

[EphraimElgrabli]

Thank you so much for digging into this, @tpae — I really appreciate it.
In the meantime, I implemented a temporary workaround on the Osaurus side by adding a custom parser that translates LFM2’s textual tool-call tokens into executable tool calls. It’s admittedly a bit hacky and still buggy, but it allows experimentation for now.
I have to say: despite the tooling friction, this model is extremely impressive for its size. Even with the workaround, its ability to decide when and how to call tools is surprisingly strong. Once native support lands in mlx-swift-lm, this should be a fantastic fit.

[tpae]

Yep! I agree. I might take a stab at it just to have it available for us. In the meantime, will keep this issue open to track it