Would like some help to understand this

[RayBytes]

So I've installed the project and ran it, and on my M1 Pro 16gb I'm currently getting 32.5 tk/s with a 88% acceptance for the prompt I ran (Qwen 3.5 4B). However, running the MLX Qwen 3.5 4B in LM Studio was about this speed.

So my question is, since if I'm understanding it correctly, the models I'm running are not the MLX optimized versions of the models, is it possible to run an MLX optimized version to see the speed gains mentioned in the benchmarks?

Thank you, any help is appreciated.

[Vapets75]

Hey,

After several experiments, I also don't understand the point of this project.
The exact same model: mlx-community/Qwen3.5-35B-A3B-4bit
Apple M1 Max - 64Gb

LM Studio - 52 t/s - 58s
DFlash - 21 t/s - 160s

What is the point?

Thank you, any comments would be appreciated.

[Charlie-Louis]

The server seems not faster than the standard server, just chat/prompt with the 9B is indeed 3x faster … in bf16 … the difference is less in MoE’s and quantized …

[Vapets75]

Hmm... Then I tried mlx-community/Qwen3.5-9B-MLX-bf16 vs Qwen/Qwen3.5-9B.
LM Studio - 19 tok/s
DFlash - 18 tok/s

I still don't understand what I'm doing wrong, and I don't see any increase in tokens per second. Perhaps this isn't suited for my MacBook model.