regarding performance between mllm v1 and v2, with Snapdragon npu backend

[LHB116]

rt. Is there performance (token rate, cpu, mem etc.) comparison between mllm v1 and v2 (npu) on llm, especially token rate, ignore great mem usage in v1.
I'd like to know whether cpu-npu heterogeneous can achieve better performance than aot (almost only npu).
thanks a lot.

[chenghuaWang]

IIRC, cpu-npu heterogeneous is much better in prefill speed due to both PUs can be pipelined. The prefill speed of AOT can be found in our technical report blog. cc @liang1232018 @oreomaker can you provide llm.npu prefill speed?

Also, v1's modeling decoding stage is on CPU, which has more memory bandwith than NPU. The decoding speed may slightly better than currently AOT method(without shape buckting).

[LHB116]

Thanks @chenghuaWang ，I get that the performance of qwen3 1.7b llm model as follows:
setting: W4A16KV8 quantization configuration, chunk=32, pad=1024 on SM8650 device
results: 758.28 tokens/s for prefill and 25.84 tokens/s for decode with 640 prefill tokens and 128 decode tokens.

I would be grateful if you could provide the test results of mllm-v1 under the same settings. @liang1232018 @oreomaker