vllm serve

[YuchaoXu]

when i use "vllm serve" to launch a model, the "api proc" must be 1?

[WilhelmTr]

Hi,

Thanks for bringing this issue to light. This implementation only works for Batch-size=1, and is a known limitation. We are working on a batched version, and will give you updates on the progress as soon as we can!