how to export onnx from pt model?

[aa851182472]

I already have the pt file and need to perform inference using TensorRT. How can I convert it to ONNX?

[cherubicXN]

Hi @aa851182472,

This should be straightforward, as it mainly involves converting the Vision Transformer to ONNX. We plan to look into this in the near future, and we’d also welcome a pull request if you have a better implementation.

Thank you for your question.

[dycoene03]

I +1 this! I tried converting to ONNX myself but kept running into problems. The main problems were relating to xformers / flash attention. When I got a working ONNX after some monkey-patching, inference went from sub 2 seconds to 15 seconds, so I'm sure I did something wrong. I would like to run this model on a Triton server.