How to inference LLM models with OpenVino EP

[ZhangWei125521]

I have found that openvino relate code has been merged into main branch in latest commit history.
I wonder how can I infer with openvino EP.
If you can provide the openvino detail self build and infer guidance like cuda directml in this website(https://github.com/microsoft/onnxruntime-genai/blob/36d8e68f020b74cc2484264f67d92163318399ae/examples/c/README.md)

[baijumeswani]

@RyanMetcalfeInt8 would you please provide some documentation around how one can build an openvino model that can run with onnxruntime-genai?

Maybe we can add it to the onnxruntime.ai tutorials.

[RyanMetcalfeInt8]

Hi @baijumeswani,

ORT GenAI support is currently enabled with the WinML Runtime 2.0 Experimental and Windows ML Runtime Intel OpenVINO Execution Provider only, which are both part of the VS Code AI Toolkit. Please refer to the model conversion workflow in AI Toolkit for examples.

For those users who are not using the WinML Runtime directly, the features from OpenVINO EP that are required to run with onnxruntime-genai will be part of the next release of onnxruntime (currently WIP to get these upstreamed).