AutoGPTQ or EXLLAMA support? #6
Description
Hey I was trying to run quantized models using AutoGPTQ or EXLLAMA but it did not work at all. So I am asking if you are thinking about creating a version compatible with those?
{{ message }}
Hey I was trying to run quantized models using AutoGPTQ or EXLLAMA but it did not work at all. So I am asking if you are thinking about creating a version compatible with those?