Skip to content

请教一下,手动设置Q&DQ节点的规则,以减少非必要的数据转换 #47

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lzcchl opened this issue Jun 3, 2025 · 1 comment

Comments

@lzcchl
Copy link

lzcchl commented Jun 3, 2025

我看到 #8 中提到,如果直接无脑生成Q&DQ节点,那么网络中就会有很多非必要的int8->fp16 和fp16->int8 数据类型转换,这个比较好理解;

然后我看到手动设置规则是在 https://github.com/NVIDIA-AI-IOT/cuDLA-samples/blob/main/export/yolov5-qat/quantization/rules.py#L59 这个find_quantizer_pairs函数是核心,这个的作用如下图所示,是将绿色圆圈内的量化参数设置成一样的,但是我无法理解,为什么这么做之后,tensorrt就能够不产生数据类型转换节点;

这里能请您解释一下么?

Image

因为理解完之后,我还需要修改我当前的网络,yolov11-det如下图所示,这里面情况又不一样,我尝试了类似的方式使得某些节点的量化参数相同,但是最终生成的engine中还是没消除这些数据类型转换节点;

Image

@lzcchl
Copy link
Author

lzcchl commented Jun 4, 2025

通过观察对yolov5自动、手动放置QDQ节点,我大概知道需要将下面红框内的参数设置成相等,但是这条规律这好像在我这个模型中不起作用,而且我设置完它们相等之后,为什么没有完全合并,为什么右边会有连续的两个QDQ节点,搞得头大。

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant