找不到代码中的Position-Aware Attention Scaling部分

[Falanke21]

您好。在论文Towards Robust Vision Transformer中提到了Position-Aware Attention Scaling和Wp矩阵，但我并没有在代码中找到这个特性。请问是这部分的代码没有发布吗？

谢谢！

Hello. Position-Aware Attention Scaling and Wp matrix are mentioned in the paper Towards Robust Vision Transformer, but I did not find this feature in the code. Is this part of the code not published?
Thank you!

[pic4xiu]

作为一个路过的人回答下这个问题，self.att_mask就是Wp矩阵，参与到后边的attention计算，也就是下方的~：

if self.use_mask:
    attn = attn * torch.sigmoid(self.att_mask).expand(B, -1, -1, -1)