Tks for your brilliant works! Have you try to parallel 'DAB' and 'SAB' instead of putting them in sequence?

[lyf1212]

It is very sharp of you to discover the potential of sparse attention mechanism and purpose the DAB in your paper~ I'd like to know have you try to put the 'DAB' and 'SAB' in parallel when constructing the arch of the model instead of putting them in sequence?

[gladzhang]

I have not tried this. It may be nice to design a parallel structure to use DAB and SAB once you find a good motivation.