Lambda_m trainable

[Serega6678]

Hello,
In the paper, you say that lambda_m and lamda'_m are trainable parameters. Could you please clarify where are they located in the code? To me, it seems that weight=0.1 (which is not trainable) in the forward method of the model and this is the exact weight used to compute L_aux.
What am I missing?
Many thanks!

[Mckysse]

Dear,
Good question! As you can see at the Section 4.1 of the paper, we had a footnote that illustrated "We did study semantic-wise weights by projecting the [CLS] token embeddings to a set of trainable parameters, but no further improvement could be achieved". So, we finally used the weight=0.1 as the fixed parameter.
Thanks for your attention!
Best,
Chen