Conversation
… fix filling w/ -inf in wav2vec2, minor cleanups
…item in log_scalar, minor cleanups
alexeib
left a comment
alexeib
left a comment
There was a problem hiding this comment.
cant see anything that is obviously wrong, though see some comments inline (in particular, diversity penalty is prob broken, but that wouldnt be root cause of what we see).
one issue is that you are sampling negatives also from padded states now. this is something you are addressing in next pr but that one has issues (see comments there)
have you checked that the generated masks are different each time and have sufficient number of masked spans, etc?
taylanbil
left a comment
taylanbil
left a comment
There was a problem hiding this comment.
one issue is that you are sampling negatives also from padded states now. this is something you are addressing in next pr but that one has issues (see comments there)
When I find time I intend to clean this pr and roll some changes from the other one into this.
have you checked that the generated masks are different each time and have sufficient number of masked spans, etc?
no, I have not. how do I check that?
|
just print out the masks and/or statistics about the masks from the collater? |
|
Addressed comments, pushed. What stats are you interested in? |
4 participants
No description provided.