Negative Sampling

[lingjieyi]

In the cpc/criterion/criterion.py file, between line 181 and 201 as shown below. The sampling negative part.
In the paper, it's said that "sampling negative within speaker".

For extIdx, the total length is batchSize * self.negativeSamplingExt * windowSize. And if it samples within each speaker, then the first self.negativeSamplingExt * windowSize elements of extIdx should always point to speaker 1 within this batch. If so, following the default settings, batchSize= 8, negativeSamplingExt=128, windowSize=116, then values of the first self.negativeSamplingExt * windowSize (128 * 116) elements of extIdx should be between [0, 128) because the first 128 (116 training + 12 testing) samples in the negExt are data of speaker 1. Samely, values of the second self.negativeSamplingExt * windowSize elements of extIdx should be between [128, 256).

However, when I check the values from the first self.negativeSamplingExt * windowSize elements of extIdx and the second self.negativeSamplingExt * windowSize elements of extIdx, both have range [0, 1024) instead of [0, 128) & [128, 256) respectively. Sampling among [0, 1024) means sampling across all different speakers.

Can you please advise?

code:

batchIdx = torch.randint(low=0, high=batchSize, size=(self.negativeSamplingExt * windowSize * batchSize, ))

seqIdx = torch.randint(low=1, high=nNegativeExt, size=(self.negativeSamplingExt * windowSize * batchSize, ))

baseIdx = torch.arange(0, windowSize, device=encodedData.device)
baseIdx = baseIdx.view(1, 1, windowSize).expand(batchSize, self.negativeSamplingExt, windowSize)
seqIdx += baseIdx.contiguous().view(-1)
seqIdx = torch.remainder(seqIdx, nNegativeExt)

extIdx = seqIdx + batchIdx * nNegativeExt
negExt = negExt[extIdx].view(batchSize, self.negativeSamplingExt, windowSize, dimEncoded)