Custom Verifiers for speaker identification

[jhbruhn]

Hi! Custom Verifiers in general are suggested to do a way of speaker identification. Have you already looked into what model suits best?

My thoughts are, for a case of two Speakers A and B:

• An ensemble of multiple custom verifiers, trained on
  • Speaker A Samples as positive, and non-speech samples as negative, or
  • Speaker A Samples as positive, and non-speech and Speaker B samples as negative.
• One custom verifier, trained as multiclass Logistic Regression, with three classes for Speaker A, Speaker B and non-speech.

In addition to that, did you experiment with other classifiers than logistic regression?

I want to implement automatic training and CV functionality into wyoming-openwakeword, and currently have to decide on the approach to take.

[jhbruhn]

I have started a multi-speaker detection implementation here: https://github.com/jhbruhn/wyoming-openwakeword/tree/custom-verifier based on the multiclass LogisticRegression approach.
But it does not seem to work well yet, the classifier mostly classifies the voices as negative samples, as finding appropriate thresholds for activation gets even more difficult with this. There also is still a bug about the speaker identification choosing the entry with maximum probability, which usually is still the negative sample.

[dscripka]

The customer verifier models can work as speaker identification models in some circumstances, but I'm not sure how much general potential there is for this task. Is your training data for each speaker all collected from the same environment? And what is the balance between your positive and negative data.

I did try several other model types besides Logistic regression, but performance didn't seem to improve substantially. I also only focused on binary models at the time.

As for choosing the entry with maximum probability, that isn't necessarily a bug, as the verifier models are primary intended to reduce false-activations. But to your point, if the goal is instead primary speaker identification, this behavior is not ideal.

[jhbruhn]

Yes, upon further testing it, I came to that conclusion as well. I want to test using the LogisticRegression as a custom verifier and then using a second classifier to differentiate between speakers, that will hopefully yield better results. The LogisticRegression is so computationally cheap that stacking two of them should make no big difference in performance.