Improper decision between voiced and unvoiced frames

[KurtWoloch]

At the default setting, when I upload a WAV of speech data, nearly all frames get voiced, and almost none get unvoiced, although the speech doesn't only contains vowels, but also consonants, which should lead to unvoiced frames.

Maybe this is because the voiced/unvoiced decision is made from the first parameter k1 if I'm correct, which I don't think is a reliable criterion.
Changing the Unvoiced Threshold in Buzzer Studio doesn't help here either... normally I'd assume that if I set it to 0, all frames should get voiced, and if I set it to 1, no frames should get voiced, but that's not what happens... there doesn't seem to be much of a difference how I set it, even if I set it to 9 (which can't be done with the arrows), still almost all frames get voiced.

I have a different algorithm in my code which I uploaded to the other issue, and I'll explain that algorithm here in more detail:

In my code there are multiple criteria which may cause a frame to be called unvoiced:
To understand this, my code does a de-emphasis on the wave data before encoding. But the wave data before the de-emphasis already gets analyzed on its energy content per frame, which is called original_energy. The energy of the de-emphasized version is then called energy_emphasized. Teh energy is calculated by calculating the sum of the squares of sample values throughout each frame and then taking the square root of this sum * 0.002.

To decide if a frame will be unvoiced, the following things are checked:

• If the value for the original energy of the frame is < 3, the frame will be unvoiced.
• If the value for the original energy of the frame divided by the value of its de-emphasized version is < 1.2, the frame will be unvoiced as well.
• If, on estimating the pitch of the frame, no strong pitch is found, the frame is called unvoiced as well. This is decided by the EstimatePeriod subroutine by multiple criteria which I haven't quite seen through myself. ;-)

[cgreening]

I'm working on a version with a lot of debug information exposed which should make it easy to see what decisions the algorithm is making - hopefully get it up tomorrow.

[KurtWoloch]

Yeah, that's a good idea. I think especially useful would be a list of frames generated with their actual values (pitch and k values - the voiced/unvoiced decision can be inferred from the pitch or from how many k values are there).

[cgreening]

ok, that's done and live. You can play with the various parameters. Let me know if you hit any weird issues (there will be some!)

[KurtWoloch]

OK, I've played now a bit with it. In my opinion, it sounds way worse than it did before. It seems like in the resulting file, the energy of the unvoiced frames is much higher than that of the voiced frames, at least in comparison... the unvoiced ones sound unnaturally loud. I don't know what the "unvoiced multiplier" does, but if it's for reducing or (gasp) increasing the volume of unvoiced frames, then its effect isn't strong enough, not even at its lowest setting of 0.5. With some of the settings, it's also unclear what they mean... maybe it would be good to have a mouse-over on the labels that explains the given feature in a bit more depth. It's hard to tell what the actual energy of a frame is though... it's given as a bar graph, but the final encoded energy value isn't given in the big mouse-over with the frame details.

I've uploaded the file I use for testing this as an attachment to this posting. It's a pretty silly fake station announcement in German I did myself. If I put this into the encoder, we have a full energy bar at frame 7 (an unvoiced frame) although the original energy is only 0.024, and the energy after emphasis is 0.003. In contrast, Frame 43 is one of the louder frames, it's voiced with an original energy of 0.134 and an emphasized energy of 0.074, but the energy bar of it is only about half full.

I just had an idea how to solve the energy issue... I can't try it myself because there's no "override energy" feature, but what you could try is the following: try encoding the file setting the energy of every frame to the same setting and then take note of each frame's energy when decoding it (with emphasis). The final energy of that frame then should be adjusted depending on the relative energy of that frame.
Here's an example:
Let's assume we have set the energy of each LPC frame to 50%, but the resulting energy is still different because of the different contents. Let's assume that frame 7 has an original energy of 0.024, and the resulting energy of the first encoding/decoding pass is 0.66. Let's further assume that Frame 43 has an original energy 0f 0.134, and the resulting energy of the first encoding/decoding pass is 0.9. We assume that in the final output, the louder frame should have an energy of 1, so it should be 11% louder than in the first pass, while frame 7, if scaled properly, should have an energy of 0.024 / 0.134 = 0,179, but it now has an energy of 0.66, so it should have 27.1% of its energy in the first pass. Thus, we adjust frame 43 to the loudest setting, which would be 1, which now corresponds to a gain of +11%, so Frame 43 has to be adjusted to 0,271/1,11 = 0,244, so 0.244 should be its final energy value. I hope this example wasn't too confusing as well...

Kurt Woloch - Stationsdurchsage.WAV

[cgreening]

ok, found a bug - the unvoiced frames were not being scaled properly.

[KurtWoloch]

OK, it's close now. It could maybe be a bit improved by making the "unvoiced multiplier" slider go a bit lower, let's say to 0.2 or 0.3. And thank you for adding a "stop" button so that it's possible to stop the playback if you're already sure you want to change something. However, I found some more bugs, but I'll open separate issues for them.

[cgreening]

Good idea. I'm just adding better playback controls and more fine grained control.

[KurtWoloch]

That's a good idea as well. Especially in the frame display, it's hard to tell what frame belongs to what part of the wave, and you can't play back individual frames or a range of frames which would make this easier.

[cgreening]

Should now be possible to really see what causes the resulting waveform. I think the next thing would be to be able to tweak the results - maybe toggle voiced/unvoiced or other params.