关于模型训练数据

[tanggang1997]

您好，我现在再您的基础上进行了模型的改进，建模特征参考了其他音频生成模型，但是数据量没您这么多，想先跑通一个小规模版本的，请问多少数据量合适呀

[viewfinder-annn]

For a small-scale version, several hundred hours should be sufficient in my view, though this is an approximate estimate rather than based on dedicated small-scale experiments.

[tanggang1997]

For a small-scale version, several hundred hours should be sufficient in my view, though this is an approximate estimate rather than based on dedicated small-scale experiments.

您好，我是第一次训练flowmatching模型，想问您相关的训练经验，据我了解flowmatching并不是完全看loss是否收敛来判断生成质量，想问您一般是如何去判断模型训练已经完成的

[viewfinder-annn]

For a small-scale version, several hundred hours should be sufficient in my view, though this is an approximate estimate rather than based on dedicated small-scale experiments.

您好，我是第一次训练flowmatching模型，想问您相关的训练经验，据我了解flowmatching并不是完全看loss是否收敛来判断生成质量，想问您一般是如何去判断模型训练已经完成的

by ears...judgment tends to rely more on subjective evaluation (e.g., listening to samples) rather than loss convergence alone.

By the way, studies like F5-TTS also explored FM-based TTS model convergence behavior by tracking metrics like WER across training steps, which can serve as a useful reference (Figure 2 in https://arxiv.org/pdf/2410.06885).

[tanggang1997]

For a small-scale version, several hundred hours should be sufficient in my view, though this is an approximate estimate rather than based on dedicated small-scale experiments.

您好，我是第一次训练flowmatching模型，想问您相关的训练经验，据我了解flowmatching并不是完全看loss是否收敛来判断生成质量，想问您一般是如何去判断模型训练已经完成的

by ears...judgment tends to rely more on subjective evaluation (e.g., listening to samples) rather than loss convergence alone.

By the way, studies like F5-TTS also explored FM-based TTS model convergence behavior by tracking metrics like WER across training steps, which can serve as a useful reference (Figure 2 in https://arxiv.org/pdf/2410.06885).

您好，感谢您的回复，我已经复现了这个工作，但是用的都是单声道数据，数据噪声也比较多，感觉我的模型好像过拟合了,我目前是在跑小批量的数据，大概1000条数据，
这是输入人声：

abjones_2_09_vocal.mp3

这是生成的结果：

output.wav

这是混合后的结果：

mixed_torch.wav
不知道对于数据上您有什么好的建议吗？分离过后的音频数据都还是会有一些损失，带有底噪，但是听您的样列音频感觉很干净

[viewfinder-annn]

For a small-scale version, several hundred hours should be sufficient in my view, though this is an approximate estimate rather than based on dedicated small-scale experiments.

您好，我是第一次训练flowmatching模型，想问您相关的训练经验，据我了解flowmatching并不是完全看loss是否收敛来判断生成质量，想问您一般是如何去判断模型训练已经完成的

by ears...judgment tends to rely more on subjective evaluation (e.g., listening to samples) rather than loss convergence alone.
By the way, studies like F5-TTS also explored FM-based TTS model convergence behavior by tracking metrics like WER across training steps, which can serve as a useful reference (Figure 2 in https://arxiv.org/pdf/2410.06885).

您好，感谢您的回复，我已经复现了这个工作，但是用的都是单声道数据，数据噪声也比较多，感觉我的模型好像过拟合了,我目前是在跑小批量的数据，大概1000条数据， 这是输入人声：

abjones_2_09_vocal.mp3

这是生成的结果：

output.wav

这是混合后的结果：

mixed_torch.wav 不知道对于数据上您有什么好的建议吗？分离过后的音频数据都还是会有一些损失，带有底噪，但是听您的样列音频感觉很干净

I recommend the UVR series models (UVR-MDX-NET-Inst_HQ_3.onnx to UVR-MDX-NET-Inst_HQ_5.onnx) — they are quite good.
However, UVR does not remove reverb. If reverb removal is needed, you can try the MelBand Roformer (anvuew edition) model mentioned in this link: https://github.com/ZFTurbo/Music-Source-Separation-Training/blob/main/docs/pretrained_models.md.

[tanggang1997]

For a small-scale version, several hundred hours should be sufficient in my view, though this is an approximate estimate rather than based on dedicated small-scale experiments.

您好，我是第一次训练flowmatching模型，想问您相关的训练经验，据我了解flowmatching并不是完全看loss是否收敛来判断生成质量，想问您一般是如何去判断模型训练已经完成的

by ears...judgment tends to rely more on subjective evaluation (e.g., listening to samples) rather than loss convergence alone.
By the way, studies like F5-TTS also explored FM-based TTS model convergence behavior by tracking metrics like WER across training steps, which can serve as a useful reference (Figure 2 in https://arxiv.org/pdf/2410.06885).

您好，感谢您的回复，我已经复现了这个工作，但是用的都是单声道数据，数据噪声也比较多，感觉我的模型好像过拟合了,我目前是在跑小批量的数据，大概1000条数据， 这是输入人声：
abjones_2_09_vocal.mp3
这是生成的结果：
output.wav
这是混合后的结果：
mixed_torch.wav 不知道对于数据上您有什么好的建议吗？分离过后的音频数据都还是会有一些损失，带有底噪，但是听您的样列音频感觉很干净

I recommend the UVR series models (UVR-MDX-NET-Inst_HQ_3.onnx to UVR-MDX-NET-Inst_HQ_5.onnx) — they are quite good. However, UVR does not remove reverb. If reverb removal is needed, you can try the MelBand Roformer (anvuew edition) model mentioned in this link: https://github.com/ZFTurbo/Music-Source-Separation-Training/blob/main/docs/pretrained_models.md.

感谢您的指导，时隔一段时间，我自己基于您的论文进行了一些修改，这是我跑出来的一些结果，想和您分享一下

mixed_torch8.wav

mixed_torch6.wav

mixed_torch2.wav

mixed_torch1.wav