Skip to content

[datapipe] Support wenet datapipe#182

Merged
cdliang11 merged 7 commits intomainfrom
datapipe
Feb 24, 2025
Merged

[datapipe] Support wenet datapipe#182
cdliang11 merged 7 commits intomainfrom
datapipe

Conversation

@mlxu995
Copy link
Collaborator

@mlxu995 mlxu995 commented Feb 5, 2025

No description provided.

@a122760
Copy link

a122760 commented Feb 5, 2025

可以支持正负样本平衡的dataloader吗,当前框架下有什么好的实现方式呢?就是每个iter都平衡正负样本。正负样本分开做两个list?

@mlxu995
Copy link
Collaborator Author

mlxu995 commented Feb 6, 2025

可以支持正负样本平衡的dataloader吗,当前框架下有什么好的实现方式呢?就是每个iter都平衡正负样本。正负样本分开做两个list?

好需求,目前想到的也是通过分成两个list来做

@mlxu995 mlxu995 marked this pull request as ready for review February 21, 2025 07:44
@mlxu995 mlxu995 requested a review from cdliang11 February 21, 2025 07:50
@cdliang11
Copy link
Contributor

Good Job!

@mlxu995
Copy link
Collaborator Author

mlxu995 commented Feb 21, 2025

  • result of e2e loss on keyword "hi xiao wen"
    image
    and on keyword "ni hao wen wen"
    image
    image

  • result of ctc loss on keyword "hi xiao wen"
    image
    and on keyword "ni hao wen wen"
    image

@cdliang11 cdliang11 merged commit c5da435 into main Feb 24, 2025
4 checks passed
@cdliang11 cdliang11 deleted the datapipe branch February 24, 2025 06:51
@mlxu995
Copy link
Collaborator Author

mlxu995 commented Nov 6, 2025

可以支持正负样本平衡的dataloader吗,当前框架下有什么好的实现方式呢?就是每个iter都平衡正负样本。正负样本分开做两个list?

maybe this is helpful https://arxiv.org/pdf/1912.04486

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants