How to stop text generation after obtaining the answer

[BennyTMT]

I tried "GRPO_From_Scratch"—and learned a lot, Thanks!

A small issue: During training/inference, even after Qwen1.5 has reached the answer, it continues generating text.

. . . <answer>66</answer>Human: In a classroom there are 30 students who all need individual attention from the teacher due to special needs. The school has two types of chairs available - standard . . . 

I tried this during training/inference on math tasks, and it usually had no impact. But for some tasks, it might affect reward calculation.
Have anyone considered how to prevent this?

[BennyTMT]

There's a simple approach for this.
In my task, the model is first "warmed up" using SFT before GRPO training.
By simply adding Qwen’s end-of-sequence token during the SFT phase. The model will learn to stop reasoning after get the answer.

<|im_start|>
_Question_
<reasoning>_Process_</reasoning>
<answer>_Answer_</answer><|im_end|>

[SuYihang66]

you can add another format_reward_func to punish the answer length after .