qwen3-32b-dflash

[gq112]

We trained a Qwen3-32B-DFlash model using the SpecForge framework. Everyone is welcome to test its performance and share feedback.
https://huggingface.co/sxf-aicp/qwen3-32b-dflash-en-zh

[Ofir408]

We trained a Qwen3-32B-DFlash model using the SpecForge framework. Everyone is welcome to test its performance and share feedback.
https://huggingface.co/sxf-aicp/qwen3-32b-dflash-en-zh

Can you share the training configuration in SpecForge?

[gq112]

--num-epochs 6
--batch-size 1
--learning-rate 6e-4
--warmup-ratio 0.04
--max-grad-norm 1.0
--max-length 4096
--chat-template qwen
--block-size 16
--num-draft-layers 5
--num-anchors 512
--loss-decay-gamma 7.0 \

[jianc99]

Given only 40K training samples, the results are promising!

[gq112]

@jianc99 .Actually, just to clarify — the model was trained on 400K samples, not 40K.Sorry for the confusion!

[Sylvan820]

I observed that dflash's performance on mt bench is average. What are the reasons for this? Can you report the performance on the subtasks of mt bench?