VLR Vision Language Model | Large Scale Training

FINETUNING

• Go to finetune.sh and change the model type
• Using finevision (data path should be a directory with parquet files.
• Just run ./finetune.sh

The code is optimized for the Marenostrum5 HPC system, with H100s.

Features

• Qwen2.5-VL & Qwen3-VL Support
• distributed checkpoints
• optimizer & scheduler checkpoints
• compile
• deterministic
• better args + config
• data parallel
• FSDP
• compile
• static shape compile (fullgraph)
• FSDP multinode
• data packing

Models Supported

• Qwen3-VL series
• Qwen2.5-VL series
• Qwen2-VL series

DISCLAIMER

This code was originally the Qwen3-VL codebase developed by Qwen team, Alibaba Cloud. We didnt change the license.