Add gradient accumulation and AMP to training scaffold

[Copilot]

Implements gradient accumulation and automatic mixed precision (AMP) for the RT-DETR pose trainer. Gradient clipping already existed via --clip-grad-norm.

Changes

Gradient Accumulation (--gradient-accumulation-steps N)

• Scales loss by 1/N to maintain effective learning rate
• Defers optimizer step to accumulation boundaries: if (steps + 1) % accum_steps == 0
• Logs unscaled loss to preserve metrics accuracy

AMP (--use-amp)

• Initializes torch.cuda.amp.GradScaler on CUDA devices, warns and disables on CPU
• Wraps forward passes (main and MIM teacher) with autocast() context
• Unscales gradients before clipping: scaler.unscale_(optim) → clip_grad_norm_() → scaler.step()

Interaction

• Gradient clipping + AMP: unscales before clipping to avoid clipping scaled gradients
• Gradient accumulation + AMP: loss scaling happens after accumulation scaling

Usage

# Effective batch size 16 with AMP and clipping
python rtdetr_pose/tools/train_minimal.py \
  --device cuda:0 \
  --batch-size 4 \
  --gradient-accumulation-steps 4 \
  --clip-grad-norm 1.0 \
  --use-amp

All features are backward compatible (default disabled) and can be used independently.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• images.cocodataset.org
  • Triggering command: /usr/bin/python3 python3 /home/REDACTED/work/YOLOZU/YOLOZU/tools/fetch_coco128_official.py --out /home/REDACTED/work/YOLOZU/YOLOZU/data/coco128 --insecure (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

gradient clipping
gradient accumulation
AMP

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.