Training

Training FloWM and baseline models

Training will produce a wandb link that leads to the logs, and also a local directory to where the model checkpoints are saved. To resume the a run with your checkpoint, fill in the model training checkpoint path to load= or the model weights path to algorithm.load_model_state. To resume the same wandb run, fill in resume={the_wandb_run_id}.

Make sure to first download the data, see the Dataset page.

• These commands assume ckpt_map=default (see configurations/ckpt_map/default.yaml).
• +name=... is the run name (used for logging / output directories); feel free to change.
• Feel free to change the batch size or number of devices in the shortcode config if it does not match your server setup.

Blockworld Training

Below are training commands for Blockworld, under /shortcode/exp/blockworld/**/train/...

Split	Method	Command
Dynamic Blockworld	FloWM	python -m main shortcode=exp/blockworld/flowm/train/dynamic +name=blockworld_flowm_10m_dynamic_50c90p algorithm=flowm_video dataset=blockworld ckpt_map=default
	DFoT	python -m main shortcode=exp/blockworld/dfot/train/dynamic_70ctf_140wf +name=blockworld_dfot_140ctx algorithm=dfot_video dataset=blockworld ckpt_map=default
	DFoT-SSM	python -m main shortcode=exp/blockworld/block-ssm/train/dynamic_70ctf_140wf +name=blockworld_ssm_70ctf_140wf algorithm=dfot_video dataset=blockworld ckpt_map=default
Textured Blockworld	FloWM	python -m main shortcode=exp/blockworld/flowm/train/tex +name=blockworld_flowm_10m_tex_50c90p algorithm=flowm_video dataset=blockworld ckpt_map=default
	DFoT	python -m main shortcode=exp/blockworld/dfot/train/tex_70ctf_140wf +name=blockworld_dfot_tex_140ctx algorithm=dfot_video dataset=blockworld ckpt_map=default
	DFoT-SSM	python -m main shortcode=exp/blockworld/block-ssm/train/tex_70ctf_140wf +name=blockworld_ssm_tex_140ctx algorithm=dfot_video dataset=blockworld ckpt_map=default
Static Blockworld	FloWM	python -m main shortcode=exp/blockworld/flowm/train/static +name=blockworld_flowm_10m_static_50c90p algorithm=flowm_video dataset=blockworld ckpt_map=default
	DFoT	python -m main shortcode=exp/blockworld/dfot/train/static_70ctf_140wf +name=blockworld_dfot_static_140ctx algorithm=dfot_video dataset=blockworld ckpt_map=default
	DFoT-SSM	python -m main shortcode=exp/blockworld/block-ssm/train/static_70ctf_140wf +name=blockworld_ssm_static_140ctx algorithm=dfot_video dataset=blockworld ckpt_map=default

MNIST World Training

Below are training commands for MNIST World, under /shortcode/exp/mnist_world/**/train/...

Split	Method	Command
Dynamic PO MNIST World	FloWM	python -m main shortcode=exp/mnist_world/flowm/train/dynamic_po_50ctf_70wf +name=mnistworld_flowm_dynamic algorithm=flowm_video dataset=mnist_world ckpt_map=default
	DFoT	python -m main shortcode=exp/mnist_world/dfot/train/dynamic_po_50ctf_70wf +name=mnistworld_dfot_dynamic algorithm=dfot_video dataset=mnist_world ckpt_map=default
	DFoT-SSM	python -m main shortcode=exp/mnist_world/block-ssm/train/dynamic_po_50ctf_70wf +name=mnistworld_ssm_dynamic algorithm=dfot_video dataset=mnist_world ckpt_map=default