Parquet magic bytes not found in footer

[kanna12580]

Hi there,
I'm encountering an issue when trying to train a model using the LeRobot library on a dataset recorded with teleop_se3_agent.py. During training, I get the following error:
'Parquet magic bytes not found in footer'

Upon inspecting the end of the .parquet file:
tail -c 32 /data/s101/datasets/isaac/isaac_s101_liftcube_0202/meta/episodes/chunk-000/file-000.parquet | xxd -g 1
The output shows:

00000000: 00 00 00 00 00 00 18 08 00 00 00 00 00 00 00 00  ................
00000010: 11 11 00 00 00 09 20 02 00 00 00 14 01 01 14 00  ...... .........

The expected "PAR1" magic bytes are missing from the file footer, indicating that the Parquet file was likely truncated during creation or transfer.

I suspect this is because I terminated the recording by pressing Ctrl+C, as I couldn’t find any other documented way to gracefully stop the recording process. Could this be the root cause? If so, what is the correct method to properly end a recording session?

For reference, here’s the command I used to start recording:

python scripts/environments/teleoperation/teleop_se3_agent.py \
  --teleop_device=so101leader \
  --port=COM3 \
  --task=LeIsaac-SO101-LiftCube-v0 \
  --num_envs=1 \
  --device=cuda \
  --enable_cameras \
  --rendering_mode=performance \
  --record \
  --use_lerobot_recorder \
  --lerobot_dataset_repo_id=isaac_s101_liftcube_0202 \
  --lerobot_dataset_fps=30

My system is running Windows.
Any guidance on how to avoid this issue—whether through a proper shutdown procedure or configuration changes—would be greatly appreciated!
Thank you very much for your help.

[EverNorif]

Could you try using this tool to visualize the dataset you collected?

[kanna12580]

Could you try using this tool to visualize the dataset you collected?

Thank you for your reply. I haven't used such a tool for visualization, as my working environment is quite restricted—uploading datasets to Hugging Face is not easy for me. Since I'm training models using LeRobot, and the training fails due to incomplete files (as indicated by the error), I assumed visualization wouldn't work either—although I can directly watch the recorded videos.
However, I managed to solve this issue by first recording the data in HDF5 format and then converting it to the LeRobot v3 format. The converted dataset now works correctly for LeRobot model training.

[EverNorif]

The LeRobot recorder in leisaac is a recently added feature, there may indeed be some issues. We’ll try to reproduce it locally.