Cannot load checkpoint #6
Description
Hi, I was trying to run the teacher policy by using the pth from transic-model/rl/ and I run the following:
python main/rl/train.py task=Stabilize test=true checkpoint=/home/xinkai/extern/transic-models/rl/stabilize.pth save_rollouts=true.
It shows one bug about invalid load key like:
=> loading checkpoint '/home/xinkai/extern/transic-models/rl/stabilize.pth' Exception invalid load key, 'v'. when trying to execute functools.partial(<function load at 0x7e74b126cee0>, map_location='cpu') with args:('/home/xinkai/extern/transic-models/rl/stabilize.pth',) and kwargs:{}... Traceback (most recent call last): File "main/rl/train.py", line 218, in launch_rlg_hydra runner.run( File "/home/xinkai/miniconda3/envs/transic/lib/python3.8/site-packages/rl_games/torch_runner.py", line 135, in run self.run_play(args) File "/home/xinkai/extern/transic/transic/rl/runner.py", line 41, in run_play _restore(player, args, is_train_restore=False) File "/home/xinkai/extern/transic/transic/rl/runner.py", line 17, in _restore agent.restore(args["checkpoint"]) File "/home/xinkai/extern/transic/transic/rl/player.py", line 574, in restore checkpoint = load_checkpoint(fn) File "/home/xinkai/extern/transic/transic/rl/player.py", line 28, in load_checkpoint state = safe_load(filename) File "/home/xinkai/extern/transic/transic/rl/player.py", line 23, in safe_load return safe_filesystem_op(partial(torch.load, map_location="cpu"), filename) File "/home/xinkai/miniconda3/envs/transic/lib/python3.8/site-packages/rl_games/algos_torch/torch_ext.py", line 69, in safe_filesystem_op raise RuntimeError(f'Could not execute {func}, give up after {num_attempts} attempts...') RuntimeError: Could not execute functools.partial(<function load at 0x7e74b126cee0>, map_location='cpu'), give up after 5 attempts...
I am not sure if it is the problem of checkpoints is out of date or something else. Could you help me with that? Thanks!