add policy client for GR00T N1.6#104
Conversation
|
@12e21 Thank you for your contribution. |
|
@12e21 Hi, I’m currently testing this PR, but I ran into an issue when fine-tuning gr00t n1.6. I’m fine-tuning on a 4090, but even when I set |
|
@EverNorif I tried finetuning with 4090 24GB, indeed OOM with batch size 1; on A100 or A6000, it works with about 40GB VRAM |
|
@EverNorif I can share my config and upload finetuned policy model to huggingface, so you can test it |
Okay, that would be better. |
|
@EverNorif I have upload configs, scripts, and checkpoint, find at 12e21/gr00t_n1d6_leisaac_pick_orange |
Summary:
add Gr00t16ServicePolicyClient to support GR00t N1.6 new policy server api #103 , compatibility with other policy types
, use as:
python scripts/evaluation/policy_inference.py \ --task=LeIsaac-SO101-PickOrange-v0 \ --eval_rounds=10 \ --policy_type=gr00tn1.6 \ --policy_host=localhost \ --policy_port=5555 \ --policy_timeout_ms=5000 \ --policy_action_horizon=16 \ --policy_language_instruction="Pick up the orange and place it on the plate" \ --device=cuda \ --enable_camerasVerification:
Tested locally: Confirmed successful deploy of finetuned GR00T N1.6 model