Add leader-follower mode for SO101#314
Add leader-follower mode for SO101#314SinedYuk wants to merge 3 commits intoPositronic-Robotics:mainfrom
Conversation
SinedYuk
force-pushed
the
so101-leader-mode
branch
from
d96a014 to
b0ab533
Compare
SinedYuk
force-pushed
the
so101-leader-mode
branch
from
b0ab533 to
599b0b7
Compare
| def run(self, should_stop: pimm.SignalReceiver, clock: pimm.Clock) -> Iterator[pimm.Sleep]: | ||
| self.motor_bus.connect() | ||
| # Initialize kinematics in the subprocess (placo.RobotWrapper is not picklable) | ||
| _ = self.kinematic |
There was a problem hiding this comment.
Very hacky. Find a better way to achieve that. For example make _solve_ik and _forward_kinematics static methods that take kinematics as argument
| q_norm = self.rad_to_norm(qpos) | ||
| q_with_gripper = np.concatenate([q_norm, [self.target_grip.value]]) | ||
| self.motor_bus.set_target_position(q_with_gripper) | ||
| case roboarm_command.NormalizedJointPosition(qpos): |
There was a problem hiding this comment.
We cannot afford introducing new commands that eaily. The solution would be to convert normalised position into radians in the leader, and send this command to follower.
In other words, I want you to remove NormalizedJointPosition.
Moreover, I suppose that the leader should emit its state with the real joint values. This is the API that robotic arms use.
| self.robot_meta_in: pimm.SignalReceiver = pimm.FakeReceiver(self) | ||
|
|
||
| def run(self, should_stop: pimm.SignalReceiver, clock: pimm.Clock) -> Iterator[pimm.Sleep]: | ||
| from positronic.dataset.ds_writer_agent import DsWriterCommand, DsWriterCommandType |
There was a problem hiding this comment.
Import this on top level pls. BTW I know that claude has this tendency.
| from positronic.drivers.roboarm import command as roboarm_command | ||
|
|
||
|
|
||
| class LeaderFollower(pimm.ControlSystem): |
There was a problem hiding this comment.
QQ – do we really need to a separete ControlSystem. Can we just connect two SO101 arms together, where we will read positions from the first one and send them to the second one?
May be just adding passive=True to the arm driver. There's a question on how to enable data collection things, but it's a data collection script concern, not the driver's concern
2 participants
Add leader-follower mode for SO101
Adds physical leader-follower teleoperation for the SO101 arm, enabling data collection without a VR headset.
What's new
LeaderFollowercontrol system (positronic/drivers/roboarm/so101/leader_follower.py)Reads joint positions from a passive leader arm at 100 Hz and emits them as
NormalizedJointPositioncommands to the follower. Recording is toggled via ENTER key with debounce, with sound feedback on start/stop.NormalizedJointPositioncommand (positronic/drivers/roboarm/command.py)New command type for direct normalized
[0,1]motor position control, bypassing IK. Fully wired throughto_wire/from_wire.Lazy kinematics init in SO101
Robot(positronic/drivers/roboarm/so101/driver.py)Kinematicsis now initialized inrun()inside the subprocess rather than in__init__, sinceplaco.RobotWrapperis not picklable. Also handlesNormalizedJointPositioncommands.so101_leaderdata collection entry point (positronic/data_collection.py)New
main_leader_follower()function andso101_leaderconfig registered in the CLI. Deduplication fix forbg_cslist (relevant whenrobot_armandgripperare the same object).Misc updates:
LinuxVideodriver — Recording episodes now works correctly with USB cameras (Arducam, Sonix, etc.)sonixcamera config for USB2.0 CAM1so101_leaderdefault camera — Now uses Sonix camera out of the boxUsage
uv run positronic-data-collection so101_leader --output_dir=~/datasets/so101_runs