Trying VLA (Part 6): Controlling LeRobot with a SpaceMouse

Mapping SpaceMouse Controls to SO-101 Movements In the previous article, I connected the SpaceMouse to the PC and confirmed that all six types of input could be detected correctly. https://dev.to/takeofuture/trying-vla-part-5-setting-up-and-testing-a-spacemouse-eio Forward / Backward Left / Right Up / Down Pitch Roll Yaw During the SpaceMouse test, I confirmed the following input values. Forward horizontal = +Y Backward horizontal = -Y Left horizontal = -X Right horizontal = +X Up = +Z Down = -Z Forward tilt = +Pitch Backward tilt = -Pitch Left tilt = -Roll Right tilt = +Roll Left twist = -Yaw Right twist = +Yaw An important point here is that these values from the SpaceMouse are not sent directly to individual SO-101 motors . Conceptually, the flow from the SpaceMouse to the SO-101 looks like this: SpaceMouse ↓ x / y / z / roll / pitch / yaw ↓ SpaceMouse Teleoperator ↓ target_x / target_y / target_z target_wx / target_wy / target_wz ↓ Inverse Kinematics (IK) ↓ SO-101 Joint Positions ↓ SO-101 The SpaceMouse plugin treats the 6DoF input from the SpaceMouse as movement of the End Effector in Cartesian coordinates. The target movement is then converted into the required SO-101 joint angles using IK, or Inverse Kinematics . In other words, instead of directly specifying something like: "Move this motor by 5 degrees" we provide commands such as: "Move the End Effector slightly forward" "Move the End Effector slightly upward" "Rotate the End Effector slightly" The SpaceMouse provides these commands, and IK calculates how the individual joints need to move. Mapping Between SpaceMouse and LeRobot Coordinates There is one thing we need to be careful about here. The x and y values displayed by the SpaceMouse test do not directly become LeRobot's target_x and target_y . With the default SpaceMouse plugin configuration, the axes are mapped as follows: SpaceMouse y -> target_x SpaceMouse x -> target_y SpaceMouse z -> target_z SpaceMouse roll -> target_wx SpaceMouse pitch -> targ