Getting Started with SO 101
The aim of this assignment is to get you started with our new SO 101 robots. You will undoubtedly run into problems trying to get all this stuff working. Feel free to use AI or ask a friend or ask us for help. As an experiment, we are going to try using Github Issues for you to ask questions about this project (instead of EdStem or Canvas), so please post any technical problems or questions there.
- First we want to install LeRobot, an open source software framework for robot learning, with many common algorithms and datasets. Follow the instructions to install LeRobot. Verify that you can run
lerobot-teleoperatewith no arguments and get a help message about how to run the program (vs file not found because it can't find the program). - We will next explore a 2D simulation domain that is low-dimensional to make sure everything is working. This task is called the Push T task because it involves pushing a 2D T shaped block to a goal location. Follow this tutorial to install the Push T task. Verify you can see the robot moving the block around in 2D space.
- Now we will install IsaacSim and LeIsaac, which is a 3D simulator that can simulate the SO 101. Follow these instructions to install. Verify you can see the pickup the red cube task, even if it's running with random actions.
- Use your phone to control the robot. Verify you can make the robot move with your phone in LeIsaac to pick up the red cube.
- Assemble the SO-101. Verify you can move it with your phone!
- We did not yet buy cameras for the robot. Spend 20 minutes researching camera options for the SO 101 and add your results to the Pugh chart on the wiki. If necessary, add more columns to the wiki.
- Extra credit: Collect 10 demonstrations of a touch target task, moving the end effector from a random starting location to a fixed end location. Train a diffusion policy on this task. Demonstrate the policy running on the robot. </ul>