How to build Yertle: the whole process
74 steps to build Yertle, from buying parts, printing and assembly to software, calibration and training, with the bill of materials, shop links and the official sources.
Yertle is a 3D-printed quadrupedal robot for locomotion research. The repository provides 3D-printed parts, assembly instructions, and a bill of materials, along with ESP32 firmware, Python control software, ROS 2, simulation, and reinforcement-learning training pipelines.
Buy parts
- Select the quadruped robot components
The robot has four three-degree-of-freedom limbs driven by hobby servos, with legs approximately 20 cm long. Prepare a 5000mAh battery, a nine-axis accelerometer/gyro, a current and voltage sensor, an ESP32s microcontroller, and an optional RPi4 single-board computer.
Assemble
- Connect the joint command interface
Have /joint_commands receive target joint angles in radians and connect them to a ros2_control forward position controller in Gazebo/RViz or to a small relay that forwards them to the ESP32.
- Choose the joint command destination
Use a ros2_control forward position controller for simulation; on hardware, learning.deploy.servo_command_string can forward the targets to the ESP32 firmware over UDP.
Software
- Choose a Robot Control Method
Run the Python GUI script on a remote device and connect to the robot via WiFi; this is the recommended option. Alternatively, use a remote desktop application to run the script on the RPi.
- Modify IP Addresses in Code
The code contains several IP address references. Modify them in the Python file and the .ino file.
- Set Startup WiFi Connection
The robot needs to connect to WiFi when it starts. The default login details are SSID: dog-net and PASSWORD: dognet123.
- Write firmware in Arduino IDE
Write the ESP32 firmware in C++ using the Arduino IDE.
- Configure two FreeRTOS tasks
Use FreeRTOS to separate tasks across ESP32 cores: one handles WiFi and serial communication, while the other handles I2C communication and inverse kinematics calculations.
- Run the Python tkinter GUI
Use Python3 and the tkinter library to generate a GUI for controlling the robot.
- Connect by WiFi or serial
Connect to the robot through either WiFi or serial communication.
- Exchange sensor and foot data
The robot sends sensor information; the control software generates the feet's positions and sends them back to the robot in real time.
- Read the ROS 2 instructions
Use the ROS 2 package by following the linked ros2/README.md.
- Install reinforcement-learning dependencies
Install the dependencies in requirements.txt and learning/requirements-rl.txt, including Gymnasium, stable-baselines3, PyTorch, and others.
- Configure 12 residual actions
Use 12 joint-angle residuals around a fixed standing pose, tracked by a position controller.
- Configure 48-value observations
Include base linear and angular velocity, projected gravity, the velocity command, joint positions and velocities, and the previous action.
- Configure reward terms
Track commanded forward, lateral, and yaw body velocity while penalizing energy, action rate, orientation, and falling.
- Randomize simulation domains
Randomize base mass, foot friction, and observation noise per episode, and apply random pushes.
- Export trained policy weights
Export the trained policy weights; the residual-around-stance action design maps directly to servo angles.
- Run onboard policy inference
Run inference at the control rate on a Raspberry Pi or laptop, building the observation from the IMU and commanded joint state.
- Stream joint targets over UDP
Stream the resulting joint targets to the ESP32 over the existing UDP path, using the f/a command strings in software/ESP32/firmware.
- Configure the PyBullet reinforcement-learning environment
Wrap the same URDF in a Gymnasium environment with a PyBullet backend, train with PPO, and apply domain randomization to mass, friction, sensor noise, and pushes.
- Install reinforcement-learning dependencies
Install the base requirements and the reinforcement-learning requirements.
- Prepare control software environment
Use Python 3.9 or newer. Clone the repository, enter its directory, and install the Python dependencies in requirements.txt.
- Install tkinter system package
On Debian or Ubuntu, tkinter is a separate system package; install it with the provided apt command.
- Launch control GUI
Run yertle_ui.py from the repository root to launch the Python control GUI.
- Prepare and open ESP32 firmware
Install the ESP32 board package and the FaBoPWM_PCA9685 and MPU9250 libraries, then open firmware.ino in the Arduino IDE.
- Configure WiFi and IP addresses
Set the WiFi credentials and IP addresses near the top of yertle_lib.cpp.
- Create the ROS 2 workspace
In a sourced ROS 2 Linux workspace, create yertle_ws/src and link the repository's yertle_rl package into src.
- Build and source yertle_rl
Build only the yertle_rl package, then source the generated workspace environment.
- Create the Windows ROS 2 environment
After installing Miniforge, create the ros2 environment from conda-forge and robostack-staging, then install pybullet and gymnasium in it.
- Set the maintainer email
Before building, set the maintainer email in package.xml and setup.py; the current value is a placeholder.
- Build the ROS package on Windows
Enter yertle\ros2, run colcon in the ros2 environment, and place build and install outputs under the user's ros_ws directory.
- Install Isaac Lab Dependencies
Use Python 3.10/3.11 and an NVIDIA RTX GPU, and create a dedicated Python 3.11 virtual environment referred to as `<isaac_venv>`. Install the specified PyTorch, TorchVision, and Isaac Lab versions according to Isaac Lab's pip instructions.
- Convert the robot model from URDF to USD
Run the conversion script to convert `simulation/yurtle.urdf` into `simulation/usd/yertle.usd`, merging joints and running headless. This conversion is already done, and the USD file lives in `simulation/usd/`.
- Run the ROS 2 bridge
Run `ros2_bridge.py` to publish `/joint_states` and `/clock` and subscribe to `/joint_command`, connecting the GPU simulation to a ROS 2 stack.
- Convert Yertle URDF to USD
Use Isaac Lab's importer to convert simulation/yertle.urdf into simulation/usd/yertle.usd with merged joints and headless mode. The generated usd/ directory is regenerable and is not committed.
Calibrate and test
- Calibrate and save IMU settings
Initially calibrate the IMU, save the settings in an .ini file, and reuse them when the robot restarts.
- Zero servos before construction
Use the GUI to zero the servos before fully constructing the robot.
- Center servos in control tab
Use the servo control tab to visually center the servos.
- Get PWM and set servo offsets
Press the Get PWM button to obtain the current PWM values, then use those values in the .ino file to set servo offsets.
- Tune standing pose and actuator gains
If the robot spawns collapsed, tune the default standing pose and actuator gains in `yertle_cfg.py`.
- Check model stability
After any URDF or actuator change, command the standing pose with zero action and confirm that the base height holds steady.
- Check the simulation environment
Run the fast environment check with random actions; it requires only pybullet, gymnasium, and numpy, with no reinforcement-learning dependencies.
- Sanitize URDF physics
The raw URDF has invalid inertia tensors and diverges when simulated freely. On load, override each link's inertia with an estimate from its mass and set foot friction.
- Test controller movement in simulation
Use the simulation built into the Python software to test controller movement without a physical robot.
- Run the reinforcement-learning smoke test
Run the smoke test to check the reinforcement-learning environment.
- Check the standing pose output
When the node runs without policy_path, it outputs the standing pose, which is useful for checking the plumbing.
- Configure IMU and joint feedback
Wire the IMU and joint feedback following the sim-to-real notes in learning/README.md, and zero-fill base linear velocity as a privileged term on hardware.
Data and training
- Train the walking policy
Train the walking policy with PPO for 3,000,000 timesteps using 8 environments.
- Deploy the trained policy through UDP
Deploy the trained policy to the physical robot through its existing UDP command path; no firmware change is needed.
- Convert URDF and add Isaac task
In Isaac Sim 5.1 with PhysX, convert the URDF containing firmware joint limits and actuator caps to USD and add it to Isaac Lab's velocity-tracking locomotion task.
- Train in parallel on an RTX GPU
Train with rsl_rl PPO across 4,096 parallel environments on one RTX GPU; the walking policy converges in about ten minutes at roughly 80,000 simulation steps per second.
- Run the rough-terrain task
Run the rough-terrain task using procedural terrain and a height scanner.
- Distill an onboard-sensor-only policy
Perform teacher-student distillation to remove the privileged base-velocity observation without performance loss, producing a policy that runs using onboard sensors only.
- Train flat-terrain locomotion
Run the flat-terrain training command in the Isaac Sim Python environment from the repository root, using 4096 environments for 1500 iterations.
- Train rough-terrain locomotion
Run the rough-terrain training command in the Isaac Sim Python environment from the repository root, using 4096 environments for 1000 iterations.
- Play and record trained policy
Run the flat task with the specified model.pt checkpoint, enable 16 environments, and record a video.
- Distill teacher policy
Run the distillation script in headless mode with model.pt specified as the teacher model.
- Run a short training smoke test
Train PPO for 20,000 timesteps with 2 environments to check the training pipeline.
- Run full policy training
Train for 3,000,000 timesteps with 8 environments; runtime depends on the hardware and ranges from minutes to hours.
- Watch the trained policy
Load the trained policy saved at learning/runs/ppo_yertle/policy.zip and watch it run.
- Record an 8-second PPO demo
Run the video recorder with the PPO policy file and set the duration to 8 seconds.
- Plot training progress
Read the Yertle PPO training log and plot the training progress.
- Distill privileged observations away
Use teacher-student distillation in the GPU pipeline at isaac_lab/distill.py. After removing the base linear velocity term, deployment needs only onboard signals.
- Train the flat policy with GPU
Run from the repo root, accept the Isaac Sim EULA, and run the scripts as plain files rather than with `-m`. On Windows, the scripts handle importing heavy libraries before the Kit app launches, avoiding the DLL-conflict crash. The flat task uses 4096 environments and 1500 iterations.
- Train the rough policy with GPU
- Play and record the flat policy
Use `play.py` to play the flat task with a checkpoint path, 16 environments, and video recording.
- Distill a deployable policy
The flat policy observes the base linear velocity, which the real robot cannot measure. Distillation trains a student from a deployable observation set with no base linear velocity and no height scan to imitate the trained teacher. On this robot, the student mean reward is 40.4 versus the teacher's 40.5, with an imitation loss of about 1e-3.
- Run the ROS 2 locomotion policy
Use the ROS 2 package ros2/yertle_rl to run a trained locomotion policy as a node.
- Connect ROS 2 live topics
Have the policy node subscribe to /cmd_vel and /imu/data and publish 12 joint targets.
- Run walking demo and Isaac bridge
Run a closed-loop walking demo over live topics and use the bidirectional Isaac Sim bridge.
- Run the policy node
Add the repository root to PYTHONPATH so the node can import the learning package, then run the policy node with the specified policy.zip.
Use it
- Run and drive digital twin
After launching the GUI, press Start Simulation to open the PyBullet digital twin, then drive the robot with the arrow keys.
- Control the Robot in PyBullet
The Python control software includes a PyBullet simulation where the robot can be controlled through the UI. Use the keyboard direction keys to move the robot around.