Connecting to sources…

How to build Stanford Pupper v3: the whole process

105 steps to build Stanford Pupper v3, from buying parts, printing and assembly to software, calibration and training, with the bill of materials, shop links and the off…

The docs put a self-build at about $2000; the BOM sheet totals about $2,158.91 (minimum $1,785.91, of which 12 GIM4305 joints are about $1,320). Present Perfection full kit $3,400, assembled $3,800 (2026‑10‑31, before tax and shipping)

Buy parts

  • Purchase Parts From the BOM

    When building Pupper from scratch, refer to the linked bill of materials and purchase each required part individually.

  • Buy a Pupper Kit

    Alternatively, purchase a kit listed in the document.

  • Source the Two Custom PCBs

    Sourcing the two custom PCBs individually, rather than in a kit, is a little tricky.

  • Prepare the Pupper v3 control board

    Prepare a Pupper v3 control board and a Raspberry Pi 4 running Ubuntu 22.04 with ROS2 Humble. The recommended realtime image is 22.04.1_v5.15.39-rt42-raspi_ros2_humble.

Assemble

  • Copy the CSV animation file

    Copy the csv file to ros2_ws/src/animation_controller_py/launch/animations.

Software

  • Install a Docker engine

    Install Docker Desktop or another Docker engine before building the Pupper OS image.

  • Update Docker

    Make sure Docker is up to date; older Docker versions will not work after upgrading MacOS.

  • Create the local environment file

    Create `.env.local` in the `pupper_image_builder` directory based on `.env.example`, and include a Github personal access token (PAT) for reliable Git LFS use.

  • Build the base PiOS image

    Run the base PiOS image build script.

  • Build the full PiOS image

    Run the full PiOS image build script.

  • Build the PiOS AI image

    Run the AI image build script.

  • Bake cloud AI keys into the image

    To use chat AI capabilities out of the box, add the relevant cloud AI keys to `.env.local`.

  • Build the full image with keys

    Run the full image build script with the `--include-keys` flag to include the cloud AI keys from `.env.local` in the image.

  • Enable automatic login

    Optionally enable the Pi to log in automatically as the `pi` user on boot by editing `/etc/gdm3/custom.conf`.

  • Blacklist DualSense controller audio

    Edit /etc/modprobe.d/blacklist.conf and add blacklist snd_usb_audio to resolve the lack of sound from the Pupper speaker on PiOS.

  • Increase Docker Desktop memory limit

    Open the resources option in Docker Desktop settings and increase the RAM memory limit; this is primarily needed when building ROS on PiOS.

  • Disable NetworkManager wait-online service

    Run the command after booting the actual RPi to disable NetworkManager-wait-online.service, which adds one minute to startup time.

  • Configure the PiOS boot command line

    Write the complete shown line into cmdline.txt. It selects HDMI-A-1 at 720x720M@60D with 270-degree rotation, sets the JP wireless regulatory domain, and runs /boot/firstrun.sh at boot with a reboot after success.

  • Create the PiOS first-run script

    Save the shown script as /boot/firstrun.sh. It sets the hostname to pupper and updates /etc/hosts, enables SSH, configures the first user as pi with the shown password hash and /home/pi, adjusts automatic-login and sudo settings, and configures the wireless SSID, password, and JP.

  • Configure the turn detector

    Set `version` to `v1` for the full LiveKit Inference model or `v1-mini` for local CPU execution, or omit it for automatic selection. Optionally set `unlikely_threshold` as a scalar or language-keyed dictionary, and set `local_fallback` to control fallback from `v1` to `v1-mini`; it defaults to `True`.

  • Force the detector language

    Configure the STT model with the required language so the turn detector can apply its language-specific threshold. For example, configure Spanish with `language="es"` or `language: 'es'`.

  • Use the detector with Python

    Create an AgentSession with inference.TurnDetector() and set the realtime model's turn_detection to None. Do not add an STT plugin.

  • Use the detector with Node.js

    Create a voice.AgentSession with a new inference.TurnDetector(). Set the realtime model's turnDetection property to null.

  • Choose suitable agent compute

    Use compute-optimized instances such as AWS c6i or c7i when v1-mini runs on a self-hosted agent. Avoid burstable instances such as AWS t3 or t4g to reduce inference timeout risk from CPU credit limits.

  • Install the Python plugin

    Install the turn-detector plugin from PyPI using uv.

  • Install the Node.js plugin

    Install the LiveKit agents turn-detector plugin from npm using pnpm.

  • Initialize the Python agent session

    Create an AgentSession with MultilingualModel for turn detection and LiveKit Inference for multi-language STT.

  • Initialize the Node.js agent session

    Create a voice AgentSession with the LiveKit MultilingualModel turn detector and LiveKit Inference for multi-language STT.

  • Provide STT for realtime models

    Provide an STT plugin to the `AgentSession` when using the text turn detector with a realtime model.

  • Disable built-in realtime turn detection

    Disable the realtime model's built-in turn detection so it does not conflict with the LiveKit turn detector.

  • Configure realtime session in Python

    Configure `AgentSession` with `TurnHandlingOptions`, `MultilingualModel`, Silero VAD, Deepgram STT, and `openai.realtime.RealtimeModel` using the `alloy` voice. Set built-in turn detection and input audio transcription to `None`.

  • Configure realtime session in Node.js

    Configure `voice.AgentSession` with `livekit.turnDetector.MultilingualModel`, Silero VAD, Deepgram STT, and `openai.realtime.RealtimeModel` using the `alloy` voice. Set built-in turn detection and input audio transcription to `null`.

  • Clone repository and install dependencies

    Enter the `agent-starter-python` directory and use `uv sync` to install dependencies into a virtual environment.

  • Create the local environment file

    Copy `.env.example` to `.env.local`, then fill in the required environment variable values.

  • Set the LiveKit URL

    Set `LIVEKIT_URL` to a LiveKit Cloud address or to a LiveKit service you host yourself.

  • Set the LiveKit API key

    Enter the `LIVEKIT_API_KEY` in `.env.local`.

  • Set the LiveKit API secret

    Enter the `LIVEKIT_API_SECRET` in `.env.local`.

  • Configure the OpenAI key

    Enter `OPENAI_API_KEY` in `.env.local`, or use your preferred LLM provider.

  • Configure the Deepgram key

    Enter `DEEPGRAM_API_KEY` in `.env.local`, or use your preferred STT provider.

  • Configure the Cartesia key

    Enter `CARTESIA_API_KEY` in `.env.local`, or use your preferred TTS provider.

  • Load the LiveKit environment automatically

    Use the LiveKit CLI to load the LiveKit environment automatically from `.env.local`.

  • Download models for the first run

    Before the first run, download required models such as Silero VAD and the LiveKit turn detector.

  • Flash the Raspberry Pi 5 image

    Follow the linked software installation instructions to flash the Raspberry Pi 5 with the project's custom image.

  • Install development dependencies

    Install git-lfs, initialize Git LFS, clone the monorepo with submodules, and run install_dev_dependencies.sh.

  • Build the ROS 2 workspace

    Enter the ros2_ws directory and run build.sh to build the workspace.

Calibrate and test

  • Set the VAD silence duration

    Enable VAD for the audio turn detector and set `min_silence_duration` to at least `0.25` seconds (250 ms). The Silero VAD default of `0.55` seconds already satisfies this requirement.

  • Use the default English thresholds

    When STT is not configured, such as with a realtime model, the turn detector uses English thresholds. The supported detector languages are English, Arabic, German, Spanish, French, Hindi, Indonesian, Italian, Japanese, Korean, Dutch, Portuguese, Turkish, and Chinese.

  • Set a global Python threshold

    Set `unlikely_threshold` to `0.5` when creating the Python `TurnDetector`. Lower values make the detector more eager to respond, while higher values make it more patient.

  • Set a global Node.js threshold

    Set `unlikelyThreshold` to `0.5` when creating the Node.js `TurnDetector`. Lower values make the detector more eager to respond, while higher values make it more patient.

  • Set per-language Python thresholds

    Provide an `unlikely_threshold` mapping to override English and Japanese thresholds; unmapped languages retain their defaults. If the session falls back from `v1` to `v1-mini`, the override is rescaled to preserve its relationship to the active model's calibrated defaults.

  • Set per-language Node.js thresholds

    Provide an `unlikelyThreshold` mapping to override English and Japanese thresholds; unmapped languages retain their defaults. If the session falls back from `v1` to `v1-mini`, the override is rescaled to preserve its relationship to the active model's calibrated defaults.

  • Choose the detector model version

    Leave version unset to let the detector select a version based on the environment. Set version explicitly, for example to "v1", to use the full model in a self-hosted agent or pin one version everywhere.

  • Configure full-model authentication

    Provide the LiveKit Cloud project's LIVEKIT_API_KEY and LIVEKIT_API_SECRET when using the full model. Self-hosted or local full-model usage counts toward the monthly development allowance.

  • Check default turn detection

    An AgentSession uses TurnDetector by default when turn_detection is omitted. The exception is a realtime LLM that provides its own server-side turn detection.

  • Use fixed endpointing timing

    Set the endpointing mode to fixed so the configured min_delay and max_delay determine the delay. The default mode is fixed.

  • Use dynamic endpointing timing

    Set the endpointing mode to dynamic to adapt the delay within the min_delay and max_delay range using session pause statistics based on an exponential moving average of between-utterance and between-turn pauses.

  • Set the minimum endpointing delay

    Set min_delay, which defaults to 0.5 seconds, as the minimum time to wait after the last detected speech. With dynamic endpointing it is the lower bound; in VAD mode it behaves like max(VAD silence, min_delay), while in STT mode it applies after the STT end-of-speech signal.

  • Set the maximum endpointing delay

    Set max_delay, which defaults to 3.0 seconds, as the maximum time to wait before terminating the turn. It prevents the agent from waiting indefinitely for the user to continue speaking.

  • Record the default credentials

    The default hostname is `pupper`, the user is `pi`, and the password is `rhea123`.

  • Reduce first-boot wait time

    Disable `NetworkManager-wait-online.service` to reduce boot time from about 90 seconds to about 30 seconds; do not do this when using AI.

  • Pair the Bluetooth controller

    Pair a PS4 or PS5 controller over Bluetooth.

  • Connect to Wi-Fi

    Optionally connect to Wi-Fi.

  • Customize joystick buttons

    Start the recorder with the record button set to 4 and the stop button set to 5.

  • Customize the output directory

    Start the recorder with the bag output directory set to /custom/path/bags.

  • Configure recorder parameters

    record_start_button is the joystick button index for starting recording, with a default of 6; record_stop_button is the joystick button index for stopping recording, with a default of 7. bag_output_dir is the directory for saving bag files, with a default of ~/bags.

  • Talk to Agent Locally

    Talk to the newly created agent locally.

  • Test with Simulated Users

    Test the agent against simulated users.

  • Adjust camera frame rate

    The camera FPS is 10hz by default. Adjust it in ros2_ws/src/neural_controller/launch/config.yaml with the FrameDurationLimits: [100000, 100000] parameter.

  • Match animation frame rate and fade time

    If editing the animation frame rate or fade time, edit both config.yaml and pupster.yaml.

  • Configure ros2_control for realtime

    Configure ros2_control for realtime operation using the linked ros2_control Controller Manager documentation.

  • Configure the ros2_control hardware interface
  • Test configuration

    Run test_config.py to test the configuration.

Data and training

  • Start recording animation data

    Hold L1 until the BAG status icon turns green, indicating that an mcap bag recording is in progress.

  • Move Pupper through the motion

    Move Pupper through the desired motion.

  • Stop the recording

    Press R1 to stop recording.

  • Verify the animation in Foxglove

    View the recorded mcap file in Foxglove to verify the animation.

  • Store the recorded bag

    Move the recorded bag to pupperv3-monorepo/bags.

  • Convert mcap to CSV

    On the robot, run scripts/mcap_to_csv.py with the mcap path, absolute start time, and absolute end time to convert it to csv.

  • Run basic training

    Run pupper_mjx_rl_training.py for basic training.

  • Reduce training timesteps

    Set fewer training timesteps with training.ppo.num_timesteps=100000000.

  • Use a smaller network

    Use a smaller network with policy.hidden_layer_sizes=[128,64,64,64].

  • Enable height field

    Enable the height field with training.height_field_random=true.

  • Load from checkpoint

    Load from the specified checkpoint with training.checkpoint_run_number=238.

  • Read Agent Sessions

    Read the agent's sessions to find and fix problems.

Use it

  • Speak to the agent in the terminal

    Run the command to speak to the agent directly in your terminal.

  • Run the agent for a frontend or telephony

    Use the `dev` command to run the agent for use with a frontend or telephony.

  • Start the agent in production

    In production, run the agent with the `start` command.

  • Filter raw image topics

    The recorder excludes topics containing image_raw, camera/image_raw, rgb/image_raw, or depth/image_raw to save storage space.

Parts

  • 12 Steadywin GIM4305 joints (4005 brushless motor with 10:1 planetary gearbox, ~3.5 Nm peak, ~1.0 Nm continuous, ~30 rad/s)
  • Raspberry Pi 5 8GB with an M.2 slot for an AI accelerator (Raspberry Pi AI Kit optional in the BOM)
  • Custom Pi HAT control board and power distribution board (JLCPCB assembly; the BNO086 must come through global sourcing)
  • 4-inch LCD for facial animations, 3W speaker, IMX296 fisheye camera, 9g ear servos
  • Two DeWalt 20V PowerStack 1.7Ah batteries, a PS5 controller
  • PETG and TPU printed structure, M2.5 heat-set inserts, squash balls as feet

Tools

  • 3D printer (PETG and TPU)
  • Soldering iron (250°C for heat-set inserts)
  • Electric screwdriver and super glue (BOM)

Skills

  • Flashing motor-driver and control-board firmware
  • Ordering PCB assembly (JLCPCB global sourcing plus PCBA)
  • ROS 2 and Linux (SSH, building the workspace)
  • Reinforcement learning and MuJoCo/MJX simulation (CS 123 course)

Software

  • pupperv3-monorepo (ROS 2 control, neural gait controller, AI agent)
  • pupperv3-mjx (MJX reinforcement-learning training)
  • pupOS Raspberry Pi images (full and AI builds)
  • Foxglove Studio visualisation

Documents