Connecting to sources…

How to build OpenBot (open-source smartphone robot car): the whole process

687 steps to build OpenBot (open-source smartphone robot car), from buying parts, printing and assembly to software, calibration and training, with the bill of materials…

The README puts the body at about $50 (bring your own phone); the BOM links to Amazon and AliExpress with no total. The RTR‑520 four-pack on Amazon US was unavailable with no offer on 2026‑10‑31; the RTR‑TT link goes to Taobao and was not checked

Buy parts

  • Choose an Arduino robot car kit

    Use a basic robot car kit as a starting point; it provides a chassis, motors, and accessories.

  • Obtain a phone mount

    Purchase a phone mount to hold the controlling phone on the robot.

  • Obtain the required components

    Obtain all components listed under Required components; the document provides purchasing links for Germany, the United States, and AliExpress.

  • Buy one Arduino Nano

    Buy one Arduino Nano.

  • Buy four TT motors with tires

    Buy four TT motors with tires.

  • Buy three 18650 batteries

    Buy three 18650 batteries.

  • Buy one 18650 battery holder

    Buy one 18650 battery holder.

  • Buy one USB OTG cable

    Buy one USB OTG cable.

  • Buy a spring or rubber band

    Buy one spring or rubber band.

  • Buy sixteen M3x25 screws

    Buy sixteen M3x25 screws.

  • Buy sixteen M3 nuts

    Buy sixteen M3 nuts.

  • Buy six M3x5 screws

    Buy six M3x5 screws.

  • Buy Dupont cables

    Obtain Dupont cables for the DIY wiring.

  • Provide USB-to-TTL serial

    Provide 1x USB-to-TTL serial interface for communication with the smartphone.

  • Provide motor PWM outputs

    Provide 4x PWM outputs for motor control.

  • Provide battery monitoring pin

    Provide 1x analog pin for battery monitoring.

  • Provide speed sensor pins

    Provide 2x digital pins for the speed meters.

  • Provide ultrasonic sensor pin

    Provide 1x digital pin for the ultrasonic sensor; it is optional.

  • Order an RTR OpenBot vehicle

    RTR OpenBot vehicles can be ordered from openbot.info and are available as RTR_TT and RTR_520.

  • Buy parts for a self-built RTR

    Building an OpenBot RTR yourself requires printing the chassis, manufacturing the PCBs, and buying motors and a phone mount.

  • Buy a ready-to-run OpenBot

    Purchase an RTR-TT or RTR-520 ready-to-run OpenBot on Amazon.

  • Select a compatible alternative MCU

    Choose an MCU with USB-to-TTL serial, four PWM outputs, one analog battery-monitoring pin, two speed-sensor pins, one ultrasonic-sensor pin, and two indicator-LED pins. The ultrasonic sensor and indicator LEDs are optional.

  • Prepare a high-end training computer

    Use a high-end laptop or a workstation with a large amount of RAM and a dedicated GPU, especially when training with larger datasets.

  • Read the disclaimer

    Read the disclaimer before starting the build.

Print and make

  • Manufacture the OpenBot RTR PCBs

    Manufacture the PCBs needed for a self-built OpenBot RTR.

  • Print the shell_bottom chassis

    Print the shell_bottom chassis; the document provides STL and STEP files.

  • Print the shell_top cover

    Print the shell_top cover; the document provides STL and STEP files.

  • Print the phone_mount

    Print the phone_mount; the document provides STL and STEP files.

  • Choose base board variant A

    Choose the bare-board option, using connectors for external motor-driver boards and an external microcontroller board.

  • Choose base board variant B

    Choose the modular board and connect an external microcontroller through its pin header.

  • Choose integrated base board variant C

    Choose the fully integrated base board, which the document recommends for most users.

  • Prepare TT-motor base board

    Prepare one Arduino base board for the TT-motor version.

  • Prepare status LED board

    Prepare one status LED board for the TT-motor version.

  • Prepare bump sensor board

    Prepare the front, top, and back bump sensor boards for the TT-motor version.

  • Prepare speed sensor boards

    Prepare four Arduino speed sensor boards for the TT-motor version.

  • Prepare 520-motor base board

    Prepare one ESP32 base board for the 520-motor version.

  • Prepare status LED board

    Prepare one status LED board for the 520-motor version.

  • Prepare bump sensor board

    Prepare the front, top, and back bump sensor boards for the 520-motor version.

  • Get status LED board files

    Get the status LED board's gerber, BOM, and centroid files.

  • Get top bump board files

    Get the top bump sensor board's gerber, BOM, and centroid files.

  • Get back bump board files

    Get the back bump sensor board's gerber, BOM, and centroid files.

  • Get CH340G front board files

    Get the gerber, BOM, and centroid files for the front bump sensor board using the CH340G USB driver.

  • Get CP2102N front board files

    Get the gerber, BOM, and centroid files for the front bump sensor board using the CP2102N USB driver.

  • Make the ESP32 DRV8870 Motor Board

    Use the linked Gerber, BOM, and pick-and-place centroid files to make the ESP32 DRV8870 motor driver board.

  • Print the blocky body

    Choose the blocky body designed by @Christos-Ps. It has variants with extra internal shell space and a LEGO-compatible top, and requires a 221mm x 150mm print area.

  • Print the phone mount bottom

    Print phone_mount_bottom using the provided STL or STEP file.

  • Print the phone mount top

    Print phone_mount_top using the provided STL or STEP file.

  • Print the tilting phone mount

    To tilt the phone upward by up to 30°, print the tilting version instead of phone_mount_bottom.

  • Clean the printed parts

    Clean the 3D print before proceeding with the build.

Assemble

  • Build a custom chassis and phone mount

    Build an OpenBot chassis and phone mount from a material of your choice, such as wood, cardboard, or styrofoam.

  • Choose the DIY or PCB assembly

    Choose the DIY approach, which uses the L298N motor driver and is recommended for hobbyists with electronics experience, or the custom PCB for a cleaner build or multiple robots.

  • Build a 5V and GND power distributor

    Solder a 6x2 male header to a perfboard. Connect the power distributor to the 5V and GND of the motor driver.

  • Prepare motors and encoder disks

    Solder wires to the motors. Add encoder disks to the two front motors if using the speed sensors.

  • Connect motors to the L298N

    Connect the two left motors to OUT1 (+) and OUT2 (-) on the L298N. Connect the two right motors to OUT4 (+) and OUT3 (-) on the L298N.

  • Mount the motors

    Mount the motors using eight M3x25 screws and nuts.

  • Mount the L298N board

    Mount the L298N board using four M3x5 screws.

  • Install the optional ultrasonic sensor

    Install the ultrasonic sensor. Replace its angled connector with a straight one, or carefully bend the pins.

  • Install optional indicator LEDs

    Install the orange LEDs for the indicator signals.

  • Attach the phone mount base

    Mount the bottom of the phone mount to the top plate using two M3x25 screws and nuts.

  • Complete the phone mount

    Insert the top section of the phone mount. Install the spring or rubber band.

  • Mount the speed sensors

    Replace the angled connector with a straight one, or carefully bend the pins. Mount each speed sensor with one M3x5 screw.

  • Install the battery case

    Install the battery case, for example using velcro.

  • Insert the optional power switch

    Place the on/off switch in the current path. Push it into the appropriate opening until it clicks.

  • Solder the switch into the power path

    Solder the battery case and power cable red wires (12V) to separate switch pins. Connect the black wires (GND) and cover the connections with heatshrink.

  • Secure the power cables

    Fix the cables with tape.

  • Attach the OLED display

    Optionally attach the OLED display.

  • Connect the L298N PWM inputs

    Connect the L298N PWM inputs to Arduino pins D5, D6, D9, and D10.

  • Power the speed and ultrasonic sensors

    Connect the speed sensors and ultrasonic sensor to 5V and GND.

  • Connect the speed sensor outputs

    Connect speed sensor pin D0 to Arduino pin D2 for the left sensor and D3 for the right sensor.

  • Connect the ultrasonic sensor signals

    Connect the ultrasonic sensor Echo and Trigger pins to Arduino pins D11 and D12, respectively.

  • Connect the indicator LEDs

    Optionally connect the left LED to D4 and the right LED to D7, with both connected to GND. A series 150 Ohm resistor is recommended to limit current draw.

  • Connect the battery voltage divider

    Optionally connect the voltage divider to Arduino pin A7 for measuring battery voltage.

  • Connect the OLED over I2C

    Optionally connect the SSD1306 OLED display to the Arduino Nano through the I2C bus.

  • Connect OLED power pins

    Connect the display VIN and GND pins to 5V and GND.

  • Connect the OLED clock pin

    Connect the display SCL pin to Arduino pin A5.

  • Connect the OLED data pin

    Connect the display SDA pin to Arduino pin A4.

  • Power the L298N

    Connect the power cables to the L298N’s +12V and GND terminals.

  • Route the Arduino USB cable

    Connect the USB cable to the Arduino and route it through the top cover.

  • Mount the top cover

    Insert six M3 nuts into the bottom plate and mount the top cover using six M3x25 screws.

  • Install the wheels

    Install the wheels.

  • Prepare the motors

    Solder Micro JST PH 2.0 connector wires to the motors. Add encoder disks to the two front motors if speed sensors will be used.

  • Mount the motors

    Mount the motors with eight M3x25 screws and nuts.

  • Connect the motors to the PCB

    Connect the left two motors to M3 and M4, and the right two motors to M1 and M2.

  • Mount the PCB and motors

    Mount the PCB with four M3x5 screws and mount the motors with eight M3x25 screws and nuts.

  • Continue the DIY assembly steps

    Follow steps 5 through 12 from the DIY option.

  • Connect the ultrasonic sensor

    Connect the ultrasonic sensor VCC/+, Trig, Echo, and GND/- pins to the PCB’s four-pin SONAR header.

  • Connect the indicator signals

    Connect the orange LEDs to the PCB’s SIGNAL_L and SIGNAL_R two-pin headers. The longer leg is positive and the shorter leg is negative.

  • Connect the speed sensors

    Connect the left and right speed sensor VCC/+, GND/-, and D0 pins to the PCB’s SPEED_L and SPEED_R three-pin headers.

  • Connect the OLED to IO2

    Optionally connect the SSD1306 OLED display to the PCB’s IO2 header.

Software

  • Configure Firmware in the Browser

    Open tools.openbot.org/firmware, select the robot and parts, then use Chrome or Edge on a computer. Flash the firmware directly or download the setup file for Arduino IDE.

  • Edit Firmware Settings Manually

    When configuring the firmware directly, change only the SETTINGS section at the top of openbot.ino. Do not change anything below it.

  • Preserve Settings During Updates

    To preserve settings when updating the firmware, copy the lines between #else and #endif in the SETTINGS section into a new openbot_settings.h file beside openbot.ino and edit them there. If openbot_settings.h exists, the firmware uses it instead of the section in openbot.ino, and Git ignores the file so it is not pushed accidentally.

  • Select the Robot Body

    Set `OPENBOT` as the robot body in Arduino IDE and select a board supported by that body. The firmware detects the board and stops the build with an error on an unsupported board.

  • Use the Updated DIY Body

    Use `DIY` instead of `DIY_ESP32` when the selected body is an ESP32 board.

  • Enable No Phone Mode

    Set `NO_PHONE_MODE 1` to use No Phone mode instead of normal phone-app control.

  • Enable Gamepad Control

    On ESP32, set `HAS_GAMEPAD 1` to operate the robot with a gamepad.

  • Configure Coast and Debug Modes

    Set `COAST_MODE` to `1` for motor coasting or `0` for braking when there is no control input. Set `DEBUG 1` to display debug output in the serial monitor.

  • Set the Bluetooth Name

    Set `BLE_NAME` to the robot name shown in the app. The name may contain up to 20 characters, numbers, spaces, hyphens, or underscores, such as `#define BLE_NAME "Team 3"`.

  • Review Component Defaults

    Each component in the settings defaults to the value for the selected body, such as `#define HAS_SONAR DEFAULT_HAS_SONAR`. Review the body-specific defaults listed for the components.

  • Enable or Disable Components

    Override a component default with `1` to enable it or `0` to disable it.

  • Disable an Uninstalled Sonar

    If the sonar sensor is not installed, set `#define HAS_SONAR 0` so the app does not show meaningless readings such as 300 cm.

  • Enable ESP32 Bluetooth

    Bluetooth connection with the phone is also enabled on ESP32.

  • Set Battery Voltage Limits

    The app battery display ranges from `VOLTAGE_LOW` for an empty battery to `VOLTAGE_MAX` for a fully charged battery. Keep the selected body's defaults or change them when using a different battery.

  • Set the Low Voltage Value

    Set the empty-battery threshold in the configuration, for example `#define VOLTAGE_LOW 6.8f`.

  • Set the Voltage Divider Factor

    If the voltage divider differs from the body's, set `VOLTAGE_DIVIDER_FACTOR` to (R1 + R2) / R2.

  • Set the Minimum Motor Voltage

    Set `VOLTAGE_MIN` to the lowest voltage at which the motors still rotate. The firmware uses this value only in No Phone mode.

  • Install PinChangeInterrupt

    Install the PinChangeInterrupt library for the speed and ultrasonic sensors on an Arduino Nano.

  • Install OLED libraries

    If the OLED is enabled, install the Adafruit_SSD1306 and Adafruit_GFX Library libraries.

  • Install libraries in Arduino IDE

    Open the Library Manager, enter the required library name in the search box, select the latest version, and click Install. If the library is already installed, the IDE shows it and allows an update.

  • Install the WCH340 driver

    A Chinese Nano clone may require the WCH340 driver from the chip manufacturer. Download the driver for Windows, Linux, or Mac using the corresponding link.

  • Open Arduino preferences

    In Arduino IDE, go to File > Preferences.

  • Add the ESP32 board URL

    Enter https://dl.espressif.com/dl/package_esp32_index.json in the Additional Board Manager URLs field, then click OK. If an ESP8266 board URL is already present, separate the two URLs with a comma.

  • Open the Board Manager

    Go to Tools > Board > Board Manager to open the Board Manager.

  • Install ESP32 version 2.0.17

    Search for ESP32, find ESP32 by Espressif Systems, verify that version 2.0.17 is selected in the drop-down, and click Install. Version 3.x.x does not work.

  • Verify the ESP32 setup

    The Arduino development environment now has everything needed to successfully flash OpenBot's ESP32 board.

  • Select the ESP32 Dev Module

    To flash OpenBot with new code, select ESP32 Dev Module from Tools > Board > ESP32 Arduino.

  • Reduce the upload speed

    If upload problems occur, go to the Tools menu and select an upload speed of 115200.

  • Configure the Arduino Nano

    Select Arduino AVR Boards > Arduino Nano, processor ATmega328P (Old Bootloader), and the USB port. If the firmware cannot be uploaded, change the processor to ATmega328P.

  • Configure the ESP32

    Select ESP32 Arduino > ESP32 Dev Module and the USB port.

  • Upload the firmware

    Upload the firmware through Sketch > Upload or by pressing the Upload button, the right-pointing arrow.

  • Enable No Phone mode

    Set `NO_PHONE_MODE 1`. The car travels at normal speed (75% PWM), slows near obstacles, turns randomly near `TURN_THRESHOLD` (default 50 cm), and reverses slowly when the estimated space ahead is below the threshold.

  • Add the Bluepad32 board URL

    In Arduino IDE, open Preferences and add the stated URL under Additional Boards Manager URLs.

  • Install the Bluepad32 board package

    Install `esp32_bluepad32` version 4.1.0 from Boards Manager. It is based on the same ESP32 core as the regular firmware, version 2.0.17.

  • Select the ESP32 Dev Module

    Select **ESP32 Dev Module** from the available esp32_bluepad32 boards.

  • Enable gamepad mode and upload

    Set `NO_PHONE_MODE 1` and `HAS_GAMEPAD 1`, then upload the firmware. The gamepad uses ESP32 Bluetooth, so phone Bluetooth connection is disabled; set `HAS_GAMEPAD 0` and pair the gamepad to the phone when controlling through the phone app.

  • Show controller name in serial monitor

    Set `DEBUG 1` to display the connected gamepad's name in the serial monitor.

  • Choose the supplied firmware target

    Use the supplied openbot.ino firmware for an Arduino Nano with an ATmega328P microcontroller or for the ESP32 development kit.

  • Select the robot and components

    Open the firmware setup page and select the robot and the components used in the build.

  • Flash firmware from the browser

    Flash the firmware directly from the setup page using Chrome or Edge on a computer.

Calibrate and test

  • Set battery, parts, and phone mode

    Set the battery, parts, and whether the robot carries a smartphone so the prepared firmware matches the robot without compiling.

  • Select the Robot Body and Board

    Set `OPENBOT` for the robot body and select a supported board in the Arduino IDE. The firmware detects the board and stops the build with an error on any unsupported board.

  • Enable No-Phone and Gamepad Modes

    Set `NO_PHONE_MODE 1` for no-phone mode. With an ESP32, also set `HAS_GAMEPAD 1` to drive with a gamepad.

  • Configure Coasting and Debug Output

    Set `COAST_MODE` to `1` for coasting motors or `0` for braking them when uncontrolled. Set `DEBUG 1` to output debug information on the serial monitor.

  • Set the Bluetooth Robot Name

    Set `BLE_NAME` to the name under which the robot appears in the phone app. The name may contain up to 20 letters, numbers, spaces, hyphens, or underscores.

  • Use the Body's Default Components

    Each body enables the components it normally has, and an ESP32 also enables the Bluetooth connection to the phone. Leave each component at the default for its body unless a change is needed.

  • Enable or Disable Components

    Set a component to `1` to enable it or `0` to disable it; for example, use `#define HAS_SONAR 0` when no ultrasonic sensor is installed. An enabled component that is not installed does not damage the robot, but the app may display meaningless values such as a sonar distance of 300 cm.

  • Set the battery voltage thresholds

    Use `VOLTAGE_LOW` for the empty level and `VOLTAGE_MAX` for the full level. For another battery, set a suitable low-voltage value in the settings.

  • Set the voltage divider factor

    If the voltage divider differs from the robot body's divider, set `VOLTAGE_DIVIDER_FACTOR` to (R1 + R2) / R2. `VOLTAGE_MIN` is the lowest voltage at which the motors still turn and is used only in no-phone mode.

  • Use suitable batteries or slow speed

    At high speed, especially when starting on the ground, motors can draw enough current to cause a reboot and disconnect the app or gamepad. Use fresh or rechargeable NiMH or Li-ion batteries, or drive at a slower level.

  • Reconnect the gamepad if needed

    After the robot reboots because of a voltage drop, the gamepad may need to be paired again.

  • Check the startup configuration

    Open the serial monitor at 115200 baud and check the reported body and enabled parts. The second line identifies the settings source as `openbot_settings.h` or `openbot.ino` when no settings file exists.

  • Set the wheel odometry interval

    Send `w<time_ms>` with the time in milliseconds between wheel odometry measurements. The wheel speed is monitored continuously, and revolutions per minute are averaged over this interval; the default is 1000 ms.

  • Set the voltage measurement interval

    Send `v<time_ms>` with the time in milliseconds between voltage measurements. Voltage is filtered with a sliding average of size 10; sending this command also requests the minimum motor-drive voltage (`vmin:<value>`), minimum battery voltage (`vlow:<value>`), and maximum battery voltage (`vmax:<value>`).

  • Set the heartbeat stop timeout

    Send `h<time_ms>` with the time in milliseconds after which the robot stops if no new heartbeat message has been received. The default is -1.

  • Set the bumper trigger reset time

    Send `b<time_ms>` with the time in milliseconds after which the bumper trigger is reset. The default is 750 ms.

  • Control a status LED

    Send `n<color>,<state>` to control a status LED. Supported colors are blue (`b`), green (`g`), and yellow (`y`); `0` turns the LED off and `1` turns it on.

  • Check the battery voltage messages

    Messages beginning with `v` report battery voltage. Turning the car switch on should display the battery voltage, while turning it off should display a small value.

  • Check the wheel speed messages

    Messages beginning with `w` report speed-sensor measurements in revolutions per minute. Set the encoder disk hole count with `DISK_HOLES`; the standard 20-hole disk produces 20 counts per wheel revolution.

  • Check the ultrasonic distance messages

    Messages beginning with `s` report the estimated free space in front of the ultrasonic sensor in centimeters.

  • Check the collision messages

    Messages beginning with `b` report collisions. The codes identify the triggering sensor: `lf` front left, `rf` front right, `cf` front center, `lb` rear left, and `rb` rear right.

  • Test the left and right indicator LEDs

    Send `i1,0` and watch the left indicator blink. Then send `i0,1` to blink the right indicator, and finally send `i0,0` to switch both indicators off.

  • Enable no-phone mode

    Set `NO_PHONE_MODE 1` in the settings. The car will then drive at normal speed, corresponding to 75% PWM.

  • Observe no-phone driving behavior

    The car slows down when the ultrasonic sensor detects obstacles. Near the `TURN_THRESHOLD`, which defaults to 50 cm, it turns in a random direction and turns on the LED on that side; below the threshold, it reverses slowly with both LEDs lit.

  • Inspect serial output with tires removed

    Before running the car, remove the tires, connect the Arduino to a computer, and observe the serial monitor as described in the `Testen` section. The monitor shows the battery voltage, left and right motor RPM, and estimated free space ahead.

  • Test obstacle response with the ultrasonic sensor

    Move a large object back and forth in front of the ultrasonic sensor. Observe how the motor speed changes as the estimated free space in front of the car changes.

  • Check ultrasonic-sensor safety

    Ensure the ultrasonic sensor is installed and enabled before testing. If it is absent or disabled, the car drives forward at 75% PWM and will eventually collide.

  • Pair the gamepad

    Power on the robot and put the gamepad into pairing mode. It connects within a few seconds and connects again whenever it is powered on.

  • Pair a different gamepad

    Turn off the connected gamepad before pairing a different one. The robot only ever drives with one gamepad.

  • Identify gamepads with debug output

    Set DEBUG to 1. The serial monitor then shows the name of every gamepad that connects.

  • Re-pair controller and restart

    If an already paired gamepad does not take control, put it into pairing mode again and restart the robot. On a PS4 or PS5 controller, a blue light bar instead of green, orange, or red indicates that it has not taken control.

  • Forget all paired gamepads

    If the gamepad still does not connect, select Forget paired gamepads on the setup page while flashing. The robot then forgets all paired gamepads.

  • Configure the sonar for exploration

    Connect the sonar sensor and set `HAS_SONAR 1`. Without the sensor, the robot simply drives straight.

  • Enable the sonar obstacle stop

    Connect the sonar sensor and set `HAS_SONAR 1`. Without them, the robot never stops in front of obstacles.

  • Configure battery, components, and mode

    On the robot itself, set the battery, components, and whether the robot uses a phone. The ready-to-use firmware then suits the robot without compilation.

  • Send later setting changes only

    After initial setup, use Update settings only to send changed settings to the robot without reflashing it.

  • Edit the SETTINGS section manually

    To configure the firmware manually, change only the SETTINGS section at the top of openbot.ino. Do not modify anything below it.

  • Set the battery gauge limits

    For a different battery, set `VOLTAGE_LOW` and `VOLTAGE_MAX` in the settings. The example sets `VOLTAGE_LOW` to 6.8f.

  • Adjust the voltage divider factor

    If the voltage divider differs from the one in the body, set `VOLTAGE_DIVIDER_FACTOR` to (R1 + R2) / R2. `VOLTAGE_MIN` is the lowest voltage at which the motors still run, and the firmware uses it only in phone-less mode.

  • Prevent low-voltage disconnections

    At high speed, especially when starting on the ground, weak or alkaline AA batteries can cause a voltage drop, reboot the board, and disconnect the app or controller. Use new or rechargeable NiMH or Li-ion batteries, or use a slower mode, then pair the controller again if needed.

Data and training

  • Connect a Bluetooth game controller

    Connect a Bluetooth game controller to the phone, such as a PS4 controller. To enter pairing mode, press the PS and share buttons until the LED flashes rapidly.

  • Select the mobile command model

    Select the `CIL-Mobile-Cmd` model in the application.

  • Drive and record a dataset

    Drive the car with the game controller and record a dataset; on a PS4 controller, recording can be toggled with the X button. The network learns better when driving is skilled and consistent.

  • Locate the recorded datasets

    Find each recording in the *Documents/OpenBot* folder on the phone's internal storage. Each recording is a zip file named *yyyymmdd_hhmmss.zip*, with the name corresponding to the timestamp when recording started.

  • Create the dataset folder

    Create a folder named `dataset` in the same folder as the Jupyter notebook.

  • Create train and test folders

    Create the `train_data` and `test_data` subfolders inside `dataset`.

  • Split data 80/20

    Use 80% of the data for training and 20% for testing, as the common split.

  • Create recording-session folders

    Create a recording-session folder inside each of `train_data` and `test_data`, using names such as `my_openbot_1` and `my_openbot_2`.

  • Add extracted recordings

    Place every recording, transferred and extracted from the phone's *Openbot* folder, in its recording-session folder.

  • Use uploaded as a training session

    Treat the `uploaded` folder as one recording session and move it into `train_data`.

  • Provide a test recording session

    If `test_data` has no recording session yet, move at least one recording from `train_data/uploaded` to `test_data/uploaded`.

  • Convert training and test datasets

    Convert dataset/train_data and dataset/test_data into train.tfrec and test.tfrec under dataset/tfrecords. By default, this conversion runs automatically when training starts.

  • Open the policy learning notebook

    Open policy_learning.ipynb, which guides you through training an autonomous-driving model with your own data. A browser window opens automatically and loads the notebook.

  • Configure shell training parameters

    Available model architectures are cil_mobile, cil_mobile_fast, cil, and pilot_net; the code default is pilot_net while the help text lists cil_mobile as the default. You can use batch_size=16, learning_rate=0.0001, num_epochs=10, plus --batch_norm, --flip_aug, --cmd_aug, and --resume.

  • Train the policy with a TFRecord

    When the dataset has already been converted to a tfrecord, train the policy from the shell through the openbot.train module.

  • Create a TFRecord before training

    Add the --create_tf_record flag to convert the dataset to a tfrecord before training.

  • Train the deployment model

    Train with a large batch size and many epochs, and enable batch normalization. pilot_net is larger than the default cil_mobile and can perform better on some tasks while still running in real time on most smartphones.

  • Automatically upload session logs

    The application automatically loads logs or sessions. See Troubleshooting for more details.

  • Move a session into a dataset

    In the web application, move a session into a selected dataset.

  • Delete a session

    Delete a session from the web application when it is no longer needed.

  • Train a model with basic parameters

    Set basic training parameters to train a model, with a progress bar shown in the web application.

  • Connect a Bluetooth gamepad

    Connect a Bluetooth gamepad to the phone, such as a PS4 controller. Hold the PS and Share buttons until the LED flashes quickly to enter pairing mode.

  • Select the command model

    Select the `CIL-Mobile-Cmd` model in the application.

  • Drive and record data

    Drive the car with the gamepad and record a dataset. On a PS4 controller, press the **X** button to enable or disable recording.

  • Organize training and test data

    Create `train_data` and `test_data` subfolders inside `dataset`, and create a folder for each recording session, such as `my_openbot_1`. Put each extracted recording zip file in its matching session folder; the usual split is 80% training data and 20% test data.

  • Automatically download and extract recordings

    You can also download the logs automatically to a Python server on your computer. ZIP files will be downloaded and extracted into `dataset/uploaded`.

  • Move uploaded folder into training data

    Treat the `uploaded` folder as a recording session and move it into `train_data`. The Jupyter notebook will then recognize these recordings as training data.

  • Prepare at least one test recording

    If `test_data` does not already contain a recording session, move at least one recording from `train_data/uploaded` to `test_data/uploaded`.

  • Convert training and test datasets

    Use `openbot.tfrecord` to convert the training and test datasets to TFRecord files in `dataset/tfrecords`, named `train.tfrec` and `test.tfrec`. By default, this conversion is performed automatically at the start of training.

  • Configure data and model parameters

    Use --no_tf_record to read a file directory directly or --create_tf_record to create a new tfrecord. Model choices are cil_mobile, cil_mobile_fast, cil, and pilot_net; the batch size defaults to 16 and the learning rate to 0.0001.

  • Configure epochs and augmentation options

    The number of epochs defaults to 10, and --batch_norm enables batch normalization. --flip_aug randomly flips images and commands, --cmd_aug adds noise to command input, and --resume continues previous training.

  • Train from an existing tfrecord

    If the dataset has already been converted to tfrecord, run openbot.train from the shell to train the policy.

  • Create tfrecord before training

    Use --create_tf_record before training to convert the dataset into a new tfrecord, then begin training.

  • Train directly from files

    Use --no_tf_record to skip tfrecord conversion and train directly from the files. This method assumes the data is already in the correct location.

  • Train the final deployment model

    Train the final deployment model with pilot_net, a batch size of 128, 100 epochs, and batch normalization. pilot_net is larger than the default cil_mobile and may perform better on some tasks while running in real time on most smartphones.

  • Select and rename the TFLite model

    At the end of training, two TFLite files are generated: the checkpoint with the best validation metrics and the last checkpoint. Choose one and rename it to autopilot_float.tflite.

  • Allow time for data collection and tuning

    Obtaining a good driving policy for your custom dataset requires patience and involves data collection and hyperparameter tuning.

  • Upload logs through the Python server

    Automatically upload logs through the Python web app on your computer; the ZIP files are extracted into `dataset/uploaded`.

  • Organize training and test data

    Treat `uploaded` as a capture session and move it into `train_data`. If `test_data` has no capture session yet, also move at least one `train_data/uploaded` capture into `test_data/uploaded`.

  • Optionally convert TFRecord data

    Create TFRecords for the training and test datasets, outputting `train.tfrec` and `test.tfrec`. By default, this conversion is performed automatically at the start of training.

Use it

  • Open the main menu

    Open the app to reach the main menu, which shows all available screens.

  • Open the settings screen

    Tap the icon at the top-right corner of the main menu to open Settings.

  • Access the app screens

    Tap the other icons on the main menu to access the available app screens and their functions.

  • Use the Free Roam screen

    Free Roam gives simple robot control with realtime battery levels, three drive states (D drive, N neutral, R reverse) where the steering wheel rotates proportionally to the steering angle, a speedometer, the free distance in front of the robot in cm, and Controller, Drive Mode and Speed controls.

  • Set point-goal navigation

    This feature requires ARCore and camera permission; the app will crash if the device does not support ARCore. Enter the distance to the front and left of the robot in meters relative to its current position and orientation; negative values correspond to back and right.

  • Review model management and benchmarks

    Review the available models and their benchmark results. All models are quantized for better performance on embedded devices; benchmarks use the COCO Detection 2017 validation set, run each model for about 1 minute, and average inference time over the last 100 frames.

  • Open the DefaultActivity screen

    Use DefaultActivity to view vehicle connection status and sensor measurements on one screen. Control the robot with a standard BT game controller or another smartphone running the OpenBot controller app.

  • Select the control mode

    Use the first button to select the control mode.

  • Use a Bluetooth gamepad

    In Gamepad mode, the app receives controls from a connected BT controller.

  • Control the robot by phone or computer

    Control the robot from another smartphone with the controller app installed, or run a Python script on a computer connected to the same network.

  • Select the game drive mode

    Use the second button to select Game mode. R2 and L2 shoulder triggers provide forward and reverse throttle, while either joystick steers.

  • Select the joystick drive mode

    Use the second button to select Joystick mode, then control the robot with either joystick.

  • Select the dual drive mode

    Use the second button to select Dual mode. The left and right joysticks control the respective sides of the car using raw differential steering.

  • Select slow speed mode

    Use the third button to select Slow mode. Motor voltage is limited to 50% of the input voltage, approximately 6V.

  • Select normal speed mode

    Use the third button to select Normal mode. Motor voltage is limited to 75% of the input voltage, approximately 9V.

  • Select fast speed mode

    Use the third button to select Fast mode. No voltage limit is applied; full throttle uses the full input voltage, approximately 12V. This is the default for running neural networks.

  • Adjust speed with the controller

    With a game controller, press the right joystick down, R3, to increase the speed mode, or the left joystick down, L3, to decrease it.

  • View controls sent to the robot

    View the controls currently sent to the robot on the right side of the interface.

  • Use the person-following model

    Use the bundled `MobileNetV1-300` model for person following. It uses an SSD object detector with a MobileNet V1 backbone and is quantized for better performance on embedded devices.

  • Toggle the Neural Network

    Use the switch on the right to turn the network on and off. While it runs, it produces robot controls and disables the game controller, although controller buttons remain available, including the R1 trigger to toggle the switch and regain control.

  • Open and refresh projects

    After signing in with a Google account, the Projects Screen lists OpenBot Playground projects; when signed out, you can still scan project QR codes. If “Oops, no project found” appears, ensure the account has projects stored on Google Drive; pull down on the project screen to reload the latest projects.

  • Run a Google Drive project

    Tap the project you want to execute and wait for its file contents to be read. A pop-up then shows the project name with “Start” and “Cancel” buttons; tap “Start” to execute or “Cancel” to cancel the activity.

  • Scan a project QR code

    Tap the QR code icon in the top-right corner and grant the app camera access. After scanning, wait for the file to be read, then tap “Start” to execute or “Cancel” to stop; if an error appears, generate a new QR code in Playground and repeat the process.

  • Run and stop project commands

    After an OpenBot Playground project runs successfully, the screen displays code block names and a stop button. Use the stop button to stop execution of the Playground block commands.

  • Delete a project

    Long-press the project you want to delete to bring up a confirmation pop-up. Tap “Yes” to confirm the deletion.

  • Sign in with Google

    When signed out, tap the `Google Sign-in` button and sign in with a Google account.

  • View signed-in account options

    After signing in, the `Profile` tab displays `Edit Profile` and `Logout` buttons.

  • Edit the user profile

    Tap `Edit Profile` to open a new screen and update profile information such as the name and profile picture.

  • Log out of the account

    Tap `Logout`, log out of the account, and return to the login screen.

  • Open OpenBot Playground

    Tap the OpenBot Playground icon at the top of the toolbar to access OpenBot Playground services.

  • Run OpenBot Playground directly

    Open https://www.playground.openbot.org/ to use OpenBot Playground directly.

  • Sync Google Drive data

    Sync Google Drive with OpenBot; the data will be saved automatically on Google Drive.

  • Store data in local storage

    Store the application data in local storage.

  • Scan the output QR code

    Scan the output QR code directly in the OpenBot application to run the program.

  • Load files from Google Drive

    After logging in to the OpenBot mobile apps, retrieve saved files from Google Drive and load them with one click.

  • Design OpenBot instructions without code

    Design robot instructions for OpenBot with zero code using OpenBot Playground blocks.

  • Use the app on mobile devices

    Use OpenBot Playground through its responsive design optimized for mobile and tablet browsing.

  • Explore OpenBot Playground blocks

    Explore the blocks available in OpenBot Playground.

  • Learn how the Playground works

    Open the OpenBot Playground home page and read the carousel content to learn how the platform works.

  • View or create a project

    View the project list in the home page project section, or use the “Create new Project” button to create a new project.

Common problems

  • Autopilot model fails to drive correctly after training #31
  • Windows Python controller cannot connect or stream video #189 #190 #215
  • Mobile app cannot reliably connect and control motors #51 #268
  • TensorFlow Lite export fails while saving best and final checkpoints #39
  • Arduino firmware fails to compile with numerous errors #47
  • ESP32 repeatedly resets and fails to start #455

Parts

  • Controller: Arduino Nano (ATmega328P) or an ESP32 dev board (firmware README)
  • DIY body: four TT gear motors with tyres, an L298N motor driver (or the custom PCB V1/V2), three 18650 cells and a holder, a USB OTG cable, M3 screws and nuts
  • Options: two speed sensors, an ultrasonic sensor, indicator LEDs, an SSD1306 OLED, a power switch
  • 3D-printed body (the regular body needs a 240×150 mm bed; slim, glueable and Blocky variants exist) and a phone mount
  • RTR vehicles: splash-proof ABS shell; RTR_TT has TT motors for indoor use, RTR_520 has an ESP32, 520 motors, metal gearboxes and all-terrain wheels
  • An Android phone (the iOS app is a beta)

Tools

  • A 3D printer; an off-the-shelf Arduino car chassis, or cardboard and wood, also work
  • A soldering iron (motor leads, switch) and Dupont wires
  • A computer with the Arduino IDE (ESP32 needs core 2.0.17); building the apps yourself needs Android Studio or Xcode

Skills

  • Basic soldering and wiring (the DIY body needs a lot of it; the custom PCB cuts it down)
  • Arduino firmware configuration, flashing and serial testing
  • Sideloading APKs; training driving policies needs Python and a conda environment
  • Blockly blocks (OpenBot Playground)

Software

  • OpenBot robot app (Android, v0.9.0 released 2026‑10‑07): free roam, data collection, autopilot, tracking of 80 object classes, point-goal navigation
  • Flutter controller app, plus Node.js, Python and web-server controllers
  • OpenBot Playground: browser Blockly coding, projects saved to Google Drive or run from a QR code
  • Policy training code and the Arduino firmware (with a no-phone mode and Bluetooth gamepad driving)

Documents