Connecting to sources…

How to build XiaoZhi AI (xiaozhi-esp32): the whole process

263 steps to build XiaoZhi AI (xiaozhi-esp32), from buying parts, printing and assembly to software, calibration and training, with the bill of materials, shop links and…

Ready boards from $21.50: AtomS3R AI Chatbot Kit $21.50, CoreS3 $59.90 (M5Stack store), SenseCAP Watcher for XiaoZhi $49.90 (Seeed); shipping excluded. The breadboard BOM lives in a Feishu wiki that was not checked

Buy parts

  • Consider alternative digital microphones

    ZTS6672 and ICS43434 can be considered as alternative microphone modules.

  • Check a substitute microphone

    Before substituting a module, check its pin orientation, channel, and firmware configuration.

  • Check the HT517 module specifications

    Before using the HT517 amplifier module, check its actual input, supply, enable behavior, and speaker connector specifications.

  • Check speaker loading and power headroom

    If considering the 4Ω enclosed speaker example, separately check amplifier loading and power headroom.

  • Prepare the ESP32S3CAM board

    The hardware is based on the ESP32S3CAM board, and the code is modified from bread-compact-wifi-lcd.

  • Prepare the OV2640 camera

    Use an OV2640 camera.

  • Select a PDM microphone

    The box uses a PDM microphone.

  • Select a common-anode LED

    The box uses a common-anode LED.

  • Check Hardware Compatibility

    Check the compatibility of codec, PMIC, and touch controller combinations.

  • Get the electronBot open-source hardware

    The Licoi Open Source page provides the hardware for this version, and the electronBot official site provides the project entry point.

  • Buy the watch-form-factor kit

    Purchase the linked watch-form-factor kit, described as one of the cheapest kits.

Assemble

  • Identify the INMP441 microphone pins

    Identify VDD, GND, WS, SCK, SD, and L/R on the INMP441 digital microphone module.

  • Connect the speaker to MAX98357A

    Use the MAX98357A digital amplifier with digital I2S input and connect the speaker across its two output terminals. This guide uses an enclosed 8Ω, 3W speaker.

  • Arrange and check the breadboards

    Use two interlocking 400-point breadboards, placing the controller in the middle and modules on either side. With power disconnected, check power-rail continuity and whether the long rails are split midway.

  • Prepare the wires and optional switch

    Use short breadboard jumper wires and Dupont leads, choosing connector ends for the board and module headers. For the optional tactile switch, check whether pins on the same side are already connected before installation.

  • Check the OLED pin arrangement

    Choose a 0.91-inch 128×32 OLED or a 0.96-inch 128×64 OLED. Check the pin order and VCC/GND positions against the markings on your display.

  • Connect the microphone

    Connect the microphone SD output to the firmware DIN input. For INMP441 power and channel selection, follow the TDK datasheet. If the microphone headers are soldered on the opposite side, identify the upward-facing side and follow matching signal labels; the 44-pin and 42-pin layouts are shown separately.

  • Connect the amplifier and speaker
  • Match the finished assembly

    Use the photos to recognize overall placement, soldered or Dupont connections, and the completed OLED build. Follow the power scheme above for the amplifier and 4G; wire colors do not replace signal labels.

  • Start from a Similar Board

    Copy and tweak an existing board instead of starting from scratch.

  • Check Every Pin Assignment

    Make sure every pin defined in `config.h` matches the schematic.

  • Connect hardware by pin definition

    Make the connections according to the pin definitions in config.h. Because the camera uses many IO pins, the ESP32S3 USB 19 and 20 pins are occupied.

  • Expose the programming header

    Before assembling the product, unglue the battery from the board to access the programming header.

  • Build the ESP32-S3 breadboard project

    Follow the linked ESP32-S3 Breadboard Tutorial to build the practice project.

Software

  • Define Hardware Configuration

    Create `config.h` and define the audio sample rates, I2S pin mapping, audio codec I2C address and pins, button and LED pins, and display parameters and pins. The example uses 24000 Hz audio rates and the listed GPIO_NUM mappings.

  • Create Board Build Configuration

    Create `config.json` with `type`, `target`, and a `builds` array containing the firmware variant name and `sdkconfig_append` entries. Set `target` to the real hardware target, such as `esp32s3`, and keep `type` and `name` stable after release; both may contain only lowercase letters, digits, periods, and hyphens.

  • Add Flash, Partition, and Audio Settings

    Add the required flash-size and partition-table entries to `sdkconfig_append`. Add `CONFIG_USE_DEVICE_AEC=y` when on-device AEC is required, and do not repeat values that already match the effective project defaults; applicable targets default to 16MB flash and `partitions/v2/16m.csv`.

  • Keep target defaults

    Keep board presets on their target defaults, and place only actual board-specific overrides in `sdkconfig_append`.

  • Exclude user interface options

    Configure language and wake-word choices consistently through `menuconfig` or build-script parameters so the CLI, agent, and online builds share the same interface.

  • Exclude wake-word engine presets

    Do not add `CONFIG_USE_*_WAKE_WORD` entries or model selections to board or chip presets because `scripts/build.py` selects the wake-word engine.

  • Opt out of wake-word detection

    For memory, flash, or performance reasons, set `CONFIG_WAKE_WORD_DISABLED=y` in `sdkconfig_append`. The build script preserves this opt-out and reports the variant's `wake_word_supported` as `false`.

  • Expose only implemented displays

    Expose a `display_model` option only when the board source or config references its `CONFIG_OLED_*` or `CONFIG_LCD_*` symbol. Add panel initialization, required dimensions, and backlight settings together; fixed-display boards expose no model option.

  • Create the board source file

    Create `my_custom_board.cc` and place the board-level implementation in it.

  • Declare the board class

    Derive the custom class from `WifiBoard` or `Ml307Board`. The example uses `MyCustomBoard` derived from `WifiBoard`, with an I2C bus handle, boot button, and display pointer.

  • Add board initialization helpers

    Add initialization helpers for I2C, the display, buttons, and IoT/MCP tools. The example provides `InitializeI2c()` and begins `InitializeSpi()`.

  • Configure the I2C bus

    Configure the codec I2C bus on `I2C_NUM_0`, using `AUDIO_CODEC_I2C_SDA_PIN` and `AUDIO_CODEC_I2C_SCL_PIN`, the default clock source, glitch-ignore count 7, interrupt priority 0, transfer queue depth 0, and internal pull-ups enabled. Create the master bus with `ESP_ERROR_CHECK(i2c_new_master_bus(&i2c_bus_cfg, &codec_i2c_bus_))`.

  • Begin SPI bus initialization

    Initialize the SPI bus with `DISPLAY_SPI_MOSI_PIN`, `DISPLAY_SPI_SCK_PIN`, `GPIO_NUM_NC` for MISO and unused quad lines, and a maximum transfer size of `DISPLAY_WIDTH * DISPLAY_HEIGHT * sizeof(uint16_t)`.

  • Implement virtual board overrides

    Implement the required virtual overrides, including `GetAudioCodec()`, `GetDisplay()`, and `GetBacklight()`.

  • Register the board class

    Register the board class with `DECLARE_BOARD(ClassName)`.

  • Configure the boot button actions

    Bind the boot button so a click enters Wi-Fi configuration mode while the device is starting. Otherwise, the click toggles the chat state.

  • Initialize the ST7789 SPI display

    Create an ST7789 panel on SPI2_HOST with the configured CS and DC pins, SPI mode 2, an 80 MHz clock, a 10-item transfer queue, and 8-bit command and parameter fields. Configure RGB order, 16 bits per pixel, no reset pin, color inversion, XY swapping, mirroring, and the specified display dimensions and offsets.

  • Construct the custom board

    Construct the board with BOOT_BUTTON_GPIO and initialize I2C, SPI, the display, and the buttons. Set the backlight brightness to 100.

  • Configure the ES8311 audio codec

    Create a static Es8311AudioCodec using codec_i2c_bus_ on I2C_NUM_0. Supply the input and output sample rates, MCLK, BCLK, WS, DOUT, DIN, PA pin, and ES8311 address.

  • Return the initialized display

    Return the display instance created by InitializeDisplay.

  • Configure the PWM backlight

    Create a static PwmBacklight using DISPLAY_BACKLIGHT_PIN and DISPLAY_BACKLIGHT_OUTPUT_INVERT, then return it.

  • Add a custom board Kconfig entry

    Add an uppercase, underscore-separated board identifier to the `BOARD_TYPE` choice in `main/Kconfig.projbuild`. Restrict it with `depends on` to the matching IDF target and localize the label if desired.

  • Add the custom board CMake branch

    Open `main/CMakeLists.txt` and add an `elseif` branch for `CONFIG_BOARD_TYPE_MY_CUSTOM_BOARD`. Set the board directory, display fonts, and optional emoji collection in that branch.

  • Choose display fonts and emoji

    Choose text and icon fonts matching the display: 128x64 uses `font_puhui_basic_14_1` and `font_awesome_14_1`; 240x240 uses the 16_4 fonts; 240x320 uses the 20_4 fonts; and 480x320 or larger uses the 30_4 fonts. Choose `twemoji_32` for small screens or `twemoji_64` for large screens.

  • Set the ESP-IDF target chip

    Set the target to ESP32-S3, ESP32-C3, or ESP32 the first time or when switching targets.

  • Clean the stale configuration

    Remove the existing build configuration before configuring the board.

  • Select the board type

    Open menuconfig, navigate to `Xiaozhi Assistant -> Board Type`, and select the required board.

  • Build, flash, and monitor

    Build the firmware, flash it to the board, and start the serial monitor.

  • Build a board automatically

    Run the build script with a board directory that contains `config.json`.

  • Select language and wake word

    Build with an explicit language and ESP-SR wake-word model.

  • Choose the interface locale

    The Kconfig default and the script default are `en-US`; omitting `--language` selects English. Use `zh-CN` for Chinese or another locale listed under `main/assets/locales/`. This controls interface text and voice prompts.

  • Use the default wake word
  • Override the wake-word model

    Pass an ESP-SR model name, `nihaoxiaozhi`, `hiesp`, or `disabled`. The aliases select a compatible model for the target.

  • Match the wake word to the chip

    ESP32-C3, C5, and C6 use WakeNet9s (`wn9s_*`) models. ESP32-S3, P4, and S31 use the AFE engine and require PSRAM; ESP32 also requires PSRAM and uses the engine without AFE.

  • Check wake-word support

    Unsupported boards and variants with an explicit board opt-out remain disabled. Requests to enable wake-word detection on them are rejected before SDK configuration.

  • Replace packed speech models

    Custom and Emote assets keep their UI resources, but their packed speech models are replaced by the models selected in the current `sdkconfig`, or removed when wake-word detection is disabled. The resulting package must fit the assets partition.

  • Configure wake words manually

    For direct `idf.py` builds, select the wake-word implementation and models explicitly in menuconfig. Chip defaults do not select a model, and a fresh manual configuration leaves wake-word detection disabled.

  • Query accepted option values

    List languages and wake words as text or JSON.

  • Populate managed components first

    Run reconfigure before querying wake words if `managed_components/` has not been populated, because the list comes from the currently resolved ESP-SR component.

  • List available boards

    Running the script without arguments prints help. Use the board-list option to list board types and variants.

Calibrate and test

  • Configure partitions and device AEC

    In sdkconfig_append, select the 4MB or 8MB partition table and enable device-side AEC with CONFIG_USE_DEVICE_AEC=y. Applicable target chips use 16MB Flash and partitions/v2/16m.csv by default.

  • Leave user build options unset

    Do not select a language or specific wake word in the board's config.json. Configure these user build options through menuconfig or build-script parameters so the CLI, Agent, and online compilation interface share them.

  • Implement and Debug Incrementally

    Implement basic functions such as display first, then add more complex functions such as audio.

  • Check Hardware Compatibility

    Confirm that all chips and drivers are compatible.

  • Troubleshoot Display Problems

    If the display is not working correctly, check the SPI configuration, mirror settings, and color inversion settings.

  • Troubleshoot Missing Audio Output

    If there is no audio output, check the I2S configuration, PA enable pin, and codec address.

  • Troubleshoot Network Connection Failure

    If the network cannot be connected, check the Wi-Fi credentials and network configuration.

  • Troubleshoot Server Communication Failure

    If communication with the server fails, check the MQTT or WebSocket configuration.

  • Record the device configuration

    Record the firmware version, board ID, Flash/PSRAM, display specifications, and current provisioning state.

  • Keep account settings separate

    Keep agent settings in the Console account. Reflashing firmware and unbinding an account are different operations.

  • Identify the board interfaces

    Use the board documentation to identify its native USB and USB-UART interfaces.

  • Check for the download interface

    Confirm that the USB connector connects to the chip's download interface rather than only supplying power.

  • Check hardware revision support

    Some Tudouzi (tudouzi), KevinBox2, and other hardware revisions do not expose the required USB pins.

  • Restart and check startup prompts

    Close the connection. Press RESET/EN or power-cycle the device, then check its startup prompts.

  • Verify the successful flash

    Confirm that writing and verification succeeded. After restart, check that the expected board firmware and version run, with the expected display and startup sound.

  • Select the action name

    Set the required `action` parameter to one of the supported action names.

  • Set the action step count

    Set `steps` from 1 to 100, with a default of 3. Larger values make the action last longer.

  • Set the action speed

    Set `speed` from 100 to 3000, with a default of 700; smaller values are faster. Most actions take 500–1500 milliseconds, while special actions can differ, such as whirlwind_leg at 100–1000 and takeoff at 200–600.

  • Set the action direction

    Set `direction` to -1, 0, or 1, with a default of 1. For walk and turn, 1 means forward or left and -1 means backward or right; for bend, shake_leg, and moonwalk, 1 means left and -1 means right; for hand actions, 1 means left hand, -1 means right hand, and 0 means both hands where supported.

  • Set the action amplitude

    Set `amount` from 0 to 170, with a default of 30. Larger values produce greater movement amplitude.

  • Set the arm-swing amplitude

    Set `arm_swing` from 0 to 170, with a default of 50. It is used only by walk and turn, and 0 disables arm swinging.

  • Bring Up the Board Incrementally

    Bring up the display first, followed by audio and then the full stack.

  • Fix an Incorrect Display

    Verify the SPI configuration, mirroring, and color inversion.

  • Fix Missing Audio

    Check the I2S wiring, PA enable pin, and codec I2C address.

  • Fix WiFi Connection Failure

    Re-check the WiFi credentials and provisioning method.

  • Fix Server Connection Failure

    Verify the WebSocket or MQTT endpoint configuration.

  • Check display, prompt, and connection

    Confirm that the OLED displays correctly, the startup prompt is audible, and the device connects and appears in your Console.

  • Test the conversation controls

    Press the button to start a conversation, speak so the microphone hears you, and confirm that replies play without a restart.

  • Use the default Simplex configuration

    Use the source's default Simplex configuration for this page.

  • Confirm firmware safety limits

    The firmware automatically trims target angles outside the safe range and limits AI-generated oscillation amplitude so that `center angle +/- amplitude` stays within the safe range.

  • Configure the Large Model

    If the XiaoZhi AI chatbot device is already connected to the official server, log in to the xiaozhi.me console for configuration.

Use it

  • Call the unified action tool

    Call `self.otto.action` and specify the movement with the `action` parameter.

  • Use basic movement actions

    Use `walk` and `turn` with steps, speed, direction, and arm_swing; use `jump` with steps and speed.

  • Use special movement actions

    Use `swing` with steps, speed, and amount; `moonwalk` with steps, speed, direction, and amount; `bend` and `shake_leg` with steps, speed, and direction; `updown` and `whirlwind_leg` with steps, speed, and amount.

  • Run fixed-position actions

    Call `sit`, `showcase`, or `home` without parameters. `showcase` chains multiple actions, while `home` returns the robot to its initial position.

  • Use supported hand actions

    With hand servos configured, use `hands_up` and `hands_down` with speed and direction; `hand_wave` with direction; `windmill`, `takeoff`, and `fitness` with steps, speed, and amount; `greeting` and `shy` with direction and steps; and `radio_calisthenics` or `magic_circle` without parameters.

  • Check hand-servo availability

    The hand actions marked with an asterisk are available only when hand servos are configured.

  • Return Otto to its home pose

    Call the `self.otto.action` tool with `{"action": "home"}` to reset the robot to its initial position.

  • Return Home After Each Action

    After each action finishes, the robot automatically returns to its initial home position so the next action can begin.

  • Disable Reset for Two Actions

    The sit and showcase actions do not automatically reset after they finish.

  • Use Default Action Parameters

    All parameters have reasonable default values, so parameters that do not need customization can be omitted.

  • Run Actions in Background

    Actions run in a background task and do not block the main program.

  • Queue Multiple Actions

    An action queue is supported so multiple actions can be executed continuously.

  • Configure Hand Servos

    Hand actions require hand servos to be configured; when they are not configured, the related actions are skipped.

  • Walk With Default Parameters

    Call the Otto action tool with the walk action and omit its customizable parameters.

  • Walk Five Steps Faster

    Call the walk action with five steps and a speed of 800.

  • Turn With Large Arm Swing

    Call the turn action for two steps with an arm swing value of 100.

  • Swing With Medium Amplitude

    Call the swing action for five steps with an amount of 50.

  • Jump

    Call the jump action with one step and a speed of 1000.

  • Moonwalk

    Call the moonwalk action with three steps, speed 800, direction 1, and amount 30.

  • Wave the Left Hand

    Call the hand-wave action with direction set to 1.

  • Run Showcase Sequence

    Call the showcase action, which chains multiple actions together.

  • Sit

    Call the sit action.

  • Windmill

    Call the windmill action with ten steps, speed 500, and amount 80.

  • Takeoff

    Call the takeoff action with five steps, speed 300, and amount 40.

  • Radio Calisthenics

    Call the radio-calisthenics action.

  • Return to Home

    Call the home action to return the robot to its initial position.

  • Stop All Actions

    Call the stop tool to immediately stop all actions and reset the robot.

  • Get the Robot IP Address

    Call the get-IP tool to retrieve the robot's IP address.

  • Wave the hand

    Use the hand-wave action with the hand servos installed.

  • Stop all actions immediately

    Call the stop tool to immediately stop all Otto robot actions.

  • Get movement status

    Get the robot's movement status, returned as "moving" or "idle".

  • Get the IP address

    Request the device's IP address.

  • Get battery level

    Request the device's battery level.

  • Get device status

    Get device information such as volume, network, and battery status.

  • Set speaker volume

    Set the speaker volume to 70 using a value from 0 to 100.

  • Run a normal servo sequence

    Run a normal sequence that moves each servo step by step.

  • Swing both arms

    Run oscillator mode to swing both arms.

  • Create a side-to-side wave

    Run oscillator mode for a left-to-right swaying wave.

  • Try voice action commands

    Use voice commands for raising or lowering the hands, waving, clapping, turning the body, looking up or down, nodding, and combined actions.

  • Call tools through WebSocket

    After connecting to WiFi, access `ws://<device-IP>:8080/ws` on the same LAN and send either MCP-formatted messages or direct JSON-RPC `tools/call` messages.

Common problems

  • Camera startup, image quality, or upload fails across board variants #1588 #1039 #1394 #1457
  • MCP and IoT calls skip actions, execute partially, or target the wrong device #1142 #1011 #204
  • Audio playback or conversations stop, switch states, or disconnect prematurely #1652 #1114 #1014
  • Device provisioning, verification, or custom-theme flashing fails #1802 #1353 #1201
  • Custom wake words fail to activate, leaving the default wake word active #1191 #1190
  • Microphones have short recognition range or detect only the wake word #668 #2262
  • Upgrades cause SSL linker and I2S audio-channel build errors #991
  • The screen intermittently freezes during operation #478

Parts

  • Controller: ESP32, ESP32‑C3/C5/C6/S3 or P4 (README)
  • Microphone and speaker (full duplex on hardware with AEC)
  • Optional: OLED/LCD, camera, ML307/EC801E/NT26 Cat.1 4G module
  • Robot bodies such as StackChan, Otto and ElectronBot have board ports

Tools

  • A computer and a data-capable USB cable
  • DIY: breadboard and jumper wires

Skills

  • Beginners can flash prebuilt firmware without a toolchain
  • Custom boards: ESP‑IDF v6.0.1 or later and C++
  • MCP tools and persona prompt setup

Software

  • xiaozhi-esp32 firmware
  • xiaozhi.me console (model, voice, persona)
  • Third-party self-hosted servers such as xinnan-tech/xiaozhi-esp32-server (Python)

Documents