How to build XiaoZhi AI (xiaozhi-esp32): the whole process
263 steps to build XiaoZhi AI (xiaozhi-esp32), from buying parts, printing and assembly to software, calibration and training, with the bill of materials, shop links and…
Ready boards from $21.50: AtomS3R AI Chatbot Kit $21.50, CoreS3 $59.90 (M5Stack store), SenseCAP Watcher for XiaoZhi $49.90 (Seeed); shipping excluded. The breadboard BOM lives in a Feishu wiki that was not checked
Buy parts
- Consider alternative digital microphones
ZTS6672 and ICS43434 can be considered as alternative microphone modules.
- Check a substitute microphone
Before substituting a module, check its pin orientation, channel, and firmware configuration.
- Check the HT517 module specifications
Before using the HT517 amplifier module, check its actual input, supply, enable behavior, and speaker connector specifications.
- Check speaker loading and power headroom
If considering the 4Ω enclosed speaker example, separately check amplifier loading and power headroom.
- Prepare the ESP32S3CAM board
The hardware is based on the ESP32S3CAM board, and the code is modified from bread-compact-wifi-lcd.
- Prepare the OV2640 camera
Use an OV2640 camera.
- Select a PDM microphone
The box uses a PDM microphone.
- Select a common-anode LED
The box uses a common-anode LED.
- Check Hardware Compatibility
Check the compatibility of codec, PMIC, and touch controller combinations.
- Get the electronBot open-source hardware
The Licoi Open Source page provides the hardware for this version, and the electronBot official site provides the project entry point.
- Buy the watch-form-factor kit
Purchase the linked watch-form-factor kit, described as one of the cheapest kits.
Assemble
- Identify the INMP441 microphone pins
Identify VDD, GND, WS, SCK, SD, and L/R on the INMP441 digital microphone module.
- Connect the speaker to MAX98357A
Use the MAX98357A digital amplifier with digital I2S input and connect the speaker across its two output terminals. This guide uses an enclosed 8Ω, 3W speaker.
- Arrange and check the breadboards
Use two interlocking 400-point breadboards, placing the controller in the middle and modules on either side. With power disconnected, check power-rail continuity and whether the long rails are split midway.
- Prepare the wires and optional switch
Use short breadboard jumper wires and Dupont leads, choosing connector ends for the board and module headers. For the optional tactile switch, check whether pins on the same side are already connected before installation.
- Check the OLED pin arrangement
Choose a 0.91-inch 128×32 OLED or a 0.96-inch 128×64 OLED. Check the pin order and VCC/GND positions against the markings on your display.
- Connect the microphone
Connect the microphone SD output to the firmware DIN input. For INMP441 power and channel selection, follow the TDK datasheet. If the microphone headers are soldered on the opposite side, identify the upward-facing side and follow matching signal labels; the 44-pin and 42-pin layouts are shown separately.
- Connect the amplifier and speaker
- Match the finished assembly
Use the photos to recognize overall placement, soldered or Dupont connections, and the completed OLED build. Follow the power scheme above for the amplifier and 4G; wire colors do not replace signal labels.
- Start from a Similar Board
Copy and tweak an existing board instead of starting from scratch.
- Check Every Pin Assignment
Make sure every pin defined in `config.h` matches the schematic.
- Connect hardware by pin definition
Make the connections according to the pin definitions in config.h. Because the camera uses many IO pins, the ESP32S3 USB 19 and 20 pins are occupied.
- Expose the programming header
Before assembling the product, unglue the battery from the board to access the programming header.
- Build the ESP32-S3 breadboard project
Follow the linked ESP32-S3 Breadboard Tutorial to build the practice project.
Software
- Define Hardware Configuration
Create `config.h` and define the audio sample rates, I2S pin mapping, audio codec I2C address and pins, button and LED pins, and display parameters and pins. The example uses 24000 Hz audio rates and the listed GPIO_NUM mappings.
- Create Board Build Configuration
Create `config.json` with `type`, `target`, and a `builds` array containing the firmware variant name and `sdkconfig_append` entries. Set `target` to the real hardware target, such as `esp32s3`, and keep `type` and `name` stable after release; both may contain only lowercase letters, digits, periods, and hyphens.
- Add Flash, Partition, and Audio Settings
Add the required flash-size and partition-table entries to `sdkconfig_append`. Add `CONFIG_USE_DEVICE_AEC=y` when on-device AEC is required, and do not repeat values that already match the effective project defaults; applicable targets default to 16MB flash and `partitions/v2/16m.csv`.
- Keep target defaults
Keep board presets on their target defaults, and place only actual board-specific overrides in `sdkconfig_append`.
- Exclude user interface options
Configure language and wake-word choices consistently through `menuconfig` or build-script parameters so the CLI, agent, and online builds share the same interface.
- Exclude wake-word engine presets
Do not add `CONFIG_USE_*_WAKE_WORD` entries or model selections to board or chip presets because `scripts/build.py` selects the wake-word engine.
- Opt out of wake-word detection
For memory, flash, or performance reasons, set `CONFIG_WAKE_WORD_DISABLED=y` in `sdkconfig_append`. The build script preserves this opt-out and reports the variant's `wake_word_supported` as `false`.
- Expose only implemented displays
Expose a `display_model` option only when the board source or config references its `CONFIG_OLED_*` or `CONFIG_LCD_*` symbol. Add panel initialization, required dimensions, and backlight settings together; fixed-display boards expose no model option.
- Create the board source file
Create `my_custom_board.cc` and place the board-level implementation in it.
- Declare the board class
Derive the custom class from `WifiBoard` or `Ml307Board`. The example uses `MyCustomBoard` derived from `WifiBoard`, with an I2C bus handle, boot button, and display pointer.
- Add board initialization helpers
Add initialization helpers for I2C, the display, buttons, and IoT/MCP tools. The example provides `InitializeI2c()` and begins `InitializeSpi()`.
- Configure the I2C bus
Configure the codec I2C bus on `I2C_NUM_0`, using `AUDIO_CODEC_I2C_SDA_PIN` and `AUDIO_CODEC_I2C_SCL_PIN`, the default clock source, glitch-ignore count 7, interrupt priority 0, transfer queue depth 0, and internal pull-ups enabled. Create the master bus with `ESP_ERROR_CHECK(i2c_new_master_bus(&i2c_bus_cfg, &codec_i2c_bus_))`.
- Begin SPI bus initialization
Initialize the SPI bus with `DISPLAY_SPI_MOSI_PIN`, `DISPLAY_SPI_SCK_PIN`, `GPIO_NUM_NC` for MISO and unused quad lines, and a maximum transfer size of `DISPLAY_WIDTH * DISPLAY_HEIGHT * sizeof(uint16_t)`.
- Implement virtual board overrides
Implement the required virtual overrides, including `GetAudioCodec()`, `GetDisplay()`, and `GetBacklight()`.
- Register the board class
Register the board class with `DECLARE_BOARD(ClassName)`.
- Configure the boot button actions
Bind the boot button so a click enters Wi-Fi configuration mode while the device is starting. Otherwise, the click toggles the chat state.
- Initialize the ST7789 SPI display
Create an ST7789 panel on SPI2_HOST with the configured CS and DC pins, SPI mode 2, an 80 MHz clock, a 10-item transfer queue, and 8-bit command and parameter fields. Configure RGB order, 16 bits per pixel, no reset pin, color inversion, XY swapping, mirroring, and the specified display dimensions and offsets.
- Construct the custom board
Construct the board with BOOT_BUTTON_GPIO and initialize I2C, SPI, the display, and the buttons. Set the backlight brightness to 100.
- Configure the ES8311 audio codec
Create a static Es8311AudioCodec using codec_i2c_bus_ on I2C_NUM_0. Supply the input and output sample rates, MCLK, BCLK, WS, DOUT, DIN, PA pin, and ES8311 address.
- Return the initialized display
Return the display instance created by InitializeDisplay.
- Configure the PWM backlight
Create a static PwmBacklight using DISPLAY_BACKLIGHT_PIN and DISPLAY_BACKLIGHT_OUTPUT_INVERT, then return it.
- Add a custom board Kconfig entry
Add an uppercase, underscore-separated board identifier to the `BOARD_TYPE` choice in `main/Kconfig.projbuild`. Restrict it with `depends on` to the matching IDF target and localize the label if desired.
- Add the custom board CMake branch
Open `main/CMakeLists.txt` and add an `elseif` branch for `CONFIG_BOARD_TYPE_MY_CUSTOM_BOARD`. Set the board directory, display fonts, and optional emoji collection in that branch.
- Choose display fonts and emoji
Choose text and icon fonts matching the display: 128x64 uses `font_puhui_basic_14_1` and `font_awesome_14_1`; 240x240 uses the 16_4 fonts; 240x320 uses the 20_4 fonts; and 480x320 or larger uses the 30_4 fonts. Choose `twemoji_32` for small screens or `twemoji_64` for large screens.
- Set the ESP-IDF target chip
Set the target to ESP32-S3, ESP32-C3, or ESP32 the first time or when switching targets.
- Clean the stale configuration
Remove the existing build configuration before configuring the board.
- Select the board type
Open menuconfig, navigate to `Xiaozhi Assistant -> Board Type`, and select the required board.
- Build, flash, and monitor
Build the firmware, flash it to the board, and start the serial monitor.
- Build a board automatically
Run the build script with a board directory that contains `config.json`.
- Select language and wake word
Build with an explicit language and ESP-SR wake-word model.
- Choose the interface locale
The Kconfig default and the script default are `en-US`; omitting `--language` selects English. Use `zh-CN` for Chinese or another locale listed under `main/assets/locales/`. This controls interface text and voice prompts.
- Use the default wake word
- Override the wake-word model
Pass an ESP-SR model name, `nihaoxiaozhi`, `hiesp`, or `disabled`. The aliases select a compatible model for the target.
- Match the wake word to the chip
ESP32-C3, C5, and C6 use WakeNet9s (`wn9s_*`) models. ESP32-S3, P4, and S31 use the AFE engine and require PSRAM; ESP32 also requires PSRAM and uses the engine without AFE.
- Check wake-word support
Unsupported boards and variants with an explicit board opt-out remain disabled. Requests to enable wake-word detection on them are rejected before SDK configuration.
- Replace packed speech models
Custom and Emote assets keep their UI resources, but their packed speech models are replaced by the models selected in the current `sdkconfig`, or removed when wake-word detection is disabled. The resulting package must fit the assets partition.
- Configure wake words manually
For direct `idf.py` builds, select the wake-word implementation and models explicitly in menuconfig. Chip defaults do not select a model, and a fresh manual configuration leaves wake-word detection disabled.
- Query accepted option values
List languages and wake words as text or JSON.
- Populate managed components first
Run reconfigure before querying wake words if `managed_components/` has not been populated, because the list comes from the currently resolved ESP-SR component.
- List available boards
Running the script without arguments prints help. Use the board-list option to list board types and variants.
Calibrate and test
- Configure partitions and device AEC
In sdkconfig_append, select the 4MB or 8MB partition table and enable device-side AEC with CONFIG_USE_DEVICE_AEC=y. Applicable target chips use 16MB Flash and partitions/v2/16m.csv by default.
- Leave user build options unset
Do not select a language or specific wake word in the board's config.json. Configure these user build options through menuconfig or build-script parameters so the CLI, Agent, and online compilation interface share them.
- Implement and Debug Incrementally
Implement basic functions such as display first, then add more complex functions such as audio.
- Check Hardware Compatibility
Confirm that all chips and drivers are compatible.
- Troubleshoot Display Problems
If the display is not working correctly, check the SPI configuration, mirror settings, and color inversion settings.
- Troubleshoot Missing Audio Output
If there is no audio output, check the I2S configuration, PA enable pin, and codec address.
- Troubleshoot Network Connection Failure
If the network cannot be connected, check the Wi-Fi credentials and network configuration.
- Troubleshoot Server Communication Failure
If communication with the server fails, check the MQTT or WebSocket configuration.
- Record the device configuration
Record the firmware version, board ID, Flash/PSRAM, display specifications, and current provisioning state.
- Keep account settings separate
Keep agent settings in the Console account. Reflashing firmware and unbinding an account are different operations.
- Identify the board interfaces
Use the board documentation to identify its native USB and USB-UART interfaces.
- Check for the download interface
Confirm that the USB connector connects to the chip's download interface rather than only supplying power.
- Check hardware revision support
Some Tudouzi (tudouzi), KevinBox2, and other hardware revisions do not expose the required USB pins.
- Restart and check startup prompts
Close the connection. Press RESET/EN or power-cycle the device, then check its startup prompts.
- Verify the successful flash
Confirm that writing and verification succeeded. After restart, check that the expected board firmware and version run, with the expected display and startup sound.
- Select the action name
Set the required `action` parameter to one of the supported action names.
- Set the action step count
Set `steps` from 1 to 100, with a default of 3. Larger values make the action last longer.
- Set the action speed
Set `speed` from 100 to 3000, with a default of 700; smaller values are faster. Most actions take 500–1500 milliseconds, while special actions can differ, such as whirlwind_leg at 100–1000 and takeoff at 200–600.
- Set the action direction
Set `direction` to -1, 0, or 1, with a default of 1. For walk and turn, 1 means forward or left and -1 means backward or right; for bend, shake_leg, and moonwalk, 1 means left and -1 means right; for hand actions, 1 means left hand, -1 means right hand, and 0 means both hands where supported.
- Set the action amplitude
Set `amount` from 0 to 170, with a default of 30. Larger values produce greater movement amplitude.
- Set the arm-swing amplitude
Set `arm_swing` from 0 to 170, with a default of 50. It is used only by walk and turn, and 0 disables arm swinging.
- Bring Up the Board Incrementally
Bring up the display first, followed by audio and then the full stack.
- Fix an Incorrect Display
Verify the SPI configuration, mirroring, and color inversion.
- Fix Missing Audio
Check the I2S wiring, PA enable pin, and codec I2C address.
- Fix WiFi Connection Failure
Re-check the WiFi credentials and provisioning method.
- Fix Server Connection Failure
Verify the WebSocket or MQTT endpoint configuration.
- Check display, prompt, and connection
Confirm that the OLED displays correctly, the startup prompt is audible, and the device connects and appears in your Console.
- Test the conversation controls
Press the button to start a conversation, speak so the microphone hears you, and confirm that replies play without a restart.
- Use the default Simplex configuration
Use the source's default Simplex configuration for this page.
- Confirm firmware safety limits
The firmware automatically trims target angles outside the safe range and limits AI-generated oscillation amplitude so that `center angle +/- amplitude` stays within the safe range.
- Configure the Large Model
If the XiaoZhi AI chatbot device is already connected to the official server, log in to the xiaozhi.me console for configuration.
Use it
- Call the unified action tool
Call `self.otto.action` and specify the movement with the `action` parameter.
- Use basic movement actions
Use `walk` and `turn` with steps, speed, direction, and arm_swing; use `jump` with steps and speed.
- Use special movement actions
Use `swing` with steps, speed, and amount; `moonwalk` with steps, speed, direction, and amount; `bend` and `shake_leg` with steps, speed, and direction; `updown` and `whirlwind_leg` with steps, speed, and amount.
- Run fixed-position actions
Call `sit`, `showcase`, or `home` without parameters. `showcase` chains multiple actions, while `home` returns the robot to its initial position.
- Use supported hand actions
With hand servos configured, use `hands_up` and `hands_down` with speed and direction; `hand_wave` with direction; `windmill`, `takeoff`, and `fitness` with steps, speed, and amount; `greeting` and `shy` with direction and steps; and `radio_calisthenics` or `magic_circle` without parameters.
- Check hand-servo availability
The hand actions marked with an asterisk are available only when hand servos are configured.
- Return Otto to its home pose
Call the `self.otto.action` tool with `{"action": "home"}` to reset the robot to its initial position.
- Return Home After Each Action
After each action finishes, the robot automatically returns to its initial home position so the next action can begin.
- Disable Reset for Two Actions
The sit and showcase actions do not automatically reset after they finish.
- Use Default Action Parameters
All parameters have reasonable default values, so parameters that do not need customization can be omitted.
- Run Actions in Background
Actions run in a background task and do not block the main program.
- Queue Multiple Actions
An action queue is supported so multiple actions can be executed continuously.
- Configure Hand Servos
Hand actions require hand servos to be configured; when they are not configured, the related actions are skipped.
- Walk With Default Parameters
Call the Otto action tool with the walk action and omit its customizable parameters.
- Walk Five Steps Faster
Call the walk action with five steps and a speed of 800.
- Turn With Large Arm Swing
Call the turn action for two steps with an arm swing value of 100.
- Swing With Medium Amplitude
Call the swing action for five steps with an amount of 50.
- Jump
Call the jump action with one step and a speed of 1000.
- Moonwalk
Call the moonwalk action with three steps, speed 800, direction 1, and amount 30.
- Wave the Left Hand
Call the hand-wave action with direction set to 1.
- Run Showcase Sequence
Call the showcase action, which chains multiple actions together.
- Sit
Call the sit action.
- Windmill
Call the windmill action with ten steps, speed 500, and amount 80.
- Takeoff
Call the takeoff action with five steps, speed 300, and amount 40.
- Radio Calisthenics
Call the radio-calisthenics action.
- Return to Home
Call the home action to return the robot to its initial position.
- Stop All Actions
Call the stop tool to immediately stop all actions and reset the robot.
- Get the Robot IP Address
Call the get-IP tool to retrieve the robot's IP address.
- Wave the hand
Use the hand-wave action with the hand servos installed.
- Stop all actions immediately
Call the stop tool to immediately stop all Otto robot actions.
- Get movement status
Get the robot's movement status, returned as "moving" or "idle".
- Get the IP address
Request the device's IP address.
- Get battery level
Request the device's battery level.
- Get device status
Get device information such as volume, network, and battery status.
- Set speaker volume
Set the speaker volume to 70 using a value from 0 to 100.
- Run a normal servo sequence
Run a normal sequence that moves each servo step by step.
- Swing both arms
Run oscillator mode to swing both arms.
- Create a side-to-side wave
Run oscillator mode for a left-to-right swaying wave.
- Try voice action commands
Use voice commands for raising or lowering the hands, waving, clapping, turning the body, looking up or down, nodding, and combined actions.
- Call tools through WebSocket
After connecting to WiFi, access `ws://<device-IP>:8080/ws` on the same LAN and send either MCP-formatted messages or direct JSON-RPC `tools/call` messages.
Common problems
- Camera startup, image quality, or upload fails across board variants #1588 #1039 #1394 #1457
- MCP and IoT calls skip actions, execute partially, or target the wrong device #1142 #1011 #204
- Audio playback or conversations stop, switch states, or disconnect prematurely #1652 #1114 #1014
- Device provisioning, verification, or custom-theme flashing fails #1802 #1353 #1201
- Custom wake words fail to activate, leaving the default wake word active #1191 #1190
- Microphones have short recognition range or detect only the wake word #668 #2262
- Upgrades cause SSL linker and I2S audio-channel build errors #991
- The screen intermittently freezes during operation #478
Parts
- Controller: ESP32, ESP32‑C3/C5/C6/S3 or P4 (README)
- Microphone and speaker (full duplex on hardware with AEC)
- Optional: OLED/LCD, camera, ML307/EC801E/NT26 Cat.1 4G module
- Robot bodies such as StackChan, Otto and ElectronBot have board ports
Tools
- A computer and a data-capable USB cable
- DIY: breadboard and jumper wires
Skills
- Beginners can flash prebuilt firmware without a toolchain
- Custom boards: ESP‑IDF v6.0.1 or later and C++
- MCP tools and persona prompt setup
Software
- xiaozhi-esp32 firmware
- xiaozhi.me console (model, voice, persona)
- Third-party self-hosted servers such as xinnan-tech/xiaozhi-esp32-server (Python)