esp-hal/hil-test

hil-test

Hardware-in-loop testing for esp-hal.

Quickstart

We use embedded-test as our testing framework. This allows us to write unit and integration tests much in the same way you would for a normal Rust project, when the standard library is available, and to execute them using Cargo's built-in test runner.

Running Tests Locally

We use probe-rs for flashing and running the tests on a target device.

cargo install probe-rs-tools \
  --git https://github.com/probe-rs/probe-rs \
  --force --locked

Target device MUST connected via its USB-Serial-JTAG port, or if unavailable (eg. ESP32, ESP32-C2, ESP32-S2) then you must connect a compatible debug probe such as an ESP-Prog.

You can run all tests for a given device by running the following command from the workspace root:

cargo xtask run tests $CHIP

To run a single test on a target, run the following command from the workspace root:

# Run GPIO tests for ESP32-C6
cargo xtask run tests esp32c6 --test gpio

If you want to run a test multiple times:

# Run GPIO tests for ESP32-C6
cargo xtask run tests esp32c6 --test gpio --repeat 10

Some tests will require physical connections, please see the current configuration in our runners.

Running Tests Remotes (ie. on Self-Hosted Runners)

The hil.yml workflow builds the test suite for all our available targets and executes them.

Our self-hosted runners have the following setup:

• ESP32-C2 (esp32c2-jtag):
  • Devkit: ESP8684-DevKitM-1 connected via UART (UART port).
    • GPIO18 and GPIO9 are I2C pins.
    • GPIO2 and GPIO3 are connected.
  • Probe: ESP-Prog connected with the following connections
  • RPi: Raspbian 12 configured with the following setup
• ESP32-C3 (rustboard):
  • Devkit: ESP32-C3-DevKit-RUST-1 connected via USB-Serial-JTAG (USB port).
    • GPIO4 and GPIO5 are I2C pins.
    • GPIO2 and GPIO3 are connected.
  • RPi: Raspbian 12 configured with the following setup
• ESP32-C5 (esp32c5-usb):
  • Devkit: ESP32-C5-DevKitC-1 connected via USB-Serial-JTAG (USB port) and UART (UART port).
    • GPIO2 and GPIO3 are I2C pins.
    • GPIO9 and GPIO10 are connected.
  • RPi: Raspbian 12 configured with the following setup
• ESP32-C6 (esp32c6-usb):
  • Devkit: ESP32-C6-DevKitC-1 V1.2 connected via USB-Serial-JTAG (USB port) and UART (UART port).
    • GPIO6 and GPIO7 are I2C pins.
    • GPIO2 and GPIO3 are connected.
  • RPi: Raspbian 12 configured with the following setup
• ESP32-C61 (esp32c61-usb):
  • Devkit: ESP32-C61-DevKitC-1 V1.0 connected via USB-Serial-JTAG (USB port) and UART (UART port).
    • GPIO6 and GPIO7 are I2C pins.
    • GPIO2 and GPIO3 are connected.
  • RPi: Raspbian 12 configured with the following setup
• ESP32-H2 (esp32h2-usb):
  • Devkit: ESP32-H2-DevKitM-1 connected via USB-Serial-JTAG (USB port) and UART (UART port).
    • GPIO12 and GPIO22 are I2C pins.
    • GPIO2 and GPIO3 are connected.
  • RPi: Raspbian 12 configured with the following setup
• ESP32-P4 (esp32p4):
  • Devkit: ESP32-P4 EV Board 1.5.2 connected via USB-Serial-JTAG (USB port).
    • GPIO2 and GPIO3 are I2C pins.
    • GPIO7 and GPIO8 are connected.
  • RPi: Raspbian 12 configured with the following setup
• ESP32-S2 (esp32s2-jtag):
  • Devkit: ESP32-S2-Saola-1 connected via UART (UART port).
    • GPIO2 and GPIO3 are I2C pins.
    • GPIO9 and GPIO10 are connected.
  • Probe: ESP-Prog connected with the following connections
  • RPi: Raspbian 12 configured with the following setup
• ESP32-S3 (esp32s3-usb):
  • Devkit: ESP32-S3-DevKitC-1 connected via USB-Serial-JTAG (USB port) and UART (UART port).
    • GPIO2 and GPIO3 are I2C pins.
    • GPIO9 and GPIO10 are connected.
  • RPi: Raspbian 12 configured with the following setup
• ESP32 (esp32-jtag):
  • Devkit: ESP32-DevKitC-V4 connected via UART (UART port).
    • GPIO32 and GPIO33 are I2C pins.
    • GPIO2 and GPIO4 are connected.
  • Probe: ESP-Prog connected with the following connections
  • RPi: Raspbian 12 configured with the following setup

RPi Setup

# Install Rust:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain stable -y --profile minimal
# Source the current shell:
. "$HOME/.cargo/env"
# Install dependencies
sudo apt install -y pkg-config libudev-dev uhubctl
# Install probe-rs
cargo install probe-rs-tools --git https://github.com/probe-rs/probe-rs --force
# Add the udev rules
wget -O - https://probe.rs/files/69-probe-rs.rules | sudo tee /etc/udev/rules.d/69-probe-rs.rules > /dev/null
# Add the user to plugdev group
sudo usermod -a -G plugdev $USER
# Install espflash
ARCH=$($HOME/.cargo/bin/rustup show | grep "Default host" | sed -e 's/.* //')
curl -L "https://github.com/esp-rs/espflash/releases/latest/download/espflash-${ARCH}.zip" -o "${HOME}/.cargo/bin/espflash.zip"
unzip "${HOME}/.cargo/bin/espflash.zip" -d "${HOME}/.cargo/bin/"
rm "${HOME}/.cargo/bin/espflash.zip"
chmod u+x "${HOME}/.cargo/bin/espflash"
# Reboot the runner
sudo reboot

Adding New Tests

1. Create a new integration test file (tests/$PERIPHERAL.rs)
2. Add a corresponding [[test]] entry to Cargol.toml (MUST set harness = false)
3. Write the tests
4. Document any necessary physical connections on boards connected to self-hosted runners
5. Add a header in the test stating which targets support the given tests. Eg:

//! AES Test

//% CHIPS: esp32 esp32c3 esp32c6 esp32h2 esp32s2 esp32s3

If the test is supported by all the targets, you can omit the header.

6. Write some documentation at the top of the tests/$PERIPHERAL.rs file with the pins being used and the required connections, if applicable.

Logging in tests

The tests can use defmt to print logs. To enable log output, add the defmt feature to the test you want to run. Eg:

//! AES Test

//% CHIPS: esp32 esp32c3 esp32c6 esp32h2 esp32s2 esp32s3
//% FEATURES: defmt

Make sure to remove this addition before you commit any modifications.

NOTE: log output is disabled by default. Enabling it can introduce some timing issues, which makes some tests fail randomly. This issue affects all Xtensa devices, as well as ESP32-C2 and ESP32-C3 currently.