This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

ESP32: firmware development



Getting started

New to development for the ESP32 platform firmware? This section will help you get up-and-running with firmware development.



Adding support for an ESP32 based device

Did you receive an ESP32 based badge at an event for which support is not yet available? Did you build your own ESP32 based device or badge? This section helps get you started with adding support for your badge.



Factory tests, provisioning and OTA updates

Interested in releasing a badge using our firmware? This section explains the factory testing and provisioning features, as well as how OTA updates and other release and support releated parts of our project work.

1 - Getting started

Welcome developer! This section describes how to get your development environment set-up so that you can build the Badge.team ESP32 platform firmware for any of the supported targets.

Introduction

Our firmware has been set-up as an ESP-IDF project. The ESP-IDF is the development framework or SDK released by Espressif. Espressif is improving and updating the ESP-IDF constantly. Unfortunately these updates often introduce breaking changes. Because of this we have included the exact version of the ESP-IDF that we use as a submodule in our firmware GIT repository.

Downloading the project

You can clone the project by running git clone https://github.com/badgeteam/ESP32-platform-firmware.git --recursive. Git will then create a folder called “ESP32-platform-firmware” containing the project files.

Installing required packages (Linux only)

Debian / Ubuntu: sudo apt-get install make unzip git libncurses5-dev flex bison gperf python-serial libffi-dev libsdl2-dev libmbedtls-dev perl

Installing the toolchain

Currently we’re using ESP-IDF version 3.2 to build the firmware. After cloning our repository you will find the exact version of the ESP-IDF used in the ESP32-platform-firmware/esp-idf folder. You don’t need to download or install a version of the ESP-IDF yourself.

The toolchain is another story: the newest ESP-IDF version (v4.x and newer) uses a different toolchain than the older (v3.3 / stable) version of the IDF. Because of this you can’t simply download the “latest greatest” ESP32 toolchain, instead you need to use a specific version.

You can download the correct version of the toolchain directly from Espressif using the following URLs:

Operating systemArchitectureDownload link
LinuxAMD64 (64-bit)https://dl.espressif.com/dl/xtensa-esp32-elf-linux64-1.22.0-80-g6c4433a-5.2.0.tar.gz
LinuxI386 (32-bit)https://dl.espressif.com/dl/xtensa-esp32-elf-linux32-1.22.0-80-g6c4433a-5.2.0.tar.gz
Apple Mac OSOSXhttps://dl.espressif.com/dl/xtensa-esp32-elf-osx-1.22.0-80-g6c4433a-5.2.0.tar.gz
Microsoft WindowsWIN32https://dl.espressif.com/dl/esp32_win32_msys2_environment_and_gcc5_toolchain-20191220.zip

You can find the official toolchain installation instructions here:

The very, very short version of these instructions for Linux is as follows:

  • Extract the archive
  • Add the path to the bin folder in the archive, containing the toolchain executables, to your path export PATH="$PATH:/path/to/my/toolchain/xtensa-esp32-elf/bin"

Configuring the firmware

The firmware can be built for many different targets. Because of this the firmware has to be configured before it can be built. By default a “generic” configuration is used. Building the firmware using the generic configuration will get you to a Python shell on any ESP32 based device. However: almost all hardware support will be missing.

To apply the configuration for a specific badge navigate to the firmware/configs folder and overwrite (/create) the file firmware/sdkconfig with the configuration relevant to your badge.

(Note that running make clean to remove the build output is a bit broken / insufficient at the moment. Please remove the firmware/build folder manually after switching configurations to make sure the firmware is built correctly.)

Manually changing the configuration of the firmware is explained in the menuconfig section.

Building the firmware

After you’ve downloaded the firmware, applied the correct configuration and installed the correct toolchain you have to build the Micropython cross compiler. This extra compiler converts Python scripts into a Micropython specific binary format so that these Python scripts can be integrated inside the firmware image.

Building the Micropython cross compiler can be done by running bash mpy_cross.sh from inside the firmware folder.

After you’ve built the Micropython cross compiler you can build the firmware by navigating to the firmware folder and running make. On multi-core systems compilation speeds can be improved by adding the number of threads to be used: make -j 4.

1.1 - Menuconfig

You can start menuconfig either by running ./config.sh in the root of the repository or by executing make menuconfig in the firmware folder.

You will then be greeted by the main menu.

Main menu

The menu contains the following submenus:

  • SDK tool configuration
  • Bootloader config
  • Security features
  • Serial flasher config
  • Firmware & device configuration
  • Partition table
  • Compiler options
  • Component config

SDK tool configuration

In the SDK tool configuration menu you can configure the compiler toolchain path and prefix and the Python 2 interpretter to use. The default settings configured here use the toolchain found in your $PATH and the python2 executable found in your path as the Python interpretter. You should not have to change these settings.

Bootloader config

The bootloader config menu allows configuration of the bootloader. Changing these settings requires advanced knowledge of the ESP32 platform. The default values configured here should work.

Security features

The security features menu allows for configuring secure boot by encrypting the flash and signing the firmware. Use of these features on a badge would defeat the purpose of a hackable device and is thus not supported. Do not attempt to enable any of the options in this menu: you will brick your device!

Serial flasher config

This is the first interesting item in the list. In the serial flasher config menu you can configure the serial port to use when executing make flash, as well as the baudrate. This menu also allows you to tell the bootloader about the flash chip mode, speed an size. Most of the Badge.team badges have a 16MB flash chip, the CampZone2019 has a 8MB chip and most chinese boards have 4MB.

Firmware & device configuration

This menu allows you to configure the identity of your device.

ItemDescription
Code-name of the firmwareName of your device, lowercase and without spaces or special characters
Build number of the firmwareVersion information in the format “YYMMDDNN”: year, month, day and build
Name of the deviceHuman readable name of the device
MicroPython modules directorysubdirectory of /firmware/python_modules to use for the built-in MicroPython modules
Name of the badge on the app hatcheryName of your device or a compatible device supported by our Hatchery, lowercase and without spaces or special characters
Hostname of server for OTA updatesDomain name of the server used for OTA updating (Example: “Badge.team”)
Use HTTPS for OTA updatesIf enabled HTTPS can be used with a Letsencrypt SSL certificate. Other certificate authorities are not supported.
Port of server for OTA updatesPort to use for OTA updates. Usually 443 for HTTPS or 80 for HTTP
Path on the server for OTA updatesPath of the OTA firmware binary on the server, starting with a /
Path on the server for OTA update versionPath to the JSON file with version information (used by the update check Python app shipped with some badges)
Hostname of server for app hatchery that contains user appsDomain name at which the Hatchery is hosted (used by the installer app shipped with some badges)
Default WiFi ssidThe default WiFi SSID to use when the user has not yet configured WiFi
Default WiFi passwordThe default WiFi password to use when the user has not yet configured WiFi (leave empty for ipen/insecure network)
Default display orientationFor badges which use the same display as another type of badge but in a different orientation (explained below)

The HackerHotel 2019 badge is a direct derrivative of the SHA2017 badge, but with the display mounted in portrait mode instead of landscape. To allow for backwards compatibility with SHA2017 apps the real orientation has been set to landscape, while HackerHotel 2019 apps can call import orientation; orientation.default() to switch to the real orientation of the badge they are running on. The default orientation is configured here.

Partition table

In this menu a partition table can be selected. A set of partition tables has been provided in the /firmware/partitions folder. The partitions.ods spreadsheet can help you when designing a custom partition table. The partition table offset and the MD5 checksum options should be left at their default settings.

Compiler options

Advanced options for compiler optimization level and assertions. We suggest leaving the assertions enabled.

Component config

The component config submenu allows for configuring various components fo the firmware such as the MicroPython interpretter and the device drivers.

MicroPython configuration

To-do

Device driver configuration

The Badge.team firmware contains drivers for multiple devices such as displays and sensors. These drivers are written in C and part of the firmware itself, but they can be accessed from withing MicroPython using the bindings provided.

Is a menu empty? Does a feature not work?

  • To be able to use I2C based devices you have to enable the I2C bus driver first.
  • To be able to use SPI based devices you have to enable the VSPI driver first.
  • To be able to access displays using the “display” API from within MicroPython you have to enable the framebuffer driver.
  • The “double buffered” mode of the framebuffer driver is only relevant for devices which do not have their own buffer such as the Flipdot and HUB75 drivers. In all other cases it’s a waste of memory! Only enable this to use the display.flush() command with displays that stream directly from the framebuffer.

2.1 - Adding drivers

If you need low-level support for hardware that isn’t available yet, you can write your own drivers, and can expose them to the Python app layer.

  • Create the folder /firmware/components/driver_<category>_<name>.
  • In this folder, create files component.mk, Kconfig, and driver_<name>.c. Kconfig allows you to add configurable switches and variables from the ./config.sh step. The driver source file exposes an initialisation function that will be called upon boot. Have a look at e.g. /firmware/components/driver_bus_i2c to see how to populate these files.
  • In /main/platform.c:platform_init(), add INIT_DRIVER(<name>) to have your driver actually initialise during boot.
  • Add your driver’s header directory to firmware/micropython/component.mk, e.g. MP_EXTRA_INC += -I$(PROJECT_PATH)/components/driver_<category>_<name>/include.
  • Add python bindings to your driver by creating components/micropython/esp32/mod<name>.c (see e.g. modi2c.c).
  • Tell micropython about your bindings by adding the following to firmware/micropython/component.mk:
ifdef CONFIG_DRIVER_<NAME>_ENABLE
SRC_C += esp32/mod<name>.c
endif
  • Add the following to components/micropython/esp32/mpconfigport.h to add the module symbols to the python environment (replace i2c with your name):
#ifdef CONFIG_DRIVER_I2C_ENABLE
extern const struct _mp_obj_module_t i2c_module;
#endif
#ifdef CONFIG_DRIVER_I2C_ENABLE
#define BUILTIN_MODULE_I2C { MP_OBJ_NEW_QSTR(MP_QSTR_i2c), (mp_obj_t)&i2c_module },
#else
#define BUILTIN_MODULE_I2C
#endif
(to the define called MICROPY_PORT_BUILTIN_MODULES, add the following line after the other drivers):
BUILTIN_MODULE_I2C \

3 - Factory tests and provisioning

(To-Do)

4 - Device status

BadgeCurrent OTA firmwarePlatform support status
SHA2017ESP32 platform Build 50 (“Rise of skywalker”)Fully supported
Disobey 2019Legacy SHA2017 firmware Unknown, OTA server is currently offline.Everything works, needs testing Not officially released yet
HackerHotel 2019Legacy SHA2017 firmwareEverything works but audio needs improvement Not officially released yet
CampZone 2019ESP32 platformFully supported
Disobey 2020ESP32 platformFully supported, audio support still in progress
Fri3dCamp 2018ESP32 platformProof of concept
OHS2018ESP32 platformDisplay works, proof of concept
Odroid GoESP32 platformProof of concept. Display and audio work. No support for buttons or SD.
TTGO LoRaESP32 platformProof of concept. Basics of the LoRa radio work, no interrupts, no WAN.