Brief Analysis of esp32 Flash Partition and OTA Function

Upgradability is very important for all embedded products. Especially when the product is mass-produced/sold, there are no conditions for manufacturers to upgrade the product, so the design of the upgrade method must be foolproof, error-proof and stable.

Espressif ESP32 , as an IoT chip that integrates Bluetooth and WIFI, is bound to be upgraded by OTA after it is developed into a product. This document briefly analyzes the Flash partition configuration of ESP32 and the OTA Demo provided by the SDK, and uses this as a reference for future product OTA process design.

This document only analyzes the upgrade process of SDKOTA Demo, and does not involve too specific Flash operations. Specific problems will be dealt with in the actual development process.


1: ESP32 Flash space partition situation


1.1 ESP32 Flash space partition configuration

The currently used ESP-WROOM-32 integrates 4MB SPI Flash. When compiling the esp32 program, there are three partition options through make menuconfig -> PartitionTable: factory program (no OTA partition)/factory program (dual OTA partition)/user-defined partition.

The configuration in menuconfig is just to modify the macros in the configuration file. In fact, the source path of the ESP32 SDK corresponding to the Flash partition configuration is:


There are files partitions_singleapp/partitions_singleapp_coredump/partitions_two_ota/partitions_two_ota_coredump in this path, all of which are used to configure the Flash partition.

Here we take the most complex partitions_two_ota_coredump as an example, and then refer to some codes, we can know the partition situation of 4M SPI Flash when partitions are configured using partitions_two_ota_coredump:  


The use of partitions requires special explanation:

<1> Core dump partition: The core dump partition is used to find software errors when the system crashes. Simply put, when the software crashes, it will flash the debug information to the Flash and save it (no power loss) for developers to analyze. There is a blog about the use of this partition: "Using ESP32's debugging tool coredump" the ESP32 debugging tool coredump.html

It can be seen that the core dump partition is useful in actual product development and needs to be retained.

<2>Factory App partition: the main program storage area when leaving the factory

<3>OTA data/OTA_0/OTA_1 is related to the OTA upgrade function, which will be described later.


1.2 Customize ESP32 Flash Partition Configuration

For product development, it is necessary to customize the configuration of the Flash partition to meet the requirements. Simply put, at least the core dump partition, the OTA-related partition, and the user data area that needs to be saved when the power is turned off are all necessary.

Now customize the Flash partition configuration based on partitions_two_ota_coredump in section 1.1, and add a 128K user data partition after the core dump partition to save user data. The method is as follows:

<1>makemenuconfig -> Partition Table -> Custom partition table CSV

After selection, you will be asked to enter a custom partition configuration csv file name, such as partition.csv.

<2> Copy a partition configuration file from the path \esp-idf-v3.0-rc1\components\partition_table, and use WordPad to modify, for example, add a 128K user data partition:

<3> Copy the newly edited partition.csv partition configuration file to the user's App directory, and then make.

<4> After burning the compiled firmware, it can be seen from the debugging information that the Flash partition has been changed. The data that the user needs to save after power off can be written to the user data area through the nvs api (nvs_open_from_partition). (not tested yet)

2: ESP32 SDK OTA Demo upgrade process


ESP32SDK path\esp-idf-v3.0-rc1\examples\system\ota is the official OTA Demo, and there is an OTA upgrade flow chart (instructions):

But obviously this is not very meaningful for embedded development.

Part of the code of ESP32SDK OTA Demo has not been debugged and tested, so it is not guaranteed that the analysis is completely correct. If there are errors, make changes.


2.1 ESP32 SDK OTA Demo upgrade strategy


The upgrade strategy is discussed in two parts: 1: Acquisition and burning of the target firmware for the upgrade; 2: OTA upgrade strategy. The latter is the key.


2.1.1 Acquisition and burning of upgrade target firmware

It is very simple to obtain and write the upgrade firmware: ESP32 connects to the HTTP server; sends a request to Get to upgrade the firmware; reads 1KB firmware data each time; writes to Flash. The upgrade process for embedded devices is basically the same.

During the abnormal processing, the upgrade is stopped and the pointer jumps in place.


2.1.2 OTA upgrade strategy


An OTA upgrade strategy is key. The lack of consideration of the upgrade strategy will kill developers, and because the end users are not professionals, the lack of fool-proof and error-proof upgrade strategies will often make product upgrade failures and bricks. Complaints and compensation are inevitable (manual smiles).

    Excluding the cumbersome code, the upgrade strategy of ESP32SDK OTA Demo is briefly described as follows: There are (at least) four partitions related to the upgrade in the ESP32 SPI Flash: OTA data, Factory App, OTA_0, OTA_1. The FactoryApp memory contains the factory default firmware.

When the OTA upgrade is performed for the first time, the OTA Demo burns the target firmware to the OTA_0 partition, and after the burn is completed, the data of the OTA data partition is updated and restarted. When the system restarts, the data of the OTA data partition is obtained for calculation, and it is decided to load the firmware of the OTA_0 partition for execution (instead of the firmware in the default Factory App partition) to realize the upgrade.

Similarly, if the ESP32 is already executing the firmware in OTA_0 after an upgrade, the OTA Demo will write the target firmware to the OTA_1 partition when upgrading again. After restarting, execute the OTA_1 partition to upgrade. And so on.

The upgraded target firmware is always burned interactively between the two partitions OTA_0 and OTA_1, and will not affect the Factory App firmware when it leaves the factory.

2.2 Brief description of ESP32 SDK OTA Demo API


According to the description of the OTA Demo upgrade strategy in 2.1.2, the relevant important function interfaces are listed here, but no detailed analysis is made. Specific problems will be dealt with when they arise during the development process:

<1> Source code path: \esp-idf-v3.0-rc1\examples\system\ota\main\ota_example_main.c



Get the Flash partition where the firmware executed by the current system is located (the difference between the two is not understood for the time being)


Get the next Flash partition of the current system (next to the currently used OTA_X partition) that can be used to burn the upgrade firmware

esp_ota_begin& esp_ota_write & esp_ota_end:

Flash the upgrade target firmware to the available Flash partition (usually the OTA_X partition)


After the upgrade is complete, update the data in the OTA data area. When restarting, load the execution target (new) firmware according to the data in the OTA data area to the Flash partition.


<2> Source path: \esp-idf-v3.0-rc1\components\bootloader\subproject\main\ bootloader_start.c

load_partition_table: Load the Flash partition table (find the OTA data area address from the partition table)

get_selected_boot_partition: Get the Flash boot partition (obtained by calculating the data in the OTA data area)

load_boot_imageunpack_load_app: Load unpacked firmware from flash boot partition and execute


2.3 Advantages and possible problems


Judging from the ESP32 SDKOTA Demo upgrade strategy, it should be relatively safe. Regardless of any abnormality during the upgrade, as long as the data in the OTA data area is not modified, the device can still load the original firmware for execution.

At present, it seems that the places that may need to be considered are:

<1> Is it possible that the OTA data data points to an area where the upgrade fails, causing the device to load damaged firmware;

<2>Because OTA requires three upgrade-related areas, the firmware size is limited to less than SPI Flash Size/3

<3>An error-proofing/retransmission/checking mechanism should also be added to obtain the upgrade target firmware; corresponding handling should also be taken when an abnormality occurs.

<4> Set aside a background control interface for modifying the OTA data area, which is convenient for the remote control program to run.



3: ESP32 SDK OTA upgrade function design


Related: Brief Analysis of esp32 Flash Partition and OTA Function