Chapter 20 Over-the-Air Upgrade (OTA) of ESP32

Pay attention to the public account of Jiayouchuang Technology

Learning goals and objectives

  1. Master the OTA working process
  2. Master the OTA programming of ESP32

OTA working process explanation

In the actual product development process, online upgrade (OTA) can remotely solve the problems introduced by product software development and better meet user needs.

 

 

Introduction to OTA for ESP32 ( Original )

  OTA (over-the-air) update is the process of loading firmware to an ESP module using a WiFi connection instead of a serial port.

 

 

There are three ways for OTA upgrade of ESP32:

  1. Arduino IDE: Mainly used in the software development stage to realize firmware programming without wiring
  2. Web Browser: Manually deliver application update modules through a web browser
  3. HTTP Server: Automatically use http server - for production applications 

In the three upgrade cases, the first firmware upload must be done through the serial port. 

The OTA process has no imposed security and needs to ensure that developers can only get updates from legitimate/trusted sources. After the update is complete, the module will restart and execute the new code. Developers should ensure that applications running on modules are closed and restarted in a safe manner.

 

 

ESP32 Flash space partition configuration

The currently used ESP-WROOM-32 integrates 4MB SPI Flash. When compiling the esp32 program, there are three partition options through make menuconfig -> PartitionTable: factory program (no OTA partition)/factory program (dual OTA partition)/user-defined partition. As shown below:

The configuration in menuconfig is only to modify the macros in the configuration file. In fact, the source path of the ESP32 SDK corresponding to the Flash partition configuration is: \esp-idf-v3.0\components\partition_table The following .csv files are used for Flash partition is configured.

  • No OTA partitions: partitions_singleapp.csv, partitions_singleapp_coredump.csv
  • Dual OTA partitions: partitions_two_ota.csv, partitions_two_ota_coredump.csv

When dual OTA partition, the partition situation of 4M SPI Flash:  

 

 

OTA upgrade strategy (HTTP)

ESP32 connects to HTTP server (can be local or cloud, OTA demo uses local server ) , sends request Get to upgrade firmware; reads 1KB firmware data each time and writes to Flash.

There are (at least) four partitions within the ESP32 SPI Flash that are relevant for upgrades:

  1. OTA data area: decide which area to run the app
  2. Factory App area: There are default apps from the factory
  3. OTA_0 area: OTA_0 App
  4. OTA_1 area: OTA_1 App

When the OTA upgrade is performed for the first time, the OTA Demo burns the target App to the OTA_0 partition, and after the burn is completed, the data of the OTA data partition is updated and restarted.

When the system restarts, the OTA data partition data is obtained for calculation, and it is decided to load the App execution in the OTA_0 partition (instead of the App in the default Factory App partition) to achieve the upgrade.

Similarly, if the ESP32 is already executing the App in OTA_0 after an upgrade, the OTA Demo will write the target App to the OTA_1 partition when upgrading again. After rebooting, perform the OTA_1 partition to upgrade. By analogy, the upgraded target App is always burned interactively between the two partitions OTA_0 and OTA_1, which will not affect the Factory App firmware when it leaves the factory, as shown in the figure below.

 

 

ConfidentialitySecurity

The module had to be wirelessly networked to get the new firmware, which allowed the module to be hacked and loaded with additional code. To reduce the chances of being hacked, consider password protecting your uploads, select certain OTA ports, encrypt the bin file as well, etc.

  •  

ESP32 OTA Demo upgrade process and instructions

Upgrade process

  1. Computer connected to router (AP)
  2. Computer running HTTP server (local)
  3. Download OTA Demo to ESP32 development board
  4. After the ESP32 is connected to the router (AP), it will access HTTP to download the new APP to the OTA area

 

OTA detailed process logic of ESP32

 

Brief description of ESP32 OTA interface

ota source code path: \esp-idf-v3.0\examples\system\ota\main\ota_example_main.c

1

esp_ota_get_boot_partition

boot

2

esp_ota_get_running_partition

Get the Flash partition where the firmware executed by the current system is located

3

esp_ota_get_next_update_partition

Get the next Flash partition of the current system (next to the currently used OTA_X partition) that can be used to burn the upgrade firmware

4

esp_ota_begin

esp_ota_write

esp_ota_end

Flash the upgrade target firmware to the available Flash partition (usually the OTA_X partition)

5

esp_ota_set_boot_partition

After the upgrade is complete, update the data in the OTA data area. When restarting, load the execution target (new) firmware according to the data in the OTA data area to the Flash partition.

 

Boot source code path: \esp-idf-v3.0\components\bootloader\subproject\main\bootloader_start.c

1

load_partition_table

Load the Flash partition table (find the OTA data area address from the partition table)

2

get_selected_boot_partition

Get the Flash boot partition (obtained by calculating the data in the OTA data area)

3

load_boot_imageunpack_load_app:

Load the decompressed firmware from the flash boot partition and execute

 

OTA partition operation process

 

software design

Introduction to the HTTP interface of ESP32, the same as the TCP interface

OTA task writing

It is basically the same as getting the city temperature through HTTP. Here, the data in the HTTP response packet is stored in the OTA partition.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

static void ota_example_task(void *pvParameter)

{undefined

    esp_err_t err;

    /* update handle : set by esp_ota_begin(), must be freed via esp_ota_end() */

    esp_ota_handle_t update_handle = 0 ;

    const esp_partition_t *update_partition = NULL;

 

    ESP_LOGI(TAG, "Starting OTA example...");

    //Get the location where the OTA app is stored

    const esp_partition_t *configured = esp_ota_get_boot_partition();

    //Get the Flash partition where the firmware executed by the current system is located

    const esp_partition_t *running = esp_ota_get_running_partition();

 

    if (configured != running) {undefined

        ESP_LOGW(TAG, "Configured OTA boot partition at offset 0x%08x, but running from offset 0x%08x",

                 configured->address, running->address);

        ESP_LOGW(TAG, "(This can happen if either the OTA boot data or preferred boot image become corrupted somehow.)");

    }

    ESP_LOGI(TAG, "Running partition type %d subtype %d (offset 0x%08x)",

             running->type, running->subtype, running->address);

 

    //Wait for the wifi to connect to the OTA, the project can make the upgrade command enter the OTA

    xEventGroupWaitBits(wifi_event_group, CONNECTED_BIT,

                        false, true, portMAX_DELAY);

    ESP_LOGI(TAG, "Connect to Wifi ! Start to Connect to Server....");

 

    //connect to http server

    if (connect_to_http_server()) {undefined

        ESP_LOGI(TAG, "Connected to http server");

    } else {undefined

        ESP_LOGE(TAG, "Connect to http server failed!");

        task_fatal_error();

    }

 

    //Group http package to send

    const char *GET_FORMAT =

        "GET %s HTTP/1.0\r\n"

        "Host: %s:%s\r\n"

        "User-Agent: esp-idf/1.0 esp32\r\n\r\n";

 

    char *http_request = NULL;

    int get_len = asprintf(&http_request, GET_FORMAT, EXAMPLE_FILENAME, EXAMPLE_SERVER_IP, EXAMPLE_SERVER_PORT);

    if (get_len < 0) {undefined

        ESP_LOGE(TAG, "Failed to allocate memory for GET request buffer");

        task_fatal_error();

    }

    int res = send(socket_id, http_request, get_len, 0);

    free(http_request);

    if (res < 0) {undefined

        ESP_LOGE(TAG, "Send GET request to server failed");

        task_fatal_error();

    } else {undefined

        ESP_LOGI(TAG, "Send GET request to server succeeded");

    }

 

    //Get the next Flash partition of the current system (next to the currently used OTA_X partition) that can be used to burn and upgrade the firmware

    update_partition = esp_ota_get_next_update_partition(NULL);

    ESP_LOGI(TAG, "Writing to partition subtype %d at offset 0x%x",

             update_partition->subtype, update_partition->address);

    assert(update_partition != NULL);

    //OTA write start

    err = esp_ota_begin(update_partition, OTA_SIZE_UNKNOWN, &update_handle);

    if (err != ESP_OK) {undefined

        ESP_LOGE(TAG, "esp_ota_begin failed, error=%d", err);

        task_fatal_error();

    }

    ESP_LOGI(TAG, "esp_ota_begin succeeded");

 

    bool resp_body_start = false, flag = true;

    //Receive complete

    while (flag) {undefined

        memset(text, 0, TEXT_BUFFSIZE);

        memset(ota_write_data, 0, BUFFSIZE);

        //Receive http packets

        int buff_len = recv(socket_id, text, TEXT_BUFFSIZE, 0);

        if (buff_len < 0) { //Package exception

            ESP_LOGE(TAG, "Error: receive data error! errno=%d", errno);

            task_fatal_error();

        } else if (buff_len > 0 && !resp_body_start) { //包头

            memcpy(ota_write_data, text, buff_len);

            resp_body_start = read_past_http_header(text, buff_len, update_handle);

        } else if (buff_len > 0 && resp_body_start) { //Data segment packet

            memcpy(ota_write_data, text, buff_len);

            //write flash

            err = esp_ota_write( update_handle, (const void *)ota_write_data, buff_len);

            if (err != ESP_OK) {undefined

                ESP_LOGE(TAG, "Error: esp_ota_write failed! err=0x%x", err);

                task_fatal_error();

            }

            binary_file_length += buff_len;

            ESP_LOGI(TAG, "Have written image length %d", binary_file_length);

        } else if (buff_len == 0) { //end packet

            flag = false;

            ESP_LOGI(TAG, "Connection closed, all packets received");

            close(socket_id);

        } else {//Unknown error

            ESP_LOGE(TAG, "Unexpected recv result");

        }

    }

 

    ESP_LOGI(TAG, "Total Write binary data length : %d", binary_file_length);

    //OTA write end

    if (esp_ota_end(update_handle) != ESP_OK) {undefined

        ESP_LOGE(TAG, "esp_ota_end failed!");

        task_fatal_error();

    }

    //Update the data in the OTA data area after the upgrade is completed. When restarting, load the execution target (new) firmware according to the data in the OTA data area to the Flash partition.

    err = esp_ota_set_boot_partition(update_partition);

    if (err != ESP_OK) {undefined

        ESP_LOGE(TAG, "esp_ota_set_boot_partition failed! err=0x%x", err);

        task_fatal_error();

    }

    ESP_LOGI(TAG, "Prepare to restart system!");

    esp_restart();

    return ;

}

 

Show results

Test process ( original )

start the server

Python in the build chain has a built-in HTTP server, which we can use directly here. We will use the example get-started/hello_world as the firmware that needs to be updated.

Open a terminal and enter the following commands to compile the example and start the server:

1

2

3

4

cd $IDF_PATH/examples/get-started/hello_world //Enter helloworld path

make //compile

cd build //Enter the compiled file.bin directory

python -m SimpleHTTPServer 8070 //Run http server (local)

After the server is running, the contents of the build directory can be browsed at http://localhost:8070/  .

NB: On some systems the command might be python2 -m SimpleHTTPServer.

NB: You may have noticed that there is nothing special about the "hello world" used for the update, this is because any .bin application compiled by esp-idf can be used as an OTA application. The only difference is whether it will be written to the factory partition or the OTA partition.

If your firewall blocks access to port 8070, open it while this example is running.

If you have any firewall software running that will block incoming access to port 8070, configure it to allow access while running the example.

 

When programming, you need to use the target erase_flash to erase the entire flash (this will delete all the data left in the ota_data partition before), and then use the serial port to program the factory:

1

make erase_flash flash //erase the entire flash

 

Modify WiFi account password, HTTP server IP (computer IP) and Port, compile and download OTA Demo code to ESP32

ESP32 automatically OTA after connecting to WiFi, and run HelloWorld program after success

 

Show results

 

OTA summary

The advantages and possible problems of the OTA method here

Judging from the ESP32 SDKOTA Demo upgrade strategy, it should be relatively safe. Regardless of any abnormality during the upgrade, as long as the data in the OTA data area is not modified, the device can still load the original firmware for execution.

At present, it seems that the places that may need to be considered are:

  1. Is it possible that the OTA data points to an area where the upgrade failed, causing the device to load corrupt firmware;
  2. Because OTA requires three upgrade-related areas, the firmware size is limited to less than SPI Flash Size/3
  3. The error prevention/retransmission/verification mechanism should also be added to obtain the upgrade target firmware; corresponding handling should also be taken when an exception occurs.
  4. Set aside a background control interface for modifying the OTA data area, which is convenient for the remote control program to run.

OTA completely uses the official source code, and it is recommended to use HTTPS in the project.

Source address: https://github.com/xiaolongba/wireless-tech

 

Click me -> more ESP32 development guide series catalog

Tags: Chapter 20 Over-the-Air Upgrade (OTA) of ESP32

ESP32 development ESP32 Development Guide ESP32

Related: Chapter 20 Over-the-Air Upgrade (OTA) of ESP32