1. Background

In the actual product development process, online upgrade can remotely solve the problems introduced by product software development and better meet user needs.

2. Introduction to OTA

An OTA (over-the-air) update is the process of loading firmware to an ESP module using a Wi-Fi connection instead of a serial port.

2.1. There are three ways to upgrade ESP32 via OTA:

• Arduino IDE: Mainly used in the software development stage to realize firmware programming without wiring
• Web Browser: Manually deliver application update modules through a web browser
• HTTP Server: Automatically use http server - for production applications 

In the three upgrade cases, the first firmware upload must be done through the serial port. 

The OTA process has no imposed security and needs to ensure that developers can only get updates from legitimate/trusted sources. After the update is complete, the module will restart and execute the new code. Developers should ensure that applications running on modules are closed and restarted in a safe manner.

2.2, Confidentiality Security

Modules must be displayed wirelessly to be updated with new sketches. This allows the module to be forcibly hacked and loaded with additional code. To reduce the chances of being hacked, consider password protecting your uploads, choosing certain OTA ports, etc.

ArduinoOTA library interface for improved security:

void setPort(uint16_t port);
void setHostname(const char* hostname);
void setPassword(const char* password);

void onStart(OTA_CALLBACK(fn));
void onEnd(OTA_CALLBACK(fn));
void onProgress (OTA_CALLBACK_PROGRESS (fn));
void onError(OTA_CALLBACK_ERROR (fn));

Certain protections are already built in and do not require any additional coding by the developer. ArduinoOTA and espota.py use Digest-MD5 to verify uploads. The integrity of the transmitted data is verified on the ESP side using the MD5 checksum.

2.2, OTA upgrade strategy - for http

ESP32 connects to the HTTP server and sends a request Get to upgrade the firmware; each time it reads 1KB of firmware data and writes it to Flash.

There are (at least) four partitions related to the upgrade in the ESP32 SPI Flash: OTA data, Factory App, OTA_0, OTA_1. The FactoryApp memory contains the factory default firmware.

When the OTA upgrade is performed for the first time, the OTA Demo burns the target firmware to the OTA_0 partition, and after the burn is completed, the data of the OTA data partition is updated and restarted.

When the system restarts, the data of the OTA data partition is obtained for calculation, and it is decided to load the firmware of the OTA_0 partition for execution (instead of the firmware in the default Factory App partition) to realize the upgrade.

Similarly, if the ESP32 is already executing the firmware in OTA_0 after an upgrade, the OTA Demo will write the target firmware to the OTA_1 partition when upgrading again. After rebooting, perform the OTA_1 partition to upgrade. By analogy, the upgraded target firmware is always burned interactively between the two partitions OTA_0 and OTA_1, which will not affect the Factory App firmware when it leaves the factory.

3. OTA instance analysis

3,1, Arduino IDE solution firmware update

Uploading modules wirelessly from the Arduino IDE is suitable for the following typical scenarios: 

Faster alternative to loading via serial during firmware development - for updating a small number of modules, only the modules are available on the same network as the Arduino IDE's computer.

Reference example:

#include <WiFi.h>
#include <ESPmDNS.h>
#include <WiFiUdp.h>
#include <ArduinoOTA.h>
 
const char* ssid = "..........";
const char* password = "..........";
 
void setup() {
  Serial.begin(115200);
  Serial.println("Booting");
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  while (WiFi.waitForConnectResult() != WL_CONNECTED) {
    Serial.println("Connection Failed! Rebooting...");
    delay(5000);
    ESP.restart();
  }
 
  // Port defaults to 3232
  // ArduinoOTA.setPort(3232);
 
  // Hostname defaults to esp3232-[MAC]
  // ArduinoOTA.setHostname("myesp32");
 
  // No authentication by default
  // ArduinoOTA.setPassword("admin");
 
  // Password can be set with it's md5 value as well
  // MD5(admin) = 21232f297a57a5a743894a0e4a801fc3
  // ArduinoOTA.setPasswordHash("21232f297a57a5a743894a0e4a801fc3");
 
  ArduinoOTA
    .onStart([]() {
      String type;
      if (ArduinoOTA.getCommand() == U_FLASH)
        type = "sketch";
      else // U_SPIFFS
        type = "filesystem";
 
      // NOTE: if updating SPIFFS this would be the place to unmount SPIFFS using SPIFFS.end()
      Serial.println("Start updating " + type);
    })
    .onEnd([]() {
      Serial.println("\nEnd");
    })
    .onProgress([](unsigned int progress, unsigned int total) {
      Serial.printf("Progress: %u%%\r", (progress / (total / 100)));
    })
    .onError([](ota_error_t error) {
      Serial.printf("Error[%u]: ", error);
      if (error == OTA_AUTH_ERROR) Serial.println("Auth Failed");
      else if (error == OTA_BEGIN_ERROR) Serial.println("Begin Failed");
      else if (error == OTA_CONNECT_ERROR) Serial.println("Connect Failed");
      else if (error == OTA_RECEIVE_ERROR) Serial.println("Receive Failed");
      else if (error == OTA_END_ERROR) Serial.println("End Failed");
    });
 
  ArduinoOTA.begin();
 
  Serial.println("Ready");
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());
}
 
void loop() {
  ArduinoOTA.handle();
}

3,2, Web Browser  solution firmware update

Scenarios for using this program:

Loading directly from the Arduino IDE is inconvenient or impossible

Users cannot expose OTA's modules from external update servers

Provide post-deployment updates to a small number of modules when setting up an update server is not feasible

Reference example:

#include <WiFi.h>
#include <WiFiClient.h>
#include <WebServer.h>
#include <ESPmDNS.h>
#include <Update.h>
 
const char* host = "esp32";
const char* ssid = "xxx";
const char* password = "xxxx";
 
WebServer server(80);
 
/*
* Login page
*/
 
const char * loginIndex =
 "<form name='loginForm'>"
    "<table width='20%' bgcolor='A09F9F' align='center'>"
        "<tr>"
            "<td colspan=2>"
                "<center><font size=4><b>ESP32 Login Page</b></font></center>"
                "<br>"
            "</td>"
            "<br>"
            "<br>"
        "</tr>"
        "<td>Username:</td>"
        "<td><input type='text' size=25 name='userid'><br></td>"
        "</tr>"
        "<br>"
        "<br>"
        "<tr>"
            "<td>Password:</td>"
            "<td><input type='Password' size=25 name='pwd'><br></td>"
            "<br>"
            "<br>"
        "</tr>"
        "<tr>"
            "<td><input type='submit' onclick='check(this.form)' value='Login'></td>"
        "</tr>"
    "</table>"
"</form>"
"<script>"
    "function check(form)"
    "{"
    "if(form.userid.value=='admin' && form.pwd.value=='admin')"
    "{"
    "window.open('/serverIndex')"
    "}"
    "else"
    "{"
    " alert('Error Password or Username')/*displays error message*/"
    "}"
    "}"
"</script>";
 
/*
* Server Index Page
*/
 
const char* serverIndex =
"<script src='https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js'></script>"
"<form method='POST' action='#' enctype='multipart/form-data' id='upload_form'>"
   "<input type='file' name='update'>"
        "<input type='submit' value='Update'>"
    "</form>"
 "<div id='prg'>progress: 0%</div>"
 "<script>"
  "$('form').submit(function(e){"
  "e.preventDefault();"
  "var form = $('#upload_form')[0];"
  "var data = new FormData(form);"
  " $.ajax({"
  "url: '/update',"
  "type: 'POST',"
  "data: data,"
  "contentType: false,"
  "processData:false,"
  "xhr: function() {"
  "var xhr = new window.XMLHttpRequest();"
  "xhr.upload.addEventListener('progress', function(evt) {"
  "if (evt.lengthComputable) {"
  "var per = evt.loaded / evt.total;"
  "$('#prg').html('progress: ' + Math.round(per*100) + '%');"
  "}"
  "}, false);"
  "return xhr;"
  "},"
  "success:function(d, s) {"
  "console.log('success!')"
"},"
"error: function (a, b, c) {"
"}"
"});"
"});"
 "</script>";
 
/*
* setup function
*/
void setup(void) {
  Serial.begin(115200);
 
  // Connect to WiFi network
  WiFi.begin(ssid, password);
  Serial.println("");
 
  // Wait for connection
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("Connected to ");
  Serial.println(ssid);
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());
 
  /*use mdns for host name resolution*/
  if (!MDNS.begin(host)) { //http://esp32.local
    Serial.println("Error setting up MDNS responder!");
    while (1) {
      delay(1000);
    }
  }
  Serial.println("mDNS responder started");
  /*return index page which is stored in serverIndex */
  server.on("/", HTTP_GET, []() {
    server.sendHeader("Connection", "close");
    server.send(200, "text/html", loginIndex);
  });
  server.on("/serverIndex", HTTP_GET, []() {
    server.sendHeader("Connection", "close");
    server.send(200, "text/html", serverIndex);
  });
  /*handling uploading firmware file */
  server.on("/update", HTTP_POST, []() {
    server.sendHeader("Connection", "close");
    server.send(200, "text/plain", (Update.hasError()) ? "FAIL" : "OK");
    ESP.restart();
  }, []() {
    HTTPUpload& upload = server.upload();
    if (upload.status == UPLOAD_FILE_START) {
      Serial.printf("Update: %s\n", upload.filename.c_str());
      if (!Update.begin(UPDATE_SIZE_UNKNOWN)) { //start with max available size
        Update.printError(Serial);
      }
    } else if (upload.status == UPLOAD_FILE_WRITE) {
      /* flashing firmware to ESP*/
      if (Update.write(upload.buf, upload.currentSize) != upload.currentSize) {
        Update.printError(Serial);
      }
    } else if (upload.status == UPLOAD_FILE_END) {
      if (Update.end(true)) { //true to set the size to the current progress
        Serial.printf("Update Success: %u\nRebooting...\n", upload.totalSize);
      } else {
        Update.printError(Serial);
      }
    }
  });
  server.begin();
}
 
void loop(void) {
  server.handleClient();
  delay(1);
}

3.3, HTTP server implementation update

The ESPhttpUpdate class can check for updates and download binaries from HTTP web servers. Updates can be downloaded from every IP or domain name address on the network or the Internet, mainly for remote server updates and upgrades.

Reference example:

/**
   AWS S3 OTA Update
   Date: 14th June 2017
   Author: Arvind Ravulavaru <https://github.com/arvindr21>
   Purpose: Perform an OTA update from a bin located in Amazon S3 (HTTP Only)
 
   Upload:
   Step 1 : Download the sample bin file from the examples folder
   Step 2 : Upload it to your Amazon S3 account, in a bucket of your choice
   Step 3 : Once uploaded, inside S3, select the bin file >> More (button on top of the file list) >> Make Public
   Step 4 : You S3 URL => http://bucket-name.s3.ap-south-1.amazonaws.com/sketch-name.ino.bin
   Step 5 : Build the above URL and fire it either in your browser or curl it `curl -I -v http://bucket-name.ap-south-1.amazonaws.com/sketch-name.ino.bin` to validate the same
   Step 6:  Plug in your SSID, Password, S3 Host and Bin file below
 
   Build & upload
   Step 1 : Menu > Sketch > Export Compiled Library. The bin file will be saved in the sketch folder (Menu > Sketch > Show Sketch folder)
   Step 2 : Upload bin to S3 and continue the above process
 
   // Check the bottom of this sketch for sample serial monitor log, during and after successful OTA Update
*/
 
#include <WiFi.h>
#include <Update.h>
 
WiFiClient client;
 
// Variables to validate
// response from S3
int contentLength = 0;
bool isValidContentType = false;
 
// Your SSID and PSWD that the chip needs
// to connect to
const char* SSID = "YOUR-SSID";
const char* PSWD = "YOUR-SSID-PSWD";
 
// S3 Bucket Config
String host = "bucket-name.s3.ap-south-1.amazonaws.com"; // Host => bucket-name.s3.region.amazonaws.com
int port = 80; // Non https. For HTTPS 443. As of today, HTTPS doesn't work.
String bin = "/sketch-name.ino.bin"; // bin file name with a slash in front.
 
// Utility to extract header value from headers
String getHeaderValue(String header, String headerName) {
  return header.substring(strlen(headerName.c_str()));
}
 
// OTA Logic
void execOTA() {
  Serial.println("Connecting to: " + String(host));
  // Connect to S3
  if (client.connect(host.c_str(), port)) {
    // Connection Succeed.
    // Fecthing the bin
    Serial.println("Fetching Bin: " + String(bin));
 
    // Get the contents of the bin file
    client.print(String("GET ") + bin + " HTTP/1.1\r\n" +
                 "Host: " + host + "\r\n" +
                 "Cache-Control: no-cache\r\n" +
                 "Connection: close\r\n\r\n");
 
    // Check what is being sent
    //    Serial.print(String("GET ") + bin + " HTTP/1.1\r\n" +
    //                 "Host: " + host + "\r\n" +
    //                 "Cache-Control: no-cache\r\n" +
    //                 "Connection: close\r\n\r\n");
 
    unsigned long timeout = millis();
    while (client.available() == 0) {
      if (millis() - timeout > 5000) {
        Serial.println("Client Timeout !");
        client.stop();
        return;
      }
    }
    // Once the response is available,
    // check stuff
 
    /*
       Response Structure
        HTTP/1.1 200 OK
        x-amz-id-2: NVKxnU1aIQMmpGKhSwpCBh8y2JPbak18QLIfE+OiUDOos+7UftZKjtCFqrwsGOZRN5Zee0jpTd0=
        x-amz-request-id: 2D56B47560B764EC
        Date: Wed, 14 Jun 2017 03:33:59 GMT
        Last-Modified: Fri, 02 Jun 2017 14:50:11 GMT
        ETag: "d2afebbaaebc38cd669ce36727152af9"
        Accept-Ranges: bytes
        Content-Type: application/octet-stream
        Content-Length: 357280
        Server: AmazonS3
                                   
        {{BIN FILE CONTENTS}}
 
    */
    while (client.available()) {
      // read line to / n
      String line = client.readStringUntil('\n');
      // remove space, to check if the line is end of headers
      line.trim();
 
      // if the the line is empty,
      // this is end of headers
      // break the while and feed the
      // remaining `client` to the
      // Update.writeStream();
      if (!line.length()) {
        //headers ended
        break; // and get the OTA started
      }
 
      // Check if the HTTP Response is 200
      // else break and Exit Update
      if (line.startsWith("HTTP/1.1")) {
        if (line.indexOf("200") < 0) {
          Serial.println("Got a non 200 status code from server. Exiting OTA Update.");
          break;
        }
      }
 
      // extract headers here
      // Start with content length
      if (line.startsWith("Content-Length: ")) {
        contentLength = atoi((getHeaderValue(line, "Content-Length: ")).c_str());
        Serial.println("Got " + String(contentLength) + " bytes from server");
      }
 
      // Next, the content type
      if (line.startsWith("Content-Type: ")) {
        String contentType = getHeaderValue(line, "Content-Type: ");
        Serial.println("Got " + contentType + " payload.");
        if (contentType == "application/octet-stream") {
          isValidContentType = true;
        }
      }
    }
  } else {
    // Connect to S3 failed
    // May be try?
    // Probably a choppy network?
    Serial.println("Connection to " + String(host) + " failed. Please check your setup");
    // retry??
    // execOTA ();
  }
 
  // Check what is the contentLength and if content type is `application/octet-stream`
  Serial.println("contentLength : " + String(contentLength) + ", isValidContentType : " + String(isValidContentType));
 
  // check contentLength and content type
  if (contentLength && isValidContentType) {
    // Check if there is enough to OTA Update
    bool canBegin = Update.begin(contentLength);
 
    // If yes, begin
    if (canBegin) {
      Serial.println("Begin OTA. This may take 2 - 5 mins to complete. Things might be quite for a while.. Patience!");
      // No activity would appear on the Serial monitor
      // So be patient. This may take 2 - 5mins to complete
      size_t written = Update.writeStream(client);
 
      if (written == contentLength) {
        Serial.println("Written : " + String(written) + " successfully");
      } else {
        Serial.println("Written only : " + String(written) + "/" + String(contentLength) + ". Retry?" );
        // retry??
        // execOTA ();
      }
 
      if (Update.end()) {
        Serial.println("OTA done!");
        if (Update.isFinished()) {
          Serial.println("Update successfully completed. Rebooting.");
          ESP.restart();
        } else {
          Serial.println("Update not finished? Something went wrong!");
        }
      } else {
        Serial.println("Error Occurred. Error #: " + String(Update.getError()));
      }
    } else {
      // not enough space to begin OTA
      // Understand the partitions and
      // space availability
      Serial.println("Not enough space to begin OTA");
      client.flush();
    }
  } else {
    Serial.println("There was no content in the response");
    client.flush();
  }
}
 
void setup() {
  //Begin Serial
  Serial.begin(115200);
  delay(10);
 
  Serial.println("Connecting to " + String(SSID));
 
  // Connect to provided SSID and PSWD
  WiFi.begin(SSID, PSWD);
 
  // Wait for connection to establish
  while (WiFi.status() != WL_CONNECTED) {
    Serial.print("."); // Keep the serial monitor lit!
    delay(500);
  }
 
  // Connection Succeed
  Serial.println("");
  Serial.println("Connected to " + String(SSID));
 
  // Execute OTA Update
  execOTA ();
}
 
void loop() {
  // chill
}
 
/*
* Serial Monitor log for this sketch
*
* If the OTA succeeded, it would load the preference sketch, with a small modification. i.e.
* Print `OTA Update succeeded!! This is an example sketch : Preferences > StartCounter`
* And then keeps on restarting every 10 seconds, updating the preferences
*
*
      rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
      configsip: 0, SPIWP:0x00
      clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
      mode:DIO, clock div:1
      load: 0x3fff0008, len: 8
      load: 0x3fff0010, len: 160
      load: 0x40078000, len: 10632
      load: 0x40080000, len: 252
      entry 0x40080034
      Connecting to SSID
      ......
      Connected to SSID
      Connecting to: bucket-name.s3.ap-south-1.amazonaws.com
      Fetching Bin: /StartCounter.ino.bin
      Got application/octet-stream payload.
      Got 357280 bytes from server
      contentLength : 357280, isValidContentType : 1
      Begin OTA. This may take 2 - 5 mins to complete. Things might be quite for a while.. Patience!
      Written : 357280 successfully
      OTA done!
      Update successfully completed. Rebooting.
      ets Jun 8 2016 00:22:57
      
      rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
      configsip: 0, SPIWP:0x00
      clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
      mode:DIO, clock div:1
      load: 0x3fff0008, len: 8
      load: 0x3fff0010, len: 160
      load: 0x40078000, len: 10632
      load: 0x40080000, len: 252
      entry 0x40080034
      
      OTA Update succeeded!! This is an example sketch : Preferences > StartCounter
      Current counter value: 1
      Restarting in 10 seconds...
      E (102534) wifi: esp_wifi_stop 802 wifi is not init
      ets Jun 8 2016 00:22:57
      
      rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
      configsip: 0, SPIWP:0x00
      clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
      mode:DIO, clock div:1
      load: 0x3fff0008, len: 8
      load: 0x3fff0010, len: 160
      load: 0x40078000, len: 10632
      load: 0x40080000, len: 252
      entry 0x40080034
      
      OTA Update succeeded!! This is an example sketch : Preferences > StartCounter
      Current counter value: 2
      Restarting in 10 seconds...
 
      ....
*
*/

refer：

http://www.yfrobot.com/wiki/index.php?title=OTA_Updates

http://www.yfrobot.com/thread-11979-1-1.html

https://www.arduino.cn/thread-41132-1-1.html

http://www.ifindbug.com/doc/id-44691/name-Brief Analysis of esp32 Flash Partition and OTA Function.html