OTA Update with node js server

[Zeni241]

I am trying to implement OTA update from my Node Js server behind NGINX. I have this code in my sketch. (I am using NodeMCU 1.0 (ESP 12E) ).

void updater(){
  if ((WiFi.status() == WL_CONNECTED)) {
Serial.setDebugOutput(true);
   // t_httpUpdate_return ret = ESPhttpUpdate.update("http://server/file.bin");
    t_httpUpdate_return  ret = ESPhttpUpdate.update("mydomain.com",80,"/updater","v1");

    switch (ret) {
      case HTTP_UPDATE_FAILED:
        Serial.printf("HTTP_UPDATE_FAILD Error (%d): %s", ESPhttpUpdate.getLastError(), ESPhttpUpdate.getLastErrorString().c_str());
        break;

      case HTTP_UPDATE_NO_UPDATES:
        Serial.println("HTTP_UPDATE_NO_UPDATES");
        break;

      case HTTP_UPDATE_OK:
        Serial.println("HTTP_UPDATE_OK");
        break;
    }
    delay(20000);
  }

My code in nodejs is:

app.post("/updater", function (req, res) {
    var h = req.headers;
    console.log(h["ESP8266 - version"]);
    console.log(h["ESP8266-mode"]);
    console.log(h["ESP8266-sketch-size"]);
//    res.send(req);
  //  return req;
});

First thing I want to assure that my request from my NodeMCU 1.0 (ESP 12E) is reaching the right address on the server. But I don’t get any console output.

The listing in Serial Monitor is as bellow:

[HTTP-Client][begin] host: mydomain.com port: 80 uri: /updater
[HTTP-Client] connected to mydomain.com:80
[HTTP-Client] connected to mydomain.com:80
17:40:47.239 -> [HTTP-Client] sending request header
17:40:47.239 -> -----
17:40:47.239 -> GET /updater HTTP/1.0
17:40:47.239 -> Host: mydomain.com
17:40:47.239 -> User-Agent: ESP8266-http-Update
17:40:47.239 -> Connection: close
17:40:47.239 -> x-ESP8266-STA-MAC: 84:F3:EB:B7:4E:8C
17:40:47.275 -> x-ESP8266-AP-MAC: 86:F3:EB:B7:4E:8C
17:40:47.275 -> x-ESP8266-free-space: 2830336
17:40:47.275 -> x-ESP8266-sketch-size: 314816
17:40:47.275 -> x-ESP8266-sketch-md5: 40f23fadc6d4efcbb22d68f7fcc9e409
17:40:47.275 -> x-ESP8266-chip-size: 4194304
17:40:47.275 -> x-ESP8266-sdk-version: 2.2.1(cfd48f3)
17:40:47.275 -> x-ESP8266-mode: sketch
17:40:47.275 -> x-ESP8266-version: v1
17:40:47.275 -> 
17:40:47.275 -> -----
[HTTP-Client][handleHeaderResponse] RX: 'HTTP/1.1 301 Moved Permanently'
17:40:49.091 -> [HTTP-Client][handleHeaderResponse] RX: 'Server: nginx/1.14.0 (Ubuntu)'
17:40:49.091 -> [HTTP-Client][handleHeaderResponse] RX: 'Date: Thu, 01 Nov 2018 12:40:49 GMT'
17:40:49.091 -> [HTTP-Client][handleHeaderResponse] RX: 'Content-Type: text/html'
17:40:49.091 -> [HTTP-Client][handleHeaderResponse] RX: 'Content-Length: 194'
17:40:49.126 -> [HTTP-Client][handleHeaderResponse] RX: 'Connection: close'
17:40:49.126 -> [HTTP-Client][handleHeaderResponse] RX: 'Location: https://mydomain.com/updater'
17:40:49.126 -> [HTTP-Client][handleHeaderResponse] RX: ''
17:40:49.126 -> [HTTP-Client][handleHeaderResponse] code: 301
17:40:49.126 -> [HTTP-Client][handleHeaderResponse] size: 194
17:40:49.126 -> [HTTP-Client][end] still data in buffer (194), clean up.
17:40:49.126 -> [HTTP-Client][end] tcp stop
17:40:49.126 -> HTTP_UPDATE_FAILD Error (-104): Wrong HTTP Code

Can someone please guide me what I am doing wrong.

[d-a-v]

You are doing nothing wrong. We don't automatically handle HTTP-redirects.

[HTTP-Client][handleHeaderResponse] RX: 'HTTP/1.1 301 Moved Permanently'
[HTTP-Client][handleHeaderResponse] RX: 'Location: https://mydomain.com/updater'

There were some efforts done to handle such redirects for OTA by @liebman, see #5009 and its handling code. @liebman could you try and fix your PR's conflicts ?

Could this be generalized to any HTTP client requests, I don't know (@devyte ?).
Maybe @darkain (#4823) would have more input on that.

[liebman]

@d-a-v I'll look at the conflicts today, but see #5009 (comment) for issues if the redirect changes between http & https.

[Zeni241]

Thanks a lot, I will try and post results.

[devyte]

Rediirects should be handled in httpClient, not in the updater.
For generalization, the main challenge is the crossover between http and https, I.e.: changin between WiFiClient and WiFiClientSecure. It should be possible with the new begin() api that takes the socket as arg, but I think there would still be one restriction.

[Zeni241]

I took the easy path, allowed traffic on port 80 (besides port 443) in NGINX. Problem solved.
Now the response code is:

[HTTP-Client][handleHeaderResponse] RX: 'HTTP/1.1 200 OK'
10:28:33.001 -> [HTTP-Client][handleHeaderResponse] RX: 'Server: nginx/1.14.0 (Ubuntu)'
10:28:33.001 -> [HTTP-Client][handleHeaderResponse] RX: 'Date: Sat, 03 Nov 2018 05:28:34 GMT'
10:28:33.001 -> [HTTP-Client][handleHeaderResponse] RX: 'Content-Type: text/html; charset=utf-8'
10:28:33.001 -> [HTTP-Client][handleHeaderResponse] RX: 'Content-Length: 26'
10:28:33.035 -> [HTTP-Client][handleHeaderResponse] RX: 'Connection: close'
10:28:33.035 -> [HTTP-Client][handleHeaderResponse] RX: 'X-Powered-By: Express'
10:28:33.035 -> [HTTP-Client][handleHeaderResponse] RX: 'ETag: W/"1a-pKKaBD9CjV01SNJgb6d99F/0eFs"'
10:28:33.035 -> [HTTP-Client][handleHeaderResponse] RX: ''
10:28:33.035 -> [HTTP-Client][handleHeaderResponse] code: 200
10:28:33.035 -> [HTTP-Client][handleHeaderResponse] size: 26
[HTTP-Client][end] still data in buffer (26), clean up.
10:28:33.142 -> [HTTP-Client][end] tcp stop
10:28:33.142 -> HTTP_UPDATE_FAILD Error (-106): Verify Bin Header Failed

My code in node js is: (note that request from esp8266 is GET and not POST)

app.get("/updater", function (req, res) {
      console.log(req.get('x-ESP8266-version'));
    console.log(req.get('x-ESP8266-mode'));
    console.log(req.get('x-ESP8266-sketch-size'));

   var version = req.get('x-ESP8266-version');
    if (version !== 'v3') {
        res.sendFile(__dirname + "/statics/updates/xxxxxxxxxxxx.bin");
    } else {
        console.log("No new version");
      response.writeHeader(304);  
      response.end();
   }
});