OTA Crashing WeMos device

[sillyfrog]

After much testing, I have been able to find one of the examples is able to replicate my issue. The most basic example as per http://marvinroger.github.io/homie-esp8266/docs/2.0.0/quickstart/getting-started/ does work OK. The TemperatureSensor does not. I think it maybe related to having two temperatureNode.advertise lines - but no idea how.

I'm using a WeMos Mini Pro v1.1.0 and a v1.0.0 (both have the same behavior - so fairly sure it's not the device). I'm sure OTA used to work on these same devices but that may have been Homie <= 1.5. I'm only recently getting back into OTA updates using the v2.0 firmware (latest stable release).

Basically, if I upload the "TemperatureSensor" example, then change the version to v1.0.1 (say), and recompile, and try to upload the newly built firmware using ota_updater.py, I get an Exception right away. The decoded exception looks like this: https://gist.github.com/sillyfrog/1e66895dd9b3ed576d31514c7bcface2

I have tried using both Arduino, and PlatformIO (each with their own checkout), and ensured all libraries are up to date.

Things appear to otherwise work OK, until you go to do an OTA update, and then things fall apart.

Happy to provide any more information you may need - and I feel I'm just doing something silly, but not sure where to start looking :/

Thanks in advance.

Cheers.

[timpur]

Currently looking into another ticket, will get to this asap

[sillyfrog]

Thanks! If you have any thoughts as to where I could start looking, happy to take a crack and start poking around - it's pretty weird! :/

[CWempe]

Same here.

I finally wanted to start using OTA, but I cannot get it work.

When I send a firmware via homie-ota server I get a reset of my Wemos D1 mini pro.

eceiving OTA payload
↕ OTA started
Triggering OTA_STARTED event...
Firmware is base64-encoded

Exception (3):
epc1=0x4024162d epc2=0x00000000 epc3=0x00000000 excvaddr=0x40031e61 depc=0x00000000

ctx: sys 
sp: 3ffffa50 end: 3fffffb0 offset: 01a0

>>>stack>>>
3ffffbf0:  4022e0d0 3fff40a4 00000000 4022e079  
3ffffc00:  00000000 00000003 00000001 00000002  
3ffffc10:  3fff251c 00000000 00000000 3fff4744  
3ffffc20:  3fff3cb4 00000000 000001e1 00000003  
3ffffc30:  00000010 3ffffcd0 3ffffcd0 3fff40a4  
3ffffc40:  00000000 400042db 00000003 402157dd  
3ffffc50:  00000000 3ffffc90 00000004 00000000  
3ffffc60:  4010500c 40245f94 00000001 3fff0684  
3ffffc70:  00064020 00000000 00000001 40241740  
3ffffc80:  40258c90 fffffe7f 00000000 40207270  
3ffffc90:  3ffef558 3ffef558 3fff0684 4024b135  
3ffffca0:  4021f244 00000001 00000000 3fff0684  
3ffffcb0:  00064020 00000000 00000001 402201b8  
3ffffcc0:  00000000 3ffeff2c 00000019 402227f4  
3ffffcd0:  00000000 00000000 00000001 3ffefd20  
3ffffce0:  3ffefda0 3fff457d 00000001 4020d8c8  
3ffffcf0:  3fff00e9 3ffe8304 00000000 401004d8  
3ffffd00:  3fff27da 00000005 3ffefd20 40225e53  
3ffffd10:  00000000 000001bd 3ffeff2c 402210e0  
3ffffd20:  00000005 3ffe89cc 3ffefd20 3fff457d  
3ffffd30:  3fff27d4 00000000 3ffefd20 4020dafe  
3ffffd40:  00085580 3fff44dc 3ffecb85 4000050c  
3ffffd50:  40000001 000001bd 0000000f 00000000  
3ffffd60:  3fff457d 3fff395c 3ffffdc0 00000000  
3ffffd70:  3fff457d 3fff3954 00000001 4022266c  
3ffffd80:  00085580 0000c000 00000218 4020bf37  
3ffffd90:  40000001 3fff3944 00000001 40222b4a  
3ffffda0:  4010690e 0712f50f 401021f2 40214000  
3ffffdb0:  00085580 3ffeab58 3ffeab58 4000050c  
3ffffdc0:  3fff4094 00000030 4020bf14 40222644  
3ffffdd0:  40000001 00000000 000003e8 00000000  
3ffffde0:  3fff3954 3fff27d4 00000000 3fff1bf8  
3ffffdf0:  3fff197c 00000445 00000445 00000000  
3ffffe00:  3ffffe70 000001bd 3fff3a0c 402229e5  
3ffffe10:  000001bd 00000000 00085580 00000001  
3ffffe20:  3fff4522 00000218 3fff01f0 40215335  
3ffffe30:  000001bd 00000000 00085580 00000001  
3ffffe40:  3fff4522 00000218 3fff01f0 40213df5  
3ffffe50:  3fff1f34 00000000 402130d8 40222954  
3ffffe60:  3fff4094 3ffee064 4021308c 402229ac  
3ffffe70:  0000005b 00000030 0000001a ffffffff  
3ffffe80:  3fff0371 3fff3a0c 3fffdcc0 00000020  
3ffffe90:  3ffed920 3ffedb98 00000002 00000000  
3ffffea0:  3fff01f0 3fff44dc 00000000 40213e28  
3ffffeb0:  00000000 00000001 3fff1bf0 40215b1c  
3ffffec0:  3fff40a4 00000001 0000001c 3fffdab0  
3ffffed0:  3ffedb70 3fffdcc0 3ffea3f8 3fff1c10  
3ffffee0:  3fff40a4 3fff1bf4 3fff1bf0 40215b60  
3ffffef0:  3fff40a4 3fff1bf4 3fff1bf5 4022ce44  
3fffff00:  2701a8c0 00000000 3ffeb2ea 00000000  
3fffff10:  00000000 0000024e 3fffff80 3fff44fa  
3fffff20:  3fff1d68 3fff44dc 3fff1e50 40230d69  
3fffff30:  00000014 000003c6 000003c6 3fff1d68  
3fffff40:  3fffdc80 3fff230c 3fff3c9c 3fff236c  
3fffff50:  00000008 3fff1d68 3fff44dc 4022a3c9  
3fffff60:  3fffdc80 3fff230c 3fff3c9c 4010453c  
3fffff70:  4024797e 3fff230c 3fff3c9c 40247990  
3fffff80:  3fff44ec 3fff44dc 00000000 3fff0730  
3fffff90:  4024285b 00000000 3fff3c9c 402498ef  
3fffffa0:  40000f49 3fffdab0 3fffdab0 40000f49  
<<<stack<<<

 ets Jan  8 2013,rst cause:2, boot mode:(3,7)

load 0x4010f000, len 1384, room 16 
tail 8
chksum 0x2d
csum 0x2d
v614f7c32
~ld

 ets Jan  8 2013,rst cause:4, boot mode:(3,7)

wdt reset
load 0x4010f000, len 1384, room 16 
tail 8
chksum 0x2d
csum 0x2d
v614f7c32
~ld
⸮

Any new information about how to fix or workaround this?
I think this is crucial and should be investigated for version v2.1.0

[luebbe]

My workaround is to use normal OTA instead of Homie OTA...

[sillyfrog]

@timpur Do you have any thoughts as to where we could start looking at this issue? I'm not advanced in low level ESP development, but always happy to have a poke around - I just have no clue with this one where to start :/
Thanks!

[timpur]

Assuming your using latest SDK and latest 2.0 release

Maybe try the different lwip options from the SDK, I have a feeling the smallest lwip option from the SDK might work

Sorry I haven't gotten to this, been so busy with new job and all. But I'm slowly still working on homie, currently trying to get 2.1 out the door, might add this bug to that release.

[CWempe]

Maybe try the different lwip options from the SDK

I would like to try this. But I have no idea what that means and where to change this lwip option. 🙄

And would I need to change this for the existing firmware or the firmware that gets uploaded? (I can find that out myself)

Anyway.
Thanks and an I would like to see this fixed in 2.1. 😃

[jmozmoz]

Maybe try the different lwip options from the SDK

I would like to try this. But I have no idea what that means and where to change this lwip option. roll_eyes

If you use platformIO as build environment:
http://docs.platformio.org/en/latest/platforms/espressif8266.html#lwip-variant

If you use Arduino IDE:
Tools->lwip Variant -> 4 different options (v2 Low Memory, v2 High Bandwidth, ...)

[CWempe]

I have tested with Arduino IDE and lwip v1.4 now.

First uploaded my sketch via USB and then (only changed homie firmware version number) via homie-ota.

Did not work. 😞

Got this messages via mqtt:

2018-09-30 21:31:39 homie/DeckenlampeTEST/$implementation/ota/status 202
2018-09-30 21:31:39 homie/DeckenlampeTEST/$implementation/ota/status 500 INTERNAL_ERROR 11

[sillyfrog]

@CWempe Be sure to do a hard reset after flashing over serial to test OTA, there is an issue with OTA where it won't work reliably after getting flashed over serial (if I remember correctly - this is at least what I observe). Be sure to select v2 Lower Memory.

For me, following the directions at http://docs.platformio.org/en/latest/platforms/espressif8266.html#lwip-variant , and using PIO_FRAMEWORK_ARDUINO_LWIP2_LOW_MEMORY exactly as per the example, things work great!

Thank you everyone! This appears to be sorted for me at least.

[CWempe]

Still no luck. 😞

I did a hard reset after every serial flash, now.
I tried LwIP v1.4 and 2.0 lower memory.

[CWempe]

btw: This could be a duplicate of #261

[CWempe]

I started using PlatformIO now.
Still no success with PIO_FRAMEWORK_ARDUINO_LWIP2_LOW_MEMORY. 😞
But I am not sure how to check if this option is actually be used while compiling. 🤔

Hopefully this will be fixed with this PR. (LwIP 2.1.0)
esp8266/Arduino#4980 (comment)

[sillyfrog]

@CWempe After compiling, are you flashing via serial, then trying an OTA update?

I'm not sure on how to check in the compiled code, but one thing I did is change the PIO_FRAMEWORK_ARDUINO_LWIP2_LOW_MEMORY to something broken (eg: PIO_FRAMEWORK_ARDUINO_LWIP2_LOW_MEMORYXXX), and try to compile, and ensured it complained that it was invalid. For me, once flashing over serial, and doing a hard reboot, OTA was working, but my sketch is not very big.