ESP32 hangs on boot - randomly

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
karlpe
New user
Posts: 5
Joined: 04 Jul 2021, 02:43

ESP32 hangs on boot - randomly

#1 Post by karlpe » 04 Jul 2021, 03:21

Dear All,

I've mostly finished my ESP Easy project on an ESP LOLIN32 Lite board (guess it's not an original WEMOS board, cannot say ..) however now I found it's not working in real circumstances...

I'm using it with a Li battery connected, so I would like to use deep sleep to wake up every 10-15 minutes. It would basically work, but the board sometimes stops when rebooting.
Initially I thought this might be a battery issue or a hardware issue (that's still possible for sure), but
- same behavior on battery or on USB (connected to PC)
- same behavior on 2 different LOLIN32 Lite boards. No other hardware connected

at the serial log I found, on both devices it always stops at the same log entry, at or after "AP Stopped" (+/- 5 characters in the serial log)

Code: Select all

rst:0x5 (DEEPSLEEP_RESET),boot:0x13 LASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DOUT, clock div:1
load:0x3fff0018,len:4
lod:0x3fff001c,len:1044
load:0x40078000,len:8896
load:0x40080400,len:5816
entry 0x400806ac
▒U38 : Info   :

INIT : Booting rsion: Karls Waage: Jun 26 2021 16:50:10 (ESP32 SDK v3.2.3-14-gd3e562907)
39 : Info   : INIT : Free RAM:296100
40 : Info   : INIT : Deep Sleep #111 - Restart Reason: CPU0: Deep Sleep, Wakeup reason (901) CPU1: for APP CPU, reseted by PRO CPU
40 : Info   : FS   : Mounti76 : Info   : CRC  : SecuritySettings CRC   ...OK
79 : Info   : WiFi : Start network scan all channels
115 : Info   : WiFi : AP Stopped
116
Originally I used a custom version, pulled from Github ~ a week ago. Today I pulled the most recent version (including the "WIFI_OFF" before deep sleep) - same issue; also same issue when using the latest released version (TEST_A).

There is no hardware connected to the dev board. And I switched off all plugins - only WIFI and the OpenHAB connector are activated.
The issue is quite random. I had cases that the boot was working 20 times in sequence and failed at the 21st; next time it took only 3 reboots. Also, I originally thought this is related to deep sleep (as a "wake up issue"); but I had the same behavior at least once after pressing the reset button on the board.


Do you have any hints for me - how could I mitigate that behavior; what would be steps to identify the root cause of it? I'm quite lost at the moment...



btw. I've been trying to compile against [core_esp32_3_2_0] today; that way I permanently messed up my PIO environment somehow due to some strange error in Arduino libraries ... Wstring.cpp. I hope I will get this working again :|.

best regards and thanks for any help!!
Karl

User avatar
Ath
Normal user
Posts: 3416
Joined: 10 Jun 2018, 12:06
Location: NL

Re: ESP32 hangs on boot - randomly

#2 Post by Ath » 04 Jul 2021, 09:41

Please check the power supply to your ESP, especially the on-board 5V to 3.3V regulator, it may be of poor spec/quality. During boot & starting WiFi the ESP uses quite some current. It will help to add an electrolitic capacitor of 220 to 470 uF close to to board on the 3.3V pin, to stabilize that side as much as possible.

When switching Arduino releases, it seems required to completely flush all PIO cache directories, (don't know the exact location, google is your friend ;), haven't switched them myself yet).
/Ton (PayPal.me)

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESP32 hangs on boot - randomly

#3 Post by TD-er » 04 Jul 2021, 11:05

I have used ESP32 core 3.1.0 for a short while and moved back to core 2.1.0
See: https://github.com/letscontrolit/ESPEas ... b8b549f1c8

Not sure if that's fixed in core 3.2.0 as I have not tested it myself.

You need to changeit in platformio_esp32_envs.ini as was done in the linked commit.

There are several folders you may want to remove if you change core levels like that.
Just assuming Windows here, Linux/Mac have similar paths, based on the user folder.

Based on where you placed the Git repository:

Delete all in C:\GitHub\TD-er\ESPEasy\.pio
Optional, delete all in C:\GitHub\TD-er\ESPEasy\build_output

Based on user folder:
Delete all in: C:\Users\gijsn\.platformio\.cache
Delete all in: C:\Users\gijsn\.platformio\packages (not all deletions are needed, but it is best to retrieve the right toolchain just to be sure)
Optional, delete all in: C:\Users\gijsn\.platformio\platforms

karlpe
New user
Posts: 5
Joined: 04 Jul 2021, 02:43

Re: ESP32 hangs on boot - randomly

#4 Post by karlpe » 04 Jul 2021, 12:03

Ath wrote: 04 Jul 2021, 09:41 It will help to add an electrolitic capacitor of 220 to 470 uF close to to board on the 3.3V pin, to stabilize that side as much as possible.
... unfortunately, this did not help (placed 470µF elco directly between GND and 3v pin of the board).


Thread open again for new suggestions :)

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESP32 hangs on boot - randomly

#5 Post by TD-er » 04 Jul 2021, 12:41

Can you lower the TX power of the ESP?
This may serve 2 purposes:
- Consuming less power
- If the WiFi antenna is badly tuned, it may reflect a lot less signal back into the ESP which may cause all kinds of issues.

When the ESP is booting and working fine, is it responsive to web page requests?

Another idea:
Can you place something like 10 uF capacitor between the ESP's EN pin and GND?
This will delay the boot of the ESP slightly allowing the power supply to stabilize before booting.

karlpe
New user
Posts: 5
Joined: 04 Jul 2021, 02:43

Re: ESP32 hangs on boot - randomly

#6 Post by karlpe » 04 Jul 2021, 14:56

TD-er wrote: 04 Jul 2021, 12:41 Another idea:
Can you place something like 10 uF capacitor between the ESP's EN pin and GND?
This will delay the boot of the ESP slightly allowing the power supply to stabilize before booting.
THAT was SO promising...!

Code: Select all

40 : Info   : INIT : Deep Sleep #298 - Restart Reason: CPU0: Deep Sleep, Wakeup reason (901) CPU1: for APP CPU, reseted by PRO CPU
298 (!) successful automatic boots from deep sleep; but then it failed again once... my last recorded highscore was 111 which was exceptional already. (currently I'm executing a testing deep sleep wake up loop from a rule; deepsleep,10 seconds after MQTT connected).
However the next time it stopped after already 26 tries...
I will try increasing the capacitor, maybe more helps more(?).
TD-er wrote: 04 Jul 2021, 12:41 When the ESP is booting and working fine, is it responsive to web page requests?
yes it is working absolutely fine - rarely with a short delay loading the page.
TD-er wrote: 04 Jul 2021, 12:41 Can you lower the TX power of the ESP?
This may serve 2 purposes:
- Consuming less power
- If the WiFi antenna is badly tuned, it may reflect a lot less signal back into the ESP which may cause all kinds of issues.
Trying that as next.
Actually, I will need some sending power as the device will be placed in the last corner of my garden :) on one of the devices I even added an external antenna...

karlpe
New user
Posts: 5
Joined: 04 Jul 2021, 02:43

Re: ESP32 hangs on boot - randomly

#7 Post by karlpe » 04 Jul 2021, 15:48

TD-er wrote: 04 Jul 2021, 11:05 There are several folders you may want to remove if you change core levels like that.
Just assuming Windows here, Linux/Mac have similar paths, based on the user folder.

Based on where you placed the Git repository:

Delete all in C:\GitHub\TD-er\ESPEasy\.pio
Optional, delete all in C:\GitHub\TD-er\ESPEasy\build_output

Based on user folder:
Delete all in: C:\Users\gijsn\.platformio\.cache
Delete all in: C:\Users\gijsn\.platformio\packages (not all deletions are needed, but it is best to retrieve the right toolchain just to be sure)
Thank you so much - that worked. At least restoring the original 2.1.0 environment.

Compiling against 3.2.0 did not work. There were some issues with the environment variables.
It's not important for me yet, hence I was curious and I was able to overcome it with a workaround.

If you are interested, here the errors shown:

Code: Select all

Compiling .pio\build\custom_ESP32_4M316k\src\src\Helpers\CompiletimeDefines.cpp.o
In file included from C:/Users/default.HP8470p/.platformio/packages/framework-arduinoespressif32/cores/esp32/Arduino.h:146:0,
                 from src/src/Helpers/CompiletimeDefines.h:4,
                 from src/src/Helpers/CompiletimeDefines.cpp:1:
[color=#BF0000]<command-line>:0:31: error: too many decimal points in number
C:/Users/default.HP8470p/.platformio/packages/framework-arduinoespressif32/cores/esp32/WString.h:39:76: note: 
in definition of macro 'FPSTR'
 #define FPSTR(pstr_pointer) (reinterpret_cast<const __FlashStringHelper *>(pstr_pointer))
                                                                            ^
C:/Users/default.HP8470p/.platformio/packages/framework-arduinoespressif32/cores/esp32/WString.h:40:34: note: 
in expansion of macro 'PSTR'
 #define F(string_literal) (FPSTR(PSTR(string_literal)))
                                  ^
src/src/Helpers/CompiletimeDefines.cpp:54:12: note: in expansion of macro 'F'
     return F(SET_BUILD_PLATFORM);
            ^
src/src/Helpers/CompiletimeDefines.cpp:54:14: note: in expansion of macro 'SET_BUILD_PLATFORM'
     return F(SET_BUILD_PLATFORM);
              ^
Compiling .pio\build\custom_ESP32_4M316k\src\src\Helpers\Convert.cpp.o
src/src/Helpers/CompiletimeDefines.cpp: In function 'const __FlashStringHelper* get_binary_filename()':
<command-line>:0:27: error: 'ESP_Easy_mega_20210704_custom_ESP32_4M316k' was not declared in this scope
C:/Users/default.HP8470p/.platformio/packages/framework-arduinoespressif32/cores/esp32/WString.h:39:76: note: 
in definition of macro 'FPSTR'
 #define FPSTR(pstr_pointer) (reinterpret_cast<const __FlashStringHelper *>(pstr_pointer))
                                                                            ^
C:/Users/default.HP8470p/.platformio/packages/framework-arduinoespressif32/cores/esp32/WString.h:40:34: note: 
in expansion of macro 'PSTR'
 #define F(string_literal) (FPSTR(PSTR(string_literal)))
                                  ^
src/src/Helpers/CompiletimeDefines.cpp:28:12: note: in expansion of macro 'F'
     return F(SET_BUILD_BINARY_FILENAME);
            ^
src/src/Helpers/CompiletimeDefines.cpp:28:14: note: in expansion of macro 'SET_BUILD_BINARY_FILENAME'
     return F(SET_BUILD_BINARY_FILENAME);
              ^
src/src/Helpers/CompiletimeDefines.cpp: In function 'const __FlashStringHelper* get_build_platform()':        
<command-line>:0:20: error: 'Windows' was not declared in this scope
C:/Users/default.HP8470p/.platformio/packages/framework-arduinoespressif32/cores/esp32/WString.h:39:76: note: 
in definition of macro 'FPSTR'
 #define FPSTR(pstr_pointer) (reinterpret_cast<const __FlashStringHelper *>(pstr_pointer))
                                                                            ^
C:/Users/default.HP8470p/.platformio/packages/framework-arduinoespressif32/cores/esp32/WString.h:40:34: note: 
in expansion of macro 'PSTR'
 #define F(string_literal) (FPSTR(PSTR(string_literal)))
                                  ^
src/src/Helpers/CompiletimeDefines.cpp:54:12: note: in expansion of macro 'F'
     return F(SET_BUILD_PLATFORM);
            ^
src/src/Helpers/CompiletimeDefines.cpp:54:14: note: in expansion of macro 'SET_BUILD_PLATFORM'
     return F(SET_BUILD_PLATFORM);
              ^
<command-line>:0:42: error: 'SP0' was not declared in this scope
C:/Users/default.HP8470p/.platformio/packages/framework-arduinoespressif32/cores/esp32/WString.h:39:76: note: 
in definition of macro 'FPSTR'
 #define FPSTR(pstr_pointer) (reinterpret_cast<const __FlashStringHelper *>(pstr_pointer))
                                                                            ^
C:/Users/default.HP8470p/.platformio/packages/framework-arduinoespressif32/cores/esp32/WString.h:40:34: note: 
in expansion of macro 'PSTR'
 #define F(string_literal) (FPSTR(PSTR(string_literal)))
                                  ^
src/src/Helpers/CompiletimeDefines.cpp:54:12: note: in expansion of macro 'F'
     return F(SET_BUILD_PLATFORM);
            ^
src/src/Helpers/CompiletimeDefines.cpp:54:14: note: in expansion of macro 'SET_BUILD_PLATFORM'
     return F(SET_BUILD_PLATFORM);
              ^
src/src/Helpers/CompiletimeDefines.cpp: In function 'const __FlashStringHelper* get_git_head()':
<command-line>:0:20: error: 'pygit2_not_installed' was not declared in this scope
C:/Users/default.HP8470p/.platformio/packages/framework-arduinoespressif32/cores/esp32/WString.h:39:76: note: 
in definition of macro 'FPSTR'
 #define FPSTR(pstr_pointer) (reinterpret_cast<const __FlashStringHelper *>(pstr_pointer))
                                                                            ^
C:/Users/default.HP8470p/.platformio/packages/framework-arduinoespressif32/cores/esp32/WString.h:40:34: note: 
in expansion of macro 'PSTR'
 #define F(string_literal) (FPSTR(PSTR(string_literal)))
                                  ^
src/src/Helpers/CompiletimeDefines.cpp:62:12: note: in expansion of macro 'F'
     return F(SET_BUILD_GIT_HEAD);
            ^
src/src/Helpers/CompiletimeDefines.cpp:62:14: note: in expansion of macro 'SET_BUILD_GIT_HEAD'
     return F(SET_BUILD_GIT_HEAD);
              ^
*** [.pio\build\custom_ESP32_4M316k\src\src\Helpers\CompiletimeDefines.cpp.o] Error 1
I "fixed" it by hard-ignoring the defined variables in src\src\Helpers\CompiletimeDefines.cpp (the error sounded for me like missing quotes, but I'm not deep into enough to understand it in detail).

Code: Select all

// This Python script will define the following defines:

//#ifndef SET_BUILD_BINARY_FILENAME << Karl commented out the IFNDEFs; 
# define SET_BUILD_BINARY_FILENAME "firmware.bin"
//#endif // ifndef SET_BUILD_BINARY_FILENAME

//#ifndef SET_BUILD_PLATFORM << Karl commented out the IFNDEFs; 
# define SET_BUILD_PLATFORM "unknown"
//#endif // ifndef SET_BUILD_PLATFORM

//#ifndef SET_BUILD_GIT_HEAD  << Karl commented out the IFNDEFs; 
# define SET_BUILD_GIT_HEAD ""
//#endif // ifndef SET_BUILD_GIT_HEAD
That way I was able to compile against 3.2.0 framework - it's running fine on my ESP and I did not see a difference to the previous version yet. However I cannot say if your specific issue with Serial.end would work now...

[edit]
Sorry it's not really correct that there is no difference:

That "AP Stopped" line in the serial log (where the boot process stopped up to now) does not appear in the serial log during boot at all anymore !?
And the "STA Started" is there only once, not twice anymore. (previously, this line always appeared twice after "AP Stopped")

[/edit]

regards, Karl

karlpe
New user
Posts: 5
Joined: 04 Jul 2021, 02:43

Re: ESP32 hangs on boot - randomly

#8 Post by karlpe » 04 Jul 2021, 21:59

Finally, to sum up (and given the possibility that I am back crying tomorrow because I'm having again that issue) - it is solved now. Lolin32 #1 rebooted now 540 times without failing, and Lolin32 #2 sits in my backyard sending data every 10 minutes successfully.

BUT, and that's what I did not expect: it seems simply using the newer PIO Espressif framework 3.2.0 did the job!

I removed all capacitors and even changed WIFI settings to "maximum TX power", and everything seems to work perfectly fine now; both the "bare" LOLIN32 Lite board and also the one with quite some hardware connected.


Thanks for your both' really fast feedback; (and thanks TD-er + Co) for that really impressive piece of software you are providing to the IoT community!!

Have a nice evening,
Karl

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESP32 hangs on boot - randomly

#9 Post by TD-er » 04 Jul 2021, 22:47

OK, thanks for the update about the 3.2.0 core for ESP32.
Will check it myself.

Always good to know there is a potential "this one fixes all" path... with all kinds of fresh new bugs ;)

Post Reply

Who is online

Users browsing this forum: No registered users and 33 guests