mqtt import disabled on reboot?

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
hamster
Normal user
Posts: 62
Joined: 27 Sep 2015, 21:01
Location: UK

mqtt import disabled on reboot?

#1 Post by hamster » 16 Sep 2018, 10:29

I notice on the latest nightly fw : ESP_Easy_mega-20180916_normal_ESP8266_4096 also ESP_Easy_mega-20180915_dev_ESP8266_4096
after a reboot mqtt import, the enable box becomes unchecked. tried on different devices with the same result?

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: mqtt import disabled on reboot?

#2 Post by TD-er » 16 Sep 2018, 11:00

There is a problem with MQTT import. See https://github.com/letscontrolit/ESPEasy/issues/1722

So therefore I added a 'recovery' plan, which simply detects a crash and counts them.
Every new crash (above crashcount of 10) will disable the next plugin, controller, notification.
So your node crashed at least 10 times at boot when you see your MQTT import disabled.
Before this recovery option, you had to start all over again (or flash an older image), which I have done way too often the last week :(

So please move back to before 20180908 if you use MQTT import

hamster
Normal user
Posts: 62
Joined: 27 Sep 2015, 21:01
Location: UK

Re: mqtt import disabled on reboot?

#3 Post by hamster » 16 Sep 2018, 11:26

thanks for the quick reply !

sasso
Normal user
Posts: 10
Joined: 16 Mar 2017, 21:48

Re: mqtt import disabled on reboot?

#4 Post by sasso » 17 Sep 2018, 22:44

TD-er

I saw this post and downloaded the20180908 release, made an update via web gui, updated ok followed by the automated reboot of the node and was back to bootloop, even after the erase of the settings and enabling all back the bootloop is still present.

I have only enabled the next things:
- controller open hab mqtt
- plugin mqtt import
- plugin nextion
- ntp

Also noticed something but will put on github as bug: command wifissid with 2 words and a space between saves only first word and the output after save is no AP found, of course because the wifi name is 2 words not 1.

Will test this a little deeper with the 0908 build which is supposed to be working ok with mqtt import.

reinhold
Normal user
Posts: 15
Joined: 19 Jun 2017, 22:52
Location: Vienna, Austria
Contact:

Re: mqtt import disabled on reboot?

#5 Post by reinhold » 18 Sep 2018, 00:23

I ran into this problem with my own custom plugin, which I'm currently developing and which is based on the mqtt import plugin. I tracked the issue down to the MQTT subscription function, thanks to the EspStackTraceDecoder.

It appears that trying to access the first enabled mqtt controller in PLUGIN_INIT is the culprit (or looping through all tasks in MQTTSubscribe_037, I'm not really sure, but at least something doesn't seem to be properly initialized in PLUGIN_INIT).
I was able to solve the problem by moving the whole MQTT connection setup from PLUGIN_INIT to PLUGIN_ONCE_A_SECOND. If the MQTTclient object has not been created (i.e. is NULL) in PLUGIN_ONCE_A_SECOND, then I create the PubSubClient, connect and subscribe to MQTT. This also makes sense, as in my tests, the wifi connection was never set up in PLUGIN_INIT, so the mqtt connection couldn't be established at that time anyway.

I haven't run into a crash since then.

Here are the changes that I did to my own plugin:
https://github.com/kainhofer/ESPEasy/co ... 0752cd908c
And this is the full code of the plugin (under development and unfinished, but the crash is gone):
https://github.com/kainhofer/ESPEasy/bl ... Bridge.ino

Dorfrog
New user
Posts: 9
Joined: 25 Feb 2019, 00:53

Re: mqtt import disabled on reboot?

#6 Post by Dorfrog » 20 Feb 2023, 08:03

I have this problem as well, with my custom build of the most recent ESP8266 4M1M release (mega-20221224).

(Reason - I needed Neopixel, CO2, MAX7219 8 digit, DS18B20, MQTT import, and HT16K33 display on the same firmware)

So long as I do not reboot, the MQTT import works fine. But after reboot, the device is disabled.

Is there some key "define" that I have not enabled in custom.h? Pretty sure I did everything right.

The MQTT import worked fine using "normal, Neopixel, and Collection A"

Sorry to bring up this old post again - Hope you can assist.

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: mqtt import disabled on reboot?

#7 Post by TD-er » 20 Feb 2023, 08:16

There are 2 possible causes for this:

First the most unlikely one:
The task was only set to enabled using the command "taskenable" and never saved.

Now the more likely one:
ESPEasy has some protection mechanism, where it detects whether the node is rebooting a lot before even completing the full boot sequence.
After 10 of such crash/reboots in a row, it starts disabling 1 task, then the next, etc. until it manages to successfully boot.

I can't tell what may cause these crashes, without knowing the reboot reason.
Is this a watchdog reboot, or an exception?
If it is an exception, it might be that your unit is running out of memory as it may receive and process a lot of messages at once, while also initializing other tasks.
Since the tasks are started in the order they are defined on the "devices" page, you might want to try putting the MQTT import either at the 1st position or the last one. (I think the last one is probably the better one)
Or you could add some simple rules to immediately disable the MQTT import task at boot and start a timer to later enable the MQTT import task(s).

Dorfrog
New user
Posts: 9
Joined: 25 Feb 2019, 00:53

Re: mqtt import disabled on reboot?

#8 Post by Dorfrog » 20 Feb 2023, 09:36

Thanks so much for the fast reply.

I am manually rebooting to test the software (whether I choose to reboot in the tools menu, or just unplug replug)

So to test, I just reloaded fresh firmware (erase firmware first).
I added ONLY the MQTT import and tested with a single topic, which was fine - the same as before.

Again, once I restart the unit, the MQTT becomes disabled.

Because this problem is not present in the release firmware (Normal, collectionA, or Display) I am certain that it is something that I have not enabled in #defines.

As suggested, I added a rule as below (I named my MQTT import "MQTT")

on System#Boot do
TaskDisable,1/MQTT
timerSet,1,10
Endon

On Rules#Timer=1 do
TaskEnable,1/MQTT
endon

This momentarily solves the problem, but as I add additional devices (MQTT Import as the last device) the restart disables device again.

However I am still curious as to why this occurs in my custom build, but not in the listed Firmware builds? Am I able to get a system dump at startup that may reveal the problem? I note in Custom.h there is a #define BUILD_NO_DEBUG

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: mqtt import disabled on reboot?

#9 Post by TD-er » 20 Feb 2023, 10:08

You have some syntax error in your rules.
A comment is marked by //, not a single slash.

The rules parser probably ignored it in this case, but in other situations it may not be ignored and cause unexpected behavior.

My first guess would be to look at the amount of free RAM.

You can also define LIMIT_BUILD_SIZE, which does disable quite a lot of stuff and typically may result in less memory usage.
However, it may also disable some MQTT import features you may want to preserve, so please check this too.

User avatar
Ath
Normal user
Posts: 3416
Joined: 10 Jun 2018, 12:06
Location: NL

Re: mqtt import disabled on reboot?

#10 Post by Ath » 20 Feb 2023, 10:53

For a Custom build, _all_ features of the MQTT Import plugins are enabled (to ease testing during development :lol:), but the disadvantage of that is that it uses quite some memory.

Around line 374 of your Custom.h file you should have enabled USES_P037 by un-commenting that line.
For any features you are not using, either mapping, filtering or JSON (though I suggest to leave that one enabled), you can disable that by un-commenting the matching line and change the 1 to a 0 (the 1 is the default, as said), that could save enough memory to make it work better.

What NeoPixel plugin(s) have you enabled? Some also take quite some memory (besides code space), like the BusFX plugin (P128).
/Ton (PayPal.me)

Dorfrog
New user
Posts: 9
Joined: 25 Feb 2019, 00:53

Re: mqtt import disabled on reboot?

#11 Post by Dorfrog » 20 Feb 2023, 12:12

Thanks again.

It was my fault:
When I moved the MQTT to the bottom of my plugin list task nr 1 became task nr 12. Damn, I forgot to change the task nr to match. Now works and I have edited using the disable/enable rules. Either combination now works beautifully.

@TD-er
I used a single slash "/" based on the Rules guide https://espeasy.readthedocs.io/en/lates ... l-commands
eg: - TaskDisable,<task nr/task name> (a single forward slash was not my attempt to comment out :oops: ).
As suggested I recompiled again by adding #define LIMIT_BUILD_SIZE to custom.h
File size now - 897328 bytes

Free Ram 11192 - Free stack 3696
This version works and all plugins are fine.

@Ath
I'm only using a neopixel strip of WS2801, as suggested I now removed the neopixel bus P128
The firmware size is was 1,013,552 bytes.
After recompiling,
1. I removed the neopixel bus,
2. Zeroed out the MQTT "mapping" and "filter" support, and just left json enabled. Only enabled Domoticz MQTT
3. in total, now 23 plugins enabled, but only 6 plugins being used.

The new file size is 1,007,936
This version works and all plugins are fine.
Free Ram 12376 - Free stack 3696

Much appreciated.

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: mqtt import disabled on reboot?

#12 Post by TD-er » 20 Feb 2023, 12:42

Dorfrog wrote: 20 Feb 2023, 12:12 [...]
@TD-er
I used a single slash "/" based on the Rules guide https://espeasy.readthedocs.io/en/lates ... l-commands
eg: - TaskDisable,<task nr/task name> (a single forward slash was not my attempt to comment out :oops: ). [...]
Ah, maybe I should split those into 2 separate lines then.
It was meant as either of these options:
- TaskDisable,<tasknr>
- TaskDisable,<taskname>

Thus an XOR, not to ask for redundant information in the command argument :)

Dorfrog
New user
Posts: 9
Joined: 25 Feb 2019, 00:53

Re: mqtt import disabled on reboot?

#13 Post by Dorfrog » 21 Feb 2023, 09:42

Thanks - Understood. I have edited the Rule accordingly.
Anyway a day later and it is working beautifully.
FYI - I added task enable MQTT to a timer assigned to do a few other things. MQTT still did not load. I created a second timer a few seconds earlier to enable MQTT import and it solved the problem.
Thanks again.

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: mqtt import disabled on reboot?

#14 Post by TD-er » 21 Feb 2023, 09:46

Great!
Good to hear you've got it working now and also found your own fix for the next problem you found :)

SirWant
Normal user
Posts: 12
Joined: 04 Feb 2023, 16:38

Re: mqtt import disabled on reboot?

#15 Post by SirWant » 22 Mar 2023, 22:01

And I turn on mqttimport that turned off after a reboot like this:

Code: Select all

On MQTT#Connected Do
  TaskEnable,mqttimportTaskName
Endon

Post Reply

Who is online

Users browsing this forum: Ahrefs [Bot], Bing [Bot] and 24 guests