How to reduce MQTT data size

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
Affinite
Normal user
Posts: 57
Joined: 03 Apr 2018, 18:46

How to reduce MQTT data size

#1 Post by Affinite » 15 Aug 2023, 20:58

Hi All
I have an ESP8266 running ESPEASY (ESP_Easy_mega_20230623_normal_ESP8266_4M1M) with an ADC measuring voltages of three batteries on my boat.
The values are transmitted every 15 minutes over an OpenHab MQTT controller using a Publish statement in rules.
The ESP8266 is connected to the Internet using a WiFi connection to a MiFi device with a standard PAYG SIM in it.
So, I am transmitting 3 x MQTT messages every 15 minutes ie 288 MQTT messages a day and, according to my ISP, I'm using about 2MB of data a day so each message is on average approx. 7KB
The ESP8266 is the only device connected to the MiFi router.

I was hoping to measure my IoT traffic in bytes not kilobytes so does anyone have any suggestions to reduce my data traffic without reducing the frequency of messages ?

Thanks

TD-er
Core team member
Posts: 8756
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: How to reduce MQTT data size

#2 Post by TD-er » 15 Aug 2023, 21:33

Keeping a connection open to a MQTT broker does also need some traffic.
And if you are subscribed to a topic which does receive data from other devices/users, you will also see extra traffic.

I think entering deep sleep may already help reducing traffic, as the ESP is then not connected to the broker.
It will also consume less energy.
But this may cause other issues if the ESP may not be able to connect to WiFi, so you could also disable the MQTT controller via rules and enable it again right before sending. (ControllerDisable and ControllerEnable)
See:
https://espeasy.readthedocs.io/en/lates ... l-commands

N.B. Using these commands, the changed state for enable/disable of the controller is only changed in memory and thus not changed permanently in the settings until it is saved.
However, as soon as anything is saved, the then current enabled/disabled state will be saved.
So when using this, you might want to set the required state explicitly in the rules on the system#boot event.
You really don't want to save it often, but better make the state explicit at boot if you might have disabled the controller via rules.

Affinite
Normal user
Posts: 57
Joined: 03 Apr 2018, 18:46

Re: How to reduce MQTT data size

#3 Post by Affinite » 16 Aug 2023, 08:27

TD-er
Thanks for quick reply.

My ESP is not subscribed to any topics. It only publishes.
I'll experiment with the ideas that you've suggested and report back.
An idea occurred to me in the night ...
I've set the ESP to receive NTP.
That must use some data right ?
I dont really see why I need time on the ESP so I think I'll turn off NTP and see if that reduces data.

TD-er
Core team member
Posts: 8756
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: How to reduce MQTT data size

#4 Post by TD-er » 16 Aug 2023, 09:31

NTP is only sending really short packets every few hours.
After the initial call at boot, the next one might be relatively quickly (like 1 hour) and then it will run at roughly 4 - 5 times a day (random interval to prevent DDoS-like behavior after a power outage)

User avatar
Ath
Normal user
Posts: 3522
Joined: 10 Jun 2018, 12:06
Location: NL

Re: How to reduce MQTT data size

#5 Post by Ath » 16 Aug 2023, 09:59

Haven't yet tried, but did you empty the 'Controller Subscribe' field in your Controller settings? (looking in the code, that's not explicitly taken care of, so I'm not sure how it will work out...). If not empty ESPEasy will subscribe to that topic, and probably receive data, that will cause TCP Ack messages to be sent, I expect.
/Ton (PayPal.me)

Post Reply

Who is online

Users browsing this forum: No registered users and 31 guests