ESPeasy cannot always handle HTML requests provided by other

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
joost d
Normal user
Posts: 53
Joined: 24 Dec 2020, 17:55
Location: Netherlands/ Haarlem

ESPeasy cannot always handle HTML requests provided by other

#1 Post by joost d » 01 Oct 2022, 11:30

It seems that ESPeasy cannot always handle incoming html requests if ESPeasy also sends html requests itself. This is my experience over the years using different ESPeasy releases. This does not seem to depend on which plugins are used.

The configuration is:
- ESPEasy-mega-20220925 with custombuild "custom_ESP32_4M316k" with :
- C001 Domoticz HTTP protocol
- P141 Nokia display (controlled by http commands provided by Domoticz)
- P005 Environment - DHT11/12/22 (with "Send to Controller" idx)
- P049 Gases - CO2 MH-Z19 (with "Send to Controller" idx)

If P005 and P049 are disabled, (No HTML commands from ESPeasy) then the Domoticz commands work without errors. For days.
As soon as P005 or P049 are enabled (or HTML commands from ESPeasy), I see this error in the Domoticz log:

Error: opening url: http://.......

Am I doing something wrong? Is this an ESPeasy imperfection. Is this a issue?

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESPeasy cannot always handle HTML requests provided by other

#2 Post by TD-er » 01 Oct 2022, 11:57

Both these tasks you made will send data to the Domoticz HTTP controller, thus disabling these will no longer send data to the controller and thus also solve any issue you may have when sending to this controller.

N.B. I wonder why you aren't using the Domoticz MQTT controller as that's a lot more light weight compared to the HTTP controller for ESPEasy.

But let's dive into some possible causes of what may be happening here.

If timing stats is included in your build (typically it isn't in the builds which may struggle with the maximum bin size), you can enable collecting timing statistics on the tools->Advanced page.
These stats are accessible via the timing stats button on the Tools page.

With this running for a while, you can see how much time each part of ESPEasy needs and how often it is called.
N.B. loading the page will reset/clear these stats.

P005 does need to disable all interrupts when processing a read.
This means it cannot handle anything other than that pin needed for reading the sensor.

The resources needed by P049 depend on whether you're using Software Serial or hardware serial.
When using Software Serial, the impact is somewhat comparable to P005 since during receiving of data, the interrupts are also partly disabled.

While interrupts are disabled, or if anything shown on the timing stats page takes longer than ~100 msec, network traffic may get lost.
For example, you try to open a connection to another host via HTTP. Then you need to send data and the host may need to reply.
However if the other host does not know where to send its reply to, it may send out an ARP packet with a question like: "Who has 192.168.1.123?" (the IP of your ESP)
The ESP will then reply something like this: "AA:BB:CC:DD:EE:FF has IP 192.168.1.123" (stating its MAC address)
But these ARP requests may happen at any time and if the ESP is too busy to handle these requests, no answer may be given.
The answers of these ARP requests are also read by switches and access points in your network so they know to which port on the switch to forward packets with that MAC address as destination.

This can be dealt with by sending out gratuituous ARP packets (tools->Advanced page at the bottom)

Another reason sending something to the Domoticz HTTP server may fail is when the timeout in the controller is set too low.
If the Domoticz server is too busy, it may take longer to reply. The default 100 msec timeout may then be too short, causing the ESP to no longer listening to any reply.

Another reason may be that -when addressing the host via a DNS name instead of an IP- the ESP may need to perform a DNS lookup.
This may also sometimes take too long and timeout.

User avatar
Ath
Normal user
Posts: 3416
Joined: 10 Jun 2018, 12:06
Location: NL

Re: ESPeasy cannot always handle HTML requests provided by other

#3 Post by Ath » 01 Oct 2022, 12:25

It seems you are trying to control ESPEasy synchronously while sending data from ESPEasy to Domoticz (you mentioned custom scripting on Domoticz before), but that's not how the internet and Domoticz work, usually that is handled a-synchronously / event driven.
When trying to display values of a sensor, connected to ESPEasy, on a display, also connected to ESPEasy, you can use the predefined Lines fields holding [sensor#value] variables on your display, and set an Interval to auto-update the display. That's the regular way of doing that. Special messages from external sources can be sent to the display on ESPEasy, either by triggering an event (preferred, like: event,myEvent=value1,"value 2",... etc.), or sending direct commands.

In Rules you can handle myEvent like this:

Code: Select all

// Expects: linenr,text1,text2,text3 // use quotes for texts with commas or spaces
on myEvent do
  pcd8544,txl,%eventvalue1%,"%eventvalue2%",,"%eventvalue3%",,"%eventvalue4%"
endon
Alternative to using Lines of the display configuration is to update the display from rules when the sensor is read:

Code: Select all

on dht22#temperature do
  pcd8544,txl,2,"T %eventvalue1%"
endon
// Repeat for MH-Z19 values to display
/Ton (PayPal.me)

joost d
Normal user
Posts: 53
Joined: 24 Dec 2020, 17:55
Location: Netherlands/ Haarlem

Re: ESPeasy cannot always handle HTML requests provided by other

#4 Post by joost d » 01 Oct 2022, 13:26

I currently do not display any values from the sensors (P005 and P049) on the display (I normally do this with [sensor#value] in the predefined Lines fields).

The situation now is:
Line-1 defined in the webform at "Line 1: "with %systime%.
The other two lines are controlled from Domoticz with 'http://esp-easy-32-c/control?cmd=pcd854 ... .addSpaces(tostring(current_power),5)..'% 22,3,%22G%20'..tostring(todayM3Gas)..'%22'
The html requests from ESPeasy go automatically from P005 and P049 via C001 Domoticz html service.

joost d
Normal user
Posts: 53
Joined: 24 Dec 2020, 17:55
Location: Netherlands/ Haarlem

Re: ESPeasy cannot always handle HTML requests provided by other

#5 Post by joost d » 01 Oct 2022, 13:36

@TD-er, Thanks for your information. I have no experience with MQTT, but I am curious about it. I'll look into that.

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESPeasy cannot always handle HTML requests provided by other

#6 Post by TD-er » 01 Oct 2022, 14:03

joost d wrote: 01 Oct 2022, 13:26 I currently do not display any values from the sensors (P005 and P049) on the display (I normally do this with [sensor#value] in the predefined Lines fields).

The situation now is:
Line-1 defined in the webform at "Line 1: "with %systime%.
The other two lines are controlled from Domoticz with 'http://esp-easy-32-c/control?cmd=pcd854 ... .addSpaces(tostring(current_power),5)..'% 22,3,%22G%20'..tostring(todayM3Gas)..'%22'
The html requests from ESPeasy go automatically from P005 and P049 via C001 Domoticz html service.
I don't think you do understand what Ton was hinting at.
And given what you're writing here, I got the impression he might be on to something.

What's suggested by Ton is this:
If you're calling a HTTP call from Domoticz, which immediately triggers a HTTP call to the Domoticz server, then you're essentially causing a blocking condition.
ESPEasy can only handle one thing at a time. Either receive a command via a HTTP GET call, or making a HTTP call to another host. Not both at the same time.

So it seems this might be happening:
1) Plugin 1 (either P005 or P049, doesn't really matter) took a sample and sends it to Domoticz via HTTP
2) Domoticz immediately replies with a call to ESPEasy to set the values on the display
3) Plugin 2 takes a sample and tries to send it to Domoticz via HTTP.
4) Domoticz immediately replies with a call to ESPEasy to set the values on the display

Problem is that the sending part in step 3 may be blocked since it is still handling the call from Domoticz.

ESPEasy will then place the value in the controller queue and try again later.
But it still may result in errors in the logs.

Why don't you place the values on the display directly from ESPEasy?

joost d
Normal user
Posts: 53
Joined: 24 Dec 2020, 17:55
Location: Netherlands/ Haarlem

Re: ESPeasy cannot always handle HTML requests provided by other

#7 Post by joost d » 01 Oct 2022, 15:49

Ok,..misunderstanding...?! :-)
The order you describe "So it seems this might be happening" is not how it works here.
No values from ESPeasy sensors are shown on the ESPeasy display! The ESPeasy sensors values are only sent to Domoticz via C001.
Domoticz receives the kWh and M3-gas values from another ESPeasy and forwards them to the ESPeasy with the Display.

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESPeasy cannot always handle HTML requests provided by other

#8 Post by TD-er » 01 Oct 2022, 15:54

Ah OK, that does make sense :)

joost d
Normal user
Posts: 53
Joined: 24 Dec 2020, 17:55
Location: Netherlands/ Haarlem

Re: ESPeasy cannot always handle HTML requests provided by other

#9 Post by joost d » 04 Oct 2022, 09:03

I took the tip from TD-er and installed MQTT on the raspberryPi.
For the communication from ESPeasy to Domoticz I now use MQTT and
for the communication from Domoticz to ESPeasy, http.
Now there are no error messages anymore :-)

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: ESPeasy cannot always handle HTML requests provided by other

#10 Post by TD-er » 04 Oct 2022, 09:12

Great!

Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 18 guests