P2P reliability

[AndyZZ]

P2P network. All ESP_Easy_mega_20240414_collection_F_ESP8266_4M1M
Unit2 is Down: WiFi, Power, Etc
Unit1 lost another Unit2, but still continues to store the last received values

img.JPG (72.27 KiB) Viewed 556 times

Why?
How can I update or reset these old irrelevant cached values (to NaN, Error, Etc) until the connection is restored and the current ones are obtained?

img2.JPG (95.82 KiB) Viewed 547 times

[TD-er]

Right now there isn't any such setting to set values when not being updated for N amount of time.

However I think you might be able to do so via rules, as the task does send out an event when it does receive an update.

So something like this:

Code: Select all

on info#update do
  TimerSet,3,600 // Task #3, set timer to 10 minutes
endon

on Rules#Timer=3 do
  taskvalueset,3,1,-1
  taskvalueset,3,2,-1
endon

You may have to check the checkbox on the tools->Advanced page to allow taskvalueset on tasks other than a Dummy task.

[jgrad]

please check if using taskvalueset within rules to override value of P2P sensor works. On tasks using "non dummy devices" you get error NOT_A_DUMMY_TASK

I was using taskvalueset within rules to set ERROR value in case that temperature sensor Dallas was disconnected and this was not reflected in setting ERROR value. Since taskvalueset is not working anymore I created viewtopic.php?t=8781 where I am waiting for resolution.
Basically this (not being able to set value on task with DS18b20 sensor) blocks my application to be upgraded to newer version of ESPEASY - I am still on 20201227 where taskvalueset on sensor task is working.

In viewtopic.php?t=10144 @ATH implemented simple solution how to detect (within rules) that one node is not responding anymore. Maybe it helps you to.

[TD-er]

@jgrad:

That's why I suggested to check the checkbox (tools->Advanced page) for "Allow TaskValueSet on all plugins:"

This should allow to set taskvalues on non-Dummy tasks.

Doesn't that work for you?

[jgrad]

@td-er

sorry - I havent noticed this suggestion in advanced settings. Seems it shall solve my problem. I will perform a test and report.

[AndyZZ]

Thanks guys for participating in this issue
It seems that those who use p2p a lot care little about the reliability of the parameters used.
This is really a problem that it is not possible to immediately configure via Error Value of a dummy remote device.
I don’t understand why the obvious functionality wasn’t implemented along with p2p.
But we are not looking for easy ways

[Ath]

It seems that those who use p2p a lot care little about the reliability of the parameters used.

Well, we seem to differ in opinion on that then, but that's no problem.

Using the P2P connected sensors without issues for a long time, and actively developing on ESPEasy, so if there would have been an issue, I'd have taken action on that.

Maybe you can share a little more on how your network is configured? (many APs?, rural area?, long WiFi distances?, poor signal quality? many other WiFi networks in close proximity?)

[AndyZZ]

Yes, this is poor quality and reliability of wifi, a lot of interference, lost power, etc... and this is very far from the reliability of wired access.
The p2p network is autonomous, and the central one collects data from all neighbor sensors, and periodically sends everything to monitoring through a periodic http get call Rule.
If one of the neighbors has completely disconnected from p2p or its sensor, the last received values that do not correspond to reality are still transmitted to the monitoring.
Sending some emergency value to monitoring instead of the last received one will indicate that the parameter is outside the permissible limits and an emergency: the data is not processed.

[TD-er]

My example to use it via rules is meant to get it working _now_ as other fixes mean adding code, making a build etc.
So by creating a working setup via rules, we also get an idea of what really is a practical use case and then decide what to do to make a universal 'fix' for this.

We do have the idea planned to add some extra settings on what to show when the sensor is in error or not connected.
This has been partly implemented for the 1Wire sensor. But I really don't like how it is done there as this means implementing it over-and-over again for every other plugin.
Thus I would like a more universal approach, which will work for any plugin. (any yielding a value, so display doesn't need it)

One approach could be to also add an "error" state to tasks which receive their data via p2p.
But then there is the question where the "error value" should be set? Is this a parameter on the sending side?
If so then it also requires changes in the p2p protocol.

Etc.

So I guess you now see why we're not immediately jumping on this to add this feature (nobody asked for in years) but first try to get an idea of what would be the actual problem or use case we need to look into.

When adding the "error value" option to all plugins, I would also like to add some indicator of some interpretation of what would be a "good" value and what isn't.
e.g. "good" air quality.
And maybe also some reliability stats (read errors, etc.)

[AndyZZ]

The receiving side, in any case, already knows how to detect a disconnection from the p2p transmitting side, and when this happens, the data from such dummy sensors is erroneous and should not be used: the source is faulty.
This may only affect dummy devices, but not all plugins.

Rules can check the validity of data from any sensors. On whose side to do this, provided p2p is available, it does not matter.

[TD-er]

Anyway, since the impact of changes needed to do it properly are quite big -as I explained-, I would like to know if the proposed rules do make it workable for your use case.
Then we can take this into account when adding the "error state values" and "error handling" and for sure also need to take p2p received values along.

For now, I really need to focus on the WiFi issues first, so it will take some time before I get to this.

[AndyZZ]

Is it possible to programmatically reduce the problems of low quality wifi channels?
This primarily affects the loading of the single esp82xx core in ESPEssy due to looping of communication attempts.

[TD-er]

One thing you might want to try is to force "b/g mode" (see tools->Advanced page)
And of course not using "ECO" mode.

[AndyZZ]

I already did this, sometimes the CPU jumps to 100% and big LC
But perhaps this is also due to four plugins sitting on one 1-wire DS18B20. However, this scheme worked fine in the sketch.
Perhaps the problem is a large CPU in two working simultaneously looptimerset:

On WiFi#Connected do

timerSet,1,10 //Set Timer 1 for the next event in 10 seconds

endon

On Rules#Timer=1 do //When Timer1 expires, do

looptimerset,2,60 //60s timer

looptimerset,3,320 //320s timer

endon


On Rules#Timer=2 do

SendToHTTP 1...

endon

On Rules#Timer=3 do

SendToHTTP 2...

endon

[TD-er]

If WiFi connection is bad, then lots of network connections may timeout.
Timeouts are 'blocking' like they keep the loop() call occupied.
ESPEasy p2p is UDP, so that should not really be blocking, but sendToHttp might be and if you have a MQTT controller configured then that is notorious for slowdowns when the WiFi connection is flaky.

[Ath]

And having "SendToHTTP wait for ack" enabled will cause similar slowdowns with poor WiFi connections.

[AndyZZ]

Right now there isn't any such setting to set values when not being updated for N amount of time.

However I think you might be able to do so via rules, as the task does send out an event when it does receive an update.

So something like this:

Code: Select all

on info#update do
  TimerSet,3,600 // Task #3, set timer to 10 minutes
endon

on Rules#Timer=3 do
  taskvalueset,3,1,-1
  taskvalueset,3,2,-1
endon

You may have to check the checkbox on the tools->Advanced page to allow taskvalueset on tasks other than a Dummy task.

What is info#update ?

img3.JPG (56.61 KiB) Viewed 345 times

[Ath]

What is info#update ?

A small typo, should be "on info#uptime do", the rest of the script is ok.

[AndyZZ]

This solution works stably, you're right.
But how can I get NaN instead of -1 (degrees) the same way it can be done into remote temperature sensors?
on Rules#Timer=3 do
taskvalueset,3,1,-1

img4.JPG (17.31 KiB) Viewed 289 times

[TD-er]

Just to be sure, you need to end the block with "endon", like this:

Code: Select all

on Rules#Timer=3 do
  taskvalueset,3,1,-1
endon

Not sure if the rules parser can understand "NaN"
Also parsing "NaN" in other platforms is a bit tricky as the only way to check for "NaN" is like this: (C++)

Code: Select all

bool isNaN(float value) {
  return value != value;
}

So not sure how well "NaN" values are supported in other platforms as "value == NaN" probably will not work unless that specific platform added support for it.
Anything you compare with a NaN will return false. (at least in C++)


By the way I came up with the following idea to add to ESPEasy as it can help in your use case and doesn't yet require a lot of code changes;
The idea is to fire an event when a node is removed from the node list as it is no longer seen.
Something like "node#disconnected=123" with "123" being the node index.
Also add an event like "node#connected=123" when it is first seen (or seen after it was removed from the node list)

And for tasks which receive values from a p2p node:
"taskname#remotelost" or something like that when the node supplying the remote feed will be removed from the node list.

[AndyZZ]

This idea is very very great.
It will cover most p2p control issues.
This will not have to be solved with significant workarounds: the basic control of node availability has already been implemented, but there are still not enough official events.

In addition, there is a lack of choice regarding the behavior of sensor values received from the remote node. This is configured in the sensor plugin (can as NaN), and should correspond to the dummy sensor appearing via p2p.

[TD-er]

I can look into the rules parser accepting NaN, but I'm a bit hesistant to mess with that piece of code as it is now working (very) well and quite fast.
And this parsing code is used everywhere in the code.

Sending task configuration to receiving nodes does require changes in the p2p code and is plugin dependent so I rather save this for a later change as it really bothers me that this would be plugin specific and I want to think of a way to make it more generic.

[AndyZZ]

You're right, we need a universal approach to any plug-in sensors available remote via p2p.

[TD-er]

I did add the p2pnode#Connected and p2pnode#Disconnected events in this build: https://td-er.nl/ESPEasy/latest/

Code: Select all

on p2pNode#Connected do
  LogEntry,'ESPEasy p2p node %eventvalue1% added: %eventvalue2% with build %eventvalue3%'
endon

on p2pNode#Disconnected do
  LogEntry,`ESPEasy p2p node %eventvalue1% not seen for a while`
endon