Question about floating-point numbers

Moderators: grovkillen, Stuntteam, TD-er

Post Reply
Message
Author
User avatar
chromo23
Normal user
Posts: 821
Joined: 10 Sep 2020, 16:02
Location: germany

Question about floating-point numbers

#1 Post by chromo23 » 13 Apr 2022, 10:53

Hi,
just wanted to ask if this is a bug or normal behaviour with floating-point precision?

Code: Select all

taskvalueset,4,1,1023.1220
fp11.png
fp11.png (8.59 KiB) Viewed 4474 times

Code: Select all

taskvalueset,4,1,1024.1220
fp12.png
fp12.png (8.51 KiB) Viewed 4474 times

Code: Select all

taskvalueset,4,1,1023.1070
fp21.png
fp21.png (8.54 KiB) Viewed 4474 times

Code: Select all

taskvalueset,4,1,1024.1070
fp22.png
fp22.png (8.42 KiB) Viewed 4474 times
As you can see the last decimal changes when the number in front of the decimals gets bigger than 1023....
(it does not happen with every decimal number but with some)

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Question about floating-point numbers

#2 Post by TD-er » 13 Apr 2022, 12:30

Yep, that's a floating point issue.
A float does have 23 bits to store the decimals. See: https://en.wikipedia.org/wiki/Single-pr ... :_binary32
The task values use float values.
The variables used in rules, use double values, exactly for this reason.
However the variables will be lost after reboot and the task values will be restored from RTC memory after a reboot. (as long as the unit remains powered)

Edit:
Storing the same decimals, without a dot (thus as large integer) will also fail the same way.
Typical use case which has come across this forum a few times, is to store the unixtime in seconds in a dummy.

User avatar
chromo23
Normal user
Posts: 821
Joined: 10 Sep 2020, 16:02
Location: germany

Re: Question about floating-point numbers

#3 Post by chromo23 » 13 Apr 2022, 12:51

TD-er wrote: 13 Apr 2022, 12:30 Storing the same decimals, without a dot (thus as large integer) will also fail the same way.
Maybe i missunderstand that but:

This seems to work

Code: Select all

taskvalueset,4,1,10231070
fp31.png
fp31.png (8.45 KiB) Viewed 4463 times

Code: Select all

taskvalueset,4,1,10241070
fp32.png
fp32.png (8.44 KiB) Viewed 4463 times

TD-er
Core team member
Posts: 8643
Joined: 01 Sep 2017, 22:13
Location: the Netherlands
Contact:

Re: Question about floating-point numbers

#4 Post by TD-er » 13 Apr 2022, 12:59

10241070 is in binary: 1001 1100 0100 0100 0010 1110

That's exactly 23 bits which are significant and a trailing zero.
So in theory, adding 1 to the value may cause a rounding error.

Edit:
The largest integer you can store without loosing a significant bit is 8388608
8388609 may cause a rounding error.

Such rounding errors may occur sooner on floating point values as some values in base 10 notation have an infinite repeating pattern in base 2.
From memory, 0.2 was such a number.
But in my code to format numerical values I try to prevent strange rounding errors so a user may not see them.

User avatar
chromo23
Normal user
Posts: 821
Joined: 10 Sep 2020, 16:02
Location: germany

Re: Question about floating-point numbers

#5 Post by chromo23 » 13 Apr 2022, 13:07

I understand (i guess ;) ).
Thank you for the explanation!

Post Reply

Who is online

Users browsing this forum: Google [Bot] and 19 guests