Bug 130202 - Floating-point number round-trip lossage
Summary: Floating-point number round-trip lossage
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
6.3.4.2 release
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-01-26 13:29 UTC by phma
Modified: 2022-11-21 08:14 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
sheet showing 'values' are there, and 'precision gap', (18.65 KB, application/vnd.oasis.opendocument.spreadsheet)
2020-11-01 13:14 UTC, b.
Details
sheet showing calculation is possible, and 'precision gap' (19.04 KB, application/vnd.oasis.opendocument.spreadsheet)
2020-11-01 13:27 UTC, b.
Details

Note You need to log in before you can comment on or make changes to this bug.
Description phma 2020-01-26 13:29:09 UTC
Enter the following numbers and formula:
A1: 31415926535897932384
A2: 31415926535897900000
A3: =A1-A2

Expected number in A3: 32768 (not 32384 because of floating-point roundoff).
Actual number in A3: 0.

Both cells display 3.14159265358979E+019 when edited. If you try to subtract 31415926535897932384 from 31415926535897967616, the latter number is turned into 3.1415926535898E+019, but the difference is still 0. The ulp of those numbers is 4096; it should be possible to enter two numbers that size whose difference is 4096.

When a floating-point binary number is converted to decimal for later conversion back to floating-point binary, it should be converted with enough digits that conversion to floating-point binary results in the same number. In this case, the number should be displayed in the edit box as 3.1415926535897932E+019.
Comment 1 Oliver Brinzing 2020-01-26 14:27:33 UTC
(In reply to phma from comment #0)
 
> Expected number in A3: 32768 (not 32384 because of floating-point roundoff).
> Actual number in A3: 0.

maybe related to:

Bug 96918 - Rounding display error for 15 digits integers

https://ask.libreoffice.org/en/question/156399/decimal-precision-how-to-have-18-decimals/
Comment 2 Eike Rathke 2020-02-18 21:46:21 UTC
The minus operator does an approxSub() within the order of magnitudes of the two operands (i.e. ties/pulls to zero), as users expect 0.3-0.2-0.1 to be 0.0

If you use =RAWSUBTRACT(A1;A2) the result is 24576, which is just the "raw" floating point subtraction of two values.

Adhoc I'm not sure if and what could be done about the input and display of such large values, as the largest unambiguous integer value representable as IEEE 754 double is 9007199254740991 or (2^53)-1.

Not directly related to bug 96918 as that was about displaying the representable integer values accurately.

> When a floating-point binary number is converted to decimal for later
> conversion back to floating-point binary, it should be converted with
> enough digits that conversion to floating-point binary results in the
> same number. In this case, the number should be displayed in the edit
> box as 3.1415926535897932E+019.
That is wishful thinking. An IEEE 754 double can not be more accurate than general 15 decimal digits, sometimes 16 and rarely 17 decimal digits. The problem with unique strings convertible back to double is that they are even less accurate and not what users expect. See http://blog.reverberate.org/2016/02/06/floating-point-demystified-part2.html

You might also be interested in some other links under https://erack.de/bookmarks/D.html#Computer_Arithmetic_and_IEEE_754 and https://erack.de/bookmarks/D.html#010203

If you have a perfect solution I'm not aware of then please tell us.
Comment 3 Buovjaga 2020-05-09 20:36:38 UTC
(In reply to Eike Rathke from comment #2)
> You might also be interested in some other links under
> https://erack.de/bookmarks/D.html#Computer_Arithmetic_and_IEEE_754 and
> https://erack.de/bookmarks/D.html#010203
> 
> If you have a perfect solution I'm not aware of then please tell us.

phma: please let's hear the perfect solution or close this report.
Comment 4 b. 2020-11-01 13:14:44 UTC
Created attachment 166902 [details]
sheet showing 'values' are there, and 'precision gap',

hello @Buovjaga, 

'phma: please let's hear the perfect solution or close this report.'

imho demanding 'the **perfect** solution' is a little overstretching, 

but *better* solutions are! possible, and i support the request of @phma demanding that ... 

see attached sheet, 

setting 'new',
Comment 5 b. 2020-11-01 13:27:17 UTC
Created attachment 166903 [details]
sheet showing calculation is possible, and 'precision gap'

sorry, calc in a way trashes the input by changing the input string and after save-load the value is gone :-(  tried to circumvent by 'value()',
Comment 6 b. 2020-11-07 23:26:13 UTC
just looked at the sheet after re-downloading: 

looks like it needs a recalc for B4 and B5 or general to show the different values, 

different differences with different versions of calc, 6.1 / 7.1, funny ...
Comment 7 QA Administrators 2022-11-08 03:47:40 UTC Comment hidden (obsolete)
Comment 8 phma 2022-11-21 08:09:22 UTC
This bug is still present in version 7.3.7.2 (package in Ubuntu Jammy) and 6.4.7.2 (package in Ubuntu Focal).
Comment 9 phma 2022-11-21 08:10:59 UTC Comment hidden (obsolete)
Comment 10 phma 2022-11-21 08:14:15 UTC Comment hidden (obsolete)