Bug 87506 - Inconsistency in how LibreOffice Calc 4.4.0BetaDevDaily handles real numbers internally or with the MOD() function or both.
Summary: Inconsistency in how LibreOffice Calc 4.4.0BetaDevDaily handles real numbers ...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.4.0.0.beta1
Hardware: x86 (IA32) Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-12-19 19:10 UTC by klsu
Modified: 2021-04-10 16:54 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description klsu 2014-12-19 19:10:31 UTC
Found this responding to comment 3 on bug 87386. Where C13 contains the number 24.09, the comment says "MOD(C13,1)*100 = 8.999999999999999". In the daily build of LibreOffice4.4Dev(en) downloaded on Dec 18, 2014, if C13 contains the number (not text) 24.09 then =MOD(C13,1)*100 does return 8.99999999999999 but displays it as 9 (regardless of cell width) and treats it as 9:

in cell A1 enter the number 24.09
in cell B1 enter =MOD(A1,1)*100 (9 is displayed)
in cell B2 enter =MOD(A1,1) (0.9 is displayed)
highlight and copy range B1:B2
move to cell C1
paste special text/numbers/date & time
in cell E1 enter the number 9
in cell E2 enter the number 0.09
in cell D1 enter =C1=$B1
copy cell D1 and paste it to cell D2 and to range F1:F2
in cell G1 enter =C1=E1
copy cell G1 and paste it to cell G2

Every cell in range F1:G2 displays TRUE, meaning that to LibreOffice Calc, 9 = 8.99999999999999 and 0.09 = 0.0899999999999999. This is not true, except for most practical purposes. In Excel, all the cells in range F1:G2 will show FALSE, which most people would expect. I do not know if this is causing bug 87386.

It is as if Excel and LibreOffice are displaying the same number of decimal places in calculation results, but Excel is calculating to a larger number of decimal places and adjusting what it displays and uses in subsequent calculations to prevent problems due to the inherent inaccuracy of calculating real numbers digitally.

I would be surprised if this problem depends on the OS version of LibreOffice used.
Comment 1 m_a_riosv 2014-12-20 00:44:40 UTC
Please take a look to this bug report just about MOD() function:

https://bugs.freedesktop.org/show_bug.cgi?id=50299

Specially Kohei comment #4.

Resolved as duplicate, please if you are not agree reopen it.

*** This bug has been marked as a duplicate of bug 50299 ***
Comment 2 klsu 2014-12-21 16:20:45 UTC
I agree that this appears to be the same as bug 87506, which is marked resolved; however, if you read that bug you will find that it is not resolved. As I understand that discussion, their resolution is, it's a hardware problem that can only be fixed in software, which isn't worth doing because it's slow (but then why does Excel, which is fast, not have the problem on the same hardware).

What I find particularly interesting is this:
in cell A6 enter 0.3
in cell B6 enter =MOD(A6*100, 10)
cell B6 displays 3.5527136788005E-015
in cell A6 enter =3/10
cell B6 displays 0

So letting the hardware create the 0.3 instead of using LO Calc's stored 0.3 gives the correct answer. That doesn't prove it's not a hardware bug, but it proves it is or is also a software bug, and I have confirmed that it exists in LO 4.4.0 Beta Dev Daily I installed 2014-12-18 AND in Excel 2003, but not as bad. More on that when I reopen 87506 with a spreadsheet example showing just how bad the LO Calc MOD() function can be.

Same result using 0.6 vs 6/10

"We can't fix it or don't think it's worth fixing, so mention it in the documentation and it's resolved" does not resolve the problem. Explaining it to the average person wouldn't help them, if they ever happen to read it (have you read and understood and remembered all the documentation of every piece of software you use?). They would understand "if you use the MOD() function, the answer might be right and it might not be, so you'd better check every use of the MOD() function manually", which means "don't use MOD(), it's not reliable".

The correct resolution would be to remove the MOD() function from LO Calc until it has been fixed.
Comment 3 b. 2021-04-10 16:54:26 UTC
@klsu: if you select cells C1 or C2 and look at the input line you will see a deviation, 
you may also get it visible with: '=RAWSUBTRACT(B2;E2)', 
calc does some 'tolerant compare' to cover such small imprecisions, but keeps the odd value and uses it for downstream calculations, one can argue the hour and the day whether well done or not, 
the source of evil is 'cancellation', and that no spreadsheet has good handling for that so far ... :-(