Bug 156858 - LibreCalc wrong subtraction of numbers 0,0000000000001
Summary: LibreCalc wrong subtraction of numbers 0,0000000000001
Status: RESOLVED DUPLICATE of bug 128312
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
7.4.5.1 release
Hardware: x86-64 (AMD64) Windows (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-08-22 12:21 UTC by dudi
Modified: 2023-08-23 21:56 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
Visualisation in 7.5.5.2 (278.20 KB, image/jpeg)
2023-08-23 21:56 UTC, ASi
Details

Note You need to log in before you can comment on or make changes to this bug.
Description dudi 2023-08-22 12:21:07 UTC
Description:
LibreCalc wrong subtraction of numbers 0,0000000000001

A	B	B-A
1730	1753,4	23,4000000000001


Steps to Reproduce:
1. put in A2 1730
2. put in B2 1753,4
3. put in C2 =B2-A2

Actual Results:
C2 result is „23,4000000000001”

Expected Results:
C2 should be 23,4


Reproducible: Always


User Profile Reset: No

Additional Info:
Version: 7.4.5.1 (x64) / LibreOffice Community
Build ID: 9c0871452b3918c1019dde9bfac75448afc4b57f
CPU threads: 8; OS: Windows 10.0 Build 19045; UI render: Skia/Raster; VCL: win
Locale: pl-PL (pl_PL); UI: pl-PL
Calc: threaded
Comment 1 Buovjaga 2023-08-22 13:29:13 UTC
There has been work on this recently. Can you see this in a master build, Win-x86_64@tb77-TDF https://dev-builds.libreoffice.org/daily/master/current.html ?

Set to NEEDINFO.
Change back to UNCONFIRMED, if the problem persists. Change to RESOLVED WORKSFORME, if the problem went away.
Comment 2 m_a_riosv 2023-08-22 15:20:53 UTC
It works fine with
Version: 7.5.7.0.0+ (X86_64) / LibreOffice Community
Build ID: f847a74a2ceffe3938736cb5ee88f81b2160a6b7
CPU threads: 16; OS: Windows 10.0 Build 22621; UI render: Skia/Raster; VCL: win
Locale: es-ES (es_ES); UI: en-US Calc: CL threaded
Comment 3 ady 2023-08-22 23:29:54 UTC
(In reply to m.a.riosv from comment #2)
> It works fine with
> Version: 7.5.7.0.0+ (X86_64) / LibreOffice Community

Are you sure? Have you tried expanding the width of the column?

My guess is that this behavior is related to floating-point math capabilities, but I'll leave it to others with relevant knowledge of the matter to answer (with a proper reply, rather than a guess).


Whether this particular subtraction is supposed to be exact, or with some inaccuracy because of floating-point math, IDK.

https://wiki.documentfoundation.org/Faq/Calc/Accuracy
Comment 4 m_a_riosv 2023-08-23 16:37:11 UTC
In Number-General format, it shows 23,4
changing to a format like 00,0000000000000 indeed it shows 23,4000000000001.
But I think the one it out of the precision.

https://wiki.documentfoundation.org/Faq/Calc/Accuracy
https://ask.libreoffice.org/t/accuracy-in-libreoffice-calc/83750/2
https://ask.libreoffice.org/t/calc-rounding-general-precision-problem/64358/2

Seems we can consider this one like a duplicate of 128312, please if you are not agree reopen it.

*** This bug has been marked as a duplicate of bug 128312 ***
Comment 5 ady 2023-08-23 17:42:56 UTC
(In reply to m.a.riosv from comment #4)
> In Number-General format, it shows 23,4
> changing to a format like 00,0000000000000 indeed it shows 23,4000000000001.

FWIW, testing with LO Dev 24.2 built 2023-08-22, there is no need to change the cell format. Using the default standard cell format with default column width, it shows 23.4 (or 23,4). By double-clicking the column right-side border in order to auto-expand the column's width, it shows 23.4000000000001 (or 23,4000000000001).

Anyway, I am just reporting the behavior I see, just in case. I still tend to think this is about floating-point math, but, as I said in a prior comment, I am guessing.
Comment 6 ASi 2023-08-23 21:56:22 UTC
Created attachment 189119 [details]
Visualisation in 7.5.5.2

same in 7.5.5.2