Bug 126552 - calc subtraction imprecision
Summary: calc subtraction imprecision
Status: RESOLVED NOTABUG
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
6.1.6.3 release
Hardware: All Windows (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-07-25 18:03 UTC by leiopar
Modified: 2019-08-14 08:50 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
screenshot (7.15 KB, image/png)
2019-07-25 18:05 UTC, leiopar
Details
calc exemple of bug (8.65 KB, application/vnd.oasis.opendocument.spreadsheet)
2019-07-25 18:06 UTC, leiopar
Details

Note You need to log in before you can comment on or make changes to this bug.
Description leiopar 2019-07-25 18:03:37 UTC
Description:
in LibreOffice Calc the operation "=581.9-582.58" say "-0.680000000000064" and not "-0.68"

Steps to Reproduce:
1.in a cell, enter : "=581.9-582.58"
2.
3.

Actual Results:
-0.680000000000064

Expected Results:
-0.68


Reproducible: Always


User Profile Reset: No



Additional Info:
Comment 1 leiopar 2019-07-25 18:05:44 UTC
Created attachment 152992 [details]
screenshot
Comment 2 leiopar 2019-07-25 18:06:19 UTC
Created attachment 152993 [details]
calc exemple of bug
Comment 3 Aron Budea 2019-07-26 00:10:36 UTC
That's how floating point arithmetic works, for details see: https://en.wikipedia.org/wiki/Floating-point_arithmetic#Accuracy_problems

However, when you set the number format to Number / General, you should see a nicely rounded number in the cell.
Comment 4 leiopar 2019-07-26 09:38:32 UTC
no ...

1. open a *NEW* Calc document with default style model of document
2. take a random cell
3. past in "=581.9-582.58"
4. enlarge the column over 4cm
5. see ...
Comment 5 Aron Budea 2019-07-26 11:07:24 UTC
Because that's the number. You can also specify the number of decimals manually.
Comment 6 leiopar 2019-07-26 12:27:40 UTC
Seriously, please !

Explaining a problem is good. But that’s only half of the way ... only half.

A simple arithmetic operation returns a false result. Masking this with formatting is not satisfactory because I want to use the value for my further development, the value, not its display.

And I’m probably not the only one in this situation...

For my project, I’m gonna do a dirty stuff ... I will add a forced ROUND, but please stop pretending that the problem is solved !
Comment 7 Aron Budea 2019-07-26 14:56:58 UTC
There's no issue here, please read the linked wikipedia article on floating point  operations. Thank you.