Bug 155646 - Decimal calculations are subject to binary float error
Summary: Decimal calculations are subject to binary float error
Status: RESOLVED NOTABUG
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
unspecified
Hardware: x86-64 (AMD64) Windows (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-06-02 10:27 UTC by Michael Whapples
Modified: 2023-06-02 11:06 UTC (History)
0 users

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Whapples 2023-06-02 10:27:04 UTC
In any cell in LibreOffice enter the following formula =1475.99-1440.32 and observe the output.
Expected: 35.67
Actual: 35.6700000000001
It appears LibreOffice is subject to the float errors found in primitive data types in most programming languages. As a Java developer I am aware of this float issue and how in Java BigDecimal is used for accurate decimal calculations. However in a spreadsheet I would expect accurate decimal calculation by default and this is the case with tools like excel or google sheets.
Even worse is that if I change the cell format to currency I still observe this float error. It is worse as currency typically should be using accurate decimal calculations, I cannot think of a case when currency should use binary float types.
The version of LibreOffice calc I have tested is 7.5.3.2.
Comment 2 Mike Kaganski 2023-06-02 11:06:41 UTC
(In reply to Michael Whapples from comment #0)
> and this is the case with tools like excel or google sheets.

This statement is wrong. Just add enough decimals to the cell format to see the real value there.