Bug 142134

Summary:	Basic math calculation errors
Product:	LibreOffice	Reporter:	Emily Tilden <emily>
Component:	Calc	Assignee:	Not Assigned <libreoffice-bugs>
Status:	RESOLVED NOTABUG
Severity:	normal	CC:	aron.budea, emily
Priority:	medium
Version:	7.1.2.2 release
Hardware:	x86-64 (AMD64)
OS:	Windows (All)
Whiteboard:
Crash report or crash signature:		Regression By:

Description Emily Tilden 2021-05-06 17:50:15 UTC

Description:
I have a cell that contains "=10.2-9.3" and the value that is returned is 0.899999999999999 instead of 0.9 exactly.

I have another cell that contains "=(16+16+10.2)-(16+16+9.3)", and it returns 0.900000000000006 instead of 0.9 exactly.



Steps to Reproduce:
1. Create a cell that contains "=10.2-9.3".
2. Create a cell that contains "=(16+16+10.2)-(16+16+9.3)".
3.

Actual Results:
Happens in new empty worksheets

Expected Results:
I should be getting exactly 0.9 or 0.900000000


Reproducible: Always


User Profile Reset: No



Additional Info:
Version: 7.1.2.2 (x64) / LibreOffice Community
Build ID: 8a45595d069ef5570103caea1b71cc9d82b2aae4
CPU threads: 6; OS: Windows 10.0 Build 19042; UI render: Skia/Vulkan; VCL: win
Locale: en-US (en_US); UI: en-US
Calc: CL

Comment 1 Aron Budea 2021-05-09 06:00:23 UTC

This is due to floating point math, and not a bug, for details please see:
https://wiki.documentfoundation.org/Faq/Calc/Accuracy

It is possible to either use rounding functions, or simply set display format to the desired number of decimals.