Bug 116853 - Simple Subtraction Error Precision
Summary: Simple Subtraction Error Precision
Status: RESOLVED NOTABUG
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
5.4.6.2 release
Hardware: x86-64 (AMD64) Windows (All)
: medium minor
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-04-06 21:22 UTC by Tiago Seibt
Modified: 2019-08-14 08:50 UTC (History)
1 user (show)

See Also:
Crash report or crash signature:

Attachments
Precision error in simple subtraction (24.25 KB, application/vnd.oasis.opendocument.spreadsheet)
2018-04-06 21:30 UTC, Tiago Seibt 		Details
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Tiago Seibt 2018-04-06 21:22:58 UTC 
Description:
If subtracting 119,26 of 119,27 results in 0,009999999999991
where the correct would be 0,01

The error happen on numbers from 119,27 to 123,88



Actual Results:  
Type 119,27 in cell A1, type 119,26 in cell A2 and type in A3 =A1-A2

Expected Results:
0,01


Reproducible: Always


User Profile Reset: No



Additional Info:
Versão: 5.4.6.2 (x64)
ID de compilação: 4014ce260a04f1026ba855d3b8d91541c224eab8
Threads da CPU:8; SO:Windows 6.19; Realizador da interface: padrão; 
Local: pt-BR (pt_BR); Calc: group


User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36
Comment 1 Tiago Seibt 2018-04-06 21:30:14 UTC 
Created attachment 141179 [details]
Precision error in simple subtraction
Comment 2 MM 2018-04-06 22:46:07 UTC 
This is a normal (processor) round off error. If you set tools / options > LO calc > calculate > 'limit decimals [.....]' to 2, you'll see the correct result.
Comment 3 Xisco Faulí 2018-04-07 13:12:48 UTC 
(In reply to MM from comment #2)
> This is a normal (processor) round off error. If you set tools / options >
> LO calc > calculate > 'limit decimals [.....]' to 2, you'll see the correct
> result.

Setting as RESOLVED NOTABUG