Bug 91503 - Spurious values when doing a simple subtraction (1 - 0.95)
Summary: Spurious values when doing a simple subtraction (1 - 0.95)
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: x86-64 (AMD64) Linux (All)
: medium major
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-05-22 23:09 UTC by Diniz Bortolotto
Modified: 2021-12-03 13:17 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments
PrintScreen of Calc with spurious numbers (12.62 KB, image/png)
2015-05-22 23:09 UTC, Diniz Bortolotto
Details
A spreadsheet with the error (9.53 KB, application/vnd.oasis.opendocument.spreadsheet)
2015-05-22 23:21 UTC, Diniz Bortolotto
Details
a spreadsheet with rounding error (9.67 KB, application/vnd.oasis.opendocument.spreadsheet)
2015-05-23 18:01 UTC, Diniz Bortolotto
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Diniz Bortolotto 2015-05-22 23:09:30 UTC
Created attachment 115864 [details]
PrintScreen of Calc with spurious numbers

I'm doing some calculation based in value status and when subtracting via IF-THEN function I received spurious numbers. See below:

                Column D        	Column E		
Line 6	0,96000000000000000000	0,01000000000000000000
Line 7	0,96000000000000000000	0,00000000000000000000
Line 8	0,95000000000000000000	0,04999999999999990000	

Function in Column E is:

        In pt-BR (my lang) or in en-US (I suppose)
Line 7	SE(E6;E6-0,01;1-D7)	 IF(E6;E6-0.01;1-D7)
Line 8	SE(E7;E7-0,01;1-D8)	 IF(E7;E7-0.01;1-D8)

PS: same data in a PrintScreen attachment.
Comment 1 Diniz Bortolotto 2015-05-22 23:21:33 UTC
Created attachment 115865 [details]
A spreadsheet with the error
Comment 2 raal 2015-05-23 07:17:05 UTC
Hello,
where is the problem?

IF(E6;E6-0.01;1-D7) -> result 0  [0,01000000000000000 - ),01]
IF(E7;E7-0.01;1-D8) -> result 0,04999999999999990000  (0,05 rounded) [1  - 0,95000000000000000]

You expect 0,05, not  0,04999999999999990000   in cell E8?

Version: 5.1.0.0.alpha1+
Build ID: e929194317a7debb1c5467282230cbbabe61a710
TinderBox: Linux-rpm_deb-x86_64@46-TDF, Branch:master, Time: 2015-05-22_06:33:18
Comment 3 Diniz Bortolotto 2015-05-23 14:46:21 UTC
Hi Raal!

Yes. I was expecting 0,05 (not 0,0499...) in cell E8.

After some others 0,01 subtractions I expect cell X8 turns 0,00 and then next cell doing the "1 - X8" portion of the IF-THEN command.

Do you know why Calc is creating that difference in a so simple subtraction like that (1 - 0,95)?
Comment 4 Diniz Bortolotto 2015-05-23 15:03:26 UTC
I forgot to say that there are some hidden rows in the attached table. These rows continues the calculus and if you show the hidden rows you will see between rows 12 and 14 that "0.00" never comes.

0,95000000000000000000	 0,00999999999999993000
0,95000000000000000000	-0,00000000000000007286
0,95000000000000000000	-0,01000000000000010000
Comment 5 Diniz Bortolotto 2015-05-23 15:10:19 UTC
Now I noticed that the problem occurs also in simple subtractions.
Not only in IF-THEN function.

See:
0,06000000000000000000	<- 1 – 0,94
0,04999999999999990000	<- 1 – 0,95
0,04000000000000000000	<- 1 – 0,96
Comment 6 raal 2015-05-23 17:45:02 UTC
Hi Diniz,
are you able to reproduce this in new spreadsheet? I tried new file and in this file is result 0,05
Comment 7 Diniz Bortolotto 2015-05-23 18:01:49 UTC
Created attachment 115909 [details]
a spreadsheet with rounding error
Comment 8 Diniz Bortolotto 2015-05-23 18:04:08 UTC
(In reply to raal from comment #6)
> Hi Diniz,
> are you able to reproduce this in new spreadsheet? I tried new file and in
> this file is result 0,05

I attached another spreadsheet.
Please try to increase your decimal appresentation.

with 15 decimals	0,050000000000000		
with 16 decimals	0,0499999999999999	    ← The error is here!!
with 17 decimals	0,04999999999999990	    When number is represented
                                                    with 16 or more decimals.
Comment 9 raal 2015-05-23 19:06:26 UTC
Reproducible with Version: 5.1.0.0.alpha1+
Build ID: e929194317a7debb1c5467282230cbbabe61a710
TinderBox: Linux-rpm_deb-x86_64@46-TDF, Branch:master, Time: 2015-05-22_06:33:18
Comment 10 QA Administrators 2016-09-20 09:41:29 UTC Comment hidden (obsolete)
Comment 11 Diniz Bortolotto 2016-09-20 20:54:51 UTC
(In reply to QA Administrators from comment #10)
> Test to see if the bug is still present on a currently supported version of
> LibreOffice 
> (5.1.5 or 5.2.1  https://www.libreoffice.org/download/
> 
> If the bug is present, please leave a comment that includes the version of
> LibreOffice and 
> your operating system, and any changes you see in the bug behavior
>  

The bug still remains active in LibreOffice Calc.
Version: 5.1.5.2
Build ID: 7a864d8825610a8c07cfc3bc01dd4fce6a9447e5
CPU Threads: 4; OS Version: Linux 4.4; UI Render: default; 
Locale: pt-BR (pt_BR.utf8); Calc: group

The bug behavior remains the same:
		1 – 0,95 ↓		
15 decimals	0,050000000000000		
16 decimals	0,0499999999999999		← The error occurs here!!
17 decimals	0,04999999999999990		      With 16 decimals
Comment 12 Diniz Bortolotto 2016-09-20 21:51:56 UTC
(In reply to QA Administrators from comment #10)
> 
> If you want to do more to help you can test to see if your issue is a
> REGRESSION. To do so:
> 1. Download and install oldest version of LibreOffice (usually 3.3 unless
> your bug pertains to a feature added after 3.3)
> 
> http://downloadarchive.documentfoundation.org/libreoffice/old/
> 
> 2. Test your bug
> 3. Leave a comment with your results.
> 4a. If the bug was present with 3.3 - set version to "inherited from OOo";
> 4b. If the bug was not present in 3.3 - add "regression" to keyword
> 

Same result with the old LibreOffice Calc.
So, I'm setting version to "inherited from OOo".

LibreOffice 3.3.2 
OOO330m19 (Build:202)
tag libreoffice-3.3.2.2

=1-0,95			decimals
0,050000000000000	15
0,0499999999999999	16
0,04999999999999990	17
Comment 13 Xisco Faulí 2017-09-29 08:50:56 UTC Comment hidden (obsolete)
Comment 14 QA Administrators 2019-12-03 14:38:57 UTC Comment hidden (obsolete)
Comment 15 QA Administrators 2021-12-03 04:37:50 UTC Comment hidden (obsolete)
Comment 16 Diniz Bortolotto 2021-12-03 13:17:28 UTC
Downloaded and opened the attached spreadsheets.
To confirm that bug is solved we need to hard recalculate formulas.

Choose Data - Calculate - Recalculate Unconditional
or
Press Shift+Ctrl+F9 to recalculate all formulas in the document.

Version: 7.2.3.2 / LibreOffice Community
Build ID: 20(Build:2)
CPU threads: 4; OS: Linux 5.10; UI render: default; VCL: kf5 (cairo+xcb)
Locale: pt-BR (pt_BR.UTF-8); UI: pt-BR
SlackBuild for 7.2.3 by Eric Hameleers
Calc: threaded