Bug 109189 - calc: calculation: statusbar, auto-sum irrit. results, different from sheet reg. rounding and ordering of operands | was: Statusbar sum and average calculation with decimal values equivalent to zero sometimes shows incorrect negative exponential value
Summary: calc: calculation: statusbar, auto-sum irrit. results, different from sheet r...
Status: RESOLVED DUPLICATE of bug 68448
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: needsDevEval
Depends on:
Blocks: Statusbar
  Show dependency treegraph
 
Reported: 2017-07-18 11:39 UTC by C.Drewke
Modified: 2021-04-13 15:30 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
sample (10.31 KB, application/vnd.oasis.opendocument.spreadsheet)
2017-07-24 20:00 UTC, Yousuf Philips (jay) (retired)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description C.Drewke 2017-07-18 11:39:32 UTC
Description:
When you select multiple numbers of which at least 6 in a column are negative and at least 2 of those have decimal values the calculated sum has a rounding error.

This bug is also present in a BASE database with values that are calculated with the SUM function.

Steps to Reproduce:
1. Enter the following values into a fresh CALC table (below each other):
144,1
-16,4
-67,3
-21,5
-11,7
-27,2

2. mark all values

3. observe the rounding error for the sum in the bottom right corner

Actual Results:  
Sum: -7,105427357601E-15

Expected Results:
Sum: 0


Reproducible: Always

User Profile Reset: No

Additional Info:
[Information automatically included from LibreOffice]
Locale: de
Module: SpreadsheetDocument
[Information guessed from browser]
OS: Linux (All)
OS is 64bit: yes
Builds ID: LibreOffice 5.3.1.2


User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0
Comment 1 Xavier Van Wijmeersch 2017-07-19 09:22:51 UTC
I can't reproduce

Version: 5.3.4.2
Build ID: SlackBuild for 5.3.4 by Eric Hameleers
CPU Threads: 8; OS Version: Linux 4.9; UI Render: default; VCL: kde4; Layout Engine: new; 
Locale: nl-BE (en_US.UTF-8); Calc: group

and

Version: 6.0.0.0.alpha0+
Build ID: 8c82e4877181f55f2fe186b341d504e5782a4f9c
CPU threads: 8; OS: Linux 4.9; UI render: default; VCL: kde4; 
Locale: nl-BE (en_US.UTF-8); Calc: group
Comment 2 Jacques Guilleron 2017-07-24 13:50:37 UTC
Hi C.Drewke,

This sum function in Status bar follows the cells format. By default, there is no value for decimal places with ths Standard format, you get so the greatest precision and no rounding.
If I enter =SUM(A1:A6) into A7, I get here 0, a rounded value, with the Standard format.

To make some differences, Excel give also a rounded value 0, when A1:A6 is selected and -7,10543E-15 precision only when Sum in A7 is selected too, still with the Standard format.
With any decimal place number, Excel give always a simple 0 for A1:A6 selection and follows the cell format when A7 is selected and formatted.

Do you expect the same behavior?
Comment 3 C.Drewke 2017-07-24 14:07:24 UTC
I am not sure I fully understand what you mean - 

When I use "=SUM(A1:A6)" in A7 it also calculates "0" for me.

When I format the cells to use only 2 decimals it shows 0,00


BUT:
When I remove the negative ("-") from the values I get a precise 288,2 when I select cells A1:A6 without formatting them previously

ALSO:
When I import these values from a database (mysql via a LibreOffice Base ODB in my case) with a SUM calculation in the SQL query it still outputs "-7,105427357601E-15". The fields that are used have the double(15,2) format.

As "-7,105427357601E-15" is also displayed when running that query from the ODB file I think this bug is not in based in Calc ...
Comment 4 Yousuf Philips (jay) (retired) 2017-07-24 20:00:05 UTC
Created attachment 134826 [details]
sample
Comment 5 Yousuf Philips (jay) (retired) 2017-07-24 20:10:40 UTC
Steps:
1. Open attachment 134826 [details]
2. Select the values in either columns B, D, or H
3. Look in the statusbar section that normally shows 'Average: ; Sum: 0' and notice that it has a negative value rather than a 0.

Version: 6.0.0.0.alpha0+
Build ID: 888f3f18cf42ca08f55a5e4826b4298f391e14aa
CPU threads: 2; OS: Linux 4.4; UI render: default; VCL: gtk2; 
Locale: en-US (en_US.UTF-8); Calc: group
Comment 6 QA Administrators 2018-07-25 02:39:07 UTC Comment hidden (obsolete)
Comment 7 Oliver Brinzing 2019-08-16 13:17:24 UTC
reproducible with:

Version: 6.3.0.4 (x64)
Build ID: 057fc023c990d676a43019934386b85b21a9ee99
CPU threads: 4; OS: Windows 10.0; UI render: default; VCL: win; 
Locale: de-DE (de_DE); UI-Language: en-US
Calc:
Comment 8 b. 2021-03-12 19:08:14 UTC
@all: careful, long, but has info with solution potential! 

@jay: nice job in narrowing down, you had the fish hooked, but i think it was too diverse an interaction to understand right away? 

imho there are five effects mixed here: 

1. calc doe's some 'rough' rounding, especially near zero results, but not or less strict for the statusbar, 

2. bin representations of most fractional decimal values are not exact but contain small deviations, 

3. these deviations may add up in calculations, 

4. especially when subtracting values of similar magnitude deviations are boosted (powered) in their relative! effect on the result, and thus sometimes become visible, 

5. above may give different results for computing the same operands in different order, 

6. calc uses different calculation engines for the sheet and for the status bar resulting in different calculation orders, and thus - sometimes - different results, ... why? ... i don't know ... 
- that for the sheet starts bottom right and calculates rightmost column bottom up, then continues with nextleft column, 
- that for the statusbar starts leftmost column top -> down, then continues with nextright column, 
- ex$el deviates and starts top row left to right, continues with rows top down, 
  (i just filed a bug for that as it undermines calc - ex$el compatibility, tdf#140691) 
- ex$el statusbar / autosum - to be honest - i haven' yet deciphered, it's sometimes by row left right top down, sometimes by column top down left right, and looks depending on the field marked, only the cells to be calculated or empty cells around included and whatever more ... might be also where you started marking, might be depending if marked by mousedrag or keyboard ... not yet understood, 

^^^^above: info for this bug - vvvv below: further reflections (important?) 

didn't check how google or similar work, but already within theese three it's deviations enough as fp-math is not in all cases associative (in fact it's only for very few cases), and thus different results come up, 

if you then try to achieve ex$el compatibility by rounding you may quickly come to a status as calc is now, rounding here and there, and nobody knows why it doesn't work ... 

the idiosyncrasy of calc dates back to 3.5.1.2 and may be former ver., i - assume - 'inherited', 

imho an important decision, 

what do we want? achieve full ex$el compatibility, or keep calc backward compatible? 

and the longer i think about i get the feeling - 'assume' (sorry@erAck) - that such might be one of the root causes for multiple calc problems, users complain about deviating from ex$el, devs don't understand why ... 'let's try some more rounding', not good!, 

step one: check if i'm right with that analysis, 

step two: spread that info, 

step three: let the big brains think about and find a solution, 

(my proposal: an option switch and two modi, 'calc - correct math' or 'ex$el - as compatible as possible', then you would have clear targets and can work straightforward ...) 

besides: repro with: 

Version: 7.2.0.0.alpha0+ (x64) / LibreOffice Community
Build ID: 62dff2844b0bf1d1bcb8eb4d6db529ef4a31bee4
CPU threads: 8; OS: Windows 6.1 Service Pack 1 Build 7601; UI render: default; VCL: win
Locale: de-DE (de_DE); UI: en-US
Calc:
Comment 9 b. 2021-04-03 15:36:41 UTC
the effects are neither limited to 

'decimal values equivalent to zero' 

nor to 

'incorrect negative exponential value' 

changing subject,  

proof: 
 1,797693134862315500E+308
-1,797693134862315500E+308
 3,995840309534720000E+291
summed in a sheet results in 
 0,000000000000000000E+000 (cancellation) while the statusbar shows 
 3,995840309534720000E+291 (different ordering of operands, no cancellation), 
putting the small value above the others changes results between sheet and statusbar, 

keying 
-2,9937604643020793E+292 in a cell shows 
-2,9937604643020800E+292 (rough rounding), keying 
 2,9937604643020800E+292 in a cell beneath and summing up results in 
 0,0000000000000000E+000 in the sheet, and 
 8,8633114604817800E+276 in the statusbar
Comment 10 b. 2021-04-13 15:30:47 UTC
IMHO dupe of 68448

*** This bug has been marked as a duplicate of bug 68448 ***