Bug 89373 - Strange behaviour of sum() function in linked or complex spreadsheet
Summary: Strange behaviour of sum() function in linked or complex spreadsheet
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.4.1.1 rc
Hardware: Other All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-02-14 08:15 UTC by andis.lazdins
Modified: 2021-04-10 17:18 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
Example of calculation approach (71.98 KB, application/vnd.oasis.opendocument.spreadsheet)
2015-02-14 08:15 UTC, andis.lazdins
Details
Updated version of example with repeatable error (59.46 KB, application/vnd.oasis.opendocument.spreadsheet)
2015-02-15 16:10 UTC, andis.lazdins
Details

Note You need to log in before you can comment on or make changes to this bug.
Description andis.lazdins 2015-02-14 08:15:00 UTC
Created attachment 113375 [details]
Example of calculation approach

I have spreadsheets, where sum() function is used to calculate, when land use conversion or other measures reach maximum. The example is in attachment. Before the 4.4.1.1 (probably, also in 4.4.0.3) I didn't have any problems with this approach. An example of reduction of carbon stock in soil follows in attachment (the problematic formulas are in range A40:BK102).
Today I recognize that sum() function is not working well in all cases. An example of the problem: I got value "1904" in cell range B41:B61, which means that sum(B41:B101) is > B102. Normally according to the formula, like in previous calc versions I would get value "1904" in cell range B41:B60 and sum(B41:B101) would be = B102.
The input values in B3:BJ16 comes from another linked spreadsheet and as soon as I moved the sheet to separate file and replaced linked values with numbers the problem was gone. Correct result can be also obtained, if original formula (like in attachment) is replaced by rounded values:
=IF($A60<B$2;"";IF(SUM(B$41:B59)>=B$102;"";B$6*$B$29*1000) replaced with =IF($A60<B$2;"";IF(ROUND(SUM(B$41:B59))>=ROUND(B$102);"";B$6*$B$29*1000)).

The problem appears only in few formulas. Unfortunately I cannot provide example with non-working formula, because the problem was gone after replacement of linked values with numbers.
Comment 1 m_a_riosv 2015-02-14 13:56:14 UTC
Hi @andis, thanks for reporting.

I think the issue it's that really you need to set up rounded values.

Precision for calculations is limited.

Please take a look to this bug report about MOD() function:

https://bugs.freedesktop.org/show_bug.cgi?id=50299#c4.

Specially Kohei comment #4.

Please see also: https://bugs.documentfoundation.org/show_bug.cgi?id=86174

Resolved as duplicate, please if you are not agree reopen it.

*** This bug has been marked as a duplicate of bug 50299 ***
Comment 2 andis.lazdins 2015-02-14 14:04:14 UTC
Thank you for response!

The problem is actually unexpected behaviour of newer version of the program. Our greenhouse gas inventory is built of libreoffice calc and it's predecessor openoffice.org since 2007 and it's the first time, when I got problems with these formulas (of course, there were other problems). It's a question of comfortable feeling using the program - should I check only my faults or should I check also results of calculations.
Comment 3 m_a_riosv 2015-02-14 17:18:08 UTC
But if with the sample file alone, issue can't be verified, no way to find what the issue begin.

Without a minimal sample files where to see the different behaviour between versions. I think very difficult if there is a way to find it.

Can a be a number or formula result that now gives a periodic decimal result, producing rounding in the last number of the precision, provoking a different result on sum the products than the product of the total, that can explain why with ROUND() function works properly.
Comment 4 klsu 2015-02-15 15:19:00 UTC
Andis' example is an excellent example of a real use of spreadsheets that most programmers don't understand and can't imagine. The SUM() bug is probably related to the same cause as bug 50299 (improper handling of significant digits in floating point calculations); however, if it is caused by something specifically in the implementation of the SUM() function, then its correction in the MOD() (and DATE()) functions will not correct it in the SUM() function.

You are fortunate; you noticed that the answers were incorrect. Since I noticed the related problem with the MOD() and DATE() functions, I have returned to using Excel and am warning people that they could get unpleasant surprises if they use these functions in open source spreadsheets. As I commented on 50299, engineers, doctors and accountants cannot use spreadsheets that give random incorrect answers when used as they would normally be expected to be used.

Does the problem still exist if the links in B3:BJ16 to values calculated in another document are replaced with links to values calculated in another worksheet in the example document? If so, you would be able to provide an example that shows the problem.

I would not mark this as resolved, but as more information required (an example that shows the problem).
Comment 5 andis.lazdins 2015-02-15 16:10:41 UTC
Created attachment 113401 [details]
Updated version of example with repeatable error
Comment 6 andis.lazdins 2015-02-15 16:22:49 UTC
Hello!

It's funny, but when I tried to reconstruct formulas in example file using another worksheet instead of linked file, I was able to repeat the error. Result of the experiments is attached to the bug report.

It is interesting, that some values are causing error ("220", "945") and some values are calculated as they should be ("375"). No problems found in version 4.3.6.2. So there is working example of the problem.

Regarding use of Libreoffice in complex calculations. I would say, that it is much, much better than excel, when there is need to use lot of sumif(), lookup(), hlookup() and other array functions. Statistical functions like ttest() are not limited to equal number of values in compared ranges and so on. I can't imagine returning to Excel now; the worst case would be returning to openoffice.org, which is much better and faster with large data ranges, because CPU use is another madness introduced somewhere with 3.5.x version.

Some time ago there was something wrong with calculations in pivot tables and since that I'm always checking results of calculations. But this is very uncommon that mistakes appear in basic functions.

Best regards
Comment 7 m_a_riosv 2015-02-15 23:25:29 UTC
Hi @andis have you tried with a hard recalc?.

Expanding decimal places at maximum, values in:
B 52:  220,2624382700770
B113: 4405,2487654015700
after a hard recalc:
B 52:  220,2624382700780
B113: 4405,2487654015600

this I think now gives the same results than in 4.3.6.2

Please can you confirm.
Comment 8 andis.lazdins 2015-02-16 05:43:07 UTC
(In reply to m.a.riosv from comment #7)
> Hi @andis have you tried with a hard recalc?.
> 
> Please can you confirm.

What is hard recalc?
Comment 9 m_a_riosv 2015-02-16 11:40:22 UTC
Forgive me, I didn't mention.

Shortcut = [Shif+Ctrl+F9]

It forces to recalculate all cells.

https://help.libreoffice.org/Calc/Recalculate

http://markmail.org/thread/vggjlicfxgpvi2xy
Comment 10 LeMoyne Castle 2015-02-20 20:34:21 UTC
Just a note in support of this report.  If confirmed, this bug may deserve a higher priority &/or severity.  

It is possible that a linked cell's current value is being taken and used without forcing an update through the link first. Or, that the value update is called for, but the calculation is not properly serialized by blocking the other parts of the calculation dependent on the pending update.  Either way, results reliant on linked data would be inconsistent and unreliable.   

I have been testing spreadsheet formulas related to other bugs and have seen results that *may* be a symptom of serialization/dependency issues in the formula interpreter.
Comment 11 andis.lazdins 2015-02-21 16:09:04 UTC
Hello!

The problem has gone after hard recalc [Shif+Ctrl+F9]. However, after saving and reopening I got the same result as before [Shif+Ctrl+F9]. Simple recalculation of cell content [f9] has no impact on result.
Comment 12 m_a_riosv 2015-02-21 20:02:14 UTC
Hi @Andis, I see the issue of your last comment, it happens when Autocalculate is activated.
Comment 13 Bill Babb 2015-07-26 01:01:48 UTC
Comment on attachment 113375 [details]
Example of calculation approach

What's wrong with this: =B3+B4+F3+F4+F7+F8+F9+F19, all fields are defined and have simple numbers in them. Calc gives "VALUE" error. SUM function gives same result.
Comment 14 Bill Babb 2015-07-26 01:14:12 UTC
Comment on attachment 113401 [details]
Updated version of example with repeatable error

Your problem confirms what I suspected: Calc is inconsistent on basic operations. Mine failed after I added the SUM function and then removed it.
Comment 15 m_a_riosv 2015-07-26 01:29:12 UTC
Please Bill what is the issue in your attached files?
Comment 16 Bill Babb 2015-07-26 04:58:56 UTC
After many tests I've found that the problem seems related to the position of the SUM function in the expression. When I place SUM as the first term in the expression the result of the calculation is wrong. When I remove it and substitute the same terms that were operands of SUM with the appropriate plus signs, the result is correct. I can't be certain about SUM being position sensitive but it is definitely suspect.
Comment 17 QA Administrators 2016-09-20 10:17:43 UTC Comment hidden (obsolete)
Comment 18 andis.lazdins 2016-09-20 11:12:21 UTC
I tried the formula on the newer version of file, where I identified the problem, with 5.1.5.2, Ubuntu 16.04, 32 bit. It looks like problem is gone or some kind of forgotten workaround is implemented.
Comment 19 b. 2021-04-10 17:18:28 UTC
@Bill Babb ... and others: 'SUM' is sensitive to the order of operands, '=SUM(1E16; -1E16; 1)' -> 1 , '=SUM(1; 1E16; -1E16)' -> 0, 
it's an effect of 'cancellation' and will truncate small operands in subtraction to different degree (between 'nearly nothing' and 'total extinction', 
calc and excel process ranges with different strategies, thus expect deviations,