Bug 141752 - calc: calculate: trap in rawsubtract, wrong calculation sequence, documentation imprecise
Summary: calc: calculate: trap in rawsubtract, wrong calculation sequence, documentati...
Status: RESOLVED DUPLICATE of bug 137679
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
6.1.6.3 release
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-04-19 03:51 UTC by b.
Modified: 2021-04-19 21:42 UTC (History)
0 users

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description b. 2021-04-19 03:51:59 UTC
Description:
rawsubtract() is a handy tool to check e.g. which deviations evolve from fp-math vs. e.g. calc rounding, and to perform precise 'unrounded' calculations,  

the documentation says, as well as the tooltip: 
'RAWSUBTRACT(Minuend, Subtrahend1, Subtrahend2, ...)' <https://help.libreoffice.org/6.1/he/text/scalc/01/func_rawsubtract.html> 

one would expect to get: 
((Minuend - Subtrahend1) - Subtrahend2) ... - Subtrahendx    and not: 
((Minuend - Subtrahendx) - Subtrahendx-1) ... - Subtrahend1 
but what you get is indeed the second calculation, reverse sequence of processing the operands. 

as the sequence matters (non associativity of fp-addition) and e.g. 
'=1E16 + 1 - 1E16' results in 0 while 
'=1E16 - 1E16 + 1' gives 1 it is: 

1. important to process the operands in a meaningful order (suppose to correct that), and 
2. important to have the documentation (and tooltip) matching the operation, which does not as it is now, 

practical sample: 
'=RAWSUBTRACT(1E+016;1E+016;1)' -> '0', should be '-1', 
'=RAWSUBTRACT(1E+016;1;1E+016)' -> '-1', should be '0' due to fp-math :-(  

assumptions: compiler optimizations? 

difficult: changing the order will break compatibility for existing sheets, not changing the order but only the documentation will trap more users in the future (as they normally don't check the documentation, but expect a program that can be operated intuitively) 

besides rawsubtract being rarely used (quite rarely know to users), it does! matter to work in a meaningful way because if, then it's used to check difficult things where precise functioning makes big difference. 

Steps to Reproduce:
1. key '=RAWSUBTRACT(1E+016;1E+016;1)' in a cell, 
2. key '=RAWSUBTRACT(1E+016;1;1E+016)' in the cell below


Actual Results:
'0'
'-1'

Expected Results:
'-1'
'0'


Reproducible: Always


User Profile Reset: No



Additional Info:
didn't check other hardware than Intel (Xeon), assume 'all', 
didn't check Linux, assume 'all', 
didn't check Versions prior 6.1.6.3, assume all versions having rawsubtract, 
didn't reset user profile as clear repro in different independent installs,
Comment 1 Mike Kaganski 2021-04-19 04:22:19 UTC
Absolutely *no* reason to introduce any specific order in this case. No matter what sequence will be used, the outcome will be equally confusing to most users, and those who know enough of FP math, will not be confused.

*** This bug has been marked as a duplicate of bug 137679 ***
Comment 2 b. 2021-04-19 21:42:29 UTC
@Mike Kaganski: sorry for objecting: 

1. users who know enough about fp-math, know that the calculating sequence matters, and do meaningful (small to big?) operand ordering acc. the sequence minuhend - subtrahend1 - subtrahend2 - subtrahend3 ... will be trapped, will be astonished that it doesn't work as expected and documented, and turn away frustrated with the idea 'fp-math is imprecise' ... we already have too much of these people, missing for making a good program, 

and it's unjustified, fp-math and IEEE 754 is! precise (to some degree), calc is messing it up in some respects, this case is clearly calc bullshitting and you defending it ... 

and it's never ever a dupe of a kahan summation enhancement request, it might be covered with kahan, might as well have / cause problems, see comment https://bugs.documentfoundation.org/show_bug.cgi?id=137679#c14, but in it's source it's different ... suggest unduping ... 

else ... file one big bug 'fp-math is imprecise', mark it as 'wontfix' and any calculation bugs as dupes and you are done ... ;-)