Bug 86174 - logic error with mod() and int() function
Summary: logic error with mod() and int() function
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.2.4.2 release
Hardware: x86 (IA32) Windows (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: Calc-Function Calculate
  Show dependency treegraph
 
Reported: 2014-11-12 00:52 UTC by Brian Prasse
Modified: 2020-11-17 22:42 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
sample_that_correct_math_is_possible_in_calc (11.47 KB, application/vnd.oasis.opendocument.spreadsheet)
2020-10-04 22:08 UTC, b.
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Brian Prasse 2014-11-12 00:52:09 UTC
Mod(2.01,int(2.01))=0.01 returns false
Comment 1 m.a.riosv 2014-11-12 01:14:31 UTC
Hi @Brian, thanks for reporting.

Really not a bug, but a hardware limitation, please take a look to https://bugs.freedesktop.org/show_bug.cgi?id=50299#c4.
Comment 2 Robinson Tryon (qubit) 2014-11-12 01:19:29 UTC
Another symptom of floating-point arithmetic being imperfect.

If you bump-up precision to 20 places, you see:

Value:                  2.01000000000000000000
INT(Value):             2.00000000000000000000
MOD(Value, INT(Value)): 0.00999999999999979000

When you compare 0.01000000000000000000 and 0.00999999999999979000, Calc (correctly) says that they're not the same.

I'm not sure if Calc has a built-in EQUAL_WITHIN_TOLERANCE() function, but you can use this for the purpose:

=(ABS($CELL_OF_FIRST_VALUE - $CELL_OF_SECOND_VALUE) < 0.0000000000001)

Making sure that two values are less than 1e-12 different should be close enough for most purposes :-)

See:
Bug 67026 - Other: Limitations in Calc precision need to be documented
Comment 3 b. 2020-10-04 22:08:25 UTC
Created attachment 166070 [details]
sample_that_correct_math_is_possible_in_calc

@m.a.riosv: 

disagree ... see attached sheet, 

hardware limitation? no, weak, but doesn't forbid meaningful calculations, 

kohei: 

'So it's a hardware problem.' - the software is responsible for what it does with the hardware, 
 
'Anything we try to do at software level would be hacky and prone to other, more serious errors,' - anything? really anything??? who gives up can no longer win,  

'including significant performance impact.' - si, but hardware and performance improve steadily, this statement is 8 years old, thus outdated by a performance factor of 16, 

'IMO we should leave this alone.' - imho not, users won't let us alone with their correct questions about incorrect results, 

math is an exact science - tries to be - we should try to keep up ... 

this bug is! a bug as the result is mathematically incorrect, and it is! a bug as better results are possible, see attachement,
Comment 4 b. 2020-10-04 22:24:25 UTC
besides ... 50299 is! resolved / fixed ...