Bug 113211 - CALC: Errors making simple arithmetic operations on big integers
Summary: CALC: Errors making simple arithmetic operations on big integers
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
(earliest affected)
Hardware: All All
: medium normal
Assignee: Mike Kaganski
Whiteboard: target:6.0.0
Depends on:
Reported: 2017-10-18 04:52 UTC by Mike Kaganski
Modified: 2017-11-19 19:41 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:

Example errors on big integers (12.69 KB, application/vnd.oasis.opendocument.spreadsheet)
2017-10-18 04:52 UTC, Mike Kaganski

Note You need to log in before you can comment on or make changes to this bug.
Description Mike Kaganski 2017-10-18 04:52:10 UTC
Created attachment 137066 [details]
Example errors on big integers

In attachments, there are some big numbers, and arithmetic operations on them (integer division by 2, and remainder of that division). With red are marked wrong results (where MOD() errors out, or integer division gives off-by-one error).

Given that the numbers are all representable in double, the operations should succeed. Please note, that there's no some threshold, because there are numbers of same magnitude, which produce correct results.

Tested with Version: (x64)
Build ID: 22b09f6418e8c2d508a9eaf86b2399209b0990f4
CPU threads: 4; OS: Windows 6.19; UI render: default; 
Locale: ru-RU (ru_RU); Calc: CL
Comment 1 Mike Kaganski 2017-10-18 04:59:17 UTC
A fix is proposed: https://gerrit.libreoffice.org/43477

I don't know if just zeroing ~3 least significant binary bits would be better maybe?
Comment 2 Mike Kaganski 2017-10-18 05:40:21 UTC
(In reply to Mike Kaganski from comment #1)
> I don't know if just zeroing ~3 least significant binary bits would be
> better maybe?

Clarification: I suppose that it is only useful to zero some trailing bits when they are not a part of integers, and contribute to some small fraction (say, <10^-5..10^-10)... but I can be wrong.

Eike, Markus, could you please advise, possibly clarify which problems the rounding solves that I might break with this change?
Comment 3 Eike Rathke 2017-10-18 14:54:54 UTC
The approxValue() rounding as used in approxFloor() and approxCeil() solves the problem that due to precision and rounding errors a floor(1.99999...) that should had been floor(2.0) actually results in 2.0 instead of 1.0

If the change affects *only* integer values to return early then it should be fine.
Comment 4 Commit Notification 2017-10-19 19:56:32 UTC
Mike Kaganski committed a patch related to this issue.
It has been pushed to "master":


tdf#113211: fix calculations with big integers

It will be available in 6.0.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:

Affected users are encouraged to test the fix and report feedback.
Comment 5 Xisco Faulí 2017-11-19 19:18:07 UTC
A polite ping to Mike Kaganski: is this bug fixed? if so, could you
please close it as RESOLVED FIXED ? Thanks
Comment 6 Mike Kaganski 2017-11-19 19:41:47 UTC
Let's call it fixed. :) Thanks Xisco!