Created attachment 137066 [details]
Example errors on big integers
In attachments, there are some big numbers, and arithmetic operations on them (integer division by 2, and remainder of that division). With red are marked wrong results (where MOD() errors out, or integer division gives off-by-one error).
Given that the numbers are all representable in double, the operations should succeed. Please note, that there's no some threshold, because there are numbers of same magnitude, which produce correct results.
Tested with Version: 18.104.22.168 (x64)
Build ID: 22b09f6418e8c2d508a9eaf86b2399209b0990f4
CPU threads: 4; OS: Windows 6.19; UI render: default;
Locale: ru-RU (ru_RU); Calc: CL
A fix is proposed: https://gerrit.libreoffice.org/43477
I don't know if just zeroing ~3 least significant binary bits would be better maybe?
(In reply to Mike Kaganski from comment #1)
> I don't know if just zeroing ~3 least significant binary bits would be
> better maybe?
Clarification: I suppose that it is only useful to zero some trailing bits when they are not a part of integers, and contribute to some small fraction (say, <10^-5..10^-10)... but I can be wrong.
Eike, Markus, could you please advise, possibly clarify which problems the rounding solves that I might break with this change?
The approxValue() rounding as used in approxFloor() and approxCeil() solves the problem that due to precision and rounding errors a floor(1.99999...) that should had been floor(2.0) actually results in 2.0 instead of 1.0
If the change affects *only* integer values to return early then it should be fine.
Mike Kaganski committed a patch related to this issue.
It has been pushed to "master":
tdf#113211: fix calculations with big integers
It will be available in 6.0.0.
The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
Affected users are encouraged to test the fix and report feedback.
A polite ping to Mike Kaganski: is this bug fixed? if so, could you
please close it as RESOLVED FIXED ? Thanks
Let's call it fixed. :) Thanks Xisco!