Bug 96821 - ROUND changes integers when it should not
Summary: ROUND changes integers when it should not
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
5.0.3.2 release
Hardware: All All
: medium normal
Assignee: Winfried Donkers (retired)
URL:
Whiteboard: target:6.1.0
Keywords:
Depends on:
Blocks: Calc-Function
  Show dependency treegraph
 
Reported: 2015-12-30 00:56 UTC by M Welinder
Modified: 2018-01-17 06:23 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description M Welinder 2015-12-30 00:56:01 UTC
ROUND(x,0) should leave integers alone.  It does not.

A1=ROUND(B1,0)
B1=C1+1
C1=1024*1024*1024*1024*1024*4
A3=A1-(C1-1000)
B3=B1-(C1-1000)

I see B3=1001 [correct] and A3=1002 [wrong].

Without looking at the code I cannot be 100% certain, but the typical
cause of this trouble is that someone used

   x = std::floor (x + 0.5);

thinking that such code rounds to the nearest integer.  It does not.
For the value in B1 -- which is 2^52+1 -- as well as the next 2^50
odd numbers, the addition of 0.5 will actually add 1 since that is
the best approximation that fits in a double.

There is one more trouble number: the smallest number that is strictly
less than 0.5.  That should round to 0, but the addition yields 1.

The solution likely is to use std::round.  However, that function might
not so the desired thing for negative values.
Comment 1 A (Andy) 2015-12-30 09:06:00 UTC
Reproducible with LO 5.1.0.1, Win 8.1
Comment 2 m.a.riosv 2015-12-30 22:03:20 UTC
Perhaps the calculation are beyond the precision limits.
Comment 3 QA Administrators 2017-01-03 19:55:08 UTC Comment hidden (obsolete)
Comment 4 mwelinder 2017-01-04 14:42:29 UTC
Still there in 5.2.3.3.  OpenLeap 42.1.

> Perhaps the calculation are beyond the precision limits.

It isn't.  (And if it had been, the result would have been 1000, not 1002.)
Comment 6 mwelinder 2017-01-04 23:40:58 UTC
Bug 67026 is unrelated.  That one is about problems arising from 1/10 not
being exactly representable in binary.  Old news.

This one is about wrong code implementing ROUND causing it to man-handle
some integers.  (And that number just below 0.5 too.)
Comment 7 m.a.riosv 2017-01-05 00:04:42 UTC
C1=1024*1024*1024*1024*1024*4 = 4503599627370496

If I'm counting well there are sixteen digits on the result.

and if I'm not wrong IEEE 754 specification it's limited to 15 digits of precision.

https://lists.freedesktop.org/archives/libreoffice-bugs/2014-August/215620.html
https://support.microsoft.com/en-us/kb/78113
Comment 8 mwelinder 2017-01-05 01:49:06 UTC
C1=1024*1024*1024*1024*1024*4

IEEE-758 [for binary formats] doesn't do decimal digits.  It does binary
digits.  C1 requires *one* binary digit of mantissa.

On page 8 of the 2008 version of the standard you will find details of
the "binary64" format that corresponds to the C "double" type on all
modern hardware.  53 bits for mantissa, 11 bits for exponent, and one
sign bit.  That adds to 65 bits, but the most significant bit for the
mantissa isn't stored as it is always 1.

So C1 is perfectly representable as is B1 (=C1+1), although B1 is
just at the limit of what can be represented.  But look what happens
if someone tries to round to nearest integer using this code with x=B1:

   x = std::floor (x + 0.5);   // WRONG

Adding 0.5 to x produces a number that is precisely midway between two
representable numbers, which are x and x+1.  IEEE-758 specifies to round
according to the current rounding mode which, typically, is round-to-even.  Therefore the rounding result will be x+1.  std::floor will not change
that.  The intent with the above code was to add 0.5 and have std::floor
throw away the decimal part.  That works fine for smaller numbers, but
not for the magnitude of B1.  It also works for numbers above 2*B1 because
they are all even, so adding 0.5 and rounding will leave the number
unchanged.

You might want to take a look at "man round", but note that it may not do
the right thing for negative numbers.
Comment 9 Winfried Donkers (retired) 2017-12-25 10:21:34 UTC
Changing A1 from =ROUND(B1,0) to =ROUND(B1,1) produces correct result (1001).
Also using either ROUNDUP(B1,0) or ROUNDDOWN(B1,0) produces a result of 1004 in A3.
All these rounding-functions use the same (rtl::math::)round() function internally.

I will look further into the code to see what exactly is happening and if it can be remedied.
Comment 10 Winfried Donkers (retired) 2017-12-27 12:15:24 UTC
It is a corner case, very close to the limit of resolution, but there is room for some improvement.
Comment 11 Commit Notification 2018-01-16 14:45:45 UTC
Winfried Donkers committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=b97a0df0f3234b4c1140ba1418d4b96a592afa4a

tdf#96821 fix corner cases for Calc function ROUND.

It will be available in 6.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.