Bug 117819 - error displaying some integer values if number of decimals is forced to more than 15
Summary: error displaying some integer values if number of decimals is forced to more ...
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
5.2 all versions
Hardware: x86-64 (AMD64) Linux (All)
: high major
Assignee: Eike Rathke
URL:
Whiteboard: target:6.2.0 target:6.1.0.1 target:6.0.5
Keywords: bibisected, bisected, regression
Depends on:
Blocks:
 
Reported: 2018-05-26 14:40 UTC by zjs.no7
Modified: 2018-05-31 08:23 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
The file is a simple calculation of the Fibonacci series. If you go to line 69 you will see that only from line 69 to line 77 is a clear display error. (24.44 KB, application/vnd.oasis.opendocument.spreadsheet)
2018-05-28 10:56 UTC, zjs.no7
Details

Note You need to log in before you can comment on or make changes to this bug.
Description zjs.no7 2018-05-26 14:40:48 UTC
Description:
inserting in a cell a number of 15 or 16 digits (eg 123456789012345) and adding the maximum number of decimals after the decimal point the display of the value will be: 1234567890,12345000000000000000 instead of: 123456789012345,00000000000000000

Steps to Reproduce:
1.insert this number 123456789012345
2.add 20 decimal numbers
3.the display of the number will be:1234567890,12345000000000000000 instead of:123456789012345,00000000000000000000

Actual Results:  
the display of the number will be:1234567890,12345000000000000000

Expected Results:
123456789012345,00000000000000000000


Reproducible: Always


User Profile Reset: No



Additional Info:


User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36
Comment 1 MM 2018-05-26 19:22:23 UTC
I think this behaviour is correct. Due to the way it works [integer round off error], you can't have more than 15 decimal places (15 zero's). Anything more than 15 decimal places, the comma is shifted.
Comment 2 Jean-Baptiste Faure 2018-05-28 07:25:51 UTC
Not sure if I understand what you mean correctly. What I do:
1/ paste 123456789012345 in the formula bar then validate => I get 1.23456789012345E+14 (correct)
2/ go back in the formula bar, it shows 123456789012345, then paste 00000000000000000000 at the end of the number the validate => I get 1.23456789012345E+34 (correct)

Tested in Version: 6.1.0.0.beta1+
Build ID: d238a5e47b8cf9a484c8f881c3c01da10b340603
Threads CPU : 4; OS : Linux 4.4; UI Render : par défaut; VCL: gtk3; 
Ubuntu_16.04_x86-64
Locale : fr-FR (fr_FR.UTF-8); Calc: threaded

Idem in Version: 6.0.4.2
Build ID: 1:6.0.4~rc2-0ubuntu0.16.04.1
Threads CPU : 4; OS : Linux 4.4; UI Render : par défaut; VCL: gtk3; 
Locale : fr-FR (fr_FR.UTF-8); Calc: 

Please, could you explain with more detail what you do to obtain your result?

Status set to NEEDINFO, please set it back to UNCONFIRMED once requested
informations are provided.

Best regards. JBF
Comment 3 zjs.no7 2018-05-28 10:56:03 UTC
Created attachment 142336 [details]
The file is a simple calculation of the Fibonacci series. If you go to line 69 you will see that only from line 69 to line 77 is a clear display error.

The file is a simple calculation of the Fibonacci series.
If you go to line 69 you will see that only from line 69 to line 77 is a clear display error.
Comment 4 Xisco Faulí 2018-05-29 11:33:43 UTC
Regression introduced by:

author	Eike Rathke <erack@redhat.com>	2016-01-13 14:40:12 +0100
committer	Eike Rathke <erack@redhat.com>	2016-01-13 14:47:57 +0100
commit 0f6203edf74832f84d8263d7a544d679203a4efc (patch)
tree f8f0062394aa9994d3e3f02b633a6ef17a348077
parent 5fdb9cd64cfb48949a8c15292eda9266155a62ea (diff)
tdf#96918 display accurate integer double values up to (2^53)-1

Bisected with: bibisect-linux-64-5.2

Adding Cc: to Eike Rathke
Comment 5 Eike Rathke 2018-05-29 16:41:56 UTC
Odd, it's related to the number of decimals in the number format. Up to 15 decimals the output is correct. Taking.
Comment 6 Eike Rathke 2018-05-29 17:33:21 UTC
Oh fun, that change triggers a bug in the number formatter.
Comment 7 Eike Rathke 2018-05-29 18:48:59 UTC
Btw, @zjs.no7, you do know that floating point double precision for integers ends at 2^53-1, don't you? So (apart from this display error here) any attempt to format them as integer does not display accurate values for numbers above and may be misleading.
Comment 8 Commit Notification 2018-05-30 10:28:03 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=8b43f58891d4b422a8934050d839b0c2c1e3a18a

Resolves: tdf#117819 append trailing '0' as needed before separator insertion

It will be available in 6.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 9 Eike Rathke 2018-05-30 10:30:36 UTC
Pending review
https://gerrit.libreoffice.org/55059 for 6-1
https://gerrit.libreoffice.org/55060 for 6-0
Comment 10 Commit Notification 2018-05-30 12:51:02 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-6-1":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=960ec1180b73a5dc0445e7da229c7df9438ce450&h=libreoffice-6-1

Resolves: tdf#117819 append trailing '0' as needed before separator insertion

It will be available in 6.1.0.1.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 11 Commit Notification 2018-05-30 14:23:51 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=a8f8cf72b2b9e912dc4a5aebef55d9b2c0969462

Add unit test for tdf#117819

It will be available in 6.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Commit Notification 2018-05-30 14:25:25 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-6-0":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=2e0c3c88865b64b04b6160f9f5955a3a9548d48c&h=libreoffice-6-0

Resolves: tdf#117819 append trailing '0' as needed before separator insertion

It will be available in 6.0.5.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 13 Xavier Van Wijmeersch 2018-05-30 19:30:01 UTC
its working with

Version: 6.2.0.0.alpha0+
Build ID: 64980d88fb75b8a34b1fce4ed1d0d64d0f2d5e09
CPU threads: 8; OS: Linux 4.14; UI render: default; VCL: kde4; 
Locale: nl-BE (en_US.UTF-8); Calc: group threaded
Comment 14 Xisco Faulí 2018-05-31 08:23:32 UTC
(In reply to Xavier Van Wijmeersch from comment #13)
> its working with
> 
> Version: 6.2.0.0.alpha0+
> Build ID: 64980d88fb75b8a34b1fce4ed1d0d64d0f2d5e09
> CPU threads: 8; OS: Linux 4.14; UI render: default; VCL: kde4; 
> Locale: nl-BE (en_US.UTF-8); Calc: group threaded

Setting to VERIFIED!