Bug 108783 - wrong display of 2^47 .. 2^53 when fractional digits are 16 .. 20
Summary: wrong display of 2^47 .. 2^53 when fractional digits are 16 .. 20
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
5.1.0.3 release
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-06-26 07:40 UTC by Carlo Verre
Modified: 2021-04-17 15:51 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
powers of 2 (2^0 .. 2^64) with fractional digits 0 .. 20, err in R49 .. V55 (15.87 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2017-06-26 07:40 UTC, Carlo Verre
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Carlo Verre 2017-06-26 07:40:39 UTC
Created attachment 134284 [details]
powers of 2 (2^0 .. 2^64) with fractional digits 0 .. 20, err in R49 .. V55

wrong display of powers of 2 from 2^47 until 2^53
when fractional digits are between 16 and 20

example:

type in A1: =2^47, you get correctly: 140737488355328

press repeatedly the '0.00+' button to increase fractional digits, you get:

15 digits: 140737488355328.000000000000000 RIGHT
16 digits: 14073748835532.8000000000000000 WRONG
17 digits: 1407374883553.28000000000000000 WRONG
18 digits: 140737488355.328000000000000000 WRONG
19 digits: 14073748835.5328000000000000000 WRONG
20 digits: 1407374883.55328000000000000000 WRONG

the wrong values are not only displayed on screen
but also written out when you save in .csv format
Comment 1 Julien Nabet 2017-06-26 09:45:08 UTC
On Win7 with LO 5.3.4 + brand new LO profile, I could reproduce this.
clicking on button '0.00+' when display value is:
1407374883,55328000000000000000
does nothing.

BTW, sum value at the bottom of window shows the same pb.
Comment 2 Jacques Guilleron 2017-06-26 10:52:22 UTC
I reproduce with
LO  5.1.0.3 Build ID: 5e3e00a007d9b3b6efb6797a8b8e57b51ab1f737
CPU Threads: 2; OS Version: Windows 6.1; UI Render: default; 
Locale : fr-FR (fr_FR)
but not with
LO 5.1.0.2 Build ID: ecd3574d51754b043f865cf5bafee286d24db7cc
CPU Threads: 2; OS Version: Windows 6.1; UI Render: default; 
Locale : fr-FR (fr_FR)
Comment 3 m_a_riosv 2017-06-26 22:28:27 UTC
Maybe because it's trying to go beyond precision limits. Fifteen digits plus sign.

The issue with csv it that it uses the formatted value not the underlying value, by default, but there is an option saving as csv to not use the formatted value.
Comment 4 QA Administrators 2018-06-27 02:48:33 UTC Comment hidden (obsolete)
Comment 5 QA Administrators 2020-06-27 03:48:59 UTC Comment hidden (obsolete)
Comment 6 b. 2021-04-17 15:02:46 UTC
looks ok with ver. 7.2.0.0.a0+, 

as well in a fresh sheet as when loading the OP sheet, 

will set WFM if no objections within next month, 

Version: 7.2.0.0.alpha0+ (x64) / LibreOffice Community
Build ID: 6e6e531b564cdc9d5b25287c215cdc5a1fcbb346
CPU threads: 8; OS: Windows 6.1 Service Pack 1 Build 7601; UI render: default; VCL: win
Locale: de-DE (de_DE); UI: en-US
Calc: CL
Comment 7 Julien Nabet 2021-04-17 15:51:06 UTC
I don't reproduce this with LO Debian package 7.0.4.2

Let's put this one to WFM then.
Of course, if someone can still reproduce this with a recent LO version (7.0.5 or 7.1.2) don't hesitate to reopen this tracker.