There's a well known issue, Calc will display this result as 1E-04 What was expected Expected 0.0001 0.0042 0.0043 =A2-A1 1E-04 If we expand the column it presently shows show 0.0000999999999999994 What's gone wrong There are various ways to fix this bug, the best is to calcuate accurately, and not display raw Double precision. A) Calc seems to be displaying raw double precision, and not using it's own accurate calculation of decimal numbers. It's pretty easy to simply have a BigNum class with an int32 and an uint32 representing the decimal precision to avoid these issues. B) The alternative, is to select appropriate formatting . eg in this case it would be #,##0.0000 Personally I favour (A), just calculate and show exactly correct values. My $5 Casio calculator displays the result correctly. There is no reason to ever show a user 1E-04 or 0.0000999999999999994 Version: 6.0.7.3 Build ID: 1:6.0.7-0ubuntu0.18.04.10 CPU threads: 8; OS: Linux 4.15; UI render: default; VCL: gtk3; Locale: en-GB (en_GB.UTF-8); Calc: group
*** This bug has been marked as a duplicate of bug 130725 ***
Created attachment 158146 [details] Google Docs doesn't have this problem
Created attachment 158335 [details] Another easy example of this working correctly in Google Docs but not Calc Another easy example of this working correctly in Google Docs but not Calc
Welcome to the wonderful world of IEEE 754 double floating point values. Go to http://weitz.de/ieee/ and input the numbers 0.0043 0.0042 and click the - minus button; see the result is 1.0000000000000026E-4 For more information visit https://erack.de/bookmarks/D.html#010203 and pick your reading. For the values in attachment 158335 [details] Calc also displays 0.8, there's no problem. Meanwhile, for Calc 7.0, for the values in attachment 158146 [details] Calc also displays 0.0001, there's no problem anymore.