Created attachment 153345 [details] A spreadsheet with a selection of subtractions showing the problem. To reproduce: Make a new spreadsheet. In A1, put '121.1' In B1, put '120.3' In C1, put '=A1-B1' Increase C1's column width to 3.23cm. C1 now displays 0.799999999999997, which it should not. Decrease C1's column width to 3.22cm. C1 now displays 0.8, which it should. If 'Format Cells' is used to set the number of decimal places to 15 or greater, the tenths-place will never show an 8.
Created attachment 153346 [details] (A screenshot of a similar spreadsheet.)
this is not a bug, this is how floating point arithmetic works. please have a look at: Bug 116853 - Simple Subtraction Error Precision Bug 126552 - calc subtraction imprecision
(In reply to Oliver Brinzing from comment #2) > this is not a bug, this is how floating point arithmetic works. > > please have a look at: > > Bug 116853 - Simple Subtraction Error Precision > Bug 126552 - calc subtraction imprecision Thank you for your feedback! I'm sorry if I'm setting the wrong Status, or if I shouldn't have changed the Status as I write the below question--from ( https://wiki.documentfoundation.org/QA/Bugzilla/Fields/Status/REOPENED ) I thought I should set this to 'NEEDINFO', but I could not see a NEEDINFO option in the list. I indeed had not realised that the linked bug reports already existed. However, if I understand correctly, that problem described should only be a function of Binary Floating-Point, where a base-10 number is inexactly represented as a base-2 number. As base 10 is being used for spreadsheet calculations, shouldn't Decimal Floating-Point be used to represent the digits exactly (within the limits of significance, and without making errors pop up in unrelated orders of magnitude)? https://en.wikipedia.org/wiki/Decimal_floating_point (It is plausible that I am very confused.)
All computers use binary internally for storage and calculation. The results are only displayed as base-10 for the convenience of human users. The issue could be mitigated by the whole package being rewritten using 128-bit floats, but all that would do is add more 9's. The issue would reappear if you made the cell wide enough, and the cost would be a huge amount of work and slow the whole software down. If 15 decimal place precision is important to you, a spreadsheet is probably not the right tool.
Sorry but not a bug, developers work hard to avoid rounding issues that comes from the floating calculations. The only way is rounding numbers with the needed decimals. Please take a look on the META about Limitations in Calc precision need to be documented https://bugs.documentfoundation.org/show_bug.cgi?id=67026