Description: I encountered some unexpected behavior in LibreOffice—using numbers in functions and arithmetic operations rounds them after a certain decimal place. For example, the result of the multiplication =0.6 * 3 will be 1.8 in the cell and in the file, which differs from the result of the calculation in Python, 1.7999999999999998. Additionally, if I use large numbers inside formulas, the result will be rounded. For instance, the result of the comparison =CLEAN(1.7999999999999998) = CLEAN(0.6 * 3) will be TRUE, and the formula will change to =CLEAN(1.8) = CLEAN(0.6 * 3). However, =CLEAN(1.79999999999998) = CLEAN(0.6 * 3) will return FALSE and will not alter the formula. Is this behavior a bug, or is rounding after a certain decimal place supposed to happen in arithmetic operations and functions? Steps to Reproduce: 1. Enter used formulas in any cell Actual Results: =1.7999999999999991 = 1.7999999999999999 - TRUE =CLEAN(1.7999999999999991) - 1.8 =CLEAN(1.799999999999991) - 1.79999999999999 Expected Results: =1.7999999999999991 = 1.7999999999999999 - FALSE =CLEAN(1.7999999999999991) - 1.7999999999999991 =CLEAN(1.799999999999991) - 1.799999999999991 Reproducible: Always User Profile Reset: No Additional Info: Used formulas: =1.7999999999999991 = 1.7999999999999999 =0.6*3 =10.01-10 =1234567890 + 0.0123456789 =CLEAN(1.7999999999999991) =CLEAN(1.799999999999991) =CLEAN(1.79999999999991) =CLEAN(0.6*3) =TRUNC(0.6*3,15) =TRUNC(1.7999999999999998, 15) =TRUNC(1.799999999999998, 15) =TRUNC(1.79999999999998, 15) =CLEAN(1.7999999999999998) = CLEAN(0.6*3) =CLEAN(1.79999999999998) = CLEAN(0.6*3)
Created attachment 199807 [details] Precision formulas
Created attachment 199808 [details] precision.ods
https://bugs.documentfoundation.org/show_bug.cgi?id=76245 https://bugs.documentfoundation.org/show_bug.cgi?id=128312 *** This bug has been marked as a duplicate of bug 128312 ***
I believe, this is about the use of things like rtl::math::approxEqual and friends [1], and also deliberate limitation of shown decimals to 15 significant digits. Yes we do some approximation, because generally people are confused by the tiny deviations resulting from the fact that 0.1 + 0.2 != 0.3. [1] https://opengrok.libreoffice.org/xref/core/include/rtl/math.hxx?r=40336ee0b4acf4e359fef696c6c2a6b3fa9718e7#323