Created attachment 91607 [details] Example of calclation that is incorrect when referncing other formulas Attached is an example of a sheet where formulas to column G should all be 0. The one the G3 gives -2.501E-012. F4 = 509.78 H4 = 1108.8 I4 = 1.18 J4 = 10500 K4 = H4 + I4 - J4 = 509.78 (correct) G4 = F4 - K4 = 509.78 - 509.78 = -2.501E-02 (incorrect, should be 0). Cheers, Steve
Comment on attachment 91607 [details] Example of calclation that is incorrect when referncing other formulas Corrected mimetype
Saving, closing, and reopening the attachment seems to give the correct answer, but reconstructing from scratch each time shows the problem. Also the problem stays in my accounts spreadsheet, even after reopening. Steve
I have upgrade to LO 4.1.4.2 and the problem still exists. FYI I'm using the English GB Language version.
A quick fix is to use the Round function, so somewhere precision is getting lost. e.g. Round(H4 + I4 - J4, 2) give 0.00 Steve
Hi Steve, I think the issue is in relation with the computations precision, please take a look to this links: https://bugs.freedesktop.org/show_bug.cgi?id=50299 http://ask.libreoffice.org/en/question/8006/libreoffice-calc-calculation-error/ As not all number have an exact representation in binary format, maybe the subtraction result. I think there is no solution, the best approximation, take care of it when designing the spreadsheet. Otherwise using a specialised software. To don't lose precision with rounding, I think rounding at doubled decimals of needed precision could be enough. *** This bug has been marked as a duplicate of bug 50299 ***
as of today i cannot find an error in the sheet, but in that calculation, it doesn't hold in any respect ... H4 = 1108.8 I4 = 1.18 J4 = 10500 K4 = H4 + I4 - J4 = 509.78 (correct) what doesn't mean fp-calculations are correct in calc or 'best possible', just this sample fails in itself ...