Created attachment 88494 [details] In this sheet I invert Modular Matrix, for testing Python implementantion of algorithms =RESIDUO(5310866044487,128) gives a result of 70.9970703125 instead of 71 and in AL74 gives a result of 26.984375 instead of 27 also in see AL55 in attached file. AQ gives the correct result 37 (these are modular functions, that has no fractional part)
Something other than the mod function is not quite right. From attachment 88494 [details] : AL65: 341139058173 (or so it seems) AQ54: 119 AM53: 128 Test: 1_ Copy cell AL65 2_ In a new spreadsheet, select A1, Paste Only number. 3_ Copy cell AQ54 4_ In the same new spreadsheet, select B1, Paste Only number. 5_ Copy cell AM53 6_ In the same new spreadsheet, select C1, Paste Only number. From now on, the following testing steps are all executed in the new spreadsheet. 7_ D1: =MOD(A1*B1,C1) , result: 26.984375. 8_ In A2 type in 341139058173 (just as a number). 9_ B2: 119 10_ C2: 128 11_ D2: =MOD(A2*B2,C2) , result 27. 12_ D3: =MOD(341139058173*119,128) , result 27. and now the more relevant formula: 13_ D4: =MOD(INT(A2)*B2,C2) , result 27. So using INT gives the adequate expected result, suggesting either: A_ that the original cell AL65 (and the new A1) is not really exactly equal to 341139058173 ; or B_ that there might be some problem with some floating point calculation. I have performed the test in Calc 4.1.3.2 (on Windows). Regards, Ady
Sorry, correction: and now the more relevant formula: 13_ D4: =MOD(INT(A1)*B1,C1) , result 27.
Hi Gravitania, I think the issue is in relation with the computations precisión, please take a look to this links: https://bugs.freedesktop.org/show_bug.cgi?id=50299 http://ask.libreoffice.org/en/question/8006/libreoffice-calc-calculation-error/ As not all number have an exact representation in binary format, maybe some of them are in the input table, this can leads in a result with decimals, in this case it is no possible to see decimal places because there is rounding in the precision bounds. In AQ53: =5310866044486,997 Doing an INT() in AQ53: =INT(MDETERM(AL44:AS51)) maybe fits better your sheet calculations. I think there is no solution, the best aproximation, take care of it when designing the spreadsheet. Otherwise using a specialised software. *** This bug has been marked as a duplicate of bug 50299 ***