Bug 94962 - The RSQ function returning values greater than one.
Summary: The RSQ function returning values greater than one.
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
5.0.2.1 rc
Hardware: x86-64 (AMD64) Windows (All)
: medium normal
Assignee: Andreas Heinisch
URL:
Whiteboard: target:7.2.0 target:7.1.4
Keywords:
: 94961 (view as bug list)
Depends on:
Blocks: Calc-Function
  Show dependency treegraph
 
Reported: 2015-10-11 17:05 UTC by moe gren
Modified: 2021-05-24 16:00 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:
Regression By:


Attachments
RSQ EXAMPLE (127.61 KB, application/vnd.oasis.opendocument.spreadsheet)
2015-10-12 08:00 UTC, moe gren
Details
same file with other tests (226.29 KB, application/vnd.oasis.opendocument.spreadsheet)
2015-10-12 09:23 UTC, GerardF
Details

Note You need to log in before you can comment on or make changes to this bug.
Description moe gren 2015-10-11 17:05:39 UTC
The RSQ function returning values greater than one. In my use it's showing >23. It could be that u need more of a description on how it is being calculated. But excel results are between 0 and 1.
Comment 1 moe gren 2015-10-11 17:11:15 UTC
*** Bug 94961 has been marked as a duplicate of this bug. ***
Comment 2 GerardF 2015-10-11 19:35:14 UTC
Can you attach a sample with a result > 1.
Comment 3 moe gren 2015-10-12 08:00:37 UTC
Created attachment 119530 [details]
RSQ EXAMPLE

RSQ GREATER than one.
Comment 4 GerardF 2015-10-12 09:23:09 UTC
Created attachment 119531 [details]
same file with other tests

Reproducible.

I get 2.58941… with RSQ
0.83332… with a scatter chart
0.941176 with LINEST

Note that the result of RSQ is the square of PEARSON, so the result of PEARSON is also false.
Comment 5 QA Administrators 2016-11-08 11:24:23 UTC Comment hidden (obsolete)
Comment 6 QA Administrators 2019-12-03 14:49:15 UTC Comment hidden (obsolete)
Comment 7 Andreas Heinisch 2021-04-13 11:25:21 UTC
I investigate the error and the problem arises during the calculation of the final result, because the involved numbers are close to zero and below the numerical limit of LO.

There are two ways in order to solve this issue. Either, if the involved numbers are below the numerical limit, just show a division by zero error, or set them to the numerical limit.

In the second case, the attached example would lead to:
RSQ Top
3,4504615480428E-15
RSQ Bottom
2,5975708894356E-14

Excel shows a division by zero error.
Comment 8 Commit Notification 2021-04-21 16:04:13 UTC
Andreas Heinisch committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/fc9919bcfcf32f2ecefa4fce18a49545b93d3499

tdf#94962 - Limit calculation of pearson coefficient

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 9 Commit Notification 2021-04-22 10:08:29 UTC
Andreas Heinisch committed a patch related to this issue.
It has been pushed to "libreoffice-7-1":

https://git.libreoffice.org/core/commit/6c9f5ddf3f905c854121b783a47959dc7618559c

tdf#94962 - Limit calculation of pearson coefficient

It will be available in 7.1.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 10 Commit Notification 2021-05-24 16:00:50 UTC
Andreas Heinisch committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/761c10bdf310fef2879874b6d5057fc51f54c0ca

tdf#94962 - Fixed german text:p of type boolean

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.