Bug 33365 - EDITING, Rand() Function
Summary: EDITING, Rand() Function
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
3.3.0 RC4
Hardware: x86-64 (AMD64) Windows (All)
: medium normal
Assignee: Kohei Yoshida
QA Contact:
URL:
Whiteboard: target:4.1.0 target:4.0.0.0.beta0
Keywords: easyHack, skillCpp
Depends on:
Blocks:
 
Reported: 2011-01-22 08:07 UTC by Jörgen
Modified: 2015-12-16 00:43 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
Calc document with Rand() (mal)function as described. (44.83 KB, application/vnd.oasis.opendocument.spreadsheet)
2011-01-22 08:07 UTC, Jörgen
Details
Document for test (30.47 KB, application/vnd.oasis.opendocument.spreadsheet)
2011-01-22 10:42 UTC, tester8
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jörgen 2011-01-22 08:07:11 UTC
Created attachment 42308 [details]
Calc document with Rand() (mal)function as described.

Rand() uses a bad Pseudo-random code for generating "random" numbers.

Even with as few records as 1000, the "random" numbers repeats itself several times.

(Version 3.2 of OpenOffice has the same "feature", so it's not a new one.)
Comment 1 Kohei Yoshida 2011-01-22 08:23:11 UTC
We call system's rand() call to make random numbers.  So, if you want to see real improvement, then you need to talk to the system developers.

Here is the code that we use:

(double)rand() / ((double)RAND_MAX+1.0)
Comment 2 Kohei Yoshida 2011-01-22 08:28:32 UTC
BTW, if you know of a good way to generate good quality random numbers, please share with us.
Comment 3 Kohei Yoshida 2011-01-22 08:29:06 UTC
I'll take this meanwhile.
Comment 4 Kohei Yoshida 2011-01-22 08:54:41 UTC
Ok.  I found a nice page summarizing various rng algorithms, both good and bad.

http://www001.upp.so-net.ne.jp/isaku/rand.html (Japanese language only)

I couldn't find an English page that contains as good info about random number generators....
Comment 5 Kohei Yoshida 2011-01-22 09:05:03 UTC
http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/SFMT/index-jp.html

This generator appears to be most statistically accurate according to the article.

The source code is released under modified BSD license.
Comment 6 tester8 2011-01-22 10:42:05 UTC
Created attachment 42313 [details]
Document for test
Comment 7 tester8 2011-01-22 10:43:34 UTC
Not reproduced on LO RC4 Ubuntu 10.04 x86.
I have generate 1000 numbers about 20 times. Every time all numbers was different (see cell D1 in attached document). Every time standart deviation was in range 0.27-0.30 when the theoretical one is equal to sqrt(3)/6=0.2887...

Average also was about theoretical value 0.5
I think it is good pseudorandom (random) numbers set.

May be late I will test this on XP.
Comment 8 tester8 2011-01-24 08:19:22 UTC
Hm. Reproduced on XP x86 LO RC4.
Every time it was 12-25 numbers that have twin.

May be Windows version of LO should has it's own random generator if standard one doesn't get expected results? Especially if MSO doesn't has this bug.
Comment 9 Rainer Bielefeld Retired 2011-04-28 10:34:19 UTC
[Reproducible] with WIN!
I am not familiar with all details of randomized numbers generators, but with tester8's sample document and "LibreOffice 3.3.2  – WIN7  Home Premium  (64bit) German UI [OOO330m19 (Build:202 / tag 3.3.2.2)]" I got in 20 refreshes an average of 27,6 duplicate Numbers in the 1000 values. That seems to be quite a lot and is very similar to tester8's results. 

It would really be interesting to know why tat works so much worse under WIN.
I doubt, but can such a problem also be related to the processor? I use 64 bit AMD Phenom II X4 955 Processor 3.2 GHz, 4GB RAM

I believe we do not need additional information from reporter?

Modified Status due to facts.
Comment 10 Michael Stahl 2012-11-28 15:56:03 UTC
seriously, Calc uses libc rand() for random numbers???

there is already rtl/random.h and Writer uses that even for boomark names...
Comment 11 Kohei Yoshida 2012-11-28 16:39:07 UTC
(In reply to comment #10)
> seriously, Calc uses libc rand() for random numbers???
> 
> there is already rtl/random.h and Writer uses that even for boomark names...

This smells like an easy hack....
Comment 12 Eike Rathke 2012-11-29 11:24:08 UTC
(In reply to comment #10)
> seriously, Calc uses libc rand() for random numbers???
> there is already rtl/random.h and Writer uses that even for boomark names...

Calc and rand() existed before rtl/random.h ;-)


(In reply to comment #11)
> This smells like an easy hack....

I just made this one..


Code pointer: replace rand() with something more sensible in sc/source/core/tool/interpr1.cxx ScInterpreter::ScRandom()
Comment 13 Eike Rathke 2012-11-29 12:14:51 UTC
Btw, MS-Excel up to 97 used this random generator: http://support.microsoft.com/kb/86523

For 2003, 2007 and 2010 it claims an algorithm developed by B.A. Wichman and I.D. Hill, see http://support.microsoft.com/kb/828795

See also:

"A Comparison of Random Number Generators Used in Business - 2004 Update" http://academic.research.microsoft.com/Paper/5668932.aspx

"Microsoft Excel’s ‘Not The Wichmann–Hill’ random number generators" http://www.sciencedirect.com/science/article/pii/S016794730800162X
Comment 14 Eike Rathke 2012-12-05 16:13:04 UTC
Apache OO just re-implemented RAND() with a B.A Wichmann and I.D Hill generator.. http://svn.apache.org/viewvc?view=revision&revision=1416271
Comment 15 Not Assigned 2012-12-08 01:34:25 UTC
tino committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=8450a99c744e9005f19173e4df35d65640bcf5c4

fdo#33365 added wrapper for boost random, use that in RAND()



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 16 Not Assigned 2012-12-10 15:36:17 UTC
Tino Kluge committed a patch related to this issue.
It has been pushed to "libreoffice-4-0":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=d421e1d25393416708f7c3fba625bd057deb9e08&g=libreoffice-4-0

fdo#33365 added wrapper for boost random, use that in RAND()


It will be available in LibreOffice 4.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 17 Johnny_M 2013-10-15 06:40:12 UTC
There seems to be a similar issue with the "Rnd" function of BASIC. Submitted bug 70474.
Comment 18 Robinson Tryon (qubit) 2015-12-16 00:43:41 UTC
Migrating Whiteboard tags to Keywords: (EasyHack,SkillCpp )
[NinjaEdit]