It seems the ScCompiler burns 7% of load-time of my (large - 227bn pseudo-cycles) of startup/load time inside 'IsValue'. I attach a photo of the fun in action =) Apart from the slow 'uppercase' call (did ScCompiler:: already uppercase the string - duplicating it in the process ?) - we seem have an: 'IsNumber' function that spends most of its time in: SvNumberFormatter::IsNumberFormat which seems to do an extraordinary amount of hard work =) is that truly necessary for parsing formulae from the file-format rather than exotic user inputs ? =) ie. would some: if (FormulaGrammar::isODFF( eGrammar )) followed by "is it a flat integer" work nicely there ? ;-) Thoughts appreciated =) I suspect the real win here comes from not parsing the formulae at all, but printing and comparing them down the column to catch long repeated columns full of ~identical formulae =) [ in the XLSX import this gave a very significant win ]
Created attachment 99517 [details] kcachegrind picture of the slowness ...
** Please read this message in its entirety before responding ** To make sure we're focusing on the bugs that affect our users today, LibreOffice QA is asking bug reporters and confirmers to retest open, confirmed bugs which have not been touched for over a year. There have been thousands of bug fixes and commits since anyone checked on this bug report. During that time, it's possible that the bug has been fixed, or the details of the problem have changed. We'd really appreciate your help in getting confirmation that the bug is still present. If you have time, please do the following: Test to see if the bug is still present on a currently supported version of LibreOffice (4.4.3 or later) https://www.libreoffice.org/download/ If the bug is present, please leave a comment that includes the version of LibreOffice and your operating system, and any changes you see in the bug behavior If the bug is NOT present, please set the bug's Status field to RESOLVED-WORKSFORME and leave a short comment that includes your version of LibreOffice and Operating System Please DO NOT Update the version field Reply via email (please reply directly on the bug tracker) Set the bug's Status field to RESOLVED - FIXED (this status has a particular meaning that is not appropriate in this case) If you want to do more to help you can test to see if your issue is a REGRESSION. To do so: 1. Download and install oldest version of LibreOffice (usually 3.3 unless your bug pertains to a feature added after 3.3) http://downloadarchive.documentfoundation.org/libreoffice/old/ 2. Test your bug 3. Leave a comment with your results; 4a. If the bug was present with 3.3 - set version to "inherited from OOo"; 4b. If the bug was not present in 3.3 - add "regression" to keyword Feel free to come ask questions or to say hello in our QA chat: http://webchat.freenode.net/?channels=libreoffice-qa Thank you for your help! -- The LibreOffice QA Team This NEW Message was generated on: 2015-06-08
** Please read this message in its entirety before responding ** To make sure we're focusing on the bugs that affect our users today, LibreOffice QA is asking bug reporters and confirmers to retest open, confirmed bugs which have not been touched for over a year. There have been thousands of bug fixes and commits since anyone checked on this bug report. During that time, it's possible that the bug has been fixed, or the details of the problem have changed. We'd really appreciate your help in getting confirmation that the bug is still present. If you have time, please do the following: Test to see if the bug is still present on a currently supported version of LibreOffice (5.1.5 or 5.2.1 https://www.libreoffice.org/download/ If the bug is present, please leave a comment that includes the version of LibreOffice and your operating system, and any changes you see in the bug behavior If the bug is NOT present, please set the bug's Status field to RESOLVED-WORKSFORME and leave a short comment that includes your version of LibreOffice and Operating System Please DO NOT Update the version field Reply via email (please reply directly on the bug tracker) Set the bug's Status field to RESOLVED - FIXED (this status has a particular meaning that is not appropriate in this case) If you want to do more to help you can test to see if your issue is a REGRESSION. To do so: 1. Download and install oldest version of LibreOffice (usually 3.3 unless your bug pertains to a feature added after 3.3) http://downloadarchive.documentfoundation.org/libreoffice/old/ 2. Test your bug 3. Leave a comment with your results. 4a. If the bug was present with 3.3 - set version to "inherited from OOo"; 4b. If the bug was not present in 3.3 - add "regression" to keyword Feel free to come ask questions or to say hello in our QA chat: http://webchat.freenode.net/?channels=libreoffice-qa Thank you for helping us make LibreOffice even better for everyone! Warm Regards, QA Team MassPing-UntouchedBug-20160920
Taking a stab at this..
Loading a test document containing 1000 formulas with only numeric constants, =12345.6789 and =123.456 alternating to not hit the formula grouping, so 1000 calls to ScCompiler::CompileString() Before: Ir Irpc Callee 9859177 9859 ScCompiler::IsValue 6246858 6246 SvNumberFormatter::IsNumberFormat 3496261 3496 SvNumberFormatter::GetStandardIndex After: 298000 298 ScCompiler::IsValue 248000 248 rtl_math_uStringToDouble
Eike Rathke committed a patch related to this issue. It has been pushed to "master": http://cgit.freedesktop.org/libreoffice/core/commit/?id=73c7e0921d752df53004ed55735f3e8888cc592f sc-perf: tdf#79023 for ODFF do not call SvNumberFormatter to determine numeric It will be available in 5.3.0. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Hi Eike, Whoot - nice work - thanks for that ! =) a nice win - hopefully with the FastParsing for ODS we'll get another win here too in a bit ... > Loading a test document containing 1000 formulas with only numeric constants, > =12345.6789 and =123.456 alternating to not hit the formula grouping, > so 1000 calls to ScCompiler::CompileString() I forget if we do the look-ahead formula parsing that is so effective with XLSX in the ODS case - which was a huge win for XLSX (I hope so). Anyhow - should we close this one ? =)
Eike Rathke committed a patch related to this issue. It has been pushed to "master": http://cgit.freedesktop.org/libreoffice/core/commit/?id=a8a8ff59c5749bbe1f2f58ea8fd42d66e6ae2a81 sc-perf: tdf#79023 do not call SvNumberFormatter also for numbers in OOXML It will be available in 5.3.0. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.