Bug 79023 - ScCompiler slowness ...
Summary: ScCompiler slowness ...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All All
: medium enhancement
Assignee: Eike Rathke
URL:
Whiteboard: target:5.3.0
Keywords: perf
Depends on:
Blocks:
 
Reported: 2014-05-21 16:42 UTC by Michael Meeks
Modified: 2022-03-16 12:20 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments
kcachegrind picture of the slowness ... (415.16 KB, image/png)
2014-05-21 16:43 UTC, Michael Meeks
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Meeks 2014-05-21 16:42:26 UTC
It seems the ScCompiler burns 7% of load-time of my (large - 227bn pseudo-cycles) of startup/load time inside 'IsValue'.

I attach a photo of the fun in action =)

Apart from the slow 'uppercase' call (did ScCompiler:: already uppercase the string - duplicating it in the process ?) - we seem have an:

'IsNumber' function that spends most of its time in:

SvNumberFormatter::IsNumberFormat

which seems to do an extraordinary amount of hard work =) is that truly necessary for parsing formulae from the file-format rather than exotic user inputs ? =)

ie. would some:

    if (FormulaGrammar::isODFF( eGrammar ))

followed by "is it a flat integer" work nicely there ? ;-)

Thoughts appreciated =)

I suspect the real win here comes from not parsing the formulae at all, but printing and comparing them down the column to catch long repeated columns full of ~identical formulae =) [ in the XLSX import this gave a very significant win ]
Comment 1 Michael Meeks 2014-05-21 16:43:03 UTC
Created attachment 99517 [details]
kcachegrind picture of the slowness ...
Comment 2 QA Administrators 2015-06-08 14:42:45 UTC Comment hidden (obsolete)
Comment 3 QA Administrators 2016-09-20 10:01:56 UTC Comment hidden (obsolete)
Comment 4 Eike Rathke 2016-09-20 17:26:24 UTC
Taking a stab at this..
Comment 5 Eike Rathke 2016-09-20 19:49:59 UTC
Loading a test document containing 1000 formulas with only numeric constants, =12345.6789 and =123.456 alternating to not hit the formula grouping, so 1000 calls to ScCompiler::CompileString()

Before:
Ir       Irpc  Callee
9859177  9859  ScCompiler::IsValue
6246858  6246  SvNumberFormatter::IsNumberFormat
3496261  3496  SvNumberFormatter::GetStandardIndex

After:
 298000   298  ScCompiler::IsValue
 248000   248  rtl_math_uStringToDouble
Comment 6 Commit Notification 2016-09-20 19:52:00 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=73c7e0921d752df53004ed55735f3e8888cc592f

sc-perf: tdf#79023 for ODFF do not call SvNumberFormatter to determine numeric

It will be available in 5.3.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 7 Michael Meeks 2016-09-20 20:19:01 UTC
Hi Eike,

Whoot - nice work - thanks for that ! =) a nice win - hopefully with the FastParsing for ODS we'll get another win here too in a bit ...
 
> Loading a test document containing 1000 formulas with only numeric constants,
> =12345.6789 and =123.456 alternating to not hit the formula grouping,
> so 1000 calls to ScCompiler::CompileString()

I forget if we do the look-ahead formula parsing that is so effective with XLSX in the ODS case - which was a huge win for XLSX (I hope so).

Anyhow - should we close this one ? =)
Comment 8 Commit Notification 2016-09-20 20:42:17 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=a8a8ff59c5749bbe1f2f58ea8fd42d66e6ae2a81

sc-perf: tdf#79023 do not call SvNumberFormatter also for numbers in OOXML

It will be available in 5.3.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.