Bug 160056 - calc threaded calculation performance issue
Summary: calc threaded calculation performance issue
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
24.2.0.0 alpha0+
Hardware: All All
: medium normal
Assignee: Caolán McNamara
URL:
Whiteboard: target:24.8.0 target:24.2.4 target:24...
Keywords: perf
Depends on:
Blocks:
 
Reported: 2024-03-05 20:12 UTC by Caolán McNamara
Modified: 2024-06-03 17:08 UTC (History)
1 user (show)

See Also:
Crash report or crash signature:


Attachments
flamegraph from before (1.83 MB, image/svg+xml)
2024-03-05 20:13 UTC, Caolán McNamara
Details
after changes addressing GetCellValueOrZero (706.31 KB, image/svg+xml)
2024-03-05 20:18 UTC, Caolán McNamara
Details
after changes addressing ScRandom (2.10 MB, image/svg+xml)
2024-03-05 20:57 UTC, Caolán McNamara
Details
IncRef/DecRef hotspot (210.96 KB, image/png)
2024-03-12 13:09 UTC, Caolán McNamara
Details
removal of hot spot (211.37 KB, image/png)
2024-03-12 13:09 UTC, Caolán McNamara
Details
flamegraph of current state (1.61 MB, image/svg+xml)
2024-03-22 16:44 UTC, Caolán McNamara
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Caolán McNamara 2024-03-05 20:12:44 UTC
Description:
There are some circumstances where we find that there is sufficient lock contention between threads that we end up bottlenecked on mutexes.

Steps to Reproduce:
1. Various sample spreadsheets

Actual Results:
SvNumberFormatter::GetFormatForLanguageIfBuiltIn featuring highly in profiling.
Some flamegraphs will be attached to illustrate.

Expected Results:
We shouldn't have a bunch of threads blocked from doing anything useful.


Reproducible: Always


User Profile Reset: No

Additional Info:
Some work to date on this is:
https://cgit.freedesktop.org/libreoffice/core/commit/?id=1245614991654b9d3ddb15f644a689585fd4c9de
https://cgit.freedesktop.org/libreoffice/core/commit/?id=bc08b1fbbd00c6e4b086f6d95249e684ace6ae25
https://cgit.freedesktop.org/libreoffice/core/commit/?id=2c3605405225de4b850064cf25b15cd64be1409b
https://cgit.freedesktop.org/libreoffice/core/commit/?id=8f70b99b4dd162df7a6a124ccd53f3814ff10b00
Comment 1 Caolán McNamara 2024-03-05 20:13:44 UTC
Created attachment 192983 [details]
flamegraph from before
Comment 2 Caolán McNamara 2024-03-05 20:18:03 UTC
Created attachment 192984 [details]
after changes addressing GetCellValueOrZero

No locks visible in GetCellValueOrZero in this one, while lock in comphelper::rng::uniform_real_distribution is still visible
Comment 3 Commit Notification 2024-03-05 20:19:35 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/8fd18bd4616ab06049aa0c760b9248c04373863f

tdf#160056 comphelper::rng takes a mutex for every random number

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 4 Caolán McNamara 2024-03-05 20:57:15 UTC
Created attachment 192985 [details]
after changes addressing ScRandom

And after addressing ScRandom
Comment 5 Caolán McNamara 2024-03-12 13:09:14 UTC
Created attachment 193076 [details]
IncRef/DecRef hotspot
Comment 6 Caolán McNamara 2024-03-12 13:09:30 UTC
Created attachment 193077 [details]
removal of hot spot
Comment 7 Commit Notification 2024-03-12 15:23:30 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/e5548dceb9086a6f0c8fa70bce8de6da0be72e40

tdf#160056 don't need to create a boost::intrusive_ptr to do search

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 9 Commit Notification 2024-03-14 11:55:50 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/479893fa6fd82a7f96df63a6b89ed24c5e0a2866

Related: tdf#160056 pass 'StarFormat' flag in instead of changing state

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 10 Commit Notification 2024-03-15 14:48:36 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/ee5b9597605176af0c80f9ea28caac9ba7980254

Related: tdf#160056 IniLnge is immutable

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 11 Commit Notification 2024-03-21 13:30:38 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/17ae8327033ee10bc997247fd1b8e75a2be50ff1

Related: tdf#160056 this SvNumberFormatter is the one from the ScDocument

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Commit Notification 2024-03-21 19:35:26 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/49189bb8823adc6a76d3f33b34c02d6a640df96a

Related: tdf#160056 this can be static

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 13 Commit Notification 2024-03-21 19:35:28 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/e8d01d2447b3350f1bd24e07580402c4c699756c

Related: tdf#160056 this can be be a local function

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 14 Commit Notification 2024-03-21 19:35:31 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/de148aa512e02457e43c6db966472a37059fbb14

Related: tdf#160056 replace with wrapper GetFormatTable()

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 15 Commit Notification 2024-03-21 21:33:01 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/97b0090938b7031cf88622c9e4aeb9cc55030953

Related: tdf#160056 can be private and not exported

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 16 Commit Notification 2024-03-22 06:57:52 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/472713123b898470a9ce54ee9f19d900dd993d2d

Related: tdf#160056 ImpSubstituteEntry can be private and not exported

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 17 Commit Notification 2024-03-22 06:57:55 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/80570de114d31f4ae5114b6fd6fd3ea365b14d63

Related: tdf#160056 these methods can be const

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 18 Caolán McNamara 2024-03-22 16:44:29 UTC
Created attachment 193245 [details]
flamegraph of current state
Comment 19 Commit Notification 2024-03-22 20:51:33 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/c6c6126aa3e8de256091b829b98b5943db6a8be6

Related: tdf#160056 refactor SvNumberFormatter

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 20 Commit Notification 2024-03-23 00:14:53 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/b38974391e8d4bf0d450abfaa86bbccbe1022995

Related: tdf#160056 do calc NumberFormatting via ScInterpreterContext

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 21 Commit Notification 2024-04-12 22:51:48 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/030b655963c182693c7b657dc6aa4d2fe85c17c6

Related: tdf#160056 don't call GetParamCount twice

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 22 Commit Notification 2024-04-12 22:52:51 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/d490efe410a54f1860578728027d1ec0439f0ba9

Related: tdf#160056 call FormulaCompiler::IsOpCodeJumpCommand just once

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 23 Commit Notification 2024-04-13 09:55:50 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/791a2489eecc3e59d7a31a4bb9e513a576db08ee

Related: tdf#160056 elide ref counting for initial tokens

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 24 Commit Notification 2024-04-13 15:06:23 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/adbbe732e7ed7f3a749e04f1d9023989c972ec22

Related: tdf#160056 all maTokens are FormulaTypedDoubleToken*

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 25 Commit Notification 2024-04-13 19:28:56 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/1d63489ae7f99545bcd8a6f41496e78ccad3273f

Related: tdf#160056 rename/rework GetDoubleAsReference to SetDouble

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 26 Commit Notification 2024-04-13 19:28:58 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/63b237e1e8147f54e6d4db4671f612a656200e2f

Related: tdf#160056 don't set nVal twice

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 27 Commit Notification 2024-04-14 10:35:14 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/7f2283c2986ff94766cc1d2c076fb34a2c88a31a

Related: tdf#160056 1 entry is more common than no entries

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 28 Commit Notification 2024-04-14 14:11:34 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-24-2":

https://git.libreoffice.org/core/commit/cce100f72dda8f5d4ae0c75c69a81603dc62e247

tdf#160056 don't need to create a boost::intrusive_ptr to do search

It will be available in 24.2.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 29 Commit Notification 2024-04-15 03:55:42 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-24-2":

https://git.libreoffice.org/core/commit/ba5cbbccefb5d55e19ed928cf900f8afb8337e9d

Related: tdf#160056 this can be static

It will be available in 24.2.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 30 Commit Notification 2024-04-15 03:55:44 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-24-2":

https://git.libreoffice.org/core/commit/992ec11cf73003d326ce7610a481c6bd19564588

Related: tdf#160056 this can be be a local function

It will be available in 24.2.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 31 Commit Notification 2024-04-15 12:08:43 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/9d29649e07a3b2c5c2b3dda1dc15c0ca1ed51ae3

Related: tdf#160056 FormulaResult may ref an initial interpreter FormulaToken

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 32 Commit Notification 2024-04-17 09:10:42 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-24-2":

https://git.libreoffice.org/core/commit/3a3ab826e73cc98988f2044e497ca55461b40c4e

Related: tdf#160056 use member init list

It will be available in 24.2.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 33 Commit Notification 2024-04-18 12:40:29 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/42c235fe8a616afea4c838901f73c52a262b1337

Related: tdf#160056 use simple ref count for temp double tokens

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 34 Commit Notification 2024-04-21 19:52:36 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/79daf83584c6faab641b3d73999871b5c235d222

Related: tdf#160056 add number format info to ScPatternAttr cache

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 35 Commit Notification 2024-04-24 08:07:47 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-24-2":

https://git.libreoffice.org/core/commit/78b62dadcc119359e9fd5ceb670e2585d18d4c3a

Related: tdf#160056 various methods that don't need to be exported

It will be available in 24.2.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 36 Commit Notification 2024-05-06 12:54:55 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-24-2":

https://git.libreoffice.org/core/commit/e3e9d51c2d4bed475f9848c62c7b0e5c72d5e125

Related: tdf#160056 various functions and objects that can be const

It will be available in 24.2.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 37 Commit Notification 2024-05-06 13:09:58 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-24-2":

https://git.libreoffice.org/core/commit/bc7198d47b4d1aef7a346c3f939122c9c764eeda

Related: tdf#160056 all maTokens are FormulaTypedDoubleToken*

It will be available in 24.2.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 38 Commit Notification 2024-05-06 13:11:01 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-24-2":

https://git.libreoffice.org/core/commit/9757290668b50827ab22597a73a4850ebf3f6521

Related: tdf#160056 don't set nVal twice

It will be available in 24.2.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 39 Commit Notification 2024-05-06 13:11:03 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-24-2":

https://git.libreoffice.org/core/commit/34126d6fceb054b7ba05ceeae76e32c89fac580f

Related: tdf#160056 don't call GetParamCount twice

It will be available in 24.2.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 40 Commit Notification 2024-05-06 13:12:05 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-24-2":

https://git.libreoffice.org/core/commit/46bd13a55492adef4bf95133d7123494a62f549a

Related: tdf#160056 1 entry is more common than no entries

It will be available in 24.2.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 41 Commit Notification 2024-05-30 16:25:37 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-24-2":

https://git.libreoffice.org/core/commit/75a6ac7428ca93d4e3a6dfc0b14f321bca243d85

Related: tdf#160056 call FormulaCompiler::IsOpCodeJumpCommand just once

It will be available in 24.2.5.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.