Bug 107945 - Performance of pivot table cache population can be improved...
Summary: Performance of pivot table cache population can be improved...
Status: ASSIGNED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
unspecified
Hardware: All All
: medium normal
Assignee: Kohei Yoshida
URL:
Whiteboard: target:5.5.0
Keywords:
Depends on:
Blocks: Pivot-Table
  Show dependency treegraph
 
Reported: 2017-05-19 00:55 UTC by Kohei Yoshida
Modified: 2019-02-27 09:29 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kohei Yoshida 2017-05-19 00:55:19 UTC
Description:
We have a customer document (xlsx file) that exhibits a slow import.  Upon further examination at least a part of the poor performance stems from pivot cache population esp. in

ScDPCache::InitFromDoc(...)

where we are not properly parsing the column values using the multi_type_vector's position iterators.

We should be able to improve the performance of this code, and I'm currently looking into this.

Actual Results:  
 

Expected Results:
 


Reproducible: Always

User Profile Reset: 

Additional Info:


User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0
Comment 1 Kohei Yoshida 2017-05-19 00:55:46 UTC
Taking it.
Comment 2 Kohei Yoshida 2017-05-19 00:57:22 UTC
Unfortunately we can't share the customer document due to its confidentiality, but we'll try to come up with a similar document to exhibit the slow performance (if we can).
Comment 3 Commit Notification 2017-05-24 03:22:33 UTC
Kohei Yoshida committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=d4cd8677889ec3807c194ef5b462f8e031807e5b

tdf#107945: properly iterate over mtv during pivot cache loading.

It will be available in 5.4.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 4 Kohei Yoshida 2017-05-24 03:44:56 UTC
FYI that's just a step one.  I'm not finished with it yet.
Comment 5 Commit Notification 2017-06-01 13:32:32 UTC
Kohei Yoshida committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=5ecb3f4625188d02e85a3daef76ccf7f26b33a61

tdf#107945: store rtl_uString pointers instead of heap OUString objects.

It will be available in 5.5.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 6 Commit Notification 2017-06-06 03:01:19 UTC
Kohei Yoshida committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=a078328e91ab9cbd78a92608c1abdc2c57ff9ac5

tdf#107945: prepare for future multi-threading of pivot cache...

It will be available in 5.5.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 7 Roman Kuznetsov 2019-02-27 09:29:42 UTC
Kohei, is this bug fixed? Please change status to FIXED if it's so