Bug 83187 - FILEOPEN: XLSX with PowerPivot Data connection hangs at 100% CPU
Summary: FILEOPEN: XLSX with PowerPivot Data connection hangs at 100% CPU
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.1.6.2 release
Hardware: Other Linux (All)
: medium normal
Assignee: Markus Mohrhard
URL:
Whiteboard: BSA target:4.5.0 target:4.4.0.2
Keywords: bibisected, perf, regression
Depends on:
Blocks:
 
Reported: 2014-08-28 09:53 UTC by Owen Genat (retired)
Modified: 2021-08-18 08:35 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments
XLSX containing PowerPivot Data connection (for convenience) (1.88 MB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2014-08-28 10:06 UTC, Owen Genat (retired)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Owen Genat (retired) 2014-08-28 09:53:05 UTC
Problem description: Report relates to an attachment in this forum thread:

http://en.libreofficeforum.org/node/8863

... attachment is:

http://en.libreofficeforum.org/sites/libreofficeforum.org/files/uploads/Agrajag_files/2030%20RAW%20TEST.xlsx

It appears to contain a PowerPivot Data connection (Microsoft_SQLServer_AnalysisServices), rather than the usual XML data structure. The file fails to open (appears to hang using ~100% CPU) under GNU/Linux using any of these versions:

- v4.1.6.2 Build ID: 40ff705089295be5be0aae9b15123f687c05b0a
- v4.2.6.2 Build ID: 185f2ce4dcc34af9bd97dec29e6d42c39557298f
- v4.3.0.4 Build ID: 62ad5818884a2fc2e5780dd45466868d41009ec0
- v4.4.0.0.alpha0+ Build ID: 37b9ea92ba81d74764a2345a9c75c65bfd272d2b TinderBox: Linux-rpm_deb-x86_64@46-TDF, Branch:master, Time: 2014-08-26_09:48:30

Under GNU/Linux using:

- v3.5.7.2 Build ID: 3215f89-f603614-ab984f2-7348103-1225a5b

... the file does open after approx. 1min 30sec using ~100% during this time.

Steps to reproduce:
1. Open attachment.

Current behavior: Calc hangs at ~100% CPU usage.

Expected behavior: File is opened as expected.

              
Operating System: Debian
Version: 4.1.6.2 release
Comment 1 Owen Genat (retired) 2014-08-28 10:06:46 UTC
Created attachment 105380 [details]
XLSX containing PowerPivot Data connection (for convenience)
Comment 2 raal 2014-08-28 14:53:31 UTC
I can confirm that the bug is present in master.
No crash, but LO freeze with 100% CPU.

Version: 4.4.0.0.alpha0+
Build ID: 37b9ea92ba81d74764a2345a9c75c65bfd272d2b
TinderBox: Linux-rpm_deb-x86_64@46-TDF, Branch:master, Time: 2014-08-26_09:48:30

I can open the file with LO 3.5, regression.
Comment 3 raal 2014-08-28 15:16:56 UTC
 5b4693bb72eca5e38e3f56d036bca425c9a21b37 is the first bad commit
commit 5b4693bb72eca5e38e3f56d036bca425c9a21b37
Author: Bjoern Michaelsen <bjoern.michaelsen@canonical.com>
Date:   Sun Dec 9 11:49:31 2012 +0000

    source-hash-e3633f60b349022994e291aa3d1a0c90c3403b2e
    
    commit e3633f60b349022994e291aa3d1a0c90c3403b2e
    Author:     Stephan Bergmann <sbergman@redhat.com>
    AuthorDate: Wed May 16 09:32:51 2012 +0200
    Commit:     Stephan Bergmann <sbergman@redhat.com>
    CommitDate: Wed May 16 09:36:38 2012 +0200
    
        fdo#46074 fdo#49948 Ignore corrupted items in Recent Documents
    
        ...following up on 4ccb4bda483eb548eb6efb5e2f1952f094522320 "fdo#46074 Ignore
        corrupted items in Recent Documents" with another problematic scenario found
        with fdo#49948.
    
        Change-Id: I3e7c803813f09c1f031defc2c18cfab6732b1621

:100644 100644 5aa1dfc68ecb9ac57316a995424b2d3683cb4774 aa42f04f09d97d387333244ba505d2fd3c3086c2 M	autogen.log
:100644 100644 72da0ea5e9ec1223cb456558a2e0254561faa98c 1829a020e51322ed60e655809575a93edd3b9032 M	ccache.log
:100644 100644 5ef3324ce1c257155c9e095fdeb7d912b2681ae1 795d8ec3e2d59c5f0a85099dac7224954a57c4f2 M	commitmsg
:100644 100644 8b14489bddefe04fcfaecb0be901837505c64b67 5e870f27775bef1e12288b413b09a4052c414870 M	dev-install.log
:100644 100644 68ac6a90c73f1f7c8776a70772a40ae1ce41e13d 78b57ac998248d89343563f89455faeeea3f57a1 M	make.log
:040000 040000 8b906c6863615fd1253b393b35b18a883201b310 e793bfa8b661936460e69be1537f15a7e99d3289 M	opt

 git bisect log
# bad: [423a84c4f7068853974887d98442bc2a2d0cc91b] source-hash-c15927f20d4727c3b8de68497b6949e72f9e6e9e
# good: [65fd30f5cb4cdd37995a33420ed8273c0a29bf00] source-hash-d6cde02dbce8c28c6af836e2dc1120f8a6ef9932
git bisect start 'latest' 'oldest'
# bad: [e02439a3d6297a1f5334fa558ddec5ef4212c574] source-hash-6b8393474974d2af7a2cb3c47b3d5c081b550bdb
git bisect bad e02439a3d6297a1f5334fa558ddec5ef4212c574
# bad: [8f4aeaad2f65d656328a451154142bb82efa4327] source-hash-1885266f274575327cdeee9852945a3e91f32f15
git bisect bad 8f4aeaad2f65d656328a451154142bb82efa4327
# good: [369369915d3582924b3d01c9b01167268ed38f3b] source-hash-45295f3cdceb4c289553791071b5d7f4962d2ec4
git bisect good 369369915d3582924b3d01c9b01167268ed38f3b
# bad: [6fce03a944bf50e90cd31e2d559fe8705ccc993e] source-hash-47e4a33a6405eb1b5186027f55bd9cb99b0c1fe7
git bisect bad 6fce03a944bf50e90cd31e2d559fe8705ccc993e
# good: [8a39227e344637eb7154a10ac825d211e64d584c] source-hash-f5080ebb7022c9f5d7d7fdca4fe9d19f9bb8cabf
git bisect good 8a39227e344637eb7154a10ac825d211e64d584c
# bad: [e4c742a9e244bd7ebeabc50c90182df28ac3daaf] source-hash-c52ba433491afbca70aa1977a624c795bdd5b9ef
git bisect bad e4c742a9e244bd7ebeabc50c90182df28ac3daaf
# good: [96a055e15ee7171a28888973a3c3a7307dd9867f] source-hash-9ca02a663c3eee2698eb360dd5dc7afb1951e743
git bisect good 96a055e15ee7171a28888973a3c3a7307dd9867f
# bad: [e87a0055deae2c9e25ae1d1a365cec8418b785ce] source-hash-67ff63988f3b8eef2cc2b5bdf917918b93c3f070
git bisect bad e87a0055deae2c9e25ae1d1a365cec8418b785ce
# bad: [5b4693bb72eca5e38e3f56d036bca425c9a21b37] source-hash-e3633f60b349022994e291aa3d1a0c90c3403b2e
git bisect bad 5b4693bb72eca5e38e3f56d036bca425c9a21b37
# good: [d101b9946a6a04e65e3923038503436c790b7e12] source-hash-18e6e7d929c2be209407ed2e56b8ec4d5e6c4900
git bisect good d101b9946a6a04e65e3923038503436c790b7e12
# first bad commit: [5b4693bb72eca5e38e3f56d036bca425c9a21b37] source-hash-e3633f60b349022994e291aa3d1a0c90c3403b2e
Comment 4 Markus Mohrhard 2014-12-15 11:58:33 UTC
Ok. That is now just horribly slow with the new matrix storage and some of our ScINterpreter functions need to be adapted.

My current idea is to implement a generic solution for it that works on all same sized matrices.

we use mdds::multi_type_matrix::walk on the original matrix and have a functor that is called for each block and calls the mdds::multi_type_matrix::set(const position_type, const _T&, const _T&) version which performs much better than the current solution.

That should bring the performance back to the old levels or maybe make it even better.
Comment 5 Markus Mohrhard 2014-12-22 00:44:51 UTC
The performance is way better but there are still optimization possibilities. However I consider this enough for this fix for now. IF anyone is interested in improving his C++ template skill please talk to me and I'll point to a few more places.
Comment 6 Commit Notification 2014-12-22 00:46:25 UTC
Markus Mohrhard committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=3d6cedd70b3c79b3ebb65c2662df420a8acb0818

improve performance of some matrix operations, related fdo#83187

It will be available in 4.5.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 7 Markus Mohrhard 2015-01-03 20:33:23 UTC
Review request for 4-4 is in gerrit.
Comment 8 Commit Notification 2015-01-08 00:54:46 UTC
Markus Mohrhard committed a patch related to this issue.
It has been pushed to "libreoffice-4-4":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=4f29a7766edd19757a8ab2e38bf7fdff36c704b7&h=libreoffice-4-4

improve performance of some matrix operations, related fdo#83187

It will be available in 4.4.0.2.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 9 Robinson Tryon (qubit) 2015-12-17 08:34:06 UTC
Migrating Whiteboard tags to Keywords: (bibisected)
[NinjaEdit]