Bug Hunting Session
Bug 97863 - FILEOPEN: XLS import hangs with 100% cpu usage
Summary: FILEOPEN: XLS import hangs with 100% cpu usage
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.4.0.0.beta1
Hardware: All All
: medium normal
Assignee: Markus Mohrhard
URL:
Whiteboard: target:5.2.0 target:5.1.3
Keywords: bisected, regression
Depends on:
Blocks:
 
Reported: 2016-02-14 20:50 UTC by tajuma
Modified: 2016-10-25 19:07 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
The original failing file (3.03 MB, application/vnd.ms-excel)
2016-03-23 20:51 UTC, tajuma
Details

Note You need to log in before you can comment on or make changes to this bug.
Description tajuma 2016-02-14 20:50:04 UTC
Using LibreOffice version 5.1.0.3 5e3e00a007d9b3b6efb6797a8b8e57b51ab1f737 (Debian download from lo.org).

Trying to open MS XLS file (3.1MB, not attaching) from http://www.tulli.fi/fi/yksityisille/autoverotus/taulukot/autot/au/012015.xls (Finnish customs statistics for imported used cars), LO "hangs" with 100% CPU usage. Not sure about its content, but it should have a sheet for every month with a few hundred rows for constants, should not have any formulas, fancy formatting or such.

I have no source but gdb backtrace points to 

(gdb) bt
#0  0x00007f2b3b1596c0 in XclImpExtName::MOper::MOper(svl::SharedStringPool&, XclImpStream&) ()
   from /opt/libreoffice5.1/program/libscfiltlo.so
#1  0x00007f2b3b15b34e in XclImpExtName::XclImpExtName(XclImpSupbook&, XclImpStream&, XclSupbookType, ExcelToSc*) ()
   from /opt/libreoffice5.1/program/libscfiltlo.so
#2  0x00007f2b3b15bfca in XclImpSupbook::ReadExternname(XclImpStream&, ExcelToSc*) () from /opt/libreoffice5.1/program/libscfiltlo.so
#3  0x00007f2b3b05f5e9 in ImportExcel8::Read() () from /opt/libreoffice5.1/program/libscfiltlo.so
#4  0x00007f2b3b03704b in ScFormatFilterPluginImpl::ScImportExcel(SfxMedium&, ScDocument*, EXCIMPFORMAT) ()
   from /opt/libreoffice5.1/program/libscfiltlo.so
#5  0x00007f2b3ce1d0de in ScDocShell::ConvertFrom(SfxMedium&) () from /opt/libreoffice5.1/program/../program/libsclo.so
...

Looking from disassembly it is spinning in the nested for loop @ https://github.com/LibreOffice/core/blob/5e3e00a007d9b3b6efb6797a8b8e57b51ab1f737/sc/source/filter/excel/xilink.cxx#L289.

Setting a breakpoint at the end of the outermost loop (nRow <= nLastRow):

(gdb) disassemble
...
   0x00007f2b3b159747 <+327>:   add    $0x1,%r15
   0x00007f2b3b15974b <+331>:   cmp    0x28(%rsp),%r15
   0x00007f2b3b159750 <+336>:   jbe    0x7f2b3b1596c0 <_ZN13XclImpExtName5MOperC2ERN3svl16SharedStringPoolER12XclImpStream+192>
...

(gdb) break *0x00007f2b3b15974b

and running it

(gdb) cont
Continuing.

Breakpoint 1, 0x00007f2b3b15974b in XclImpExtName::MOper::MOper(svl::SharedStringPool&, XclImpStream&) ()
   from /opt/libreoffice5.1/program/libscfiltlo.so

(gdb) p $r15
$42 = 19240055
(gdb) x/gx $rsp+0x28
0x7ffeadb55d08: 0xffffffffffffffff

The limit nLastRow is ULONG_MAX and the test being for "unsigned less or equal" it will spin there forever.
Comment 1 Julien Nabet 2016-02-14 22:24:33 UTC
On pc Debian x86-64 with master sources updated today, I could reproduce the hang, I noticed these logs on console:
warn:sc:21746:1:sc/source/filter/excel/xlroot.cxx:153: XclRootData::XclRootData - cannot get output device info: invalid attempt to assign an empty interface of type com.sun.star.frame.XFrame!
warn:sc:21746:1:sc/source/core/tool/scmatrix.cxx:2232: ScMatrix with 0 columns!
warn:sc:21746:1:sc/source/core/tool/scmatrix.cxx:2233: ScMatrix with 0 rows!
warn:legacy.osl:21746:1:sc/source/filter/excel/xistream.cxx:1033: XclImpStream::EnsureRawReadSize - record overread
warn:sc:21746:1:sc/source/filter/excel/xilink.cxx:284: Parsing error: 0 max possible rows, but 0 index claimed, truncating
Comment 2 Cor Nouws 2016-02-15 08:42:09 UTC
freezes 4.4.7.2 too.
opens fine in 4.3.7.2.
Comment 3 Cor Nouws 2016-02-15 08:42:36 UTC
(In reply to Cor Nouws from comment #2)
> freezes 4.4.7.2 too.
> opens fine in 4.3.7.2.

tested on 32 bits Ubuntu
Comment 4 Terrence Enger 2016-02-16 19:03:22 UTC
Working in the 44max bibisect repository, I see (whitespace added)
from `git bisect bad` ...

    a1422a6cd05a917b9187e3227e2c111ab20508f3 is the first bad commit
    commit a1422a6cd05a917b9187e3227e2c111ab20508f3
    Author: Matthew Francis <mjay.francis@gmail.com>
    Date:   Sun Mar 15 06:07:58 2015 +0800

        source-hash-05362fd2dbb481b735e8b7e97288d842a6e3ec0b
    
        commit 05362fd2dbb481b735e8b7e97288d842a6e3ec0b
        Author:     Caolán McNamara <caolanm@redhat.com>
        AuthorDate: Tue Nov 18 21:22:10 2014 +0000
        Commit:     Caolán McNamara <caolanm@redhat.com>
        CommitDate: Wed Nov 19 09:12:43 2014 +0000
    
            coverity#1242708 Untrusted loop bound
    
            Change-Id: Ic5af417ad38cafa46051789574239996a8845ffb

    :040000 040000 d87041ad5b8e8aa6c4b510aaca6ea0de50aff49e 95960e28e7f75c442e0fc26741de18bbc502f82f M	opt

and from `git bisect log` (whitespace added) ...

    # bad: [cf6ea17155fabb2a120ba07c150735591ac861d7]
        source-hash-3f94c9e9ddfd807b449f3bb9b232cf2041fa12d2
    # good: [fc71ac001f16209654d15ef8c1c4018aa55769f5]
        source-hash-c15927f20d4727c3b8de68497b6949e72f9e6e9e
    git bisect start 'latest' 'oldest'
    # good: [8cf60cc706948588e2f33a6d98b7c55d454e362a]
        source-hash-f340f0454627939f1830826fb5cc53a90e6c62a4
    git bisect good 8cf60cc706948588e2f33a6d98b7c55d454e362a
    # good: [7beddf3808dadd525d7e55c00a5a90a2b44c23d3]
        source-hash-2f10386ce577f52e139aa23d41bc787d8e0b4d59
    git bisect good 7beddf3808dadd525d7e55c00a5a90a2b44c23d3
    # good: [fb3ec529b3f37f0c7eab2e9b7a9cc695c0f27676]
        source-hash-0516d123f53917d1833c7e8a8c528a619c71a0af
    git bisect good fb3ec529b3f37f0c7eab2e9b7a9cc695c0f27676
    # good: [47a64818ddfe63bbb8e6448fcc476f55996d61b1]
        source-hash-d12efada389643ab0e13a280246d14caed273029
    git bisect good 47a64818ddfe63bbb8e6448fcc476f55996d61b1
    # good: [8f2027699192b7f2aaf83f95a02c817f2e0c8d50]
        source-hash-eb6d27321d2d5f9d069c4a3cbcc9bc6e5b4c98ab
    git bisect good 8f2027699192b7f2aaf83f95a02c817f2e0c8d50
    # good: [a318eefc5f56a820803027a525c255451cd603bd]
        source-hash-537befbb2fd5f1587f7c9cd8c55498d29b713770
    git bisect good a318eefc5f56a820803027a525c255451cd603bd
    # bad: [ae924ef4ddc9d3c4a941fa9c444f9712c59721cf]
        source-hash-7fe2a3fd370049d7599c301d2af71ca61fec1431
    git bisect bad ae924ef4ddc9d3c4a941fa9c444f9712c59721cf
    # bad: [4215aa7df7b2f632c2bcf6abc2f49388fc510042]
        source-hash-db222b74f1482870aac76d51646215d756901b8d
    git bisect bad 4215aa7df7b2f632c2bcf6abc2f49388fc510042
    # good: [6ca58fb62944a6ef674e4630740673971334f424]
        source-hash-1b9aaba0bfe8bc0872e7ea9f9aef5961e4b52f7c
    git bisect good 6ca58fb62944a6ef674e4630740673971334f424
    # good: [ea00dc01d460eb870bbbbb60ad06000b1a7da41e]
        source-hash-772a36932e4803beaffdad87200e0162e1020e94
    git bisect good ea00dc01d460eb870bbbbb60ad06000b1a7da41e
    # bad: [bc4fa52a1f8c33aa35ed98b277238fd0c631fcca]
        source-hash-b943150fe84a0158546b6a00ee330ea503c02359
    git bisect bad bc4fa52a1f8c33aa35ed98b277238fd0c631fcca
    # bad: [8c91fe808882b18c3aa1e1b20fc147f62f50ba50]
        source-hash-2ab8eb20ce82649168af96c551476cdfeb31f4ff
    git bisect bad 8c91fe808882b18c3aa1e1b20fc147f62f50ba50
    # bad: [a1422a6cd05a917b9187e3227e2c111ab20508f3]
        source-hash-05362fd2dbb481b735e8b7e97288d842a6e3ec0b
    git bisect bad a1422a6cd05a917b9187e3227e2c111ab20508f3
    # first bad commit: [a1422a6cd05a917b9187e3227e2c111ab20508f3]
        source-hash-05362fd2dbb481b735e8b7e97288d842a6e3ec0b
Comment 5 Markus Mohrhard 2016-03-23 20:06:14 UTC
And of course the file is gone now. Please always attach your test files.
Comment 6 tajuma 2016-03-23 20:51:44 UTC
Created attachment 123799 [details]
The original failing file
Comment 7 tajuma 2016-03-23 20:53:54 UTC
Now the original file attached. Changing back to NEW.
Comment 8 Markus Mohrhard 2016-03-24 11:32:48 UTC
Ok, In that case the sanitizing went wrong.
Comment 9 Commit Notification 2016-03-24 11:46:39 UTC
Markus Mohrhard committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=6aa836329d17bf0aca4aead6bb9c35a6783ec215

don't sanitize value to an insane value, tdf#97863

It will be available in 5.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 10 Commit Notification 2016-03-24 13:36:08 UTC
Markus Mohrhard committed a patch related to this issue.
It has been pushed to "libreoffice-5-1":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=cb47752029f0a2b84ed9020a24eb65e11cd32a63&h=libreoffice-5-1

don't sanitize value to an insane value, tdf#97863

It will be available in 5.1.3.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 11 tajuma 2016-03-25 14:32:32 UTC
Tested with libreoffice-5-1~2016-03-25_11.03.40_LibreOfficeDev_5.1.3.0.0_Linux_x86-64 and now it works.

Thanks.
Comment 12 Cor Nouws 2016-03-25 14:52:40 UTC
thanks for fixing and verifying!