Bug 103927 - filter:xlsx LibreOffice crashes when importing XLSX file
Summary: filter:xlsx LibreOffice crashes when importing XLSX file
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.1.0.4 release
Hardware: x86-64 (AMD64) Windows (All)
: high critical
Assignee: Kohei Yoshida
URL:
Whiteboard: target:5.4.0 target:5.3.1 target:5.2.6
Keywords: filter:xlsx, haveBacktrace, notBibisectable, regression
Depends on:
Blocks: GDI-Limit
  Show dependency tree / graph
 
Reported: 2016-11-14 16:11 UTC by paul.mitterberger
Modified: 2017-09-03 02:46 UTC (History)
2 users (show)

See Also:
Crash report or crash signature: ["VirtualDevice::ImplInitVirDev(OutputDevice const *,long,long,DeviceFormat,SystemGraphicsData const *)"]

Attachments
XLSX-File, which was sent to me and is viewable with the MS EXCEL viewer. (185.35 KB, application/xlsx)
2016-11-14 16:11 UTC, paul.mitterberger 		Details
backtrace of crash (11.92 KB, text/plain)
2016-11-14 17:48 UTC, Aron Budea 		Details
Dr. Memory log (9.35 KB, application/x-bzip)
2017-01-19 02:23 UTC, Kohei Yoshida 		Details
call stack on RuntimeException in virdev.cxx. (7.39 KB, text/plain)
2017-01-23 23:22 UTC, Kohei Yoshida 		Details
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description paul.mitterberger 2016-11-14 16:11:44 UTC 
Created attachment 128750 [details]
XLSX-File, which was sent to me and is viewable with the MS EXCEL viewer.

Libre Office crashes without any notice or crash report, when opening attached file DP_14-11-16.xlsx.
This file could be opend with the MS EXCEL viewer, which is available under https://www.microsoft.com/de-at/download/details.aspx?id=10.
Comment 1 Aron Budea 2016-11-14 17:17:55 UTC 
Confirmed with v5.2.3.3 / Windows 7.
What happened for me, it hung during loading for a while, then opened the spreadsheet, which looked really weird (large, condensed text in cells), then got Fatal Error after a bit.

Crash report was sent afterwards: http://crashreport.libreoffice.org/stats/crash_details/1a23362c-6b3f-4a18-9832-df8caf9c0671

Then another time it crashed during loading, and on a third time it opened, looked okay, but crashed after a few clicks in cells. No crash report was created in those cases.

Crashes in 4.1.0.4, doesn't crash in 4.0.0.3, so it seems to be a regression. The spreadsheet is still slow to work with in 4.0.0.3 and earlier releases, that's a separate issue.
Comment 2 Aron Budea 2016-11-14 17:48:08 UTC 
Created attachment 128754 [details]
backtrace of crash

Here's a backtrace for a crash that happened after loading the spreadsheet, and clicking in one of the merged cells (there seem to be some invisible UI elements in those cells, the anchors for them can be seen in early LO versions that don't crash).
Comment 3 Michael Meeks 2016-11-14 21:08:39 UTC 
Can you test with: export MAX_CONCURRENCY=1 ? Thanks =)
Comment 4 Aron Budea 2016-11-15 02:33:29 UTC 
With 5.2.3.3 / Windows 7, and MAX_CONCURRENCY=1, the experience is pretty much the same. The crash happens kind of randomly, sometimes the window just disappears after loading, or during movement of mouse cursor, then other times the crash comes with a dialog, or even a crash report afterwards.
Comment 5 Michael Meeks 2016-11-17 17:05:27 UTC 
Hi Aaron - the crash looks like some pure ColorListBox issue - nearly un-related to Calc =) It looks typical of an exception getting thrown during the creation of the object - is it possible to put a breakpoint into: ColorListBox::ColorListBox -- and then catch all exceptions from then on ? - would need a build with symbols of course ....
Comment 6 Kohei Yoshida 2017-01-19 02:23:30 UTC 
Created attachment 130541 [details]
Dr. Memory log

I ran it with Dr. Memory to see if there are any sort of invalid memory access issues during the load.  This is the log.  No idea how to interpret this and/or this is of any use just yet.
Comment 7 Kohei Yoshida 2017-01-19 02:59:56 UTC 
The two stacks that are not related to the DC errors.  Maybe they can point us in the right direction?

---
Error #70: INVALID HEAP ARGUMENT: allocated with Windows API layer, freed with C library layer
# 0 replace_free                                                        [d:\drmemory_package\common\alloc_replace.c:2706]
# 1 MSVCP120.dll!Cnd_destroy                                           +0x1e     (0x6361eaf9 <MSVCP120.dll+0x2eaf9>)
# 2 comphelper.dll!std::_Ref_count_obj<>::_Destroy                      [c:\program files (x86)\microsoft visual studio 12.0\vc\include\memory:944]
# 3 scfiltlo.dll!oox::xls::`anonymous namespace'::importSheetFragments  [j:\libo\my-master\sc\source\filter\oox\workbookfragment.cxx:346]
# 4 scfiltlo.dll!oox::xls::WorkbookFragment::finalizeImport             [j:\libo\my-master\sc\source\filter\oox\workbookfragment.cxx:477]
# 5 expwraplo.dll!sax_fastparser::FastSaxParserImpl::parseStream        [j:\libo\my-master\sax\source\fastparser\fastparser.cxx:823]
# 6 expwraplo.dll!sax_fastparser::FastSaxParser::parseStream            [j:\libo\my-master\sax\source\fastparser\fastparser.cxx:1378]
# 7 ooxlo.dll!oox::core::FastParser::parseStream                        [j:\libo\my-master\oox\source\core\fastparser.cxx:118]
# 8 ooxlo.dll!oox::core::FastParser::parseStream                        [j:\libo\my-master\oox\source\core\fastparser.cxx:126]
# 9 ooxlo.dll!oox::core::XmlFilterBase::importFragment                  [j:\libo\my-master\oox\source\core\xmlfilterbase.cxx:367]
#10 ooxlo.dll!oox::core::XmlFilterBase::importFragment                  [j:\libo\my-master\oox\source\core\xmlfilterbase.cxx:309]
#11 scfiltlo.dll!oox::xls::ExcelFilter::importDocument                  [j:\libo\my-master\sc\source\filter\oox\excelfilter.cxx:101]
#12 ooxlo.dll!oox::core::FilterBase::filter                             [j:\libo\my-master\oox\source\core\filterbase.cxx:486]
#13 scfiltlo.dll!oox::xls::ExcelFilter::filter                          [j:\libo\my-master\sc\source\filter\oox\excelfilter.cxx:155]
#14 sfxlo.dll!SfxObjectShell::ImportFrom                                [j:\libo\my-master\sfx2\source\doc\objstor.cxx:2263]
#15 sfxlo.dll!SfxObjectShell::DoLoad                                    [j:\libo\my-master\sfx2\source\doc\objstor.cxx:764]
#16 sfxlo.dll!SfxBaseModel::load                                        [j:\libo\my-master\sfx2\source\doc\sfxbasemodel.cxx:1831]
#17 sfxlo.dll!`anonymous namespace'::SfxFrameLoader_Impl::load          [j:\libo\my-master\sfx2\source\view\frmload.cxx:698]
#18 fwklo.dll!framework::LoadEnv::impl_loadContent                      [j:\libo\my-master\framework\source\loadenv\loadenv.cxx:1095]
#19 fwklo.dll!framework::LoadEnv::startLoading                          [j:\libo\my-master\framework\source\loadenv\loadenv.cxx:377]
Note: @0:03:19.666 in thread 9860
Note: memory was allocated here:
Note: # 0 replace_RtlAllocateHeap                                             [d:\drmemory_package\common\alloc_replace.c:3770]
Note: # 1 MSVCR120.dll!memcmp                                                +0x34c    (0x65fa11f9 <MSVCR120.dll+0x111f9>)
Note: # 2 MSVCR120.dll!calloc_crt                                            +0x13     (0x65fa121c <MSVCR120.dll+0x1121c>)
Note: # 3 MSVCP120.dll!Cnd_init                                              +0x14     (0x6361eb12 <MSVCP120.dll+0x2eb12>)
Note: # 4 comphelper.dll!std::_Ref_count_obj<>::_Ref_count_obj<><>            [c:\program files (x86)\microsoft visual studio 12.0\vc\include\memory:932]
Note: # 5 comphelper.dll!std::make_shared<>                                   [c:\program files (x86)\microsoft visual studio 12.0\vc\include\memory:1003]
Note: # 6 comphelper.dll!comphelper::ThreadPool::createThreadTaskTag          [j:\libo\my-master\comphelper\source\misc\threadpool.cxx:224]
Note: # 7 scfiltlo.dll!oox::xls::`anonymous namespace'::importSheetFragments  [j:\libo\my-master\sc\source\filter\oox\workbookfragment.cxx:318]
Note: # 8 scfiltlo.dll!oox::xls::WorkbookFragment::finalizeImport             [j:\libo\my-master\sc\source\filter\oox\workbookfragment.cxx:477]
Note: # 9 expwraplo.dll!sax_fastparser::FastSaxParserImpl::parseStream        [j:\libo\my-master\sax\source\fastparser\fastparser.cxx:823]
Note: #10 expwraplo.dll!sax_fastparser::FastSaxParser::parseStream            [j:\libo\my-master\sax\source\fastparser\fastparser.cxx:1378]
Note: #11 ooxlo.dll!oox::core::FastParser::parseStream                        [j:\libo\my-master\oox\source\core\fastparser.cxx:118]

Error #71: INVALID HEAP ARGUMENT: allocated with Windows API layer, freed with C library layer
# 0 replace_free                                                        [d:\drmemory_package\common\alloc_replace.c:2706]
# 1 MSVCP120.dll!Mtx_destroy                                           +0x21     (0x6361efc8 <MSVCP120.dll+0x2efc8>)
# 2 comphelper.dll!std::_Ref_count_obj<>::_Destroy                      [c:\program files (x86)\microsoft visual studio 12.0\vc\include\memory:944]
# 3 scfiltlo.dll!oox::xls::`anonymous namespace'::importSheetFragments  [j:\libo\my-master\sc\source\filter\oox\workbookfragment.cxx:346]
# 4 scfiltlo.dll!oox::xls::WorkbookFragment::finalizeImport             [j:\libo\my-master\sc\source\filter\oox\workbookfragment.cxx:477]
# 5 expwraplo.dll!sax_fastparser::FastSaxParserImpl::parseStream        [j:\libo\my-master\sax\source\fastparser\fastparser.cxx:823]
# 6 expwraplo.dll!sax_fastparser::FastSaxParser::parseStream            [j:\libo\my-master\sax\source\fastparser\fastparser.cxx:1378]
# 7 ooxlo.dll!oox::core::FastParser::parseStream                        [j:\libo\my-master\oox\source\core\fastparser.cxx:118]
# 8 ooxlo.dll!oox::core::FastParser::parseStream                        [j:\libo\my-master\oox\source\core\fastparser.cxx:126]
# 9 ooxlo.dll!oox::core::XmlFilterBase::importFragment                  [j:\libo\my-master\oox\source\core\xmlfilterbase.cxx:367]
#10 ooxlo.dll!oox::core::XmlFilterBase::importFragment                  [j:\libo\my-master\oox\source\core\xmlfilterbase.cxx:309]
#11 scfiltlo.dll!oox::xls::ExcelFilter::importDocument                  [j:\libo\my-master\sc\source\filter\oox\excelfilter.cxx:101]
#12 ooxlo.dll!oox::core::FilterBase::filter                             [j:\libo\my-master\oox\source\core\filterbase.cxx:486]
#13 scfiltlo.dll!oox::xls::ExcelFilter::filter                          [j:\libo\my-master\sc\source\filter\oox\excelfilter.cxx:155]
#14 sfxlo.dll!SfxObjectShell::ImportFrom                                [j:\libo\my-master\sfx2\source\doc\objstor.cxx:2263]
#15 sfxlo.dll!SfxObjectShell::DoLoad                                    [j:\libo\my-master\sfx2\source\doc\objstor.cxx:764]
#16 sfxlo.dll!SfxBaseModel::load                                        [j:\libo\my-master\sfx2\source\doc\sfxbasemodel.cxx:1831]
#17 sfxlo.dll!`anonymous namespace'::SfxFrameLoader_Impl::load          [j:\libo\my-master\sfx2\source\view\frmload.cxx:698]
#18 fwklo.dll!framework::LoadEnv::impl_loadContent                      [j:\libo\my-master\framework\source\loadenv\loadenv.cxx:1095]
#19 fwklo.dll!framework::LoadEnv::startLoading                          [j:\libo\my-master\framework\source\loadenv\loadenv.cxx:377]
Note: @0:03:19.666 in thread 9860
Note: memory was allocated here:
Note: # 0 replace_RtlAllocateHeap                                             [d:\drmemory_package\common\alloc_replace.c:3770]
Note: # 1 MSVCR120.dll!memcmp                                                +0x34c    (0x65fa11f9 <MSVCR120.dll+0x111f9>)
Note: # 2 MSVCR120.dll!calloc_crt                                            +0x13     (0x65fa121c <MSVCR120.dll+0x1121c>)
Note: # 3 MSVCP120.dll!Mtx_init                                              +0x14     (0x6361efee <MSVCP120.dll+0x2efee>)
Note: # 4 comphelper.dll!std::_Ref_count_obj<>::_Ref_count_obj<><>            [c:\program files (x86)\microsoft visual studio 12.0\vc\include\memory:932]
Note: # 5 comphelper.dll!std::make_shared<>                                   [c:\program files (x86)\microsoft visual studio 12.0\vc\include\memory:1003]
Note: # 6 comphelper.dll!comphelper::ThreadPool::createThreadTaskTag          [j:\libo\my-master\comphelper\source\misc\threadpool.cxx:224]
Note: # 7 scfiltlo.dll!oox::xls::`anonymous namespace'::importSheetFragments  [j:\libo\my-master\sc\source\filter\oox\workbookfragment.cxx:318]
Note: # 8 scfiltlo.dll!oox::xls::WorkbookFragment::finalizeImport             [j:\libo\my-master\sc\source\filter\oox\workbookfragment.cxx:477]
Note: # 9 expwraplo.dll!sax_fastparser::FastSaxParserImpl::parseStream        [j:\libo\my-master\sax\source\fastparser\fastparser.cxx:823]
Note: #10 expwraplo.dll!sax_fastparser::FastSaxParser::parseStream            [j:\libo\my-master\sax\source\fastparser\fastparser.cxx:1378]
Note: #11 ooxlo.dll!oox::core::FastParser::parseStream                        [j:\libo\my-master\oox\source\core\fastparser.cxx:118]
Comment 8 Kohei Yoshida 2017-01-23 22:52:51 UTC 
I just chatted with Michael (Meeks), and he suspects that this one probably shares the same root causes as Bug 102688.
Comment 9 Kohei Yoshida 2017-01-23 23:19:43 UTC 
I'm pretty sure that ultimately the instability is caused by a RuntimeException being thrown at vcl/source/gdi/virdev.cxx around line 150, which is in turn caused by CreateCompatibleDC(...) in WinSalInstance::CreateVirtualDevice(...) returning a null DC pointer.
Comment 10 Kohei Yoshida 2017-01-23 23:22:07 UTC 
Created attachment 130642 [details]
call stack on RuntimeException in virdev.cxx.

Here is the call stack I get when the first RuntimeException gets thrown. Note that this document contains 4233 drawing objects, and we seem to be creating a VirtualDevice instance for every single one of them, and ultimately hits Windows' resource limit.
Comment 11 Commit Notification 2017-01-26 01:51:57 UTC 
Kohei Yoshida committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=b41186a2fc49e440890b8c86e5367352ffaf9cd6

tdf#103927: Share single standard VirtualDevice instance.

It will be available in 5.4.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Kohei Yoshida 2017-01-26 02:40:34 UTC 
I call this fixed.
Comment 13 Timur 2017-01-30 09:05:16 UTC 
Please comment on backport.
Comment 14 Kohei Yoshida 2017-01-30 22:22:17 UTC 
(In reply to Timur from comment #13)
> Please comment on backport.

They are on their way.

5.3: https://gerrit.libreoffice.org/#/c/33640/
5.2: https://gerrit.libreoffice.org/#/c/33641/
Comment 15 Commit Notification 2017-02-01 13:30:41 UTC 
Kohei Yoshida committed a patch related to this issue.
It has been pushed to "libreoffice-5-3":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=48168ced891de8e4eca6fd1eb490c452534fccd2&h=libreoffice-5-3

tdf#103927: Share single standard VirtualDevice instance.

It will be available in 5.3.1.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 16 Commit Notification 2017-02-01 13:52:11 UTC 
Kohei Yoshida committed a patch related to this issue.
It has been pushed to "libreoffice-5-2":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=fc1d2236eb9ae89fb8f2ece013bc9396bafb1db1&h=libreoffice-5-2

tdf#103927: Share single standard VirtualDevice instance.

It will be available in 5.2.6.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 17 vihsa 2017-09-03 02:46:21 UTC 
endless loop
5.4.0.0.alpha0+ [ build id: 3902bb7 ]
6.0.0.0.alpha0+ [ build id: aa084cc ]
5.3.0.0.alpha1+ [ build id: 4136757 ]
5.2.0.0.alpha0+ [ build id: f6a74ce ]

os: android 5.1
device: lyf flame 3 [ ls-4001 ]