Bug 103632 - Segfault during sort in ScColumn::GetCellValue
Summary: Segfault during sort in ScColumn::GetCellValue
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
5.2.0.0.alpha1
Hardware: All Linux (All)
: medium critical
Assignee: Not Assigned
URL:
Whiteboard: target:5.3.0
Keywords: bibisectRequest, haveBacktrace, regression
Depends on:
Blocks:
 
Reported: 2016-11-02 00:05 UTC by Jim Avera
Modified: 2016-11-11 15:42 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments
test case .ods (290.70 KB, application/vnd.oasis.opendocument.spreadsheet)
2016-11-02 00:05 UTC, Jim Avera
Details
gdb stacktrace at segfault (37.76 KB, text/plain)
2016-11-02 00:06 UTC, Jim Avera
Details
bt with symbols (18.61 KB, text/plain)
2016-11-02 20:43 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jim Avera 2016-11-02 00:05:11 UTC
Description:
After some macro code runs (which I can't pin down), sort segfaults. 

Unfortunatly the unknown precursor operations are in BASIC macro code, so you have to enable macros to reproduce the problem.  I could not repro the crash doing a "manual" sort which seems equivalent to what the macro code is doing.

Fortunately I have a test case which causes the segfault every time with a single button click. 

In any case, LO should never segfault, even if BASIC macros are mis-coded.

I'll attach the gdb backtrace from running with --backtrace.  It starts with this:

Thread 1 "soffice.bin" received signal SIGSEGV, Segmentation fault.
0x00007fffb11852ae in mdds::multi_type_vector<mdds::mtv::custom_block_func3<mdds::mtv::default_element_block<52, svl::SharedString>, mdds::mtv::noncopyable_managed_element_block<53, EditTextObject>, mdds::mtv::noncopyable_managed_element_block<54, ScFormulaCell> >, sc::CellStoreEvent>::position(unsigned long) const () from program/../program/libsclo.so
#0  0x00007fffb11852ae in mdds::multi_type_vector<mdds::mtv::custom_block_func3<mdds::mtv::default_element_block<52, svl::SharedString>, mdds::mtv::noncopyable_managed_element_block<53, EditTextObject>, mdds::mtv::noncopyable_managed_element_block<54, ScFormulaCell> >, sc::CellStoreEvent>::position(unsigned long) const () from  program/libsclo.so
#1  0x00007fffb11778e0 in ScColumn::GetCellValue(int) const () from program/../program/libsclo.so
#2  0x00007fffb12f1171 in ScTable::Compare(int, int) const () from program/../program/libsclo.so
#3  0x00007fffb12fa22a in ScTable::Sort(ScSortParam const&, bool, bool, ScProgress*, sc::ReorderParam*) () from program/libsclo.so



Steps to Reproduce:
1. Start LO on the attached test.ods. 

2. [if necessary] Tools->Options->LibreOffice->Security->Macro Security... set to "Medium" and restart.

3. Enable macro execution when prompted.

4. Click the "MO, TV%" button in the top middle (near column P or Q).  This is supposed to sort the sheet by Month and then TV%


Actual Results:  
Libre Office crash dialog appears

Expected Results:
No crash


Reproducible: Always

User Profile Reset: No

Additional Info:


User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0
Comment 1 Jim Avera 2016-11-02 00:05:52 UTC
Created attachment 128416 [details]
test case .ods
Comment 2 Jim Avera 2016-11-02 00:06:28 UTC
Created attachment 128417 [details]
gdb stacktrace at segfault
Comment 3 Xisco Faulí 2016-11-02 08:33:01 UTC
Confirmed in

Version: 5.3.0.0.alpha1+
Build ID: 64d3bd38c9242c1cc114f6e68bf13475ef679b73
CPU Threads: 4; OS Version: Linux 4.2; UI Render: default; VCL: gtk2; Layout Engine: old; 
Locale: ca-ES (ca_ES.UTF-8); Calc: group

and

Version: 5.2.0.0.alpha1+
Build ID: 5b168b3fa568e48e795234dc5fa454bf24c9805e
CPU Threads: 4; OS Version: Linux 4.2; UI Render: default; 
Locale: ca-ES (ca_ES.UTF-8)

but not in

Version: 5.0.0.0.alpha1+
Build ID: 0db96caf0fcce09b87621c11b584a6d81cc7df86
Locale: ca-ES (ca_ES.UTF-8)
Comment 4 Julien Nabet 2016-11-02 20:43:04 UTC
Created attachment 128452 [details]
bt with symbols

On pc Debian x86-64 with master sources updated today, I could reproduce the crash
However, I had another bt that the one already attached.
Comment 5 Michael Meeks 2016-11-04 20:57:38 UTC
A bogus field/column offset is passed in:

/usr/include/c++/4.8/debug/vector:353:error: attempt to subscript container 
    with out-of-bounds index 12303, but container only holds 1024 elements.

(gdb) p nMaxSorts.maKeyState[0]
Attempt to extract a component of a value that is not a structure.
(gdb) p aSortParam.maKeyState[0]
[Thread 0x7fffb49a6700 (LWP 5820) exited]
$5 = (__gnu_cxx::__alloc_traits<std::allocator<ScSortKeyState> >::value_type &) @0x2c451520: {bDoSort = true, nField = 4, bAscending = false}
(gdb) p aSortParam.maKeyState[1]
$6 = (__gnu_cxx::__alloc_traits<std::allocator<ScSortKeyState> >::value_type &) @0x2c45152c: {bDoSort = true, nField = 2, bAscending = true}
(gdb) p aSortParam.maKeyState[2]
[Thread 0x7fff9ae9c700 (LWP 5823) exited]
$7 = (__gnu_cxx::__alloc_traits<std::allocator<ScSortKeyState> >::value_type &) @0x2c451538: {bDoSort = true, nField = 12303, bAscending = true}

It seems the field to sort by is completely bogus. I'm amazed this is a regression.
Comment 6 Michael Meeks 2016-11-04 21:31:51 UTC
Pushed something to at least slightly sanitize bogus OOB sorting data that happens to overcome this problem, I suspect though that it is still possible to use a range very much larger than the column storage provides:

https://gerrit.libreoffice.org/30576 tdf#103632 - sc: tolerate OOB sort fields from UNO API.
Comment 7 Michael Meeks 2016-11-10 19:25:34 UTC
Knocked up a unit test for this too as https://gerrit.libreoffice.org/30761 - not a perfect fix - but rather a silly macro bug =)
Comment 8 Michael Meeks 2016-11-11 15:42:04 UTC
Pushed fix + test.
Comment 9 Commit Notification 2016-11-11 15:42:41 UTC
Michael Meeks committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=b8fa4d1b2d135cabc9eb5162ca0c5af649f1cdaf

tdf#103632 - sc: tolerate OOB sort fields from UNO API.

It will be available in 5.3.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 10 Commit Notification 2016-11-11 15:42:48 UTC
Michael Meeks committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=99316fc0c830ae6c827d9b7f56bf057ff7731ed5

tdf#103632 - sc: OOB sort fields unit test.

It will be available in 5.3.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.