Bug 118620 - PivotTable: the last row of a worksheet can't be used in the PivotTable (crash)
Summary: PivotTable: the last row of a worksheet can't be used in the PivotTable (crash)
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
6.0.5.2 release
Hardware: All All
: highest critical
Assignee: Michael Meeks
URL:
Whiteboard: target:6.2.0 target:6.1.0.2 target:6....
Keywords: bibisected, bisected, regression
Depends on:
Blocks: Pivot-Table
  Show dependency treegraph
 
Reported: 2018-07-08 08:58 UTC by kabilo
Modified: 2018-07-20 20:47 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Screenshot with the crash message. (9.79 KB, image/png)
2018-07-08 13:56 UTC, m_a_riosv
Details

Note You need to log in before you can comment on or make changes to this bug.
Description kabilo 2018-07-08 08:58:17 UTC
Description:
the last row of a worksheet can't be used when create a new PivotTable. You receive error message.

Actual Results:
 

Expected Results:
 


Reproducible: Always


User Profile Reset: No



Additional Info:
Comment 1 m_a_riosv 2018-07-08 13:56:11 UTC
Created attachment 143388 [details]
Screenshot with the crash message.

I'm not sure if it's the same.

Selecting A1:B10485756 (from first to last row)
Menu/Insert/Pivot table
Accept current range.
LibreOffice crash with attached screenshot message.
"multi_typevector::get_block_position#816:block position not found|
(logical pos=1048576, block size 2, logical size=1048576)"

First version with the issue
Version: 6.0.5.2 (x64)
Build ID: 54c8cbb85f300ac59db32fe8a675ff7683cd5a16
CPU threads: 4; OS: Windows 10.0; UI render: GL; 
Locale: es-ES (es_ES); Calc: CL

It's a regression, doesn't happen with:
Version: 5.2.7.1
Build ID: bf0fa7b86c7c0592941ede29fca6fafff642a948
CPU Threads: 4; OS Version: Windows 6.2; UI Render: GL; 
Locale: es-ES (es_ES); Calc: group
Comment 2 m_a_riosv 2018-07-08 13:59:52 UTC
I doesn't happen also with:
Version: 5.4.6.2 (x64)
Build ID: 4014ce260a04f1026ba855d3b8d91541c224eab8
CPU threads: 4; OS: Windows 6.19; UI render: GL; 
Locale: es-ES (es_ES); Calc: group

So I guess the regression it's with version 6.
Comment 3 Xisco Faulí 2018-07-08 18:01:48 UTC
> Selecting A1:B10485756 (from first to last row)
> Menu/Insert/Pivot table
> Accept current range.
> LibreOffice crash with attached screenshot message.
> "multi_typevector::get_block_position#816:block position not found|
> (logical pos=1048576, block size 2, logical size=1048576)"

Following these steps, it started to crash ( before it used to hang ) after

https://cgit.freedesktop.org/libreoffice/core/commit/?id=d4cd8677889ec3807c194ef5b462f8e031807e5b

author	Kohei Yoshida <kohei.yoshida@collabora.com>	2017-05-22 21:08:56 -0400
committer	Kohei Yoshida <libreoffice@kohei.us>	2017-05-24 05:19:40 +0200
commit d4cd8677889ec3807c194ef5b462f8e031807e5b (patch)
tree 4b53429527d6358d145feef9912d0460924cbde0
parent 7948e84091f37fbda75f524f20138d1171918e64 (diff)
tdf#107945: properly iterate over mtv during pivot cache loading.
This reduces the total time required for populating the pivot cache
by ~60%.

Bisected with: bibisect-linux64-6.0

Adding Cc: to Kohei Yoshida
Comment 4 kabilo 2018-07-08 18:19:09 UTC
48/5000
This bug exists for any range of rows (the last row is part of the range), for example A10485754:B10485756
Comment 5 Michael Meeks 2018-07-12 20:06:11 UTC
Some analysis:

(gdb) bt 5
#0  0x00007ffff6a60580 in __cxxabiv1::__cxa_throw(void*, std::type_info*, void (*)(void*)) (obj=0x33b4fc0, tinfo=0x7ffff6d449b8 <typeinfo for std::out_of_range>, dest=0x7ffff6a75cf0 <std::out_of_range::~out_of_range()>) at ../../../../libstdc++-v3/libsupc++/eh_throw.cc:75

#1  0x00007fffa8c94d77 in mdds::detail::throw_block_position_not_found(char const*, int, unsigned long, unsigned long, unsigned long) (method_sig=0x7fffaa4851fc "multi_type_vector::get_block_position", line=816, pos=1048576, block_size=2, container_size=1048576) at /data/opt/libreoffice/master/workdir/UnpackedTarball/mdds/include/mdds/multi_type_vector_def.inl:43


(gdb) p os.str()
$2 = {static npos = 18446744073709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x2d79698 "multi_type_vector::get_block_position#816: block position not found! (logical pos=1048576, block size=2, logical size=1048576)"}}


#2  0x00007fffa8d44f41 in mdds::multi_type_vector<mdds::mtv::custom_block_func3<mdds::mtv::default_element_block<52, svl::SharedString>, mdds::mtv::noncopyable_managed_element_block<53, EditTextObject>, mdds::mtv::noncopyable_managed_element_block<54, ScFormulaCell> >, sc::CellStoreEvent>::get_block_position(mdds::__mtv::const_iterator_base<mdds::multi_type_vector<mdds::mtv::custom_block_func3<mdds::mtv::default_element_block<52, svl::SharedString>, mdds::mtv::noncopyable_managed_element_block<53, EditTextObject>, mdds::mtv::noncopyable_managed_element_block<54, ScFormulaCell> >, sc::CellStoreEvent>::const_iterator_trait, mdds::__mtv::private_data_forward_update<mdds::__mtv::iterator_value_node<unsigned long, mdds::mtv::base_element_block> >, mdds::__mtv::iterator_base<mdds::multi_type_vector<mdds::mtv::custom_block_func3<mdds::mtv::default_element_block<52, svl::SharedString>, mdds::mtv::noncopyable_managed_element_block<53, EditTextObject>, mdds::mtv::noncopyable_managed_element_block<54, ScFormulaCell> >, sc::CellStoreEvent>::iterator_trait, mdds::__mtv::private_data_forward_update<mdds::__mtv::iterator_value_node<unsigned long, mdds::mtv::base_element_block> > > > const&, unsigned long, unsigned long&, unsigned long&) const (this=0x1e80b90, pos_hint=..., pos=1048576, start_row=@0x7fffffff65f0: 1048576, block_index=@0x7fffffff65e8: 1) at /data/opt/libreoffice/master/workdir/UnpackedTarball/mdds/include/mdds/multi_type_vector_def.inl:816
#3  0x00007fffa8da40b0 in mdds::multi_type_vector<mdds::mtv::custom_block_func3<mdds::mtv::default_element_block<52, svl::SharedString>, mdds::mtv::noncopyable_managed_element_block<53, EditTextObject>, mdds::mtv::noncopyable_managed_element_block<54, ScFormulaCell> >, sc::CellStoreEvent>::position(mdds::__mtv::const_iterator_base<mdds::multi_type_vector<mdds::mtv::custom_block_func3<mdds::mtv::default_element_block<52, svl::SharedString>, mdds::mtv::noncopyable_managed_element_block<53, EditTextObject>, mdds::mtv::noncopyable_managed_element_block<54, ScFormulaCell> >, sc::CellStoreEvent>::const_iterator_trait, mdds::__mtv::private_data_forward_update<mdds::__mtv::iterator_value_node<unsigned long, mdds::mtv::base_element_block> >, mdds::__mtv::iterator_base<mdds::multi_type_vector<mdds::mtv::custom_block_func3<mdds::mtv::default_element_block<52, svl::SharedString>, mdds::mtv::noncopyable_managed_element_block<53, EditTextObject>, mdds::mtv::noncopyable_managed_element_block<54, ScFormulaCell> >, sc::CellStoreEvent>::iterator_trait, mdds::__mtv::private_data_forward_update<mdds::__mtv::iterator_value_node<unsigned long, mdds::mtv::base_element_block> > > > const&, unsigned long) const (this=0x1e80b90, pos_hint=..., pos=1048576) at /data/opt/libreoffice/master/workdir/UnpackedTarball/mdds/include/mdds/multi_type_vector_def.inl:1542
#4  0x00007fffa8e971b9 in sc::ColumnIterator::ColumnIterator(mdds::multi_type_vector<mdds::mtv::custom_block_func3<mdds::mtv::default_element_block<52, svl::SharedString>, mdds::mtv::noncopyable_managed_element_block<53, EditTextObject>, mdds::mtv::noncopyable_managed_element_block<54, ScFormulaCell> >, sc::CellStoreEvent> const&, int, int) (this=0x33ec3c0, rCells=..., nRow1=1048573, nRow2=1048575) at /data/opt/libreoffice/master/sc/source/core/data/columniterator.cxx:174


171	
172	ColumnIterator::ColumnIterator( const CellStoreType& rCells, SCROW nRow1, SCROW nRow2 ) :
173	    maPos(rCells.position(nRow1)),
174	    maPosEnd(rCells.position(maPos.first, nRow2+1))
175	{
176	}
177	

(gdb) p nRow2
$4 = 1048575
(gdb) p maPos.first
$5 = {<mdds::__mtv::iterator_common_base<mdds::multi_type_vector<mdds::mtv::custom_block_func3<mdds::mtv::default_element_block<52, svl::SharedString>, mdds::mtv::noncopyable_managed_element_block<53, EditTextObject>, mdds::mtv::noncopyable_managed_element_block<54, ScFormulaCell> >, sc::CellStoreEvent>::const_iterator_trait>> = {m_cur_node = {type = 52, position = 1048573, size = 3, data = 0x339c1b0, __private_data = {block_index = 1}}, m_pos = {<__gnu_debug::_Safe_iterator_base> = {_M_sequence = 0x1e80bb0, _M_version = 3, _M_prior = 0x33ec410, _M_next = 0x0}, _M_current = {_M_current = 0x33a0500}}, m_end = {<__gnu_debug::_Safe_iterator_base> = {_M_sequence = 0x1e80bb0, _M_version = 3, _M_prior = 0x0, _M_next = 0x33ec3e8}, _M_current = {_M_current = 0x33a0510}}}, <No data fields>}
(gdb) p nRow1
$6 = 1048573

#5  0x00007fffa8e7b1a8 in _ZN4o3tl11make_uniqueIN2sc14ColumnIteratorEJRKN4mdds17multi_type_vectorINS3_3mtv18custom_block_func3INS5_21default_element_blockILi52EN3svl12SharedStringEEENS5_33noncopyable_managed_element_blockILi53E14EditTextObjectEENSB_ILi54E13ScFormulaCellEEEENS1_14CellStoreEventEEERiSL_EEENSt9enable_ifIXntsr3std8is_arrayIT_EE5valueESt10unique_ptrISN_St14default_deleteISN_EEE4typeEDpOT0_ (args=..., args=@0x7fffffff66f4: 1048573, args=@0x7fffffff66f0: 1048575) at /data/opt/libreoffice/master/include/o3tl/make_unique.hxx:29
#6  0x00007fffa8e707f6 in ScColumn::GetColumnIterator(int, int) const (this=0x1e80ab0, nRow1=1048573, nRow2=1048575) at /data/opt/libreoffice/master/sc/source/core/data/column4.cxx:1639

1634	std::unique_ptr<sc::ColumnIterator> ScColumn::GetColumnIterator( SCROW nRow1, SCROW nRow2 ) const
1635	{
1636	    if (!ValidRow(nRow1) || !ValidRow(nRow2) || nRow1 > nRow2)
1637	        return std::unique_ptr<sc::ColumnIterator>();
1638	
1639	    return o3tl::make_unique<sc::ColumnIterator>(maCells, nRow1, nRow2);
1640	}
1641	
1642	void ScColumn::EnsureFormulaCellResults( SCROW nRow1, SCROW nRow2 )
1643	{
(gdb) p nRow2
$7 = 1048575
(gdb) p nRow1
$8 = 1048573

#7  0x00007fffa92ffd61 in ScTable::GetColumnIterator(short, int, int) const (this=0x1e81880, nCol=0, nRow1=1048573, nRow2=1048575) at /data/opt/libreoffice/master/sc/source/core/data/table7.cxx:430
#8  0x00007fffa8fe8ae7 in ScDocument::GetColumnIterator(short, short, int, int) const (this=0x1e75800, nTab=0, nCol=0, nRow1=1048573, nRow2=1048575) at /data/opt/libreoffice/master/sc/source/core/data/document10.cxx:920
#9  0x00007fffa90fe05e in (anonymous namespace)::initColumnFromDoc((anonymous namespace)::InitDocData&, (anonymous namespace)::InitColumnData&) (rDocData=..., rColData=...) at /data/opt/libreoffice/master/sc/source/core/data/dpcache.cxx:394
#10 0x00007fffa90fd89e in ScDPCache::InitFromDoc(ScDocument*, ScRange const&) (this=0x33b4470, pDoc=0x1e75800, rRange=...) at /data/opt/libreoffice/master/sc/source/core/data/dpcache.cxx:598
#11 0x00007fffa90522b0 in ScDPCollection::SheetCaches::getCache(ScRange const&, ScDPDimensionSaveData const*) (this=0x3459400, rRange=..., pDimData=0x0) at /data/opt/libreoffice/master/sc/source/core/data/dpobject.cxx:2977
#12 0x00007fffa90f99ab in ScSheetSourceDesc::CreateCache(ScDPDimensionSaveData const*) const (this=0x33b5980, pDimData=0x0) at /data/opt/libreoffice/master/sc/source/core/data/dpshttab.cxx:305
#13 0x00007fffa9042e17 in ScDPObject::GetTableData() (this=0x34530a8) at /data/opt/libreoffice/master/sc/source/core/data/dpobject.cxx:717
#14 0x00007fffa9042783 in ScDPObject::CreateObjects() (this=0x34530a8) at /data/opt/libreoffice/master/sc/source/core/data/dpobject.cxx:754
#15 0x00007fffa904e360 in ScDPObject::FillOldParam(ScPivotParam&) const (this=0x34530a8, rParam=...) at /data/opt/libreoffice/master/sc/source/core/data/dpobject.cxx:2309
#16 0x00007fffa9b29a7e in ScPivotLayoutDialog::ScPivotLayoutDialog(SfxBindings*, SfxChildWindow*, vcl::Window*, ScViewData*, ScDPObject const*, bool) (this=0x3452c00, pSfxBindings=0x1fd66f0, pChildWindow=0x2d2f330, pParent=0x1fd8160, pViewData=0x1fe3288, pPivotTableObject=0x2d2f090, bNewPivotTable=true) at /data/opt/libreoffice/master/sc/source/ui/dbgui/PivotLayoutDialog.cxx:152
#17 0x00007fffaa2bf445 in VclPtr<ScPivotLayoutDialog>::Create<SfxBindings*&, SfxChildWindow*&, vcl::Window*&, ScViewData*, ScDPObject*&, bool>(SfxBindings*&, SfxChildWindow*&, vcl::Window*&, ScViewData*&&, ScDPObject*&, bool&&) (arg=@0x7fffffff86b0: 0x1fd66f0, arg=@0x7fffffff86a8: 0x2d2f330, arg=@0x7fffffff86a0: 0x1fd8160, arg=@0x7fffffff8318: 0x1fe3288, arg=@0x1fe3838: 0x2d2f090, arg=@0x7fffffff8317: true) at /data/opt/libreoffice/master/include/vcl/vclptr.hxx:131
#18 0x00007fffaa2b8e32 in ScTabViewShell::CreateRefDialog(SfxBindings*, SfxChildWindow*, SfxChildWinInfo const*, vcl::Window*, unsigned short) (this=0x1fe31c0, pB=0x1fd66f0, pCW=0x2d2f330, pInfo=0x7fffffff8c68, pParent=0x1fd8160, nSlotId=26151) at /data/opt/libreoffice/master/sc/source/ui/view/tabvwshc.cxx:422
#19 0x00007fffaa241954 in ScPivotLayoutWrapper::ScPivotLayoutWrapper(vcl::Window*, unsigned short, SfxBindings*, SfxChildWinInfo const*) (this=0x2d2f330, pParentP=0x1fd8160, nId=26151, p=0x1fd66f0, pInfo=0x7fffffff8c68) at /data/opt/libreoffice/master/sc/source/ui/view/reffact.cxx:110
#20 0x00007fffaa23ec9c in ScPivotLayoutWrapper::CreateImpl(vcl::Window*, unsigned short, SfxBindings*, SfxChildWinInfo*) (pParent=0x1fd8160, nId=26151, pBindings=0x1fd66f0, pInfo=0x7fffffff8c68) at /data/opt/libreoffice/master/sc/source/ui/view/reffact.cxx:40
#21 0x00007ffff1a14161 in SfxChildWindow::CreateChildWindow(unsigned short, vcl::Window*, SfxBindings*, SfxChildWinInfo const&) (nId=26151, pParent=0x1fd8160, pBindings=0x1fd66f0, rInfo=...) at /data/opt/libreoffice/master/sfx2/source/appl/childwin.cxx:256
#22 0x00007ffff1ac3223 in SfxWorkWindow::CreateChildWin_Impl(SfxChildWin_Impl*, bool) (this=0x1fdbd60, pCW=0x25273a0, bSetFocus=true) at /data/opt/libreoffice/master/sfx2/source/appl/workwin.cxx:1337
#23 0x00007ffff1ac5cca in SfxWorkWindow::ToggleChildWindow_Impl(unsigned short, bool) (this=0x1fdbd60, nId=26151, bSetFocus=true) at /data/opt/libreoffice/master/sfx2/source/appl/workwin.cxx:1848
#24 0x00007ffff1ac6c76 in SfxWorkWindow::SetChildWindow_Impl(unsigned short, bool, bool) (this=0x1fdbd60, nId=26151, bOn=true, bSetFocus=true) at /data/opt/libreoffice/master/sfx2/source/appl/workwin.cxx:2046
#25 0x00007ffff206f2eb in SfxViewFrame::SetChildWindow(unsigned short, bool, bool) (this=0x1fd47d0, nId=26151, bOn=true, bSetFocus=true) at /data/opt/libreoffice/master/sfx2/source/view/viewfrm.cxx:3072
#26 0x00007fffa9a2eb15 in ScModule::SetRefDialog(unsigned short, bool, SfxViewFrame*) (this=0x1e5cfb0, nId=26151, bVis=true, pViewFrm=0x1fd47d0) at /data/opt/libreoffice/master/sc/source/ui/app/scmod.cxx:1528
#27 0x00007fffaa10a692 in ScCellShell::ExecuteDataPilotDialog() (this=0x2004c90) at /data/opt/libreoffice/master/sc/source/ui/view/cellsh1.cxx:2928

The root cause is this:

174	    maPosEnd(rCells.position(maPos.first, nRow2+1))

where nRow2 == MAXROW-1 so nRow2+1 = MAXROW which blows the container.
Comment 6 Telesto 2018-07-13 06:58:58 UTC
https://gerrit.libreoffice.org/#/c/57358/
Comment 7 Commit Notification 2018-07-13 08:28:44 UTC
Michael Meeks committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=d3387b38fe0eea3fb7ac630c026f02986e8dafc4

tdf#118620 - avoid out of bounds iterator for end of sheet pivots.

It will be available in 6.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 8 Commit Notification 2018-07-13 08:28:52 UTC
Michael Meeks committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=fdef3ecc1ddef7adcf51df465048613b7b3e4ae6

tdf#118620 - unit test.

It will be available in 6.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 9 Michael Meeks 2018-07-13 09:30:31 UTC
Needs a follow-on cleanup in mdds for master really.
Comment 10 Commit Notification 2018-07-13 17:55:00 UTC
Michael Meeks committed a patch related to this issue.
It has been pushed to "libreoffice-6-1":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=4e4421325d406cc555f15c2b8b5bbab443eb6a7d&h=libreoffice-6-1

tdf#118620 - avoid out of bounds iterator for end of sheet pivots.

It will be available in 6.1.0.2.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 11 Commit Notification 2018-07-13 17:55:11 UTC
Michael Meeks committed a patch related to this issue.
It has been pushed to "libreoffice-6-0":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=4a8ecfd9fdb6d8460bc7eee8773580d4ae550b18&h=libreoffice-6-0

tdf#118620 - avoid out of bounds iterator for end of sheet pivots.

It will be available in 6.0.7.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Xisco Faulí 2018-07-14 10:57:15 UTC
@Michael Meeks, should it be backported to 6.0.6 branch as well ?
Comment 13 Xisco Faulí 2018-07-16 18:20:05 UTC
Verified in

Version: 6.2.0.0.alpha0+
Build ID: f543b6a0ac6cf30922c1a1ae9bfce1d605f1d4f1
CPU threads: 4; OS: Linux 4.13; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); Calc: group threaded

@Michael Meeks, Thanks for fixing this!!
Comment 14 Commit Notification 2018-07-20 20:47:22 UTC
Michael Meeks committed a patch related to this issue.
It has been pushed to "libreoffice-6-0-6":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=9ae56cdf78af1ff1b5c5178168b4aaa9a37c3e04&h=libreoffice-6-0-6

tdf#118620 - avoid out of bounds iterator for end of sheet pivots.

It will be available in 6.0.6.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.