Steps to reproduce: 1. Open attachment 53869 [details] from bug 43263 2. Change to sheet MI_Zopa -> soffice.bin: /home/xisco/libreoffice/vcl/source/outdev/map.cxx:253: tools::Long ImplLogicToPixel(tools::Long, tools::Long, tools::Long, tools::Long): Assertion `nMapNum == 0 || std::abs(n) < std::numeric_limits<tools::Long>::max() / nMapNum / nDPI' failed. Unspecified Application Error Reproduced in Version: 7.4.0.0.alpha0+ / LibreOffice Community Build ID: 49ee1c889665c3539fa9a1c99a865a42fc08ee97 CPU threads: 8; OS: Linux 5.10; UI render: default; VCL: x11 Locale: es-ES (es_ES.UTF-8); UI: en-US Calc: threaded
https://gerrit.libreoffice.org/c/core/+/133145
Created attachment 179680 [details] bt with debug symbols On pc Debian x86-64 with master sources updated today, I could reproduce this.
If it can help: (gdb) frame 5 #5 0x00007f093849688c in OutputDevice::ImplLogicToDevicePixel (this=0x6097d30, rLogicRect=...) at vcl/source/outdev/map.cxx:419 419 rLogicRect.IsWidthEmpty() ? 0 : ImplLogicToPixel( rLogicRect.Right()+maMapRes.mnMapOfsX, mnDPIX, maMapRes.mnMapScNumX, maMapRes.mnMapScDenomX )+mnOutOffX+mnOutOffOrigX, (gdb) p rLogicRect $1 = (const tools::Rectangle &) @0x7ffd946c45d8: {static RECT_EMPTY = -32767, nLeft = 0, nTop = 0, nRight = 3913623887751, nBottom = 473490318} (gdb) frame 6 #6 0x00007f0937f35f2b in vcl::Window::Invalidate (this=0x6094da0, rRect=..., nFlags=InvalidateFlags::NoErase) at vcl/source/window/paint.cxx:1158 1158 tools::Rectangle aRect = pOutDev->ImplLogicToDevicePixel( rRect ); (gdb) p rRect $2 = (const tools::Rectangle &) @0x7ffd946c45d8: {static RECT_EMPTY = -32767, nLeft = 0, nTop = 0, nRight = 3913623887751, nBottom = 473490318} (gdb) frame 7 #7 0x00007f093bca7466 in SdrPaintView::InvalidateOneWin (this=0x45b81c0, rDevice=..., rRect=...) at svx/source/svdraw/svdpntv.cxx:870 870 rDevice.GetOwnerWindow()->Invalidate(rRect, InvalidateFlags::NoErase); (gdb) p rRect $3 = (const tools::Rectangle &) @0x7ffd946c45d8: {static RECT_EMPTY = -32767, nLeft = 0, nTop = 0, nRight = 3913623887751, nBottom = 473490318} (gdb) frame 8 #8 0x00007f093bca73a7 in SdrPaintView::InvalidateAllWin (this=0x45b81c0, rRect=...) at svx/source/svdraw/svdpntv.cxx:855 855 InvalidateOneWin(rOutDev, aRect); (gdb) p rRect $5 = (const tools::Rectangle &) @0x7ffd946c4648: {static RECT_EMPTY = -32767, nLeft = 0, nTop = 0, nRight = 3913623887751, nBottom = 473490318} (gdb) frame 9 #9 0x00007f093bc8cd47 in SdrPageView::InvalidateAllWin (this=0x5f8a790) at svx/source/svdraw/svdpagv.cxx:201 201 GetView().InvalidateAllWin(aRect);
Also: (gdb) p GetPage()->GetWidth() $6 = 36991431 (gdb) p GetPage() $7 = (SdrPage *) 0x36ae720 (gdb) p *GetPage() $8 = {<SdrObjList> = {_vptr$SdrObjList = 0x7f0923f00c50 <vtable for ScDrawPage+16>, maSdrObjListOutRect = {static RECT_EMPTY = -32767, nLeft = 344, nTop = 819, nRight = 3913623887751, nBottom = 217429}, maSdrObjListSnapRect = {static RECT_EMPTY = -32767, nLeft = 344, nTop = 820, nRight = 33834, nBottom = 217329}, maList = std::__debug::deque with 101 elements = {0xa1a7870, 0xa17f510, 0xa2937d0, 0xa3217c0, 0xa32edd0, 0xa32de80, 0xa33b9a0, 0xa2b7010, 0xa2c1370, 0xa2c2230, 0xa22a910, 0xa224640, 0xa2b9730, 0xa2bad30, 0xa200110, 0xa200ae0, 0xa209400, 0xa2082f0, 0xa201d30, 0xa21adb0, 0xa305af0, 0xa2cb450, 0xa3106d0, 0xa2d3900, 0xa2ed0e0, 0xa302c70, 0xa2edee0, 0xa3026d0, 0xa2e1790, 0xa352a70, 0xa1b6fd0, 0xa352f50, 0xa36e890, 0xa377af0, 0xa36e1a0, 0xa1b11c0, 0xa361ea0, 0xa386760, 0xa385b20, 0xa38db10, 0xa385180, 0xa38c880, 0xa396ff0, 0xa395f80, 0xa394c60, 0xa3a4a00, 0xa3a40d0, 0xa3af160, 0xa3a96f0, 0xa3b7de0, 0xa3b7260, 0xa3ab410, 0xa3c0080, 0xa3be570, 0xa3c9260, 0xa3b6700, 0xa3b6ab0, 0xa3b5d10, 0xa3d2c00, 0xa3e1150, 0xa3f6e10, 0xa3f7860, 0xa40a0d0, 0xa3f72d0, 0xa3fa660, 0xa4248d0, 0xa40fcb0, 0xa421730, 0xa4341e0, 0xa415ec0, 0xa4449b0, 0xa428900, 0xa428e20, 0xa437ef0, 0xa4370b0, 0xa461f00, 0xa462630, 0xa46b3e0, 0xa462ee0, 0xa463250, 0xa43a490, 0xa471240, 0xa46c950, 0xa4902f0, 0xa4877f0, 0xa49b640, 0xa490890, 0xa4c6180, 0xa4b16d0, 0xa4e0960, 0xa4f16c0, 0xa4cc7c0, 0xa4ccf80, 0xa502060, 0xa4ffb10, 0xa50bec0, 0xa5031e0, 0xa511e30, 0xa512cf0, 0xa506c90, 0xa518ff0}, mxNavigationOrder = std::optional<std::__debug::vector<tools::WeakReference<SdrObject>>> [no contained value], mbObjOrdNumsDirty = true, mbRectsDirty = false, mbIsNavigationOrderDirty = false}, <cppu::OWeakObject> = {<com::sun::star::uno::XWeak> = {<com::sun::star::uno::XInterface> = { _vptr$XInterface = 0x7f0923f00d30 <vtable for ScDrawPage+240>}, <No data fields>}, m_refCount = 1, m_pWeakConnectionPoint = 0x0, m_pReserved = 0x0}, maPageUsers = std::__debug::vector of length 0, capacity 0, mpViewContact = std::unique_ptr<sdr::contact::ViewContact> = {get() = 0x3705950}, mrSdrModelFromSdrPage = @0x3722f70, mnWidth = 36991431, mnHeight = 473490318, mnBorderLeft = 0, mnBorderUpper = 0, mnBorderRight = 0, mnBorderLower = 0, mbBackgroundFullSize = false, mpLayerAdmin = std::unique_ptr<SdrLayerAdmin> = {get() = 0x368af70}, mpSdrPageProperties = std::unique_ptr<SdrPageProperties> = {get() = 0x35b4040}, mxUnoPage = uno::Reference to (ScPageObj *) 0x34be9e0, mpMasterPageDescriptor = std::unique_ptr<sdr::MasterPageDescriptor> = { get() = 0x0}, nPageNum = 4, mbMaster = false, mbInserted = true, mbObjectsNotPersistent = false, mbPageBorderOnlyLeftRight = false} (gdb) p GetPage()->GetHeight() $9 = 473490318
I added a breakpoint tools::Rectangle::Union(tools::Rectangle const&) at tools/source/generic/gen.cxx:107 and retrieved a bt when nRight has a value > 2147483647 I get this: (gdb) p nRight $1 = 663607481563 (gdb) p rRect $2 = (const tools::Rectangle &) @0xb78ff88: {static RECT_EMPTY = -32767, nLeft = 14921, nTop = 32153, nRight = 663607481563, nBottom = 34419} (gdb) bt #0 tools::Rectangle::Union(tools::Rectangle const&) (this=0x4af0708, rRect=...) at tools/source/generic/gen.cxx:107 #1 0x00007f47c0b4fbc9 in SdrObjList::RecalcRects() (this=0x4af0700) at svx/source/svdraw/svdpage.cxx:251 #2 0x00007f47c0b532f4 in SdrObjList::GetAllObjBoundRect() const (this=0x4af0700) at svx/source/svdraw/svdpage.cxx:746 #3 0x00007f47c0b78d26 in SdrPageView::InvalidateAllWin() (this=0xb35a6e0) at svx/source/svdraw/svdpagv.cxx:200 #4 0x00007f47c0b96293 in SdrPageView::SetLayerVisible(rtl::OUString const&, bool) (this=0xb35a6e0, rName="hidden", bShow=false) at include/svx/svdpagv.hxx:188 #5 0x00007f47c0b91ebc in SdrPaintView::SetLayerVisible(rtl::OUString const&, bool) (this=0x609e850, rName="hidden", bShow=false) at svx/source/svdraw/svdpntv.cxx:419 #6 0x00007f47ac2ab66d in ScTabView::UpdateLayerLocks() (this=0x5fc7488) at sc/source/ui/view/tabview5.cxx:384 #7 0x00007f47ac2aaa3e in ScTabView::TabChanged(bool) (this=0x5fc7488, bSameTabButMoved=false) at sc/source/ui/view/tabview5.cxx:291 #8 0x00007f47ac29d24e in ScTabView::SetTabNo(short, bool, bool, bool) (this=0x5fc7488, nTab=4, bNew=false, bExtendSelection=true, bSameTabButMoved=false) at sc/source/ui/view/tabview3.cxx:1935 #9 0x00007f47ac2b3627 in ScTabViewShell::Execute(SfxRequest&) (this=0x5fc7250, rReq=...) at sc/source/ui/view/tabvwsh3.cxx:526 when going in frame 1 at: SdrObject* pObj=GetObj(i); (in svx/source/svdraw/svdpage.cxx) i = 46 But now I don't know how to keep on the investigation.
The problem begins at SdrCaptionPrimitive2D::SdrCaptionPrimitive2D, where it creates an inversion of the matrix to transform the tail polygon to unit. The inversion fails (ludcmp returns true, because on one of iterations, fValBB is 0 - don't ask, I do not understand what it means :)); and the SdrCaptionPrimitive2D ctor happily applies the uninverted matrix to the thing, scaling it up instead of scaling it down. The construction of the matrix happens in ViewContactOfSdrCaptionObj::createViewIndependentPrimitive2DSequence, using createScaleShearXRotateTranslateB2DHomMatrix(21000-14922, 32253-32253, 0, 0, 14922, 32253) It looks like the problem is with the zero vertical scale. Possibly in case of such a failure, tail should be reset to empty? Would it be a reasonable thing for something that is intended to have no height (or no width), i.e., invisible? Then in SdrCaptionPrimitive2D::create2DDecomposition, it would not be scaled up using the same matrix, and would not result in some huge values...
(In reply to Mike Kaganski from comment #6) Or better just don't try to scale the tail to the unit. It is only done to later scale back in the create2DDecomposition, and we can avoid that scaling back altogether. The matrix is not modified after assigning to the object, so ...
Created attachment 179683 [details] Minimal reproducer
https://gerrit.libreoffice.org/c/core/+/133217
Mike Kaganski committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/e44f71dd3439cd1c473b2e32516e33986de042f3 tdf#148669: do not try to transform tail to unit rectangle It will be available in 7.4.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
I confirm initial file indicated doesn't assert anymore with Mike's patch. Idem for minimal reproducer file. Thank you Mike! :-)
Xisco Fauli committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/c5de4d0277e18fbf79f8f80fbe49d2d25be1fb4c tdf#148669: sc_uicalc: Add unittest It will be available in 7.4.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.