Bug Hunting Session
Bug 119388 - Calc freezes in Linux and very slow in Windows after trying to delete column or line in much compressed ods with lot of draw:custom-shape
Summary: Calc freezes in Linux and very slow in Windows after trying to delete column ...
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:6.4.0 target:6.3.0.1 target:6....
Keywords: perf
Depends on:
Blocks: CPU-AT-100% Shapes-Custom
  Show dependency treegraph
 
Reported: 2018-08-20 13:32 UTC by Dainius
Modified: 2019-09-02 06:24 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
LibreOffice freez after try delete column or line (76.76 KB, application/vnd.oasis.opendocument.spreadsheet)
2018-08-20 13:34 UTC, Dainius
Details
perf flamegraph (68.33 KB, application/x-bzip)
2019-08-19 18:31 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Dainius 2018-08-20 13:32:49 UTC
Description:
freez after try delete column or line

Steps to Reproduce:
1. Open file
2. Select column or line
3. From contest menu select Delete Rows or Delete Columns

Actual Results:
LibreOffice freeze

Expected Results:
Delete Columns or Rows


Reproducible: Always


User Profile Reset: No



Additional Info:
Comment 1 Dainius 2018-08-20 13:34:34 UTC
Created attachment 144330 [details]
LibreOffice freez after try delete column or line

LibreOffice freez after try delete column or line
Comment 2 Timur 2018-08-20 14:45:13 UTC
Looks like LO freezes in Linux and not in Windows, but very slow to delete and yields a dump on fileopen and delete. Test with 6.2+.
Comment 3 Timur 2018-08-20 14:55:28 UTC
File looks small but is very compressed and it has large contents.xml with a lot of draw:custom-shape elements. 

Fileopen in Windows:
FOLLOWUP_IP: 
svxcorelo!SdrCaptionObj::setSuppressGetBitmap+8d94
000007fe`da1d6b34 4c891424        mov     qword ptr [rsp],r10

Delete in Windows:
FOLLOWUP_IP: 
svxcorelo!SvxShape::Notify+31
575d6d01 0f8481020000    je      svxcorelo!SvxShape::Notify+0x2b8 (575d6f88)
Comment 4 Xisco Faulí 2018-08-21 12:59:11 UTC
Also reproduced in

Version: 4.3.0.0.alpha1+
Build ID: c15927f20d4727c3b8de68497b6949e72f9e6e9e

LibreOffice 3.3.0 
OOO330m19 (Build:6)
tag libreoffice-3.3.0.4
Comment 5 Commit Notification 2019-06-26 09:22:41 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/8fb35f06ddec551a6de887ce269d9196d9fe2337%5E%21

tdf#119388 calc, slow removing column, improve ScChildrenShapes

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 6 Commit Notification 2019-06-28 13:09:59 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "libreoffice-6-3":

https://git.libreoffice.org/core/+/d28b76a7fd88dcd9d8d601ba1d4b31d2d91b96ac%5E%21

tdf#119388 calc, slow removing column, improve ScChildrenShapes

It will be available in 6.3.0.1.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 7 Commit Notification 2019-07-17 18:40:46 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/aab10bb21f230a0d0d6acf59ccab50dd3c2c3d0a%5E%21

fix "tdf#119388 calc, slow removing column, improve ScChildrenShapes"

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 8 Commit Notification 2019-07-18 10:06:33 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "libreoffice-6-3":

https://git.libreoffice.org/core/+/4d285cd1dd8155ca41e5f3b94dbe4142bff8972d%5E%21

fix "tdf#119388 calc, slow removing column, improve ScChildrenShapes"

It will be available in 6.3.0.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 9 Xisco Faulí 2019-08-19 09:50:06 UTC
it takes ~2 minutes in

Version: 6.4.0.0.alpha0+
Build ID: cbe9601e5972ec1c466311e0b148a9fbe8cda4d1
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded
Comment 10 Xisco Faulí 2019-08-19 09:50:45 UTC
@Julien, Would it be possible to have a perf chart ?
Comment 11 Julien Nabet 2019-08-19 09:52:46 UTC
(In reply to Xisco Faulí from comment #10)
> @Julien, Would it be possible to have a perf chart ?

No pb but I'll be able to do this only after my daytime job.
Comment 12 Julien Nabet 2019-08-19 18:29:15 UTC
First I gave a try with a debug build.
I noticed these logs when opening:
warn:sc:20667:20667:sc/source/filter/orcus/orcusfiltersimpl.cxx:149: Unable to load styles from xml file! boost::filesystem::file_size: No such file or directory

A lot of these:
warn:legacy.tools:20667:20667:xmloff/source/draw/shapeimport.cxx:744: Shape sorting failed
(whereas I don't see any shapes in the file)
Comment 13 Julien Nabet 2019-08-19 18:31:14 UTC
Created attachment 153513 [details]
perf flamegraph

Here's a Flamegraph created from the moment I began to delete the column.
(on a not debug build)
Comment 14 Commit Notification 2019-08-20 16:38:22 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/80d8fda9c092ea77869c7c162d0acac854a0738d%5E%21

tdf#119388 don't broadcast when locked

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 15 Commit Notification 2019-08-20 18:35:22 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/89ea9340aca77a87ef10785a734201d2f400a933%5E%21

tdf#119388 speed up listener broadcast

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 16 Commit Notification 2019-08-23 13:12:22 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/ec940941e0bd7db15c5cf7d43df82226e0d849dc%5E%21

tdf#119388 add new UNO listener/broadcaster

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 17 Commit Notification 2019-08-23 17:36:09 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/79e1d1353b4bed9a120c44b3a0436d4b041c30f7%5E%21

tdf#119388 use virtual to bypass dynamic_cast

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 18 Commit Notification 2019-08-23 20:18:26 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/04991eaf9f6767d94ad7c4e47708f47615beccf6%5E%21

tdf#119388 only enqueue events we actually process

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 19 Noel Grandin 2019-08-24 07:05:59 UTC
This is as good as I can get it
Comment 20 Commit Notification 2019-08-27 19:20:18 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/+/61c97e941b1d73bf94293f05fdd26636b55a4bce%5E%21

no need to add a new virtual method (tdf#119388 related)

It will be available in 6.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 21 Timur 2019-08-29 08:24:41 UTC
Looks good. Test in Windows. Just a few seconds stall on deletion. 
I don't see dumps either on fileopen and row/column delete.
Comment 22 Xisco Faulí 2019-09-02 06:24:56 UTC
Deletion takes 20 seconds in

Version: 6.4.0.0.alpha0+
Build ID: 41cd3e8e817c8c33a13608e62eeb06ce2c6977e4
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

@Noel, Thanks for fixing this issue!