Bug 69977 - freeze/crash when creating a diagram from many thousand cells
Summary: freeze/crash when creating a diagram from many thousand cells
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Chart (show other bugs)
Version:
(earliest affected)
4.0.5.2 release
Hardware: All All
: high major
Assignee: Not Assigned
URL:
Whiteboard: target:5.2.0 target:6.1.0 target:6.2...
Keywords:
: 134922 (view as bug list)
Depends on:
Blocks: Chart
  Show dependency treegraph
 
Reported: 2013-09-30 18:50 UTC by ibramlab
Modified: 2024-03-03 08:52 UTC (History)
11 users (show)

See Also:
Crash report or crash signature:


Attachments
Used csv database (200.16 KB, application/x-7z-compressed)
2013-09-30 18:50 UTC, ibramlab
Details
100k data to create a chart (1.26 MB, application/vnd.oasis.opendocument.spreadsheet)
2014-07-14 17:41 UTC, Thomas Arnhold
Details
strace log (1.61 MB, application/gzip)
2016-06-17 08:57 UTC, pietro.pangallo
Details
perf flame graph (1.95 MB, image/svg+xml)
2018-03-27 12:17 UTC, Noel Grandin
Details

Note You need to log in before you can comment on or make changes to this bug.
Description ibramlab 2013-09-30 18:50:11 UTC
Created attachment 86861 [details]
Used csv database

LibreOffice Calc freezes after I load csv file with 11248 items (1.8 mb) and then click diagram button. The diagram wizard is opened after ~3min and works very very slow. Same version of LO under Windows just unexpectedly quits after freezing without any errors and user should manually run it again.
Used csv is attached in archive.
Comment 1 tommy27 2013-10-06 21:34:56 UTC
loaded test .csv in Calc 4.1.2.3 and 4.0.5.2
pressed "select all" menu item
then clicked on "diagram" button

LibO 4.1.2.3 freezes for a few seconds then crashes.
LibO 4.0.5.2 stays frozen untile Ctrl-Alt-Canc kill process

haven't tried earlier releases.
Comment 2 Thomas Arnhold 2014-07-14 17:41:03 UTC
Created attachment 102787 [details]
100k data to create a chart

Another example, in ods format. When creating a chart libo takes hours to create the chart.
Comment 3 QA Administrators 2015-07-18 17:44:26 UTC Comment hidden (obsolete)
Comment 4 Buovjaga 2015-10-23 16:49:53 UTC
Confirmed with attachment 86861 [details]. Could not get a backtrace. Windbg just said the process exited.

Win 7 Pro 64-bit, Version: 5.0.2.2 (x64)
Build ID: 37b43f919e4de5eeaca9b9755ed688758a8251fe
Locale: fi-FI (fi_FI)

Version: 5.1.0.0.alpha1+
Build ID: fcc2415ade6ae93710bbbda9f7e163045e323105
TinderBox: Win-x86@62-merge-TDF, Branch:MASTER, Time: 2015-10-21_16:55:13
Locale: fi-FI (fi_FI)
Comment 5 Commit Notification 2015-12-10 10:33:39 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=2afead78b346f13030ad43589da9bc389c3c4c32

tdf#69977: constructing and destructing AccessibleElementInfo...

It will be available in 5.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 6 Commit Notification 2015-12-11 08:12:37 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=58d8d8ac67aa9b907f1304a48efa0f7a473d9de4

tdf#69977: uno::Sequence is expensive

It will be available in 5.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 7 pietro.pangallo 2016-06-17 08:57:06 UTC
Created attachment 125699 [details]
strace log

strace generated from:
Version: 5.3.0.0.alpha0+
Build ID: a8bd44573b75d1399257d6f5d052611439607189
CPU Threads: 2; OS Version: Linux 4.1; UI Render: default; 
TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time: 2016-06-13_23:46:49
Locale: it-IT (it_IT.UTF-8)
OS: openSUSE Leap 42.1 (x86_64)

with master i can open the file while libreoffice hangs inserting a chart.

with version:
Version: 4.3.7.2
Build ID: 8a35821d8636a03b8bf4e15b48f59794652c68ba
OS: openSUSE Leap 42.1 (x86_64)

LO hangs before opening the file.
Comment 8 pietro.pangallo 2016-06-17 08:59:37 UTC
according to comment 7:
arch change to all all.
Comment 9 Xisco Faulí 2017-01-13 12:48:26 UTC
Hello,
Is this bug fixed?
If so, could you please close it as RESOLVED FIXED?
Comment 10 QA Administrators 2018-01-14 03:24:29 UTC Comment hidden (obsolete)
Comment 11 Commit Notification 2018-03-27 11:27:44 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=3acc5a2383f5b0458e3caf1505fe6b8ad7dc3fb0

tdf#69977 improve creation of large charts

It will be available in 6.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Noel Grandin 2018-03-27 12:17:40 UTC
Created attachment 140908 [details]
perf flame graph
Comment 13 Noel Grandin 2018-03-27 12:19:14 UTC
The flame graph shows that the bulk of the time is in creating labels and tick marks which seem to use some rather heavyweight text machinery
Comment 14 Commit Notification 2018-10-04 19:31:50 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=1371c2f4edab5086af6d0661de046252def29455

Do not let end row creep above start row, tdf#69977 tdf#119305 follow-up

It will be available in 6.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 15 Commit Notification 2018-10-05 10:20:22 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-6-1":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=9bc004d6187da610e146527585c678b5cd9432ae&h=libreoffice-6-1

Do not let end row creep above start row, tdf#69977 tdf#119305 follow-up

It will be available in 6.1.3.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 16 Roman Kuznetsov 2019-03-18 21:55:52 UTC
Eike, is this commit fixed?
Comment 17 Eike Rathke 2019-03-19 14:33:39 UTC
My commit fixed a glitch from the other commit (see commit message). Whether this performance bug is fixed I don't know and don't have time to check or work on, but from comment 13 I'd deduce there's still a bottle neck.
Comment 18 Roman Kuznetsov 2019-03-19 15:01:18 UTC
still repro freeze in

Version: 6.3.0.0.alpha0+ (x64)
Build ID: de024e572dd7a588f82b84c68daa2051ec6b20e9
CPU threads: 4; OS: Windows 10.0; UI render: GL; VCL: win; 
TinderBox: Win-x86_64@42, Branch:master, Time: 2019-03-18_01:13:57
Locale: ru-RU (ru_RU); UI-Language: en-US
Calc: threaded
Comment 19 Noel Grandin 2021-01-31 12:15:30 UTC
I suggest a handful of possibilities here:

(*) several of our chart2 types (e.g. pie) should not allow more than X categories before just putting all remaining data into a "remaining" category

(*) possibly we should choose Data Range before choosing Chart Type

(*) if choosing a really large data range, the user should be warned that the chart may be slow

(*) if the user does not explicitly choose a Data Range before opening the chart wizard, we should only select the currently visible page or some other small subset of the data

(*) some of the internals of the chart wizard should just "do nothing" or add some warning message if presented with too much data e.g. if the user does something that ends up creating > 1000 tick marks, just display a single tick that says "too may ticks"

(*) there are definitely some bugs here e.g. the tick creation logic is supposed to self-limit by increasing the tick-interval until the ticks stop overlapping, but that is not happening here

Adding tag for UX team to check my suggestions
Comment 20 Heiko Tietze 2021-02-01 12:43:39 UTC
(In reply to Noel Grandin from comment #19)
> (*) several of our chart2 types (e.g. pie) should not allow more than X
> categories before just putting all remaining data into a "remaining" category

Sounds reasonable but not so easy to implement unless the data is sorted. Otherwise you end up with 10x 0.1% plus 99% in the remaining category.
 
> (*) possibly we should choose Data Range before choosing Chart Type

Sure, naturally the first step. Not sure it solves the issue however. Most users may just click over this step.

> (*) if choosing a really large data range, the user should be warned that
> the chart may be slow

That's what we do in similar situations. For instance recently for bug 134779.

> (*) if the user does not explicitly choose a Data Range before opening the
> chart wizard, we should only select the currently visible page or some other
> small subset of the data

Error-prone, people are used to get everything selected.

> (*) some of the internals of the chart wizard should just "do nothing" or
> add some warning message if presented with too much data e.g. if the user
> does something that ends up creating > 1000 tick marks, just display a
> single tick that says "too may ticks"

> (*) there are definitely some bugs here e.g. the tick creation logic is
> supposed to self-limit by increasing the tick-interval until the ticks stop
> overlapping, but that is not happening here

Thought we have this Automatic function. What I have in mind is the minor interval (set to 2 in the example at comment 0). But changing this to 10 has no effect and is always reverted to automatic.


(*) another option is to kick the can down the road: a pie chart with 5k entries is done within <5s on my machine; the maximum of 100k takes a bit longer...
Comment 21 Heiko Tietze 2021-02-26 12:59:30 UTC
Had this on the design meeting agenda but no further input. Guess any of the options is an improvement.
Comment 22 Timur 2022-01-26 09:54:14 UTC
*** Bug 134922 has been marked as a duplicate of this bug. ***
Comment 23 QA Administrators 2024-01-27 03:14:50 UTC
Dear ibramlab,

To make sure we're focusing on the bugs that affect our users today, LibreOffice QA is asking bug reporters and confirmers to retest open, confirmed bugs which have not been touched for over a year.

There have been thousands of bug fixes and commits since anyone checked on this bug report. During that time, it's possible that the bug has been fixed, or the details of the problem have changed. We'd really appreciate your help in getting confirmation that the bug is still present.

If you have time, please do the following:

Test to see if the bug is still present with the latest version of LibreOffice from https://www.libreoffice.org/download/

If the bug is present, please leave a comment that includes the information from Help - About LibreOffice.
 
If the bug is NOT present, please set the bug's Status field to RESOLVED-WORKSFORME and leave a comment that includes the information from Help - About LibreOffice.

Please DO NOT

Update the version field
Reply via email (please reply directly on the bug tracker)
Set the bug's Status field to RESOLVED - FIXED (this status has a particular meaning that is not 
appropriate in this case)


If you want to do more to help you can test to see if your issue is a REGRESSION. To do so:
1. Download and install oldest version of LibreOffice (usually 3.3 unless your bug pertains to a feature added after 3.3) from https://downloadarchive.documentfoundation.org/libreoffice/old/

2. Test your bug
3. Leave a comment with your results.
4a. If the bug was present with 3.3 - set version to 'inherited from OOo';
4b. If the bug was not present in 3.3 - add 'regression' to keyword


Feel free to come ask questions or to say hello in our QA chat: https://web.libera.chat/?settings=#libreoffice-qa

Thank you for helping us make LibreOffice even better for everyone!

Warm Regards,
QA Team

MassPing-UntouchedBug