Bug Hunting Session
Bug 36779 - LibreOffice Calc is very slow at handling charts with ~3000*6 points
Summary: LibreOffice Calc is very slow at handling charts with ~3000*6 points
Status: RESOLVED INVALID
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Chart (show other bugs)
Version:
(earliest affected)
3.4.1 RC1
Hardware: x86 (IA32) Linux (All)
: medium critical
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
: 39760 85096 (view as bug list)
Depends on:
Blocks:
 
Reported: 2011-05-02 10:42 UTC by Ruslan Kabatsayev
Modified: 2014-11-04 16:59 UTC (History)
9 users (show)

See Also:
Crash report or crash signature:


Attachments
Test case with ~11000 lines of data. (238.16 KB, application/vnd.oasis.opendocument.spreadsheet)
2011-06-20 14:17 UTC, Thomas Arnhold
Details
10k rows of data, to draw a simple line chart (1.26 MB, application/vnd.oasis.opendocument.spreadsheet)
2014-07-12 15:18 UTC, Kevin Suo
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ruslan Kabatsayev 2011-05-02 10:42:22 UTC
See test document. It has a chart at the end of the table.
Here's a set of ways one can reproduce Calc hangs with this document for about half a minute:
1. First, Calc hangs after loading the test case (as it seems, on rendering phase).
2. After the document has been loaded, another hang will occur when you try to scroll it so that the chart is clipped, then scrolled back to rerender.
3. Yet another hang is when you double-click the chart and try to edit a line.
4. After editing has finished (e.g. line width changed), pressing OK to close the dialog makes Calc hang again.

This might seem reasonable for such a large table, but it appears that MS Excel works with the same document (converted to xls via Calc) without any single hang (or, well the hangs are less than for half a second).
Comment 1 Ruslan Kabatsayev 2011-05-02 10:46:38 UTC
Test document: http://www.filedropper.com/test_35
Comment 2 Ruslan Kabatsayev 2011-05-03 12:11:50 UTC
A strange fact: if i create similar-sized table in MS Excel, and then make a chart there, save as xls and then open via LibreOffice, then i get almost no performance problems.
Comment 3 Thomas Arnhold 2011-06-20 14:15:30 UTC
Same for me. When doing charts with 10.000 elements calc is unusable (high cpu usage).

Tested this with 3.4.0 and 3.4.1 RC1 on x86_64. I've attached a sample ods file. The issue is the same for xls files.
Comment 4 Thomas Arnhold 2011-06-20 14:17:44 UTC
Created attachment 48210 [details]
Test case with ~11000 lines of data.
Comment 5 Stefan Knorr (astron) 2011-07-24 04:57:13 UTC
Can you check again with 3.4.2 RC2, please? I have absolutely no problems with it on Ubuntu x86-64.
Comment 6 Stefan Knorr (astron) 2011-07-24 05:14:10 UTC
Absolutely no problem might have been a bit of an overstatement:
* I see a 3 to 5 second freeze after changing some data in sheet 1 and then switching back to sheet 2.
* Changing the line colour of the chart gives me a longer freeze of ~10 seconds.
* Scrolling the chart is a bit laggy and reported CPU usage in top rises to 20% (soffice) + 25% (Xorg) = 45% (on a 2.6 GHz C2D).
Comment 7 Thomas Arnhold 2011-07-24 06:30:26 UTC
@Aaron: You're right, it's fixed with 3.4.2 RC2 for me. Maybe this is caused by the mdds upgrade. Opening the document (test.ods) is really fast! Only double-clicking on it to edit the chart takes some time (about 10 secs for me).

Great work! Thanks Kohei!
Comment 8 Stefan Knorr (astron) 2011-07-25 13:57:36 UTC
Let's close this then.
@Ruslan: feel free to reopen if your test case is still slow with 3.4.2.
Comment 9 albert.kurucz 2011-08-22 08:21:14 UTC
I have tried Thomas Arnold's test case with ~11000 lines of data.
3.4.2 hangs. Please reopen.
I have a table at production with only 7200 lines, which also hangs terrible. If you need more test data, let me know.
Comment 10 Thomas Arnhold 2011-08-25 13:49:47 UTC
Albert: Did you try it with a clean Libo profile? For me the attached test case works fine. What are your system specs?
Comment 11 Adriano Afonso 2012-07-15 01:17:15 UTC
Here the problem is:
Any Graphic edition takes at least 10s 14s
Win 7 64bits 4GbRam AMD LibreOffice 3.5.5
Comment 12 Ruslan Kabatsayev 2013-01-02 22:09:42 UTC
Thomas Arnhold's test takes 66 seconds to get rendered first time on my system with Version 4.1.0.0.alpha0+ (Build ID: ca3aba4cdf766cf33b0dbbe495c9e6f57f06982). Until this happens, I have unresponsive LibO window without any controls.
So still unacceptably slow.
Comment 13 Thomas Arnhold 2013-05-01 02:10:15 UTC
*** Bug 39760 has been marked as a duplicate of this bug. ***
Comment 14 Kevin Suo 2014-07-12 15:03:06 UTC
I added some bugs which may be related/duplicate as see also.
There is also another one, reported in June 2012, but was set as INVALID: bug 51160.

As we can see, users are always complaining about the chart issue on big/large data source, but there is always no improvements.
Comment 15 Kevin Suo 2014-07-12 15:18:25 UTC
Created attachment 102671 [details]
10k rows of data, to draw a simple line chart

The attached test ods file contains 10,000 rows / 2 columns of data in the first sheet.

Try to see if you can draw a similar simple line chart as the 2nd sheet.
(When creating the chart, remember to choose the option "First Column as Label")

For me, it's impossible, because Calc freezes during the process.
Comment 16 Thomas Arnhold 2014-07-14 17:37:01 UTC
Comment on attachment 102671 [details]
10k rows of data, to draw a simple line chart

Not related to this one, see Bug 69977
Comment 17 Thomas Arnhold 2014-07-14 17:37:21 UTC
Comment on attachment 102671 [details]
10k rows of data, to draw a simple line chart

Not related to this one, see Bug 69977
Comment 18 Thomas Arnhold 2014-07-14 17:39:21 UTC
Kevin, Bug 69977 matches more for your bug file (btw your file has 100k entries). This bug here, is more about loading a existing chart which was created by many entries, see my test file.
Comment 19 Tom 2014-08-10 13:56:36 UTC
Hi, this problem still exists in:
4.1.3.2 (freezes for ~32 seconds to resize the chart with anti-aliasing enabled, ~17 seconds with anti-aliasing disabled)
4.3.0.4 (roughly same timing as above)

Any operation on the graph will result in a freeze for a considerable amount of time, even opening and closing the 'Chart Type' dialogue without making any changes results in a freeze.

Interestingly, after reducing the range to 3000 rows it takes around 3 seconds to redraw the graph (and anti-aliasing doesn't seem to be making a big difference here).

I have tested it using the test.ods file (11k+ rows, 2 columns).

By anti-aliasing I mean the settings as per "Options -> View -> Graphics output -> Use anti-aliasing".

Also, it looks like the following reports are related to the same/similar issue:
"Bug 79927 - LibreOffice Calc very slow to plot xy line graphic"
https://bugs.freedesktop.org/show_bug.cgi?id=79927
"Bug 70872 - Calc very slow with charts with large numbers"
https://bugs.freedesktop.org/show_bug.cgi?id=70872
"Bug 80677 - UI: Very sluggish with larger data sets"
https://bugs.freedesktop.org/show_bug.cgi?id=80677
Comment 20 m.a.riosv 2014-10-16 20:21:06 UTC
*** Bug 85096 has been marked as a duplicate of this bug. ***
Comment 21 Jean-Baptiste Faure 2014-10-16 21:40:51 UTC
(In reply to Thomas Arnhold from comment #18)
> Kevin, Bug 69977 matches more for your bug file (btw your file has 100k
> entries). This bug here, is more about loading a existing chart which was
> created by many entries, see my test file.

Fe me it is the same problem, in both test file the main problem comes from the use of too many labels.

You should try with a chart of XY type.

That said what is the usefulness of drawing a chart with 100k points when you have a number of pixels on the width of your screen less than 2% of these 100k?
That means you try to draw from 50 to 100 points of your chart on each column of pixels on your screen. Even if you zoom up to 600% you can't see something.

Workaround to generate the chart: the idea is to shortcut the initial work by the wizard because it tries to draw a preview with the wrong type of chart. To do that start with a partial chart, say with 100 points. Then edit the chart and change the data range.

Best regards. JBF
Comment 22 Diggory Hardy 2014-10-17 07:44:16 UTC
Mr Faure,

> That said what is the usefulness of drawing a chart with 100k points when you
> have a number of pixels on the width of your screen less than 2% of these 100k?

Maybe one has a long time series (for example, the x-axis being a time step and the y-axis being a count of events or parasite density in the blood or some such). In this case being able to get a rough view of the chart can be important even though the user will obviously not be able to read off the exact time (on the x-axis) of some feature in the plot.

Note that in such a plot it probably makes sense to somehow reduce the number of points before drawing the graph, but you can't simply sample some points and ignore the rest since that could ignore a rare event in a long series.
Comment 23 rlk 2014-10-17 18:52:14 UTC
I have poor performance with drawing with both line and scatter (x-y) plots.  The ones with on the order of 10000 data points are really bad, but even much smaller ones, with only a few hundred points, are annoyingly slow.  This is an issue both for switching to a page with a graph or with scrolling if any of the graph is visible.
Comment 24 Joel Madero 2014-11-04 03:26:36 UTC
This bug is a disaster. I'm closing it. For anyone still experiencing similar problems PLEASE:

1. Report a CLEAN BUG that is A SINGLE ISSUE (the first bug reports has 4 issues, this is not helpful for a bug report) WITH REPRODUCIBLE STEPS and CLEAN AND SIMPLE TEST DOCUMENT.

Thanks - sorry for the caps - but wanted to be clear to avoid having to close more bugs similar to this one.

Also please always include:
Your version of LibreOffice;
Your operating system.

Please do not reopen this bug - it's not helpful for getting anything fixed. Closing as INVALID as the original report had 4 reports in one and since then this bug has gotten worse and worse.
Comment 25 Diggory Hardy 2014-11-04 10:04:14 UTC
Joel, are you claiming that issue described by the first post is not reproducible? It is for me. Or that the creation of a document from scratch is not described? If so I will create a new bug report.
Comment 26 Joel Madero 2014-11-04 15:19:22 UTC
I'm saying that the first bug report reports 4 different problems -- and they are enumerated as such. Although they are all "hangs" they are hanging at different points and therefore should be reported separately. With millions of lines of code, just because there is a hang does not mean that it is related to other hangs - therefore, every different issue should be written independent of one another.
Comment 27 Ruslan Kabatsayev 2014-11-04 15:25:04 UTC
(In reply to Joel Madero from comment #26)
> I'm saying that the first bug report reports 4 different problems -- and
> they are enumerated as such.

Well, when enumerating these hangs, I was talking of them not as separate issues, but rather as different ways one could reproduce the single issue — unusable slowness with the large document. I'd bet they all stem from a single source — rendering of the chart.

You, on the other hand, don't seem to even have tried to reproduce this bug in any way, and just closed it because it has an enumeration of symptoms.

Well, I'll leave it for someone else to re-report — someone who will squash these symptoms into one and hopefully make you happy.
Comment 28 Joel Madero 2014-11-04 15:31:57 UTC
...you clearly don't get it so that's fine, someone else can report. A hang at 4 different periods in time are likely NOT the same areas of code that are causing the hang. I suspect if you took a look at the 10,000,000+ lines of code you'd understand why that's the case.

Anyways, that's fine, leave as INVALID - let someone else report.
Comment 29 Diggory Hardy 2014-11-04 16:59:03 UTC
Here: https://bugs.freedesktop.org/show_bug.cgi?id=85867

And please don't using insulting language just because you disagree with someone.