Bug 123640 - Sort dialog takes forever to load when many columns are selected
Summary: Sort dialog takes forever to load when many columns are selected
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
6.2.0.1 rc
Hardware: All Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-02-22 08:58 UTC by Marcel Waldvogel
Modified: 2020-04-01 14:51 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marcel Waldvogel 2019-02-22 08:58:58 UTC
Description:
When having selected many columns (e.g. by selecting entire rows or after "Select All"), displaying the "Data→Sort…" dialog takes seemingly forever (>1 minute; operating systems asks whether to force quit LibreOffice).

Steps to Reproduce:
1. Open a new spreadsheet
2. Select several (or all) rows
3. Choose "Data→Sort…"
4. Wait

Actual Results:
LibreOffice is blocked for more than a minute, the operating system asks whether to force quit the application (which I accepted twice, before becoming patient enough). This can lead to user data loss.

Expected Results:
The process should be much faster (on the order of a second).



Reproducible: Always


User Profile Reset: No



Additional Info:
Despite not really knowing what LibreOffice does in that time, I still have some suggestions on what could be done:

1. Speed up the loop inserting the column names into the dialog (maybe it is even an O(n²) algorithm?)
2. Only insert columns which do have real data into the dialog
Comment 1 Durgapriyanka 2019-02-22 19:37:33 UTC
Thank you for reporting the bug, but I cannot reproduce the bug in

Version: 6.3.0.0.alpha0+
Build ID: b6b28931435e44aca92b8c0e1659f701e3ed1a87
CPU threads: 2; OS: Windows 6.1; UI render: default; VCL: win; 
TinderBox: Win-x86@42, Branch:master, Time: 2019-01-30_06:57:04
Locale: en-US (en_US); UI-Language: en-US
Calc: threaded
Comment 2 Oliver Brinzing 2019-02-23 10:21:24 UTC
cannot reproduce with an empty new spreadsheet as described above.

Please attach a sample document, as this makes it easier for us to verify the bug. 
(Please note that the attachment will be public, remove any sensitive information before attaching it. 
See https://wiki.documentfoundation.org/QA/FAQ#How_can_I_eliminate_confidential_data_from_a_sample_document.3F for help on how to do so.)

I have set the bug's status to 'NEEDINFO'. Please change it back to 'UNCONFIRMED' once the requested document is provided.
Comment 3 QA Administrators 2019-09-02 09:30:57 UTC Comment hidden (obsolete)
Comment 4 Marcel Waldvogel 2019-09-02 11:33:43 UTC
A new empty spreadsheet works perfectly well.

It could however be a Linux-only problem. Could you please try this on Linux?

I am still having the same problem with 6.3 on Ubuntu 19.04 x86_64. On a fairly recent machine (Lenovo T460s), "New Spreadsheet", "Select All", "Data->Sort" takes about a minute before the "Sort" dialog appears.

The waiting time is roughly quadratic to the number of columns selected:

Columns | Delay
--------+------
    200 |    3s
    400 |   10s
    600 |   20s
    800 |   40s
   1024 |   60s

("1024" is what "Select all" gives; the delays are only estimates)

Up to 200 columns, the delay is acceptable. But then the delays start becoming annoyingly long. It seem that there is quadratic behavior in one of the data structures involved in creating the "Sort Key 1/2/3" drop-down lists which hurts performance when many columns are involved.
Comment 5 QA Administrators 2019-09-03 16:13:12 UTC Comment hidden (obsolete)
Comment 6 Roman Stingler 2019-11-11 10:21:26 UTC
I can confirm this.




If creating an empty sheet and press the sort button

the soffice thread is maxed out on one thread (could be parallelized)
for a long time.

https://youtu.be/SfWMa8SD3NM

6.3.2.2
Comment 7 Marcel Waldvogel 2019-11-11 10:39:43 UTC
Roman, thank you for the confirmation!

It seems that Windows is not bitten by the bug, so I changed the platform accordingly.

I do not think that parallelizing would work here. It seems to be related to using a wrong (inefficient) GTK(?) function when adding the next column name to the drop-down list, which does not seem to efficiently append the column name, but possibly compare against all the other elements (judging from the O(n^2) timing).

Adding 1024 strings to a list, even if all strings need to be compared against each other on insertion (~1 Million string comparisons) should not take around a minute on a modern processor.

So there are probably two optimization steps to investigate:

* Why is one of the operations involved in adding a column label to the drop-down so slow? (>0.1 ms per invocation?)
* Why does adding a column label cause this function to be called *again* for every label that has previously been added (quadratic behavior; i.e., adding the n-th label causes this slow operation to be invoked n times)?

Strictly thinking, neither of those two seems to be necessary, assuming reasonable APIs and optimal usage of those APIs.
Comment 8 Leslie Walter 2019-12-28 00:16:40 UTC
Running LibreOffice Version 6.3.3.2.0+ on 64-bit Fedora 31, Gnome Version 3.34.2. LibreCalc takes an inordinate amount of time to return the sort dialogue when selecting data by row (all columns as part of selection.) Occasionally Fedora times out before the sort dialogue appears and warns that LibreOffice is not responding and allows a forced quit. Not sure when this first appeared but it did not used to have this problem.
Comment 9 Rifki Affandi 2020-03-03 04:27:54 UTC
I can confirm this

test with LibreOffice 6.3.3.2.0, with openSUSE Leap. 

But this does not happen with openSUSE Tumbleweed with LibreOffice 6.4
Comment 10 Marcel Waldvogel 2020-03-06 08:53:50 UTC
It is still slow with LibreOffice 6.4.0.3 on Ubuntu 19.10.

So it seems LibreOffice 6.4 does not fix the problem, but maybe an update to one of the system libraries did the magic?
Comment 11 Jean-Baptiste Faure 2020-03-29 21:08:48 UTC
Not reproducible for me with LO 6.4.4.0+ under Ubuntu 18.04.

I did the following :
1/ open a new empty spreadsheet
2/ ctrl+A to select all
3/ menu Data > Sort

Result: the sort dialog show up in ~2 seconds

Version : 6.4.4.0.0+
Build ID : 4183a7f7423998d5497c67752624731c2fb00346
Threads CPU : 4; OS : Linux 4.15; UI Render : par défaut; VCL: gtk3; 
Ubuntu_18.04_x86-64
Locale : fr-FR (fr_FR.UTF-8); Langue IHM : fr-FR
Calc: threaded

Best regards. JBF
Comment 12 Xisco Faulí 2020-03-31 12:13:56 UTC
Thank you for reporting the bug. To be certain the reported issue is not related to corruption in the user profile, could you please reset your Libreoffice profile ( https://wiki.documentfoundation.org/UserProfile ) and re-test?

I have set the bug's status to 'NEEDINFO'. Please change it back to 'UNCONFIRMED' if the issue is still present
Comment 13 Marcel Waldvogel 2020-04-01 14:26:33 UTC
On Ubuntu 19.10, `libreoffice6.4 --safe-mode` (6.4.0.3) shows the same behavior as in Comment 10 (>1 minute, Ubuntu complaining about "Application does not respond. Force Quit?").
Comment 14 Marcel Waldvogel 2020-04-01 14:40:47 UTC
Just upgraded to LO 6.4.2.2 (from 6.4.0.3, on Ubuntu 19.10). Now the dialog appears after about 1s, perfectly acceptable.

So it seems that bug has been fixed (inadvertently?) for Linux somewhere between 6.4.0.3 and 6.4.2.2, when looking at all the reports.
Comment 15 Jean-Baptiste Faure 2020-04-01 14:51:04 UTC
Thank you for the tests. As we do not know what fixed the problem, the correct status is WorksForMe.

Best regards.