Bug 133878 - Slow performance in autofilter if the record contains many unique values (10 seconds after 5.4 vs. 2 seconds before)
Summary: Slow performance in autofilter if the record contains many unique values (10 ...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
5.4.0.3 release
Hardware: All All
: high major
Assignee: Luboš Luňák
URL:
Whiteboard: target:7.2.0 target:7.1.0.0.beta2
Keywords: bibisected, bisected, perf, regression
: 138248 (view as bug list)
Depends on:
Blocks: AutoFilter
  Show dependency treegraph
 
Reported: 2020-06-10 20:19 UTC by Telesto
Modified: 2020-12-01 13:31 UTC (History)
6 users (show)

See Also:
Crash report or crash signature:


Attachments
Bibisect log (2.90 KB, text/plain)
2020-06-10 20:20 UTC, Telesto
Details
Example file (269.46 KB, application/vnd.oasis.opendocument.spreadsheet)
2020-06-10 20:43 UTC, Telesto
Details
Minimal test file (35.38 KB, application/vnd.oasis.opendocument.spreadsheet)
2020-09-25 11:11 UTC, Kevin Suo
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Telesto 2020-06-10 20:19:28 UTC
Description:
Slow performance autofilter 10 seconds after 5.4 before 2 seconds

Steps to Reproduce:
1. Open the attached file
2. Expand the autofilter drop down for column A
3. Deselect " 1"  and press OK

Actual Results:
10 seconds delay

Expected Results:
2 seconds


Reproducible: Always


User Profile Reset: No



Additional Info:
Found in
Version: 7.0.0.0.beta1+
Build ID: c344de1b9985b6ca10b354e24151d0bdf92dc20e
CPU threads: 2; OS: Linux 5.3; UI render: default; VCL: x11
Locale: en-US (en_US.UTF-8); UI: en-US
Calc: threaded

and in
5.4
Comment 1 Telesto 2020-06-10 20:20:41 UTC
Created attachment 161860 [details]
Bibisect log

Bisected to
author	Kohei Yoshida <kohei.yoshida@collabora.com>	2017-01-30 22:01:45 -0500
committer	Kohei Yoshida <libreoffice@kohei.us>	2017-02-02 05:01:53 +0000
commit 42e472b5870278058537d43d03d457dc80b16166 (patch)
tree 68f9c7526463dc004f58a5c9a3bc81547b2b233f
parent 5f33745c1d6750126e53a02b2a95355a3e5b16a0 (diff)
tdf#105629: Let's move the advanced-filter specific stuff ...
to the place where we actually parse the filtering rules for advanced
filter queries.

https://cgit.freedesktop.org/libreoffice/core/commit/?id=42e472b5870278058537d43d03d457dc80b16166
Comment 2 Telesto 2020-06-10 20:43:07 UTC
Created attachment 161861 [details]
Example file
Comment 3 Telesto 2020-06-10 20:44:02 UTC
Note:
All autofilter drop down buttons are slow too after deselecting " 1" and pressing OK
Comment 4 Roman Kuznetsov 2020-06-13 11:39:50 UTC
confirm in

Version: 7.1.0.0.alpha0+
Build ID: 26483950760f0aac7bc45e93db4127f66a98fdc6
CPU threads: 4; OS: Mac OS X 10.15.5; UI render: default; VCL: osx
Locale: ru-RU (ru_RU.UTF-8); UI: en-US
Calc: threaded
Comment 5 Kevin Suo 2020-09-25 11:11:45 UTC
Created attachment 165833 [details]
Minimal test file

Filtering in column A takes 14s in my PC. 

This may due to column A contains too many unique values. I do encounter many cases when autofilter takes a lot of time when the field contains a lot of unique values in my daily work.

Attached is a minimal test case. Column A contains too many unique values: filtering in this column takes a lot of time. Column B contains only 3 unique values: filtering in this column takes only < 1s.
Comment 6 Xisco Faulí 2020-11-12 12:07:28 UTC
*** Bug 137086 has been marked as a duplicate of this bug. ***
Comment 7 Xisco Faulí 2020-11-12 12:19:54 UTC
Hi Luboš, Noel,
I thought you might be interested in this performance issue. The file from bug 137086 hangs for minutes
Comment 8 Kevin Suo 2020-11-16 00:54:29 UTC
*** Bug 138248 has been marked as a duplicate of this bug. ***
Comment 9 Kevin Suo 2020-11-16 01:32:31 UTC
Set importance to HIGH MAJOR as this regression impacts core functioning in Calc.
Comment 10 Kevin Suo 2020-11-29 12:50:37 UTC
I see Luboš Luňák has submitted a patch on gerrit:
https://gerrit.libreoffice.org/c/core/+/106594

With this patch applied the filter is not done within 2 seconds. Those who can build by yourself should also test if there are regressions.
Comment 11 Commit Notification 2020-12-01 10:19:56 UTC
Luboš Luňák committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/85cb9262644ebe376f9c5af1f01a0216a51a6d6d

bring back optimized calc querying by value (tdf#133878)

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Commit Notification 2020-12-01 13:31:24 UTC
Luboš Luňák committed a patch related to this issue.
It has been pushed to "libreoffice-7-1":

https://git.libreoffice.org/core/commit/8d29d0cee0b16df71134cd82d3fc0dbcdcd3dde3

bring back optimized calc querying by value (tdf#133878)

It will be available in 7.1.0.0.beta2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.