Bug 136838

Summary: AutoFilter Improvement: Use "Not Equal" (i.e., <>) when we selected all but one item
Product: LibreOffice Reporter: Kevin Suo <suokunlong>
Component: CalcAssignee: Not Assigned <libreoffice-bugs>
Status: RESOLVED DUPLICATE    
Severity: enhancement CC: buzea.bogdan, l.lunak
Priority: medium    
Version: 6.4.6.2 release   
Hardware: All   
OS: Linux (All)   
See Also: https://bugs.documentfoundation.org/show_bug.cgi?id=146037
Whiteboard: target:7.3.0
Crash report or crash signature: Regression By:
Bug Depends on:    
Bug Blocks: 103512    
Attachments: test ODS file with autofilter
Another test ods file

Description Kevin Suo 2020-09-17 08:47:07 UTC
Created attachment 165612 [details]
test ODS file with autofilter

Assume a column contains n items. When we open the autofilter and un-selected one, then we have n-1 items selected. Doing this filter with AutoFilter is slow because it seems to be using the "x in n-1" logic. However, doing this filter with Standard Filter is fast if we use "Not Equal" logic.

Steps to Reproduce:

1. Open the attached test ODS file

2. Click the autofilter in column A, and de-select the 1st item (i.e., #N/A), then click OK.
--> Slow, it takes more than 8s.

3. Use Standard Filter, and set condition "POMatch <> #N/A", then click OK.
--> Fast, it takes 1s.

Apparently, if we deselect only one item, the condition is the same as <> that item. It should be more expensive to compare the value to see whether it is in the n-1 list.

Actually, if we de-select several items (e.g., 3 items), it would still be faster to use "(x <> a) and (x <> b) and (x <> c)". However, if the deselected items are quite a lot, the "x in n-1" logic should be faster.
Comment 1 Kevin Suo 2020-09-17 12:48:37 UTC

*** This bug has been marked as a duplicate of bug 133835 ***
Comment 2 Kevin Suo 2021-11-21 15:53:24 UTC
Created attachment 176404 [details]
Another test ods file

Deselecting autofilter items in columns O, P, S will freeze Calc as there are many unique values in these columns.
Comment 3 Commit Notification 2021-11-25 13:28:32 UTC
Luboš Luňák committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/7b0aabe71d2455f6f643553a07f1056935cf190f

sort, cache and binary search query items if they're many (tdf#136838)

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.