Bug 78664 - Having over 100 000 rows autofilter does not open dialog box (hanging) (possibly related to decimal places)
Summary: Having over 100 000 rows autofilter does not open dialog box (hanging) (possi...
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.2.4.2 release
Hardware: All Windows (All)
: medium critical
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: AutoFilter
  Show dependency treegraph
 
Reported: 2014-05-13 13:58 UTC by andrey
Modified: 2020-11-23 21:39 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Sample file to test (2.85 MB, application/xml)
2014-05-13 23:49 UTC, m_a_riosv
Details
This video shows the process of how it happens (1.57 MB, video/avi)
2015-08-07 07:24 UTC, andrey
Details
The file from video (3.22 MB, application/vnd.oasis.opendocument.spreadsheet)
2015-08-07 07:25 UTC, andrey
Details
Sample file modified, numbers with only 2 decimals. (278.02 KB, application/vnd.oasis.opendocument.text)
2015-08-07 17:09 UTC, m_a_riosv
Details
The file with the reduced accuracy (2.04 MB, application/vnd.oasis.opendocument.spreadsheet)
2015-08-10 06:53 UTC, andrey
Details

Note You need to log in before you can comment on or make changes to this bug.
Description andrey 2014-05-13 13:58:14 UTC
When I set autofilter on any column, i want to select a condition for the autofilter. But Libreoffice Calc is hanging. 
(I have over 100 000 rows in this column. 
Data in this column are represented as distinct float values)
Comment 1 Kohei Yoshida 2014-05-13 16:41:54 UTC
Back to UNCONFIRMED.  This needs to be properly triaged.
Comment 2 m_a_riosv 2014-05-13 23:49:59 UTC
Created attachment 99001 [details]
Sample file to test

Hi Andrey, thanks for reporting.

with attached file, it's really slow with 8192 rows (all different), and becomes unusable with more rows.
With:
Win7x64Ultimate
Versión: 4.2.4.2 Id. de compilación: 63150712c6d317d27ce2db16eb94c2f3d7b699f8


Works really nice with:
Version: 4.2.5.0.0+ Build ID: c4e10ad83fe1fa92a115327c1909218f5dc8c8bd
  TinderBox: Win-x86@42, Branch:libreoffice-4-2, Time: 2014-05-12_10:29:35
a bit slower (what I think is expected for master) with:
Version: 4.3.0.0.alpha1+ Build ID: 224b235971a01971de626d38ccc8506d0a55771b
   TinderBox: Win-x86@39, Branch:master, Time: 2014-05-11_23:55:17
Comment 3 m_a_riosv 2014-05-13 23:54:01 UTC
I think the proper status is Resolved Worksforme, because it is solved for the next release.

Daily builds can be found in: http://dev-builds.libreoffice.org/daily/

Please if you are not agree, reopen it.
Comment 4 andrey 2015-08-06 12:53:52 UTC
Just I checked on 5 version. The error remained.
Windows XP 32-bit.
Comment 5 m_a_riosv 2015-08-06 13:43:36 UTC
Hi @andrey, not for me with:
Win7x64
Version: 4.4.5.2 Build ID: a22f674fd25a3b6f45bdebf25400ed2adff0ff99
Version: 5.0.0.5 (x64) Build ID: 1b1a90865e348b492231e1c451437d7a15bb262b

Can you attach a sample file, remember to clean any private information.

Please try resetting the user profile, sometimes solves strange issues.
https://wiki.documentfoundation.org/UserProfile
Comment 6 andrey 2015-08-07 07:24:27 UTC
Created attachment 117729 [details]
This video shows the process of how it happens
Comment 7 andrey 2015-08-07 07:25:54 UTC
Created attachment 117730 [details]
The file from video
Comment 8 m_a_riosv 2015-08-07 17:09:49 UTC
Created attachment 117753 [details]
Sample file modified, numbers with only 2 decimals.

Looking for possible strange characters in the data, I think, I have found that it depends on numbers with a lot of decimals, as more decimal places worse behavior.

Reducing decimal places to only two seems allows to work fine.

See the attached file, it's the sample file with only two decimal places.

Reproducible, with versions 4.2-and 5.0.0.5
Comment 9 andrey 2015-08-10 06:53:39 UTC
Created attachment 117799 [details]
The file with the reduced accuracy

Does not help. I think the case in the amount of data. When exceeding a certain size - everything works.
Comment 10 m_a_riosv 2015-08-10 11:17:29 UTC
Forgive me @andrey, I forgot to set up as NEW in my previous comment.

Of course amount of data have influence, but I think the issue is in relation the decimal places of the numbers.
Comment 11 Cor Nouws 2015-09-11 13:52:05 UTC
lets set version to first reported one
Comment 12 m_a_riosv 2015-09-11 14:02:49 UTC
Thanks Cor, I forgot to do it.
Comment 13 Kevin Suo 2015-10-13 05:55:59 UTC
Reproduced with
Version: 5.0.3.1
Build ID: fd8cfc22f7f58033351fcb8a83b92acbadb0749e
Locale: zh-CN (zh_CN)
Win10 x86.

This is a possible regression.
Comment 14 Kevin Suo 2015-10-13 06:04:10 UTC
Per discussing in bug 91307, maybe the delay/hang is related to the size of the cell value? with long text / numbers with many dicimal places, the delay is obvious.
Comment 15 Haico 2016-08-24 09:14:58 UTC
I started bug 91307. The spreedsheet file (I gave as an example) which I'm still working on now exceeds the value of 20.000 rows.
I'm still on version 4.1.6.2 because of this bug. 
I still don't have any trouble with this version. It works like a charme! No delays and hangups whatsoever. So the size doesn't have to be a problem!
After v.4.1.6 the autofilter got a new design. That's when my file frooze and crashed every time I wanted to use autofilter in newer releases.
Comment 16 Haico 2016-08-25 07:56:55 UTC
Aron, I tried v5.2. and b u g 9 1 3 6 4 is NOT FIXED. I'm not going to reopen it because I will stay with v.4. I give up!

Are you sure this is a different bug. I still have the same problems as described here, only my file has 20.000+ rows not 100.000+!! 

I've waited a very long time and I'm getting a bit tired of trying yet another release with the same results as the last. My bug shouldn't have been closed in the first place. I think I have every right to complain. I really hoped there was a fix. And there isn't one.

Probably if this bug will be fixed bug 91364 will be fixed for sure. :-)

(in reply to Aron)
Haico, the bugfix is in v5.2. If the fix didn't get rid of your bug, reopen this bug report instead of complaining in a different one (bug 78664) without giving any new detail.


(In reply to Haico from comment #15)
> I started bug 91307. The spreedsheet file (I gave as an example) which I'm
> still working on now exceeds the value of 20.000 rows.
> I'm still on version 4.1.6.2 because of this bug. 
> I still don't have any trouble with this version. It works like a charme! No
> delays and hangups whatsoever. So the size doesn't have to be a problem!
> After v.4.1.6 the autofilter got a new design. That's when my file frooze
> and crashed every time I wanted to use autofilter in newer releases.
Comment 17 Haico 2016-08-25 08:08:26 UTC
Since the redesign of the autofilter after v. 4.1.6 it can't handle files with large number of rows. That's the problem in my opinion!

(In reply to Haico from comment #16)
> Aron, I tried v5.2. and b u g 9 1 3 6 4 is NOT FIXED. I'm not going to
> reopen it because I will stay with v.4. I give up!
> 
> Are you sure this is a different bug. I still have the same problems as
> described here, only my file has 20.000+ rows not 100.000+!! 
> 
>
Comment 18 Aron Budea 2016-08-25 23:12:04 UTC
(In reply to Haico from comment #16)
> Aron, I tried v5.2. and b u g 9 1 3 6 4 is NOT FIXED. I'm not going to
> reopen it because I will stay with v.4. I give up!

Apparently you've written three comments about the issue since "giving up".

Honestly, reopening an issue is two clicks, plus giving some form of constructive feedback, for example that you tried the new version, and this-and-this was your observation. It takes no more time than complaining.
Comment 19 Haico 2016-08-29 19:19:24 UTC
(In reply to Aron Budea from comment #18)
> (In reply to Haico from comment #16)
> > Aron, I tried v5.2. and b u g 9 1 3 6 4 is NOT FIXED. I'm not going to
> > reopen it because I will stay with v.4. I give up!
> 
> Apparently you've written three comments about the issue since "giving up".
> 
> Honestly, reopening an issue is two clicks, plus giving some form of
> constructive feedback, for example that you tried the new version, and
> this-and-this was your observation. It takes no more time than complaining.

In the first place I was just giving my opinion. Your an .... (not a nice word).
Comment 20 QA Administrators 2017-09-01 11:19:24 UTC Comment hidden (obsolete)
Comment 21 Cor Nouws 2017-09-13 10:30:00 UTC
using test file from comment #2, 
- opening a filter drop down takes ~1 or 2 second (s)
- unchecking all and selecting one criteria is OK in a second or so
- but unchecking three values and then OK, freezes the whole UI..
 
Version: 6.0.0.0.alpha0+
Build ID: 5bbfa7ab8ded08d73dcb86c5e9fa3692b629e5bf
CPU threads: 4; OS: Linux 4.10; UI render: default; VCL: gtk2; 
TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time: 2017-09-11_08:32:29
Locale: nl-NL (nl_NL.UTF-8); Calc: group
Comment 22 QA Administrators 2019-05-19 02:50:44 UTC Comment hidden (obsolete)
Comment 23 Cor Nouws 2019-05-24 13:59:50 UTC
using test file from comment #2, 
- opening one of the filter drop downs takes < 1 second
- unchecking all and selecting one criteria is OK in less then a second
- unchecking one or two values and then OK, freezes the whole UI for 7 minutes or so


Version: 6.3.0.0.alpha1+
Build ID: b26b6cab5d8147d35f76a21c333719c80840d08d
CPU threads: 4; OS: Linux 5.0; UI render: default; VCL: gtk3; 
TinderBox: Linux-rpm_deb-x86_64@86-TDF, Branch:master, Time: 2019-05-20_23:15:15
Locale: nl-NL (nl_NL.UTF-8); UI-Language: en-US
Calc: threaded
Comment 24 Kevin Suo 2020-11-20 03:32:32 UTC
(In reply to Cor Nouws from comment #23)

> opening one of the filter drop downs takes < 1 second
This should be the issue as originally reported by the bug reporter, and it is now fixed. I don't remember which one fixed this. See also bug 122419.

So, should we set this as WORKSFORME?

> unchecking all and selecting one criteria is OK in less then a second
> unchecking one or two values and then OK, freezes the whole UI for 7 minutes or so
This is a different issue and should be tracked separately, and we already have many, see bug 133835 for example.
As I commented in bug 133835, when deselecting one, it is fast to use the compare condition which is like "a <> x". Actually you can see this if you set the condition in Standard Filer (e.g, the column Not Equal to your deselected item.
Comment 25 Cor Nouws 2020-11-23 21:39:59 UTC
(In reply to Kevin Suo from comment #24)
> (In reply to Cor Nouws from comment #23)
> 
> > opening one of the filter drop downs takes < 1 second
> This should be the issue as originally reported by the bug reporter, and it
> is now fixed. I don't remember which one fixed this. See also bug 122419.
> 
> So, should we set this as WORKSFORME?
Good idea

> This is a different issue and should be tracked separately, and we already
> have many, see bug 133835 for example.
Linked there.
Thanks.