Bug 119637 - XLSX: Small perf delay filtering file with 1500 comments in cells
Summary: XLSX: Small perf delay filtering file with 1500 comments in cells
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.1 all versions
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: filter:xlsx, haveBacktrace, perf
Depends on:
Blocks: XLSX-Autofilter
  Show dependency treegraph
 
Reported: 2018-09-01 18:46 UTC by Roman Kuznetsov
Modified: 2023-08-13 09:48 UTC (History)
11 users (show)

See Also:
Crash report or crash signature:


Attachments
Example XLSX file with 1500 comments (60.02 KB, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)
2018-09-01 18:47 UTC, Roman Kuznetsov
Details
Example ODS file with 1500 comments (24.54 KB, application/vnd.oasis.opendocument.spreadsheet)
2018-09-01 18:49 UTC, Roman Kuznetsov
Details
bt with debug symbols (21.82 KB, text/plain)
2019-04-27 12:01 UTC, Julien Nabet
Details
Flamegraph (273.11 KB, application/x-bzip)
2019-06-10 07:50 UTC, Julien Nabet
Details
perf flamegraph (107.48 KB, application/x-bzip)
2021-11-26 12:35 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Roman Kuznetsov 2018-09-01 18:46:57 UTC
Description:
Very long time works autofilter in XLSX file with 1500 comments in cells. About 30 sec in

Version: 6.2.0.0.alpha0+ (x64)
Build ID: d0425778eef7ea20ccc19834c07d0b265f58baf0
CPU threads: 4; OS: Windows 10.0; UI render: GL; 
TinderBox: Win-x86_64@42, Branch:master, Time: 2018-08-08_03:06:53
Locale: ru-RU (ru_RU); Calc: CL

In ODS file with same content, autofilter works very fast

Steps to Reproduce:
1. Open XSLX file from attach and try filter content with autofilter in columnt A
2.
3.

Actual Results:
Very long time works autofilter in XLSX file with 1500 comments in cells

Expected Results:
autofilter in XLSX file with 1500 comments in cells works very fast, as in ODS with same content


Reproducible: Always


User Profile Reset: No



Additional Info:
Comment 1 Roman Kuznetsov 2018-09-01 18:47:28 UTC
Created attachment 144598 [details]
Example XLSX file with 1500 comments
Comment 2 Roman Kuznetsov 2018-09-01 18:49:51 UTC
Created attachment 144599 [details]
Example ODS file with 1500 comments
Comment 3 m_a_riosv 2018-09-01 23:49:47 UTC
Repro
Version: 6.2.0.0.alpha0+ (x64)
Build ID: 414ef6cb187dd3bbcc917dbedf3c0c1cc8668f60
CPU threads: 4; OS: Windows 10.0; UI render: GL; 
TinderBox: Win-x86_64@42, Branch:master, Time: 2018-08-21_00:13:04
Locale: es-ES (es_ES); Calc: CL
Comment 4 Xisco Faulí 2018-09-03 16:38:11 UTC
Undoing the filtering makes LibreOffice to hang...
Comment 5 Xisco Faulí 2018-09-03 17:24:41 UTC
it's much faster in gen than in gtk3 or gtk...
Comment 6 Xisco Faulí 2018-09-03 17:31:33 UTC
Also reproduced in

Version 4.1.0.0.alpha0+ (Build ID: efca6f15609322f62a35619619a6d5fe5c9bd5a)
Comment 7 Roman Kuznetsov 2018-09-05 10:06:43 UTC
Ilhan, may be you are interested of this bug
Comment 8 Roman Kuznetsov 2018-11-21 14:42:02 UTC
in

Версия: 6.2.0.0.beta1
ID сборки: d1b41307be3f8c19fe6f1938cf056e7ff1eb1d18
Потоков ЦП: 4; ОС:Windows 6.1; Отрисовка ИП: по умолчанию; VCL: win; 
Локаль: ru-RU (ru_RU); UI-Language: ru-RU
Calc: threaded

It takes only 7 sec for autofilter's work

Can to matter status of OpenGL?
Comment 9 Roman Kuznetsov 2019-02-11 17:08:41 UTC
3 sec in

Version: 6.3.0.0.alpha0+
Build ID: cec3fe24dd4569bf04b13e0284823bad129dbb86
CPU threads: 4; OS: Windows 10.0; UI render: GL; VCL: win; 
TinderBox: Win-x86@42, Branch:master, Time: 2019-02-02_03:18:07
Locale: ru-RU (ru_RU); UI-Language: en-US
Calc: threaded
Comment 10 Xisco Faulí 2019-02-11 17:10:15 UTC
(In reply to Roman Kuznetsov from comment #9)
> 3 sec in
> 
> Version: 6.3.0.0.alpha0+
> Build ID: cec3fe24dd4569bf04b13e0284823bad129dbb86
> CPU threads: 4; OS: Windows 10.0; UI render: GL; VCL: win; 
> TinderBox: Win-x86@42, Branch:master, Time: 2019-02-02_03:18:07
> Locale: ru-RU (ru_RU); UI-Language: en-US
> Calc: threaded

Compare to the initial report, it's 1/10 of the time time. I would say it's RESOLVED WORKSFORME
Comment 11 Julien Nabet 2019-04-27 12:01:04 UTC
Created attachment 151039 [details]
bt with debug symbols

On pc Debian x86-64 with master sources updated today, it seems to hang.
I retrieved a bt at random during loading.
Comment 12 Julien Nabet 2019-06-09 15:21:13 UTC
On pc Debian x86-64 with master sources updated today + enable-symbols, I don't reproduce this.

I still reproduce the slowness with enable-dbgutil but perhaps it's not relevant.

I think all Noel's optimizations may have helped here.
Roman: could you give a try to a recent daily master build?
Comment 13 Roman Kuznetsov 2019-06-09 16:59:23 UTC
(In reply to Julien Nabet from comment #12)
> On pc Debian x86-64 with master sources updated today + enable-symbols, I
> don't reproduce this.
> 
> I still reproduce the slowness with enable-dbgutil but perhaps it's not
> relevant.
> 
> I think all Noel's optimizations may have helped here.
> Roman: could you give a try to a recent daily master build?

I got 6 sec in

Version: 6.4.0.0.alpha0+ (x64)
Build ID: b170256fb6ebaf774b02b89835b19d9f3a1afb89
CPU threads: 4; OS: Windows 10.0; UI render: default; VCL: win; 
TinderBox: Win-x86_64@42, Branch:master, Time: 2019-06-07_03:30:35
Locale: ru-RU (ru_RU); UI-Language: en-US
Calc: threaded

It is still slow I think
Comment 14 Julien Nabet 2019-06-10 07:50:47 UTC
Created attachment 152072 [details]
Flamegraph

Flamegraph done on pc Debian x86-64 with master sources updated yesterday + enable-symbols
Comment 15 Xisco Faulí 2019-06-10 09:28:28 UTC
Around 6 seconds in

Version: 6.4.0.0.alpha0+
Build ID: ec905d131374f0860bac77c52873eed984b1966f
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded

as said in comment 4, it hangs undoing the filtering. I'll create another bugreport
Comment 16 Xisco Faulí 2019-06-10 12:37:03 UTC
Adjusting Summary. Very long time doesn't seems to be accurate. it takes 6 seconds... it doesn't seem like very long time to me...
Comment 17 Roman Kuznetsov 2020-01-10 13:39:13 UTC
(In reply to Xisco Faulí from comment #16)
> Adjusting Summary. Very long time doesn't seems to be accurate. it takes 6
> seconds... it doesn't seem like very long time to me...

takes ~7 sec in

Version: 6.5.0.0.alpha0+ (x64)
Build ID: b6295e4a1b7735c148174f44f6d28221f4f52302
CPU threads: 4; OS: Windows 10.0 Build 18362; UI render: default; VCL: win; 
Locale: ru-RU (ru_RU); UI-Language: en-US
Calc: threaded
Comment 18 Kevin Suo 2021-11-26 10:40:22 UTC
There is a delay when filtering in the first attachment (i.e. the XLSX document, attachment 144598 [details]), but the second one in instant, with the latest master builds which includes the improvements in autofilter (see bug 133835).

@Luboš Luňák and @Noel Grandin, I think you may also be interested in this one?
Comment 19 Julien Nabet 2021-11-26 12:35:27 UTC
Created attachment 176520 [details]
perf flamegraph

Here's a Flamegraph retrieved on pc Debian x86-64 with master sources updated today (b18c2a0024c6d33cdf142ed2adf0d127483411e8) with gen rendering on xlsx file.
Accessing filter is instant, then I deselected all (instant too) and finally chose 1 value (some seconds)
Comment 20 Roman Kuznetsov 2022-01-06 12:47:19 UTC
Still 6 sec delay in

Version: 7.4.0.0.alpha0+ (x64) / LibreOffice Community
Build ID: 1be170d0629cf761f0ee4173007a3c021966546e
CPU threads: 4; OS: Windows 6.1 Service Pack 1 Build 7601; UI render: default; VCL: win
Locale: ru-RU (ru_RU); UI: en-US
Calc: CL

I still use an old CPU here - Intel Core 2 Quad 9450
Comment 21 Roman Kuznetsov 2023-08-13 09:48:48 UTC
Finally it was fixed
I checked in my own build

Version: 24.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: d635e73308dc6fcea9897855b5167be09f52f840
CPU threads: 16; OS: Linux 6.2; UI render: default; VCL: gtk3
Locale: ru-RU (ru_RU.UTF-8); UI: en-US
Calc: threaded

Filter took less than 1 sec

Closed as WFM (I think it was Caolan's patch https://gerrit.libreoffice.org/c/core/+/155569, but I won't check it)