Bug 133801 - Sorting a column uses 600 MB at peak, with LibO 4.2 90 MB (with autofilter on an empty row)
Summary: Sorting a column uses 600 MB at peak, with LibO 4.2 90 MB (with autofilter on...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
(earliest affected)
4.3 all versions
Hardware: All All
: medium minor
Assignee: Not Assigned
Keywords: bibisected, bisected, regression
Depends on:
Blocks: Memory
  Show dependency treegraph
Reported: 2020-06-08 18:27 UTC by Telesto
Modified: 2022-06-27 03:29 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:
Regression By:

Example file (48.07 KB, application/vnd.oasis.opendocument.spreadsheet)
2020-06-08 18:27 UTC, Telesto
Bibisect log (3.54 KB, text/plain)
2020-06-08 18:28 UTC, Telesto
Bibisect log (4.24 KB, text/plain)
2020-06-08 18:37 UTC, Telesto
Example file (30.31 KB, application/vnd.oasis.opendocument.spreadsheet)
2020-06-09 09:36 UTC, Telesto
Screencast (760.53 KB, video/mp4)
2020-06-25 10:17 UTC, Telesto
sorting col F and E with 7.1, win7, no mem peak, (92.03 KB, image/jpeg)
2020-06-25 12:16 UTC, b.

Note You need to log in before you can comment on or make changes to this bug.
Description Telesto 2020-06-08 18:27:27 UTC
Sorting a column uses 600 MB, with LibO 4.2 90 MB

Steps to Reproduce:
1. Open the attached file
2. Sort column F ascending

Actual Results:
600 MB ram

Expected Results:
90 MB ram

Reproducible: Always

User Profile Reset: No

Additional Info:
Found in
Build ID: 9bc848cf0d301aa57eabcffa101a1cf87bad6470
CPU threads: 2; OS: Linux 5.3; UI render: default; VCL: x11; 
Locale: en-US (en_US.UTF-8); UI-Language: en-US
Calc: threaded

but not in
Comment 1 Telesto 2020-06-08 18:27:46 UTC
Created attachment 161773 [details]
Example file
Comment 2 Telesto 2020-06-08 18:28:45 UTC
Created attachment 161774 [details]
Bibisect log

Bisected to
commit 89c92c5812cbfb70120b588d1f52d3d8dfcacce3
Author: Matthew Francis <mjay.francis@gmail.com>
Date:   Thu May 28 21:17:43 2015 +0800

    commit f4a075728f62f0083a4bffd40d3c02265082d962
    Author:     Kohei Yoshida <kohei.yoshida@collabora.com>
    AuthorDate: Fri Apr 18 00:29:55 2014 -0400
    Commit:     Kohei Yoshida <kohei.yoshida@collabora.com>
    CommitDate: Wed Apr 23 21:08:20 2014 -0400
        Avoid using SwapRow() when sorting.
        Instead, build a data table prior to sorting, swap rows in this data table
        as we sort, then transfer the results back to the document in one step.  This
        significantly speeds up the sort performance.
        Formula cells are yet to be handled. I'll work on that next.
        Change-Id: I59bde1a243dc8940411d1a33eef147671b060cd0
Comment 3 Telesto 2020-06-08 18:29:59 UTC
And the memory usage is sticky too.. close the document.. still in use
Comment 4 Telesto 2020-06-08 18:37:33 UTC
Created attachment 161775 [details]
Bibisect log

The first bibisect is about bump from 89 to 430 mb

This bibisect is for the bump from 430 to 580

ommit ef183cd2ffc39d3d0baa53a2ce02763530b86129
Author: Matthew Francis <mjay.francis@gmail.com>
Date:   Thu May 28 21:17:52 2015 +0800

    Bibisect: This commit covers the following source commit(s) which failed to build
    commit c72f76fcd1107a2e5542b9a43fc535914a210b17
    Author:     Kohei Yoshida <kohei.yoshida@collabora.com>
    AuthorDate: Wed Apr 23 15:29:56 2014 -0400
    Commit:     Kohei Yoshida <kohei.yoshida@collabora.com>
    CommitDate: Wed Apr 23 21:08:26 2014 -0400
        Set mdds 0.10.3 as the new package requirement.
        Change-Id: Ide0e10fa528d53a7e732d00b54c940111beebe19

:040000 040000 794d8214a07008c5d90296cba968e839f7964984 36bbb850acf98abc2b23d305588eca506cca8933 M	opt
Comment 5 Telesto 2020-06-09 09:36:13 UTC
Created attachment 161792 [details]
Example file
Comment 6 b. 2020-06-25 00:39:32 UTC
no repro with below versions, linux only? 

found no data in col. F, tried sorting 'E' instead and sorting empty col. F, both less than 10 MB of mem-use, 

ver and, both winx64,
Comment 7 Telesto 2020-06-25 10:17:35 UTC
Created attachment 162394 [details]

Only about the peak
Version: (x64)
Build ID: aadcd6f90916bd2b9734ae793141d0c77cc5b46c
CPU threads: 4; OS: Windows 6.3 Build 9600; UI render: default; VCL: win
Locale: nl-NL (nl_NL); UI: en-US
Calc: CL
Comment 8 b. 2020-06-25 12:16:58 UTC
Created attachment 162398 [details]
sorting col F and E with 7.1, win7, no mem peak,

hi @Telesto, 

thanks for the video, see behaviour, but not reproducible here, 

see attached screenshot, marks 1: load of calc, 2: close of program, inbetween load of file, nearly no mem usage, sort of col. F ascending, no visible mem usage, sort of col E ascending, no visible mem usage ... 

above with: 
Version: (x64)
Build ID: a201ab6f47c2d5a7ba4c5f998b0aa231cae82010
CPU threads: 8; OS: Windows 6.1 Service Pack 1 Build 7601; UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: en-US
Calc: CL

selecting whole column and using [data | sort] with undo steps 100 - standard, 

similar with ver. linx64 (debian/kali 2020.2), no visible impact or peak, 

thus: bug is, i believe in your video, but no repro here, still 'unconfirmed' ...
Comment 9 b. 2020-06-25 12:33:24 UTC
sorry, seen on recheck, 

you used [autofilter | sort], 

that indeed has some weaknesses, mem usage here short moment ~370 MB, not critical, but unneccessary as [data | sort] proves it can be done better, 

'new' but minor importance,
Comment 10 Telesto 2020-06-25 13:53:16 UTC
Comment 11 b. 2020-06-26 05:43:38 UTC
i've put some bugs under 'see also', if someone is working in this area it might be appropriate to check for them. users - especially me - often have problems when similar functionalities act differently if they are reached in different ways. Especially when these differences happen 'hidden' without the user getting a hint. (in everyday life not every user has the time and in complex tables only few users have the possibility to check the results of an operation in detail).

in this case it is that 'sort' behaves differently when you call it via [data - sort] than when you activate it via the dropdown-button of an autofilter and then sort [as|des]cending. 

the selection of the range to which the sort is applied is probably done differently, i mean i saw a very nice hint menu in [data - sort] the other day 'there is data in adjacent ranges if you want to include it' or something like that, and users who accidentally shredded their data with [autofilter - sort] would have been much less annoyed if they had received a similar warning there. 

Of course, it is a complex task for the usability to strive for both consistency and compatibility with Excel, this may sometimes conflict, is there a 'general guideline' for this somewhere?
Comment 12 Xisco Faulí 2020-06-26 06:30:38 UTC
Removing perf keyword since this issue is about memory usage and not about performance
Comment 13 QA Administrators 2022-06-27 03:29:05 UTC
Dear Telesto,

To make sure we're focusing on the bugs that affect our users today, LibreOffice QA is asking bug reporters and confirmers to retest open, confirmed bugs which have not been touched for over a year.

There have been thousands of bug fixes and commits since anyone checked on this bug report. During that time, it's possible that the bug has been fixed, or the details of the problem have changed. We'd really appreciate your help in getting confirmation that the bug is still present.

If you have time, please do the following:

Test to see if the bug is still present with the latest version of LibreOffice from https://www.libreoffice.org/download/

If the bug is present, please leave a comment that includes the information from Help - About LibreOffice.
If the bug is NOT present, please set the bug's Status field to RESOLVED-WORKSFORME and leave a comment that includes the information from Help - About LibreOffice.

Please DO NOT

Update the version field
Reply via email (please reply directly on the bug tracker)
Set the bug's Status field to RESOLVED - FIXED (this status has a particular meaning that is not 
appropriate in this case)

If you want to do more to help you can test to see if your issue is a REGRESSION. To do so:
1. Download and install oldest version of LibreOffice (usually 3.3 unless your bug pertains to a feature added after 3.3) from https://downloadarchive.documentfoundation.org/libreoffice/old/

2. Test your bug
3. Leave a comment with your results.
4a. If the bug was present with 3.3 - set version to 'inherited from OOo';
4b. If the bug was not present in 3.3 - add 'regression' to keyword

Feel free to come ask questions or to say hello in our QA chat: https://web.libera.chat/?settings=#libreoffice-qa

Thank you for helping us make LibreOffice even better for everyone!

Warm Regards,
QA Team