Bug 131204 - Exceptionally slow SORTING
Summary: Exceptionally slow SORTING
Status: RESOLVED NOTABUG
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
6.3.5.2 release
Hardware: x86-64 (AMD64) All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-03-07 12:55 UTC by Colin
Modified: 2021-02-09 12:51 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments
Spreadsheet filled with genuine data (413.84 KB, application/vnd.oasis.opendocument.spreadsheet)
2020-03-07 13:15 UTC, Colin
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Colin 2020-03-07 12:55:58 UTC
Description:

A sheet with 416 rows 37 columns containing mostly numerical data of two digits custom sorting on an alpha column of 3 digits takes substantially longer following the update - it's almost comparable with watching paint dry.
Fifteen of the numerical columns are the result of minor formulae and the 3 character alpha column being sorted is also formula driven

Steps to Reproduce:
1.Using sheet "All Hits from the attached file EJResults.ods
2.Select data from A11 to bottom of column AK (Select the first two rows and then select to end of range with CTRL + SHIFT + Down)
3.Select sort button
4.Sort on Column L with option Custom sort order
5.The custom sort is the final table - scroll down to the bottom of the list
6.Allow it to sort



Actual Results:
Takes many times longer with the update today then previously

Expected Results:
Faster sort - comparable to the previous release


Reproducible: Always


User Profile Reset: No



Additional Info:
Version: 6.3.5.2 (x64)
Build ID: dd0751754f11728f69b42ee2af66670068624673
CPU threads: 4; OS: Windows 10.0; UI render: default; VCL: win; 
Locale: sv-SE (en_GB); UI-Language: en-GB
Calc: threaded
Comment 1 Mike Kaganski 2020-03-07 13:13:09 UTC
You seem to have forgotten to attach the sample?
Comment 2 Colin 2020-03-07 13:15:19 UTC
Created attachment 158467 [details]
Spreadsheet filled with genuine data
Comment 3 Colin 2020-03-07 13:16:57 UTC
(In reply to Mike Kaganski from comment #1)
> You seem to have forgotten to attach the sample?

I misread the file size limits and was desperately trying to trim it to 30K
Comment 4 m_a_riosv 2020-03-07 16:47:49 UTC
I think it's in relation with references in other places to data in the sorted range, e.g. Payout.AN FREQUENCY function, a lot of calculation needs to be done while sorting.
Comment 5 Colin 2020-03-07 17:09:39 UTC
(In reply to m.a.riosv from comment #4)
> I think it's in relation with references in other places to data in the
> sorted range, e.g. Payout.AN FREQUENCY function, a lot of calculation needs
> to be done while sorting.

Does change control identify anything that has changed in the FREQUENCY function since the previous release?

The data has been accumulated and maintained weekly for eight years. Whilst loading from startup had sometimes been a cause for concern, the increased functionality of the various analyses, as the volume of data increased or new analytical functions (& sheets)were added, has never caused a slowdown of this magnitude.

Perhaps it should also be noted that I misinterpreted the upload file metrics and systematically removed all sheets except first and last. I then trimmed the analyses on the last sheet so just the source numbers were maintained (the larger font data and concatenation beneath). I then had to remove over 350 data rows in sheet one. An insignificant change in sorting speed was observed.

Would it be pertinent to run a comparative test on an earlier version of Calc?
Comment 6 Colin 2020-03-07 17:23:21 UTC
Not sure if it's even a consideration but if, previously, the sorting took place on the data I was viewing and then all the other areas were updated retrospectively on completion of the sort, I wouldn't have noticed the processing times and updates of all the other areas as I wouldn't have been viewing them.
Just a thought, quite happy to be told "go play and let the professionals deal with it".
Comment 7 m_a_riosv 2020-03-08 16:56:38 UTC
With 6.2.8 I have the same times than with 6.3 or 6.4

Please what version was quicker.
Comment 8 Colin 2020-03-08 17:21:21 UTC
(In reply to m.a.riosv from comment #7)
> With 6.2.8 I have the same times than with 6.3 or 6.4
> 
> Please what version was quicker.

It would only have been one release earlier 6.3.4.2 - I generally update as soon as it's notified. Worst case 6.3.3.2

Also,

After the Option sort and some manual processing, I would then just re-select the data and use the button sort [Descending].

I just recorded my times - virtually 8 seconds for the option sort on column L and also 8 seconds for the descending sort on Col A. Unfortunately, I didn't take the times when first reported.

What times are you achieving? I have an i5 @ 3.4 8GB Ram.

I would normally have firefox with perhaps 3 windows open as well - currently the same but with a paused YouTube video instead of just ordinary web links
Comment 9 QA Administrators 2020-03-09 02:54:50 UTC Comment hidden (obsolete)
Comment 10 Colin 2020-03-14 05:43:46 UTC
This is embarrassing.
I have just processed the sheet with today's update and the sort was practically instantaneous.
As the job is always performed as the first task on a Saturday morning I would have expected my own PC environmental status to be consistent.
This makes me suspect there may have been a massive Win10 automatic update process running in the background. Hindsight is a wonderful thing.
A learning process, next time check the damn task manager for additional activities - Look before I Leap.
My apologies, I have also set the status to RESOLVED, hope that's the right thing to do.
Comment 11 Colin 2020-03-14 05:52:37 UTC
I just noticed the slowdown was occurring over the first two days of the bug report. I ran the thing again on day two with the 8-second timing. Would that preclude any background effects from a WIN update or is it conceivable the update broke something and got fixed during the last 5 days?