Bug 98695 - Data sorting fails when you have many rows of data
Summary: Data sorting fails when you have many rows of data
Status: RESOLVED INSUFFICIENTDATA
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
5.0.5.2 release
Hardware: x86-64 (AMD64) Windows (All)
: medium normal
Assignee: Not Assigned
URL: http://jmp.sh/fVLanbg
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-03-15 22:11 UTC by home
Modified: 2017-05-31 10:47 UTC (History)
1 user (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description home 2016-03-15 22:11:42 UTC
Steps to get error:
1. Open a .csv file with over 1,048,576 row of data. 
2. Selected all the columns A:F
3. Then selected Data > Sort. Sort Key 1 (column c) Sort Key 2 (column d) > Ok.

You will always get Fatal Error:
multi_type_vector::get_block_position#676: block position not found! (logical pos=1048576, block size=1, logical size=1048576)

The sort function doesn't seem to be able to sort as many rows of data as Excel can sort. Maybe the array sizes need to be increased so they can use more memory?
Comment 1 raal 2016-03-16 11:51:09 UTC Comment hidden (obsolete)
Comment 2 home 2016-03-16 20:23:37 UTC
The data doesn't really matter as long as you have well over 1 million rows.

I created a file with random numbers and test it, the sort still fails with the same error message.

The file was over 200MB zipped (bigger than the attachment upload size limit).

I loaded it here, not sure how long before it gets deleted.
http://jmp.sh/fVLanbg
Comment 3 raal 2016-03-17 05:54:56 UTC
Error message whne loading: The data could not be loaded completely because the maximum number of rows per sheet was exceeded.


File loaded, select A:D, sort D descending. Without error in Version: 5.2.0.0.alpha0+ (x64)
Build ID: 042f16a19e3d5f884759dae71264433b988df0e6
CPU Threads: 1; OS Version: Windows 6.1; UI Render: default; 
TinderBox: Win-x86_64@62-TDF, Branch:MASTER, Time: 2016-03-16_10:03:38

Seems to be fixed in dev version. Please could you test with dev version?
http://dev-builds.libreoffice.org/daily/master/
Thank you
Comment 4 home 2016-03-17 16:07:37 UTC
You didn't try sorting more than one column as described in step 3.

I installed 5.2.0.0.alpha0+ (x64) and was able to sort the test file.

Next I tried sorting on a file with real data that is 42 columns wide.

I got the error: The exception unknown software exception (0x40000015) occurred in the application at location 0x52a77676. 

Click on OK to terminate the program.
Comment 5 raal 2016-03-17 19:45:52 UTC
(In reply to home from comment #4)
> You didn't try sorting more than one column as described in step 3.
> 
> I installed 5.2.0.0.alpha0+ (x64) and was able to sort the test file.
> 
> Next I tried sorting on a file with real data that is 42 columns wide.
> 
> I got the error: The exception unknown software exception (0x40000015)
> occurred in the application at location 0x52a77676. 
> 
> Click on OK to terminate the program.

I copied columns in your test file and tested again  / sort A:AP, sort two columns. No crash.
Please attach test file which crash in dev version. We need example to verify.
Please note that the attachment will be public, remove any sensitive information before attaching it.
How can I eliminate confidential data from a sample document?
https://wiki.documentfoundation.org/QA/FAQ#How_can_I_eliminate_confidential_data_from_a_sample_document.3F
Thank you
Comment 6 Xisco Faulí 2016-10-10 11:25:32 UTC Comment hidden (obsolete)
Comment 7 home 2016-10-11 14:55:19 UTC
(In reply to Xisco Faulí from comment #6)

I can't attached the original file because it has private data. But when I tested with the new version it no longer crashed.
Comment 8 QA Administrators 2017-05-02 11:37:33 UTC Comment hidden (obsolete)
Comment 9 QA Administrators 2017-05-31 10:47:10 UTC
Dear Bug Submitter,

Please read this message in its entirety before proceeding.

Your bug report is being closed as INSUFFICIENTDATA due to inactivity and
a lack of information which is needed in order to accurately
reproduce and confirm the problem. We encourage you to retest
your bug against the latest release. If the issue is still
present in the latest stable release, we need the following
information (please ignore any that you've already provided):

a) Provide details of your system including your operating
   system and the latest version of LibreOffice that you have
   confirmed the bug to be present

b) Provide easy to reproduce steps – the simpler the better

c) Provide any test case(s) which will help us confirm the problem

d) Provide screenshots of the problem if you think it might help

e) Read all comments and provide any requested information

Once all of this is done, please set the bug back to UNCONFIRMED
and we will attempt to reproduce the issue. Please do not:

a) respond via email 

b) update the version field in the bug or any of the other details
   on the top section of our bug tracker

Warm Regards,
QA Team

MassPing-NeedInfo-20170531