Bug 126678 - Empty cells with formatting not included in sort even though "Include formats" option is checked (see comment 6)
Summary: Empty cells with formatting not included in sort even though "Include formats...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
5.0.0.5 release
Hardware: All All
: medium normal
Assignee: Andreas Heinisch
URL:
Whiteboard: target:7.2.0
Keywords: bibisected, bisected, regression
: 88002 (view as bug list)
Depends on:
Blocks: Sorting
  Show dependency treegraph
 
Reported: 2019-08-02 15:22 UTC by Adalbert Hanßen
Modified: 2022-02-11 10:10 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
Test case as described, with screenshots and pasted spreadsheets (50.22 KB, application/vnd.oasis.opendocument.text)
2019-08-02 15:24 UTC, Adalbert Hanßen
Details
the spreadsheet as ods-file (iun its final state) (12.71 KB, application/vnd.oasis.opendocument.spreadsheet)
2019-08-02 15:25 UTC, Adalbert Hanßen
Details
Minimal test case (9.24 KB, application/vnd.oasis.opendocument.spreadsheet)
2020-04-19 15:23 UTC, Buovjaga
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Adalbert Hanßen 2019-08-02 15:22:24 UTC
Description:
This bug came to my attention using a version of LibreOffice 5.1. Since it is an old version, I tried it out with 

Version: 6.1.6.3
Build ID: 5896ab1714085361c45cf540f76f60673dd96a72
CPU threads: 4; OS: Linux 4.4; UI render: default; VCL: gtk2; 
Locale: de-DE (de_DE.UTF-8); Calc: group threaded

and with the latest development version on my computer which is

Version: 6.4.0.0.alpha0+
Build-ID: 9ee5ad5a0b84bfa652da34694ba4f75668f06087
CPU-Threads: 4; BS: Linux 4.4; UI-Render: Standard; VCL: gtk3; 
TinderBox: Linux-rpm_deb-x86_64@86-TDF, Branch:master, Time: 2019-07-30_13:21:44
Gebietsschema: de-DE (de_DE.UTF-8); UI-Sprache: de-DE
Calc: threaded

I would not be surprised if this bug is in all previous versions of LibreOffice too.

I have a spreadsheet in LO Calc, the interesting part of which looks like this:

The relevant part of the LO Calc spreadsheet pasted into this LO Writer document (see attached LO Writer document with screenshots)

The properties typeface, boldness, color, slant and background have been assigned to the whole rows by marking them by a click on the row number in LO Calc and then assigning the properties: e.g. the whole line 7 has got a yellow background. All this was done in LO Calc, the results were copied via clipboard to the attached file
.
Then I use Daten>Sortieren and sort the spreadsheet by column B. Because the second column was numbered downward and it is an even number of rows, every row changes its place:

Picture: The relevant part of the LO Calc spreadsheet after sorting.

By the way: After selecting Daten>Sortieren, estimated 5 seconds pass until I can select the sort criterium to be by column B. I think this delay should be sped up!

Despite the fact that the whole spreadsheet was selected for sorting, only the first columns with entries were sorted, as can be seen by the highlight formerly belonging to row 7 which has now moved up, but only for the maximum used column extent (Column D in the example).

I consider this to be an error, since the selection was on everything, not just the used part of the lines. The same bug also happens, if whole rows lines are selected.

This spreadsheet was meant to be used for a similar one to collect data by pushing the old entries in a row to the right if a new measurement is done. Of course, more than one line was meant to be highlighted by a background color. Sorting the table will be a part of the use of it, but if that overthrows the colors and other character properties, that is not as it was expected.
When I add new entries to the existing lines to the right, I expect them to inherit all character formatting properties (e.g. size, font, color, slant, underline and background color) from the respective line. But that’s not what happens: I enter some new text to the empty fields 


Conclusion: The inheritance of color, background, slant and other properties is messed up after sorting a LibreOffice spreadsheet. It looks like the properties are only properly assigned to the cells after the sorting procedure up to the rightmost used column of the spreadsheet.

If the spreadsheet has a maximum size known ahead, this gives a hint how to circumvent the problem in many use cases: Just put an entry to a cell to the utmost right lower corner of the maximum size the spreadsheet will ever grow to. Then there can't be columns to the right of the spreadsheet. - But this circumvention is not a real remedy, it is a bug and it should be corrected:
During sorting, all character display properties (e.g. font, slant, boldness, size, color, background, underlining and possibly all others which may be present) should be also transported to the “sorted positions” - regardless if there is an entry or not!
If the internal program logic maintains a “lower right cell coordinate” and sorting only applies the display properties to the cells for the active cells as they were before, it should extend the display properties of the rightmost column actually treated during the sorting process and if there is a leftover, it should extend the display properties from the rightmost active cell of each row to its very end. This is analogous to what happens after “Insert Column to the right”. This is consistent with the general result of extending a spreadsheet: Display properties are always propagated from the left to the right and from top to bottom: Insert Column left inserts a new column with properties  inherited from the old column left of the insertion column, Insert Column right inserts a new column with the display properties derived from the current column. Insert row above and row below behave this way also following the top to bottom rule.

Steps to Reproduce:
1.Make a table in LO Calc with some column by which the entries will be rearranged after sorting by this column
2.Highlight one row which will change its place after applying Sort to some part of the spreadsheet.
3.Mark a part in this spreadsheet to be sorted, the item which is highlighted with the background color will change its place must be in the selection.
4. Sort the selected items.

Actual Results:
4. Sorted table is ok, regarding only the input in the cells. But highlights are broken after sorting. Same applies to other cell display features like boldnes, italics, font, size, slant, color and maybe more if they exist.

Expected Results:
I expect that the rows of a spreadsheet maintain their properties after sorting, even in the part right to the most rightmost cell of the to be sorted part or the whole spreadsheet.


Reproducible: Always


User Profile Reset: Yes



Additional Info:
Comment 1 Adalbert Hanßen 2019-08-02 15:24:35 UTC
Created attachment 153109 [details]
Test case as described, with screenshots and pasted spreadsheets
Comment 2 Adalbert Hanßen 2019-08-02 15:25:24 UTC
Created attachment 153110 [details]
the spreadsheet as ods-file (iun its final state)
Comment 3 Joel Madero 2019-08-03 16:24:17 UTC
@Aldabert -

Just a suggestion, in the future keep bug reports much shorter and less of a book than a bulleted list of how to reproduce. I can spend 10 minutes triaging a couple bugs, or spend 30 reading this one ;) Any chance you can summarize your novel into a couple bullets?

Thanks :)
Comment 4 Adalbert Hanßen 2019-09-05 16:27:22 UTC
(In reply to Joel Madero from comment #3)
> @Aldabert -
> 
> Just a suggestion, in the future keep bug reports much shorter and less of a
> book than a bulleted list of how to reproduce. I can spend 10 minutes
> triaging a couple bugs, or spend 30 reading this one ;) Any chance you can
> summarize your novel into a couple bullets?
> 
> Thanks :)

The bug is still present in the development Version: 6.4.0.0.alpha0+
Build ID: 0fb2927a8fe06e6c3255544b8e4c4c9c0f5a67d3
CPU threads: 4; OS: Linux 4.4; UI render: default; VCL: gtk3; 
TinderBox: Linux-rpm_deb-x86_64@86-TDF, Branch:master, Time: 2019-08-27_22:50:23
Locale: de-DE (de_DE.UTF-8); UI-Language: en-US
Calc: threaded

I don't think that this following numbered form or even one with only ullets to the steps one would be easier to grasp and to reproduce than my original one and probably you would have to read more carefully to understand it, what it is all about (unfortunately this is not a bug of the type to let LO Calc crahs):

1. make a LO Calc document with different line lengths
2. take care that there is at least one column of data which are such that after sorting by this column no row maintains its place
3. assign different character sets, sizes, colours, boldness, slant and the like to whole lines by first marking the line on the left border, then assign the property,
4. sort the whole table by first clicking into the field left to and above A1 and then sort the whole table by the column specified in 2.
5. add information to lines the right of the existing data in some rows. Observe if it inherits the colour, slant, boldness and the like from the left,
6. Also do this for columns for which no row had any data in the original spreadsheet.

Expected result: in step 5 and 6 all added fields inherit their formatting properties from the existing part left to them, since the properties were assigned to the whole rows, nut just a part of it. 

Unfortunately, during sorting only the parts in the data with the minimum rectangle possible to describe all data are sorted. The rest is left as it was before. Thus after sorting, the lines no longer have uniform formatting properties as they had before. This is buggy, since the formatting properties were assigned to the whole lines, not just parts of them. Sorting was not confined to the part of the file with data in it.
Comment 5 QA Administrators 2019-09-06 05:03:04 UTC Comment hidden (obsolete)
Comment 6 Buovjaga 2020-04-19 15:23:25 UTC
Created attachment 159719 [details]
Minimal test case

Steps with this test case:

1. Select A1:E10
2. Data - Sort
3. In the Options tab, verify that "Include formats" is checked
4. Sort key Column B

Result: the empty cell with yellow background does not move with the sort.

Bibisected with Linux 50max repo to https://git.libreoffice.org/core/+/c50d8bf596f527bb53b86dd04da652ed72f2c549%5E!/

"Fix high memory usage when sorting cells in Calc. (fdo#81501)

Empty columns were being unnecessarily included in the sorting
range which caused extra allocation. This happens when you select
the entire sheet to sort. There was already code to skip empty
trailing cells in rows but not to skip empty columns."

Looks like this small change did not take into account the option "Include formats".
Comment 7 Commit Notification 2021-04-14 06:46:37 UTC
Andreas Heinisch committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/774a61afa9fc281290e7bfad4e28c05978b82d73

tdf#126678 - Consider "Include formats" option during sort

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 8 Xisco Faulí 2022-02-11 10:10:20 UTC
*** Bug 88002 has been marked as a duplicate of this bug. ***