Bug 67128 - Auto Filter Sort Results in Crash
Summary: Auto Filter Sort Results in Crash
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.2.0.0.alpha0+ Master
Hardware: All Linux (All)
: highest blocker
Assignee: Kohei Yoshida
URL:
Whiteboard:
Keywords: regression
: 67141 (view as bug list)
Depends on:
Blocks: mab4.2
  Show dependency treegraph
 
Reported: 2013-07-21 04:18 UTC by Joel Madero
Modified: 2013-09-02 22:29 UTC (History)
8 users (show)

See Also:
Crash report or crash signature:


Attachments
File Demonstrating Problem (187.31 KB, application/vnd.oasis.opendocument.spreadsheet)
2013-07-21 04:18 UTC, Joel Madero
Details
Backtrace (6.76 KB, text/plain)
2013-07-22 18:41 UTC, Arnaud Versini
Details
typescript with backtrace and valgrind (272.39 KB, text/plain)
2013-07-31 22:12 UTC, Terrence Enger
Details
backtrace from signal 6, SIGABRT (5.30 KB, text/plain)
2013-08-01 14:03 UTC, Terrence Enger
Details
backtrace from later segfault, executing on a 64-bit system (54.03 KB, text/plain)
2013-08-14 23:08 UTC, Terrence Enger
Details
simpler test document (7.53 KB, application/vnd.oasis.opendocument.spreadsheet)
2013-08-15 14:09 UTC, Terrence Enger
Details
console + bt with symbols on master sources (10.67 KB, text/plain)
2013-08-18 17:03 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Joel Madero 2013-07-21 04:18:04 UTC
Created attachment 82759 [details]
File Demonstrating Problem

Version: 4.2 master
Build Date: Thu Jul 18 14:51:17 2013 +0200


Reproducible Steps:

1. Open Attached File

2. Select entire sheet (UNFORMAT)

3. Data -> Filter -> Auto Filter

4. Go to Column X

5. Sort



Expected: Sort happens

Observed: Crash



Works fine on 4.1 RC3
Comment 1 Julien Nabet 2013-07-21 08:03:41 UTC
Comment on attachment 82759 [details]
File Demonstrating Problem

Mimetype fixed
Comment 2 m.a.riosv 2013-07-21 10:35:50 UTC
I can not reproduce.
Win7x97i3Ultimate
Version: 4.2.0.0.alpha0+
Build ID: 2cce78137bbf9810421b9fdedd45127b019b1188
TinderBox: Win-x86@6-debug, Branch:master, Time: 2013-07-18_23:43:36

Maybe a bit detailed step by step and OS information, to do exactly the same, could help.
Have you tried resetting the user profile?
Comment 3 Joel Madero 2013-07-21 19:21:18 UTC
ah apologies: 

Bodhi Linux x64

Not sure how much more detailed I can be than those steps that were provided. Every time I do those steps results in a crash - clean profile = same crash.


The only additional note is that you must first turn off the autofilters that are on - in other words, if you just open the file as is and sort by column X on sheet "UNFORMATED", no crash occurs. First turn off the auto filter

1. Data -> Filters -> Autofilters

2. Select the entire sheet

3. Repeat step 1 to turn auto filters back on

4. Sort by column X


Results in nasty crash.

Marking as UNCONFIRMED again - if no crash on Windows perhaps Linux only bug. If those steps are still not clear I'll hop into dev channel or QA channel and walk someone through  :)
Comment 4 Joel Madero 2013-07-21 20:40:27 UTC
Tested on Windows 8: master~2013-07-11_04.05.08_LibreOfficeDev_4.2.0.0.alpha0_Win_x86.msi 

No crash - marking as Linux(all)
Comment 5 Cor Nouws 2013-07-22 15:56:51 UTC
Works fine on 4.1.0.3 on Ubuntu
Comment 6 Arnaud Versini 2013-07-22 18:41:06 UTC
Created attachment 82836 [details]
Backtrace

Confirmed on Ubuntu 64
Comment 7 Terrence Enger 2013-07-31 22:12:44 UTC
Created attachment 83396 [details]
typescript with backtrace and valgrind

Some points of potential interest:

  - l 14:    start of execution under gdb.  The program issues lots of
             warnings.

  - l 1046:  SIGSEGV

  - L 1055: thread apply all backtrace full.  At top of active thread is
            __gnu_debug::_Safe_iterator_base::_M_attach_single(__gnu_debug::_Safe_sequence_base*, bool)

  - l 1430:  start of execution under valgrind.

  - l 2666:  invalid read.  At top of stack is the same function

       ...:   more valgrind errors follow

  - l 2806:  SIGSEGV.  At top of stack is the same function.


My LibreOffice is master commit 1cf0999, pulled around 2013-07-28 0208 UTC,
built and running on ubuntu-natty (11.04) 32-bit, configured with ...

    --enable-option-checking=fatal
    --enable-dbgutil
    --enable-crashdump
    --without-system-postgresql
    --without-myspell-dicts
    --with-extra-buildid
    --without-doxygen
    --with-external-tar=/home/terry/lo_hacking/git/src
Comment 8 Terrence Enger 2013-08-01 14:03:07 UTC
Created attachment 83439 [details]
backtrace from signal 6, SIGABRT

I have been trying to bibisect this bug using the daily bibisect repo,
and the results are less clear-cut than I would like.  Thus, I am
witholding keyword "bibisected" until somebody finds my results
adequately useful.

For purposes of bibisecting, I treated two conditions as "bad":
(*) segfault, as evident in version latest
(*) signal 6, SIGABRT, with terminal messages 
        terminate called after throwing an instance of 'std::out_of_range'
          what():  vector::_M_range_check
    but only after the progress bar started to move across the bottom
    of the screen.  Among the LO versions which bibisect probed, this
    appeared in 3791268.  The current attachment is from that crash.
Moreover, as I went through the versions of LibreOffice, I sorted on
column B and took that one result--crash or no crash--as the answer.
This despite my vague memory of one version of LibreOffice which
succeeded in sorting on column A but crashed sorting on column B.

This is my first experience using AutoFilter.  I saw some surprising
behaviour, even some which I think is outright wrong, but in the absence
of a crash I deemed the LO version "good".


Let me continue with the standard report of bibisect results.  The
terminal output from the final `git bisect good` is ...

    3791268ce3e6f9e570f02c09d586fd8e9f2485c3 is the first bad commit
    commit 3791268ce3e6f9e570f02c09d586fd8e9f2485c3
    Author: Jean-Baptiste Lallement <jean-baptiste.lallement@canonical.com>
    Date:   Tue Jun 25 08:38:33 2013 +0000

        source-hash-51daa4de4fbb86903aeb9cdfefbb089e8d00c001
    
        commit 51daa4de4fbb86903aeb9cdfefbb089e8d00c001
        Author:     Jelle van der Waa <jelle@vdwaa.nl>
        AuthorDate: Sat Jun 22 12:52:31 2013 +0200
        Commit:     Michael Stahl <mstahl@redhat.com>
        CommitDate: Mon Jun 24 21:44:24 2013 +0000
    
            fdo#43460 sd,rsc,ucb,sdext: use isEmpty()
    
            Change-Id: I7a7a77c26b74078f7fc160fbaa1c8d4e912b844e
            Reviewed-on: https://gerrit.libreoffice.org/4442
            Reviewed-by: Michael Stahl <mstahl@redhat.com>
            Tested-by: Michael Stahl <mstahl@redhat.com>

    :100644 100644 e0cc68ae32874b64ac7d808f00880e1a79e714dc df3b23445f820f871a2d81e85b1e77e498f3ad23 M	ccache.log
    :100644 100644 37f2ae4d811759d268b9b1038112d19960894a41 dc625f5b9d41121b4fa888e776a25b5cdbf462a8 M	commitmsg
    :100644 100644 7873383e78698b5969775797224dea2257429d1a b85c93b24f073f04203e6d8492b6ae63d3039c30 M	dev-install.log
    :100644 100644 8c4471055dd337a79af1aea93ba257edc45bbae3 062a117b360ac5cff020aae921e81cb3ca6578d9 M	make.log
    :040000 040000 1cf15df9ad150422e1e8c4c13350c7fd64c88fd1 dcff4929a651e1e77268fb14b8688ae249b041ed M	opt

and `git bisect log` shows ...

    # bad: [893c8a30f0419e92c94e9c6c435ee2531a6f748c] source-hash-2fc0fa62b26ce34675fcb94de59194592421eeb5
    # good: [9342a3ee4ef4df633567057ea46530d79254b6d2] source-hash-0d61e5dcdbe2cec9df33ac22e3f0e4fbd6a07517
    git bisect start '893c8a3' '9342a3e'
    # good: [159a7f006fe0759d61076916d2ddc1bec2e0fa83] source-hash-bddf3bba1fa13b57a69f2bd5f7c7f96bb945066d
    git bisect good 159a7f006fe0759d61076916d2ddc1bec2e0fa83
    # bad: [6e4f7bdbe1bf434b9ae111c5e83f524113b7c4b8] source-hash-f237f1a616d973397511575c1eb033731d6007f7
    git bisect bad 6e4f7bdbe1bf434b9ae111c5e83f524113b7c4b8
    # bad: [4e06b05aa294d89809817c3ac5743e4328fe363a] source-hash-26a45c1886d9167d8f9ae9aad6234a3702768d8a
    git bisect bad 4e06b05aa294d89809817c3ac5743e4328fe363a
    # bad: [3791268ce3e6f9e570f02c09d586fd8e9f2485c3] source-hash-51daa4de4fbb86903aeb9cdfefbb089e8d00c001
    git bisect bad 3791268ce3e6f9e570f02c09d586fd8e9f2485c3
    # good: [a5620bdfe00015bdba86ee895e26fb4a13721dfe] source-hash-97f71c5f8be85f47d7978259a2d82708412043fd
    git bisect good a5620bdfe00015bdba86ee895e26fb4a13721dfe


Along the way I saw some funny things ...

(*) The down-arrow icon in the first row throws up an untitled
    dialog. The dialog sometimes appears immediately and sometimes
    appears after a delay of a few seconds.  I think that the delayed
    presentation of the dialog is associated with crashing versions of
    LibreOffice, but I was well into the bibisect process before I
    noticed the connection.  Treat the observation as tentative.

(*) In some versions of LibreOffice, that unnamed dialog captures
    <alt>-<tab> to move focus among its own controls instead of moving
    focus between applications.

(*) After I told the dialog box arising from Data > Filter >
    AutoFilter to use first row as headings, the sort moved the
    headings down to row 148, after col B = 1 and before col B =
    blank.  This seems just plain wrong.  Joel?

(*) 1 sorts before blank.  Again, this seems just plain wrong.  Joel?
Comment 9 Kohei Yoshida 2013-08-13 19:21:04 UTC
I can't reproduce this, on Linux.

Does anyone test with a non-dbgutil build?

Also, how are you sorting? There are 3 ways to sort by a column.  I need to know which sorting method was used.  Please be specific as to which menu item, which dialog, which buttons are clicked, which icon was pressed, and where the cell cursor is at the time of the sort.

Thanks.
Comment 10 Terrence Enger 2013-08-13 20:23:40 UTC
The versions of LibreOffice in the daily bibisect are non-dbgutil.
But perhaps, Kohei, you want a distributed version?

The sort I used is "Sort Ascending" on column B.  For completeness,
the whole procedure with version "latest" (4118d73...
source-hash-5bd6a51...) from the daily bibisect is ...

(1) Run soffice from the command line, naming the example file
    attached to this bug.  Program displays window "Unconfirmed.ods -
    LibreOffice Calc", sheet UNFORMATED, with cell I12 selected.  The
    cells in row 1 each have a down-arrow button.

(2) Take menu options Data > Filter.  Note that AutoFilter is selected
    (the icon at the left of AutoFilter is against a dark background).

(3) Click sub-sub-menu option AutoFilter.  Program presents dialog box
    "Autofilter not possible".  In row 1 of "Unconfirmed.ods...", the
    cells do not have down-arrow buttons.

(4) Click <OK> in the dialog.  Focus returns to "Uconfirmed.ods...".

(5) Select the whole sheet by clicking on the rectangle above row
    numbers and to the left of column numbers.  (Surely this rectangle
    has a name?)  Program colours all visible cells to show selection.

(6) Take menu option Data > Filter > AutoFilter.  Program presents
    dialog "LibreOffice Calc: The range does not contain column
    headers. Do you want the first line to be used as column header?".

(7) Click <Yes> in the dialog.  Focus returns to "Unconfirmed.ods...",
    each visible cell in row 1 has a down-arrow button, all visible
    cells are shaded to show selection, and cell I12 is outlined.

(8) Click the down-arrow button in cell B1.  After about five seconds,
    the program presents pop-up with options "Sort Ascending", "Sort
    Descending", "Top 10", "Empty", "Not Empty", "Standard Filter...".

(9) Click "Sort Ascending".  Program produces segfault.
Comment 11 Kohei Yoshida 2013-08-13 21:01:35 UTC
Thanks Terrence.  Did exactly that and ensured that your descriptions matched mine in each step.  And it still doesn't crash...

Perhaps this no longer happens in the latest master?
Comment 12 Terrence Enger 2013-08-13 22:40:42 UTC
I am starting git fetch now.  Should have something to report within 24 hours.
Comment 13 Terrence Enger 2013-08-14 23:08:54 UTC
Created attachment 84075 [details]
backtrace from later segfault, executing on a 64-bit system

With master commit ecb842e, pulled around 2013-08-13 22:30 UTC, built
with --enable-dbgutil, the results are different, but further actions
lead anyway to a segmentation fault.  So, picking up from step (9) of
comment 10, my revised report is ...

( 9) Click "Sort Ascending".  Program resorts the rows so that all
     cells visible in column B have 1.  Note that the header row has
     been sorted out-of-sight.

(10) Type ctrl-Z.  Row 1, in particular, is once again the header row
     with down-arrow buttons.

(11) Click the down-arrow button in cell B1.  Program present the
     pop-up.

(12) Click "Sort Descending".  Segfault.

The backtrace again shows function

    __gnu_debug :: _Safe_iterator_base :: _M_attach_single
        ( __gnu_debug :: _Safe_sequence_base *
        , bool
        )

So, I conclude that
    (a) this is, in some ill-defined sense, the same crash; and
    (b) a non-dbgutil build is unlikely to fail the same way.

Still, if the non-dbgutil build is not somehow unclean--perhaps
relying on undocumented behaviour of the STL, or something--it would
suggest a problem in STL.  Unless our code really is different with
--enable-dbgutil; I saw a commit fixing one of those bugs in the last
few days.
Comment 14 Terrence Enger 2013-08-15 14:09:45 UTC
Created attachment 84101 [details]
simpler test document

Here is a simpler way to reach a segfault:

(1) Download and open the attached file, simpler.ods.  Or, in an empty
    worksheet, just enter in rows 1 to 3 and columns A to C ...

        number   en    fr
             1   one   un
             2   two

    and turn on AutoFilter.

(2) Sort on column A descending.

(3) Sort on column B descending.


Emptiness of cell C3 is essential to the segfault: type "deux" in that
cell and there is no crash.
Comment 15 Kohei Yoshida 2013-08-15 14:10:19 UTC
Thanks. At this point I'll assume that this crash only happens with a dbgutil build, which likely means some mis-use of singular iterator instances which normally don't crash in the release build, but that itself is an error.
Comment 16 Kohei Yoshida 2013-08-15 14:28:12 UTC
I'll take it.
Comment 17 Kohei Yoshida 2013-08-17 17:44:46 UTC
Well, I just built myself a dbgutil build, convinced that that would reproduce this crash.  I followed Terrence's simpler example in Comment 14, but still doesn't crash... :-(
Comment 18 Kohei Yoshida 2013-08-17 17:56:15 UTC
Well, this is disappointing.  I also tried the original scenario with my dbgutil build, and still no crash...  No idea what's going on here...
Comment 19 Kohei Yoshida 2013-08-17 18:17:07 UTC
I'll put to back to the default owner for now.
Comment 20 Julien Nabet 2013-08-18 16:24:48 UTC
Terrence: on pc Debian x86-64 with master sources updated today (+ brand new LO profile), I don't reproduce this.
Could you tell precisely what you use to sort the columns?
Here what I did:
- select column A (by clicking "A")
- click Sort descending icon => LO asked me if I wanted to
  1) Extend selection
  2) Current selection
  3) Cancel
I clicked on "Extend selection" and it turned off the Autofilter.
Then I kept on the same way with Column B and no crash.

I also tried with ("Current selection"), Autofilter turned off again and no crash

config elements:
gcc (Debian 4.7.3-4)
ldd (Debian EGLIBC 2.17-92)
autogen.input includes --enable-debug and --enable-dbgutil
Comment 21 Kohei Yoshida 2013-08-18 16:33:32 UTC
(In reply to comment #20)
> Terrence: on pc Debian x86-64 with master sources updated today (+ brand new
> LO profile), I don't reproduce this.
> Could you tell precisely what you use to sort the columns?

Julien, follow Terrence's very detailed steps in Comment 10. That's almost as good as we could ask for.
Comment 22 Julien Nabet 2013-08-18 17:03:44 UTC
Created attachment 84205 [details]
console + bt with symbols on master sources

Thank you Kohei, I've been able to reproduce the problem.
I attached console logs (which showed a lot of errors) + bt but this last one seems exactly the same as Terrence's one.
Comment 23 Terrence Enger 2013-08-18 17:29:57 UTC
Julien,

You do not say whether you started with the first attachment (a normal
workbook from Joel Madero) or the later one (my trivial worksheet).
In the latter, the "Sort Descending" icon gives me a segfault, but
only after a "Sort Descending" on another column.

However, I can make a segfault on the first sort after loading the
sheet by entering row 4 with values 3,three,drei before sorting on
column C.

Kohei,

Is it the case that an empty cell should work well in a swap-cells
operation?
Comment 24 Julien Nabet 2013-08-18 17:32:08 UTC
(In reply to comment #23)
> You do not say whether you started with the first attachment (a normal
> workbook from Joel Madero) or the later one (my trivial worksheet).
> In the latter, the "Sort Descending" icon gives me a segfault, but
> only after a "Sort Descending" on another column.
Sorry Terrence, it was with the Joel Madero's one.
I'll give a try with yours and keep you informed of course:-)
Comment 25 Kohei Yoshida 2013-08-18 19:47:32 UTC
(In reply to comment #23)
> Kohei,
> 
> Is it the case that an empty cell should work well in a swap-cells
> operation?

Of course. We even have a unit test case for that with empty cells.
Comment 26 Kohei Yoshida 2013-08-18 20:02:05 UTC
How many of you have live spell check enabled when the crash happens?  Does turning that off make any difference?
Comment 27 Julien Nabet 2013-08-18 20:13:30 UTC
(In reply to comment #23)
> Julien,
> 
> You do not say whether you started with the first attachment (a normal
> workbook from Joel Madero) or the later one (my trivial worksheet).
> In the latter, the "Sort Descending" icon gives me a segfault, but
> only after a "Sort Descending" on another column.
> 
> However, I can make a segfault on the first sort after loading the
> sheet by entering row 4 with values 3,three,drei before sorting on
> column C.
I reproduced the problem with your file but only with second method (row 4).
With first method, I chose "Sort Descending" for 1 column, LO proposed me Extend/Current/Cancel, I chose Extend, then another column, did again "Sort Descending", no crash.
Comment 28 Julien Nabet 2013-08-18 20:20:02 UTC
(In reply to comment #26)
> How many of you have live spell check enabled when the crash happens?  Does
> turning that off make any difference?
Kohei: I had spell check enabled (it seems the default option since I used a brand new LO profile).
Anyway, I disabled it and gave a new try with Terrence's file method 2 (row 4) and no crash!

Then I also gave a new try to comment 10 and initial file, no crash too when live spellchecking disabled!
Comment 29 Terrence Enger 2013-08-18 20:42:47 UTC
I found Tools > Options > Languages > "Writing Aids" > "Check spelling
as you type" selected.  When I deselect it, the segfault goes away;
when I select it the segfault returns.
Comment 30 Terrence Enger 2013-08-18 20:48:51 UTC
> With first method, I chose "Sort Descending" for 1 column, LO proposed me
> Extend/Current/Cancel, I chose Extend, then another column, did again "Sort
> Descending", no crash.

For me to make the crash, "another column" must be column C, and (I
think) the sort must move the empty cell.
Comment 31 Julien Nabet 2013-08-18 20:51:55 UTC
(In reply to comment #30)
> > With first method, I chose "Sort Descending" for 1 column, LO proposed me
> > Extend/Current/Cancel, I chose Extend, then another column, did again "Sort
> > Descending", no crash.
> 
> For me to make the crash, "another column" must be column C, and (I
> think) the sort must move the empty cell.

Thank you Terrence, I reproduced the crash with first method. If live spellcheck (as you said Tools > Options > Languages > "Writing Aids" > "Check spelling
as you type") is disabled, no crash.
Comment 32 Kohei Yoshida 2013-08-19 23:43:06 UTC
Ok. That narrows it down for me. Having live spell check on (which btw is also referred to as "online spell check") is known to cause hard-to-reproduce problems because it kicks in at random and modifies cell content.
Comment 33 Julien Nabet 2013-08-22 17:43:35 UTC
*** Bug 67141 has been marked as a duplicate of this bug. ***
Comment 34 Julien Nabet 2013-08-22 17:46:30 UTC
Kohei: I found that fdo#67141 had the same bt and disabling "checking spelling" prevents LO from crashing.
The description of the bug https://bugs.freedesktop.org/show_bug.cgi?id=67141#c0 gives a even more straighforward process to reproduce the bug (at least for me :-))
Comment 35 Alex Thurgood 2013-08-23 12:31:53 UTC
FWIW, I can not reproduce on 
Version: 4.2.0.0.alpha0+
Build ID: 22d1beb78a475e4846af945afde1c4d6c263b5d6

on OSX 10.8.4.

My master build is a 64bit OSX version from 22/08/2013.

Alex
Comment 36 Alex Thurgood 2013-08-23 12:32:49 UTC
I tested with simpler.ods, and it made no difference whether I turned live spellchecking on/off.

Alex
Comment 37 Kohei Yoshida 2013-09-01 18:52:15 UTC
I've made a whole bunch of changes on the master branch just now, to rework the way how calc's online spell check works.  The new implementation doesn't modify the document content, so it should make this use case a lot more stable.
Comment 38 Terrence Enger 2013-09-02 20:54:32 UTC
@Kohei

Thank you for the fix.

I have built on ubuntu 32-bit and tested on ubuntu 32-bit and 64-bit
with no observed problem.


@Joel

Is it a bug that the sort should move the heading row to the bottom in
the example file that you attached.  That does not happen with my
trivial worksheet.


Terry.
Comment 39 Joel Madero 2013-09-02 21:26:32 UTC
Yeah it is - but I believe it's been reported on another bug. No clue the # unfortunately
Comment 40 Kohei Yoshida 2013-09-02 21:51:53 UTC
I'll park this bug on me since I fixed it.
Comment 41 Joel Madero 2013-09-02 22:29:37 UTC
Thanks Kohei - glad it got fixed before release.