Download it now!
Bug 61520 - EDITING: Calc freezes/crashes while opening a data validity list in a cell
Summary: EDITING: Calc freezes/crashes while opening a data validity list in a cell
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
4.0.0.3 release
Hardware: Other All
: high major
Assignee: Eike Rathke
URL:
Whiteboard: target:4.3.0 target:4.2.3 target:4.1.6
Keywords:
Depends on:
Blocks:
 
Reported: 2013-02-26 17:48 UTC by rpr
Modified: 2014-03-07 03:46 UTC (History)
10 users (show)

See Also:
Crash report or crash signature:


Attachments
test XLS file (13.50 KB, application/vnd.ms-excel)
2013-02-26 17:48 UTC, rpr
Details
bts at random with master sources (3.47 KB, application/bzip2)
2013-03-10 16:51 UTC, Julien Nabet
Details
Windbg output 4.0.3 (23.02 KB, text/plain)
2013-04-10 07:38 UTC, blank
Details

Note You need to log in before you can comment on or make changes to this bug.
Description rpr 2013-02-26 17:48:14 UTC
Created attachment 75589 [details]
test XLS file

How to reproduce:

(1) In Calc open the XLS file in the attachment created with MS Office 2003. On Sheet2 it has a couple of values in the first column. The values are named as list_data cell range. The first cell (A1) on Sheet1 has data validation enabled that allows only values from the list_data cell range.

(2) Click the drop-down list in the A1 cell in order to select a value from the list. Calc freezes at once (does not respond to any keyboard or mouse command) and causes high CPU usage.

I tested this in LibreOffice 4.0 on MS Windows 7 64-bit and in LibreOffice 3.6.5 on Ubuntu 12.04 64-bit.
Comment 1 rpr 2013-02-27 09:14:09 UTC
Oh, this also happen if the data validation list is created in Calc:

(1) Start a new spreadsheet in Calc. On Sheet2 enter a couple of values in the first column.

(2) Add a name for the range of cells: select the cells just entered and in the menus choose Data > Define Range and enter a name for the range, e.g. mydata.

(3) In the first cell of Sheet1 define a data validation:
Data > Validity > Criteria tab
Allow: Cell range 
Source: =mydata

As soon as you confirm this, Calc crashes.
Comment 2 Michel Rudelle 2013-03-10 12:48:33 UTC
I confirm,
This happens also if you use a named range (Insert > Names > Define) or enter a cell range (like A1:A2)
Tested with:
Version 4.0.1.2 (Build ID: 84102822e3d61eb989ddd325abf1ac077904985)
Vista-32b
Notice that usually I found the crash when using this command for the 2nd time

The crash is still there on the master correcting the bug 58630:
Version 4.1.0.0.alpha0+ (Build ID: d7ca9b5cbcac463dd1baa089180bac2a1c0e5e3)
TinderBox: Win-x86@6, Branch:master, Time: 2013-03-09_23:19:58
Vista-32b
Comment 3 Julien Nabet 2013-03-10 14:49:12 UTC
Comment on attachment 75589 [details]
test XLS file

Mimetype fixed
Comment 4 Julien Nabet 2013-03-10 16:51:08 UTC
Created attachment 76278 [details]
bts at random with master sources

On pc Debian x86-64 with master sources updated today, I reproduced the hanging problem.
I attached 3 bts at random on a tar.bz file
Comment 5 Julien Nabet 2013-03-10 16:53:56 UTC
Kohei/Markus/Eike/Caolán: calc or vcl problem I don't know because bt attached show both parts.
Comment 6 Caolán McNamara 2013-03-12 09:53:13 UTC
Looks to me that calc fills a list box with 0x10000 entries and its just ultra slow trying to do all that
Comment 7 Caolán McNamara 2013-03-12 10:00:07 UTC
Hang seems to happen in 3.6.5 as well. Looks like a calc-specific problem, so I'm out :-)
Comment 8 blank 2013-03-13 15:02:25 UTC
crash also with 4.0.1.2, also with this build Version 4.0.3.0+ (Build ID: e15488286d2f5eb4fb5414cf3a17690b36eed8a)

1- start new doc
2- in 1sheet make a list of values and define range and name it "aa" make a second list in column 2 and name it bb
3- on second sheet select cells > data > validity
4- allow > cell range
5- source > tape aa
6- make the same to column 2 in source aa is already set if I wite bb calc crash
Comment 9 blank 2013-03-13 16:44:15 UTC
I try on other computers:
windows 7 x64 + lo 4.0.0.3 = crash also
linux mint 32 with LO 4.0.1.2 or 3.6.2 = working

i try to disable java 7 update 17 on windows computer without more success.
Comment 10 José Consuegra 2013-04-08 18:42:05 UTC
Confirm that crashes also on LO 4.0.2.2

Steps to reproduce

1) Create a new doc
2) cells -> data -> validity
3) allow: Cell Interval
4) Select a range and apply values

5) when performing steps 2 and 3 again over another cell, the application shows another dialog with the previous range and when you try to close it finally crashes.
Comment 11 Julien Nabet 2013-04-08 20:02:39 UTC
blank: could you give a try to 4.0.2?

José: On pc Debian x86-64 with master sources updated today, I don't reproduce the problem after having followed your steps. On which env are you?

1) If on Linux, could you try to retrieve a bt by following this link: https://wiki.documentfoundation.org/BugReport#How_to_get_a_backtrace_on_Linux

2) on Mac, you can bring extra information by following this: https://wiki.documentfoundation.org/BugReport#How_to_get_debug_information_on_Mac_OS_X

3) On Windows, it's more complicated, see https://wiki.documentfoundation.org/BugReport#How_to_get_a_backtrace_on_Windows
Comment 12 rpr 2013-04-08 21:19:09 UTC
On Ubuntu 12.04 64-bit with LibO 4.0.2.2 I've just run the tests in Comment1 and Comment11 and the problem didn't occur.
Comment 13 rpr 2013-04-08 21:20:23 UTC
On Ubuntu 12.04 64-bit with LibO 4.0.2.2 I've just run the tests in Comment1 and Comment10 and the problem didn't occur.
Comment 14 Caolán McNamara 2013-04-09 00:08:58 UTC
The double dialog thing of comment #10 is probably bug 61948, but that's unrelated to comment #1 which is using the in-document drop-down listbox
Comment 15 José Consuegra 2013-04-09 07:28:42 UTC
@(In reply to comment #14)
> The double dialog thing of comment #10 is probably bug 61948, but that's
> unrelated to comment #1 which is using the in-document drop-down listbox

Yes, you're right... Seems that I put it in here by mistake...

My apologies... :/
Comment 16 blank 2013-04-10 06:50:03 UTC
 ON Version 4.0.2.2 (Build ID: 4c82dcdd6efcd48b1d8bba66bfe1989deee49c3)
problem still occur on windows 7 sp1 x64
Comment 17 blank 2013-04-10 07:38:52 UTC
Created attachment 77723 [details]
Windbg output 4.0.3
Comment 18 blank 2013-04-10 07:40:10 UTC
i have make the stuff with windbg and lo Version 4.0.3.0+ (Build ID: afa375d7f7b1094c96597bbc3793fbc9adb87d6)
Comment 19 blank 2013-04-19 06:44:25 UTC
I need to try with my boring users, but with this buid Version 4.0.3.1 (Build ID: a67943cd4d125208f4ea7fa29439551825cfb39) it seems now working
Comment 20 Michel Rudelle 2013-05-09 11:39:51 UTC
Hi,
Now that bug 58630 and bug 61948 are fixed, it looks clear for me that this is not a bug but a particular problem when using a too long list (if anyone can tell me who can choose an element among 1000000 ? - this is due to a permissive use of Excel where you can select an entire column)

Tests performed with:
Version 4.0.3.3 (Build ID: 0eaa50a932c8f2199a615e1eb30f7ac74279539)
Vista-32b

- on a new spreadsheet with a list A1:Axxxxxx:
time before seeing the list when I click on the drop-down arrow
100000 cells -> 4 seconds 
200000 cells -> 50 seconds

- on the  attachment 75589 [details]  (test XLS file):
  in Sheet2, enter Hello in cell A1048576 (last one on the column)
  in Sheet1, enter Hell in cell A1, 2 seconds before reading "invalid value"
                   enter Hello in cell A1, 2 seconds before data is validated
=> Validity function works perfectly

So, imho, there is nothing to do, just explain to users, or have an improvement, eg: the drop-down arrow should not be shown when the list is greater than a readable length
Comment 21 retired 2013-11-17 12:56:59 UTC
Opening test XLS file with Version: 4.2.0.0.alpha1+
Build ID: 868103846b9b32bfecd77c08055fdca69d0265c2
TinderBox: MacOSX-x86@48-TDF, Branch:master, Time: 2013-11-14_23:51:46

leads to a crash

Why is this is NEEDINFO? Imo this is NEW and valid.
Comment 22 Julien Nabet 2013-11-17 14:09:04 UTC
On pc Debian x86-64 with master sources updated yesterday, I still have the hanging. I've waited for about 2 minutes with make debugrun to retrieve the part where it hangs.
Program received signal SIGINT, Interrupt.
[Switching to Thread 0x2aaaaab1be40 (LWP 10810)]
0x00002aaab08dd419 in ImplDelData::~ImplDelData (this=0x18c4780, __in_chrg=<optimized out>) at /home/julien/compile-libreoffice/libo/vcl/source/window/window.cxx:320
320	ImplDelData::~ImplDelData()
(gdb) bt
#0  0x00002aaab08dd419 in ImplDelData::~ImplDelData (this=0x18c4780, __in_chrg=<optimized out>) at /home/julien/compile-libreoffice/libo/vcl/source/window/window.cxx:320
#1  0x00002aaab08efd86 in Window::CallEventListeners (this=0x2274f00, nEvent=1153, pData=0x6f59) at /home/julien/compile-libreoffice/libo/vcl/source/window/window.cxx:5311
#2  0x00002aaab03f2e57 in ListBox::InsertEntry (this=0x2274f00, rStr="", nPos=65535) at /home/julien/compile-libreoffice/libo/vcl/source/control/lstbox.cxx:1023
#3  0x00002aaac9640a0f in ScGridWindow::LaunchDataSelectMenu (this=0x223f8b0, nCol=0, nRow=0, bDataSelect=true)
    at /home/julien/compile-libreoffice/libo/sc/source/ui/view/gridwin.cxx:1206
#4  0x00002aaac964384d in ScGridWindow::HandleMouseButtonDown (this=0x223f8b0, rMEvt=..., rState=...)
    at /home/julien/compile-libreoffice/libo/sc/source/ui/view/gridwin.cxx:1889
#5  0x00002aaac96429bb in ScGridWindow::MouseButtonDown (this=0x223f8b0, rMEvt=...) at /home/julien/compile-libreoffice/libo/sc/source/ui/view/gridwin.cxx:1643
#6  0x00002aaab090dde3 in ImplHandleMouseEvent (pWindow=0x18c4780, nSVEvent=1, bMouseLeave=0 '\000', nX=111, nY=179, nMsgTime=23463458, nCode=1, nMode=3)
    at /home/julien/compile-libreoffice/libo/vcl/source/window/winproc.cxx:780
Comment 23 rpr 2014-02-25 10:38:08 UTC
(In reply to comment #20)
> Hi,
> Now that bug 58630 and bug 61948 are fixed, it looks clear for me that this
> is not a bug but a particular problem when using a too long list (if anyone
> can tell me who can choose an element among 1000000 ? - this is due to a
> permissive use of Excel where you can select an entire column)
> 
> [...]
> 
> So, imho, there is nothing to do, just explain to users, or have an
> improvement, eg: the drop-down arrow should not be shown when the list is
> greater than a readable length

In the XLS test file, Sheet2 contains the data range named data1 that is defined as entire column A but actually it contains only four values. That definition enables adding new values to the data range without the need to redefine it, which is quite convenient.

When user clicks to open a drop-down list in a validation cell I'd suggest that Calc checks the data range in order to find the last non-empty value in the range and then to work only with the real range of data.

I'd say there is a function that finds the last non-empty value on a sheet when you click Ctrl-End.
Comment 24 Anatoly Moskovsky 2014-02-25 10:47:14 UTC
> In the XLS test file, Sheet2 contains the data range named data1 that is
> defined as entire column A but actually it contains only four values. That
> definition enables adding new values to the data range without the need to
> redefine it, which is quite convenient.
> 
> When user clicks to open a drop-down list in a validation cell I'd suggest
> that Calc checks the data range in order to find the last non-empty value in
> the range and then to work only with the real range of data.
> 
> I'd say there is a function that finds the last non-empty value on a sheet
> when you click Ctrl-End.


Actually whole-column ranges worked fine in 3.5 (the dropdown was shown immediately if a few values only were at the beginning of the column). 
But now it hangs in 4.x.
So it looks like a regression.
Comment 25 Eike Rathke 2014-02-25 16:30:52 UTC
Following the original description and breaking/stepping through some list insertions, I believe the underlying cause is that vcl ListBox handles only sal_uInt16 elements (i.e. 64k) though the implementation container meanwhile (since 4.0?) now is a std::vector capable of holding size_t elements, but we're stuffing in 1M elements. With the old LIST_APPEND=0xffff position value that results in for all elements greater than capacity 0xffff the element will be inserted at that position instead of appended to the list, effectively having to shift all elements from that position for every element inserted ... O(N^2) ...

And no, whole column (if it contained data) did not work in 3.5 because the list was truncated after 64k elements.

I'll try to take a stab at this.
Comment 26 Anatoly Moskovsky 2014-02-25 16:41:54 UTC
(In reply to comment #25)

> And no, whole column (if it contained data) did not work in 3.5 because the
> list was truncated after 64k elements.

No sane person would ever need so much entries in a dropdown.
So even if there was some bug related to the 16-bit overflow, it is not as critical as this hang is, which occurs even for a single item.
Comment 27 Commit Notification 2014-03-05 13:31:47 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=68ec95b3f80408ae50897b043eed69a07d084df9

made ListBox handle more than 64k elements, fdo#61520 related



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 28 Eike Rathke 2014-03-05 14:31:13 UTC
With that change waiting time is already down to "only" a few seconds. Next step will be to limit the entries to the used data area instead of appending a million blank entries.
Comment 29 Commit Notification 2014-03-06 00:27:17 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=7122ef19847b26529ed1d5bad40df869e91a8495

resolved fdo#61520 do not add multiple empty strings to the validation list



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 30 Eike Rathke 2014-03-06 00:39:56 UTC
Pending review
for 4-2 at https://gerrit.libreoffice.org/8469
for 4-1 at https://gerrit.libreoffice.org/8470
Comment 31 Commit Notification 2014-03-07 03:38:35 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-4-2":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=b1119c7b80b70672f86c3415725bc0806be14981&h=libreoffice-4-2

resolved fdo#61520 do not add multiple empty strings to the validation list


It will be available in LibreOffice 4.2.3.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 32 Commit Notification 2014-03-07 03:46:09 UTC
Eike Rathke committed a patch related to this issue.
It has been pushed to "libreoffice-4-1":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=3ce626c81ac4a9b4b70c61d9fa576f6ae2c85c8f&h=libreoffice-4-1

resolved fdo#61520 do not add multiple empty strings to the validation list


It will be available in LibreOffice 4.1.6.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.