Bug 61270 - FILEOPEN: Writer Opens Documents with many Form Controls very Slow
Summary: FILEOPEN: Writer Opens Documents with many Form Controls very Slow
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
4.0.1.1 rc
Hardware: Other All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: BSA
Keywords:
Depends on:
Blocks:
 
Reported: 2013-02-22 11:24 UTC by Harald Koester
Modified: 2015-01-23 18:34 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Example document in order to reproduce the problem. (21.60 KB, application/vnd.oasis.opendocument.text)
2013-02-22 11:24 UTC, Harald Koester
Details
Larger Testdokument - just a few more form elements, now showing the problem better. (33.36 KB, application/vnd.oasis.opendocument.text)
2013-02-24 23:07 UTC, info
Details
More form elements - to fill two pages now - and an exponential growth of opening time. (40.43 KB, application/vnd.oasis.opendocument.text)
2013-02-25 00:26 UTC, info
Details
Testdokument4.odt - form elements over 2 pages, different zoom factor, 50 mins opening time in Windows 7 64 Bit Pro, LibreoOffice 3.6.5.2 (37.03 KB, application/vnd.oasis.opendocument.text)
2013-02-25 01:39 UTC, info
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Harald Koester 2013-02-22 11:24:21 UTC
Created attachment 75301 [details]
Example document in order to reproduce the problem.

Problem description: 

This bug has been reported on the German discuss mailing list by Joerg Sigle. He created a text document with a lot of form controls which is opened very slow. In his case the time length is not tolerable. 

According Joerg the load time increases if the number of form controls is increased and also the other way round.

In order to reproduce the problem Joeg prepared a reduced example document which I will attach to this bug report. Just open this file and measure the length of time to open this document (my system: ~ 15 sec).

By chance I found a workaround for this problem: If the Navigator is displayed the document opens much more faster (my system: within 2 sec). This is also the case if the Navigator is set to “be displayed”. So generally LO is able to open this kind of documents fast. Joerg stated that he can't use this workaround because LibreOffice is “called through the UNO via noa-libre etc. interface”. Hence the workaround is not obvious, also to my opinion it is not a real workaround, you have to know it.

Joerg stated that the problem does not exist in early OpenOffice versions and that it exists at least since one of the early versions of LO. He also stated that the problem appears under Linux and under Windows.

Expected behaviour: Also fast opening if the Navigator is not displayed.

My system: Win 7, LO version: 4.0.0.3
Operating System: All
Version: 4.0.0.3 release
Comment 1 Joel Madero 2013-02-22 17:26:15 UTC
Opens quickly for me even with navigator off - approximately 3 seconds to open:

Version 4.1.0.0.alpha0+ (Build ID: 9601a571d42b0199bdccf2256fc8be82e14af8a)
Bodhi Linux 2.2

Marking as WFM, adding Joren to confirm that the issue isn't present in 4.1 master

@Reporter - if you could test against a nightly build that will help tremendously, if not, if Joren confirms that in 4.1 it's fixed, we'll assume that it's been fixed and should be corrected for users by 4.1 release (maybe 4.0.1 depending on if the fix was back ported)
Comment 2 Jorendc 2013-02-22 18:47:58 UTC
(In reply to comment #1)
> Marking as WFM, adding Joren to confirm that the issue isn't present in 4.1
> master

I can confirm it opens without any delay (a popup message will ask to enable macros or not, but after enabling/disabling opens fluidly). Tested using LibreOffice 4.0.1.1 rc and LibreOffice Version 4.1.0.0.alpha0+ (Build ID: 99501a839f6d777c24bc9210787fd14dc3aad67).
Comment 3 Harald Koester 2013-02-24 18:13:06 UTC
Tested again under Win 7 Prof. with:

Version 4.1.0.0.alpha0+ (Build ID: 74f74aa5470fe631c7827897742c0ccbddcf6ad)
TinderBox: Win-x86@6, Branch:master, Time: 2013-02-23_23:21:01

Problem still exists how described !!!

and with

Version 4.0.1.1 (Build ID: 2c0c17a6e4bee0ee28131ea4bdc47edc700d659)

Problem also still exists !!!

I can't check the behaviour with Linux. This info I got from the original user. A problem just with Windows? 

Are there additional infos I can provide?
Comment 4 info 2013-02-24 23:07:08 UTC
Created attachment 75464 [details]
Larger Testdokument - just a few more form elements, now showing the problem better.

From your comments I assume that I made my first testdokument.odt too small = too fast on your computers.

When I had tested it on Windows with LO 3.6.5.2, it took some 30 secs. without the workaround (too long), and 4 secs with the workaround (ok).

Now, testing on Linux with the (currently delivered by Debian) LO 3.5.4, it takes about 8 secs, which is the same with or without the workaround. So I couldn't decide whether this would be correct or not.

Then, I just copy/pasted a few more form elements, so that the table on the first page is now completely filled. This is the enclosed testdokument2.odt

Under Linux, it takes 55 secs to open this, no matter whether the Navigation dialog is open or not. The Macro-Warning appears fast, then, when I click "OK", the window is not redrawn before some 50 secs are over. I used this command to test it; clicking the Macro Warning ok and closing the program after it had opened the document as fast as possible:

$ time /opt/libreoffice3.6/program/swriter /c/Users/jsigle/Desktop/Testdokument2.odt 
p11-kit: couldn't load module: /usr/lib/i386-linux-gnu/pkcs11/gnome-keyring-pkcs11.so: /usr/lib/i386-linux-gnu/pkcs11/gnome-keyring-pkcs11.so: Kann die Shared-Object-Datei nicht öffnen: Datei oder Verzeichnis nicht gefunden

real    0m55.990s                                                                                                                                  
user    0m54.378s                                                                                                                                  
sys0m0.124s                                                                                                                                   

(The file gnome-keyring is not found as this is a 64-bit system and the file is located in the -amd64 subtree, but never mind.)

So the bug exists under Linux, but the workaround does NOT help under Linux.

I'll try to test it with a more recent version as well...

Kind regards, Joerg (original bug reporter)
Comment 5 info 2013-02-24 23:29:31 UTC
I've now tested it with 4.0.1.1-1 under Linux, and the problem remains.
And as before, opening the navigation dialog would NOT help under Linux.

Once again, confirming the macro warning with ok - redraw delayed with no response from the program for some time - closing the libre office window as soon as it is possible, gives me this result:

$ time /opt/libreoffice4.0/program/swriter /c/Users/jsigle/Desktop/Testdokument2.odt 

real    0m39.289s
user    0m36.853s
sys     0m0.176s

So the overall speed of 4.0.1.1 is faster than 3.5.4, but clearly, the problem is still there.

Here is some output from "nice top" during the second attempt, after confirming the macro warning, during the phase where libre office would not respond. At some time, I also observed 103% cpu usage by libre office. Apparently, there is one process that uses one CPU completely during all of the unwanted delay, plus maybe a little bit going on in addition to that:

top - 00:16:43 up  7:32,  5 users,  load average: 0.86, 0.72, 0.57
Tasks: 211 total,   3 running, 208 sleeping,   0 stopped,   0 zombie
%Cpu(s): 13.4 us,  0.3 sy,  0.0 ni, 86.1 id,  0.0 wa,  0.0 hi,  0.1 si,  0.0 st
KiB Mem:  16370056 total,  8987272 used,  7382784 free,  3017420 buffers
KiB Swap:        0 total,        0 used,        0 free,  2197516 cached

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
14335 jsigle    20   0  792m 128m  78m R  99.7  0.8   0:22.14 soffice.bin
10797 root      20   0  172m  64m  34m S   7.0  0.4   2:54.03 Xorg
11015 jsigle    20   0  364m  30m  22m S   3.3  0.2   0:07.41 konsole
11235 jsigle    20   0 1453m 611m  36m S   1.7  3.8   4:50.95 firefox
 2550 root      20   0  4508 1080  524 S   1.0  0.0   0:16.89 acpid
...
Comment 6 info 2013-02-24 23:59:26 UTC
Sorry, I haven't found anything beyond 4.0.x - even on
http://dev-builds.libreoffice.org/daily/
there is apparently nothing like a 4.1.x version.

Can you please tell me where I could find that?
Preferrably for both Windows 7 / 64-bit and as Linux Debian installers?

Until I can test it on anything newer, I'm changing the status from "Resolved WorksForMe" back to "Reopened" - because as far as I can see, it was only my test case that was too small to show the problem on your Linux systems - with the second test document, I've confirmed the problem for the currently available 4.0.1.1 prerelease under Linux - and I cannot test anything newer.

I hope the new test document is sufficiently large to show the problem even on a fast system under Linux (mine is "only" an i7 laptop at 1.6 GHz). And please note, the workaround with the navigation Window does apparently NOT improve things under Linux.

Thanks in advance and kind regards, Joerg
Comment 7 info 2013-02-25 00:26:46 UTC
Created attachment 75467 [details]
More form elements - to fill two pages now - and an exponential growth of opening time.

One more attachment - Testdokument3.odt

This extends the table of form elements to cover (almost) another page.

I.e. there are roughly 2 x the form elements as in Testdokument2.

Here are the results with LibreOffice 4.0.1.1-1 on Linux:

$ time /opt/libreoffice4.0/program/swriter /c/Users/jsigle/Desktop/Testdokument3.odt 

real    4m42.116s
user    4m38.882s
sys     0m0.198s

I think this clearly shows that the time required to open the document grows exponentially with the number of form elements.

So effectively, LibreOffice may currently be unusable for forms with more than a couple of elements. Please reconsider your rating of the bug severity given this finding.

And... I guess it would be perfectly reasonable to ensure that your pizza has been delivered, before trying to open the same document under Windows (as long as you don't have the navigator window enabled)... :-)

Kind regards, Joerg
Comment 8 info 2013-02-25 01:36:37 UTC
My pizza guess regarding Testdokument3.odt was wrong. :-(
(But please continue reading.)

I tried it (not with pizza, but with potatoes au gratin). Double clicking on Testdokument3.odt (as saved from Linux 4.0.1.1-1) on the Desktop of a freshly booted Windows 7 64 Pro with LibreOffice 3.6.5.2 required "only" some 25 seconds to open.

This was with the Navigator displayed - but, surprisingly, it wouldn't get much faster or slower when I disabled/re-enabled that.

?!?!?!?

So I tried to save it from the Windows Version, as Testdokument4.odt, and reopen it. Roughly the same result.

Then, I changed the zoom factor from "fit horizontally + vertically (both pages at once)" to (actually, I don't know) maybe page width, or 100%. And saved it again.

First, this took several minutes for saving. Once again, LibreOffice would not respond, and the respective warning from Windows would appear multiple times, while I finally opened the browser to visit bugzilla.

LibreOffice finally terminated, so I tried to open the newly written Testdokument4.odt.

This is still ongoing - I've searched this error in bugzilla, not found it, opened thunderbird, come here via Haralds link, logged in, written this report, and in the background, LibreOffice is still displaying a gray Window.
(7 Minutes and counting).

Needless to say my potatoes are gone and I would have loved to have a pizza by now :-(

It looks like that forecast regarding opening time under Windows was not completely wrong - but it might also depend upon the zoom factor?!?

And obviously, the bug has multiple facets. It behaves differently between Windows and Linux, in Windows differently between with/without Navigator window, maybe also depending on the zoom factor. Altogether, I guess that really can make it a show stopper for anyone trying to work with forms.

When LibreOffice may finally return and I get the final processing time, I will also upload Testdokument4.odt - saved under Windows, with the different zoom factor. Now, it's 16 minutes past pressing the return key...

Mind you, the actual contents of Testdokument4 should be the same as in Testdokument3, only the display scaling should differ.

I've waited 22 mins by now. I could have put a pizza in the oven and baked it by now... Grr. Why can't computing be moderately deterministic by 2013? If Humanity gets a second change, may I humbly suggest that engineers focus on reliability *only* - instead of adding new registers, memory, objective programming, and thereby merely speeding up processing a million times, and at the same time giving raise to complexity of code by a million times as well, whose outcome may still apear, or it may not - without the user knowing in advance...

To be true, I don't expect this to be an endless loop. Task manager says 13% cpu usage - i.e. 100% of one core, just what I've seen before. My previously collected numbers may suggest linearly extrapolated waiting times from 64 secs to 15 mins, but we're at 45 minutes now. So we're clearly on exponential territory by now, aggravated by the Windows environment - exactly what I feared when I first mentioned the pizza. But the possible variation is just so large that I'll cancel the measurement for now.

... Oh, whatdoisayredeterministic or not: here we are. 50 minutes opening time. Just what it needs to eat a pizza (and to make it first).

:-) :-) :-)

Kind regards, Joerg
Comment 9 info 2013-02-25 01:39:38 UTC
Created attachment 75468 [details]
Testdokument4.odt - form elements over 2 pages, different zoom factor, 50 mins opening time in Windows 7 64 Bit Pro, LibreoOffice 3.6.5.2

See previous comment. Zoom setting may have been "fit-width"; window in "normal" state (i.e. neither maximized, nor minimized) - js
Comment 10 info 2013-02-25 01:45:55 UTC
Same Testdokument4.odt, same Windows environment - Opened Writer first, enabled the Navigator window, dragged Testdokument4 over - 10 seconds opening time.

I think this *is* impressive, isn't it?

Kind regards, Joerg
Comment 11 info 2013-02-25 01:54:38 UTC
Just tried it again with Testdokument3.odt (dual-page display of complete document, zoom=fit hor + vert). Confirmed the ca. 25 secs, incl. program startup time, both with or without visible Navigator window.

While it's open:

Testdokument.odt:
4 table boxes = 2 rows filled, 40 x 4 = 160 form elements including labels.

Opening times:
Windows:
Navigator disabled:30 secs.
2 secs.

Testdokuemt
Comment 12 info 2013-02-25 02:09:57 UTC
(Sorry, tab key sent off partially written comment above.)



Just tried it again with Testdokument3.odt (dual-page display of complete document, zoom=fit hor + vert). Confirmed the ca. 25 secs, incl. program startup time, both with or without visible Navigator window.

While it's open:

Testdokument.odt: (saved under Windows)
4 table boxes = 2 rows filled, 40 x 4 = 160 form elements including labels.

Opening times (may or may not include program loading time of up to 15 seconds, but on a system with a lot of RAM = cache, so very imprecise measurements):

Windows:
Navigator disabled:  30 secs.
Navigator enabled:    2 secs.
Linux:
Navigator disabled:   8 secs.
Navigator enabled:    8 secs.

Testdokument2.odt: (saved under Linux)
14 table boxes = 7 rows filled, 40 x 14 = 560 form elements
Linux, 3.5 or 3.6
Navigator disabled:  55 secs.
Navigator enabled:   55 secs.
Linux, 4.0.1.1
Navigator disabled:  39 secs.
Navigator enabled:   39 secs.

Testdokument3.odt: (saved under Linux)
20 table boxes = 10 rows filled, 40 x 20 = 800 form elements
Linux, 4.0.1.1
Navigator ???:      4:40 min:secs (i.e. 280 secs)

Windows, 3.6.5.2
Navigator disabled:   25 secs
Navigator enabled:    25 secs

Testdokument4.odt (interim version, not published, merely saved under Windows):
Windows, 3.6.5.2
Navigator disabled:   25 secs
Navigator enabled:    25 secs  (same as above)

Change zoom factor from fit hor + vert to fit width (Fensterbreite):

Testdokument4.odt (saved under Windows - took several minutes):
Windows, 3.6.5.2
Navigator disabled:   50 min (i.e. 3000 secs)
Navigator enabled:     8 sec (excl. program loading time, maybe 25 secs incl.)

I just started out to create a Testdokument5.odt, by changing zoom factor to 100% (from Testdokument3.odt, where it is still fit hor + vert). But saving that dokument has already taken 1:50 min:sec. After re-opening, the main window area stays completely gray again, so I think the findings will be similar to those from Testdokument4.odt after changing the zoom factor. I'm not going to wait for it.

Hope that this information may all help you to improve LibreOffice.
Thank you for your reading time & your efforts & good luck! Joerg
Comment 13 Matthew Francis 2014-10-15 16:39:49 UTC
This has been fixed on 4.4 master, and a backported fix should also make it into 4.3.3

(although this bug is older, the investigation that led to the fix took place on bug 84752 and bug 85032. Other reports of the same problem will be marked duplicate)

*** This bug has been marked as a duplicate of bug 84752 ***
Comment 14 info 2014-10-17 01:49:22 UTC
Hello.

Thanks for finally fixing this. 

I tested 4.3.3.1 and 4.3.4.0.0 as available on 2014-10-16.
These still take long. (This is just for the record).

I also tested libo-master~2014-10-16_01.04.13_LibreOfficeDev_4.4.0.0.alpha0_Win_x86.msi

Inside my target application, it opens my actual-use-document in 23 secs.
That's about the same order of magnitude as before the bug had appeared. :-)

(For reasons below): I measured loading and saving times with manually operated standalone LibreOffice 4.4.0.0 and the test documents previously attached to this bug report.

The results look very nice - for reference:

                            loading    saving
Bug Many from elements.odt    2s        2s
Testdokument2.odt             4s       10s
Testdokument3.odt             8s       10s
Testdokument4.odt             8s       10s

myActualUseCase.odt           6s        8s

(Win7 64-bit, some i7 CPU)
(Remember the last document required more than 50 mins *with* the bug.)

Thank you for locating and fixing the issue!

-

N.B.: Actual reasons to make above measurements / related issues:

(1) An auto-save within myActualUseCase.odt in my target app stalled for more than half an hour before I killed it (reason yet unknown). This is reproducable. So I wasn't sure whether a similar bug would still remain on the saving-side of file-i/o. Given above save times, I don't think it's at that specific place.

(2) There is still *much* more flickering on opening and scrolling/redisplaying forms (i.e. myActualuseCase.odt and Testdokument?.odt etc.) in LibreOffice 4.4.0.0 than e.g. in OpenOffice 3.x or 2.x.

(3) When I execute the macro in myActualUseCase.odt which takes data from all input fields and processes them, 4.4.0.0 crashes in 100%.
The document can be recovered thereafter.
The recovered document then contains processed data.

OO 3.3.0, however, does *not* crash. And produce much smoother redisplaying.
Sidenote: The recovery name is the name under which the document was first opened, not under which it was last saved before the crash. That's dangerous. 

So the specific problem of bug report 61270 is probably solved. :-) :-) :-)

But its highly probable that more issues have been introduced when Field-/Macro processing went from OpenOffice 3.3.x to LibreOffice 3.x.x

I don't know whether I have time to do further research, therefore I noted these issues here even if I know I should rather file a separate bug report (or search all available ones first).

Thanks again and kind regards! Joerg
Comment 15 Matthew Francis 2014-10-17 02:05:13 UTC
Thanks for checking. If you can reproduce other issues, please raise another bug or bugs.

The specific document you are using would be useful for a new bug report, but if you're not comfortable attaching it publicly, please feel free to drop into IRC chat.freenode.net #libreoffice-qa (via http://webchat.freenode.net/?channels=libreoffice-qa if you don't have an IRC client handy) and we can help evaluate it without making the file public.
Comment 16 Harald Koester 2015-01-08 15:07:08 UTC
I checked this bug again with version 4.3.3 and 4.3.5. To my opinion this bug is not fixed in version 4.3.3 as stated in comment 13. It still lasts about 15 sec to open the first example document without displaying the Navigator and about 2 sec with displaying the Navigator.

Hence status set back to UNCONFIRMED because this bug has never been confirmed independently.
Comment 17 Robinson Tryon (qubit) 2015-01-17 23:29:46 UTC
TESTING with LO 4.4.0.2 + Ubuntu 14.04

(In reply to Harald Koester from comment #16)
> I checked this bug again with version 4.3.3 and 4.3.5. To my opinion this
> bug is not fixed in version 4.3.3 as stated in comment 13. 

REPRO Steps:
- Make sure the Navigator is not enabled
- Shut down LibreOffice
- Time how long it takes to open 'Bug many form elements.odt' (attachment 75301 [details])

Total time: about 12 seconds; Getting the Macro-warning dialog made the timing a little variable

- Activate the Navigator

- Shut down LibreOffice
- Time how long it takes to open 'Bug many form elements.odt' (attachment 75301 [details])

Total time: about 8 seconds

> It still lasts
> about 15 sec to open the first example document without displaying the
> Navigator and about 2 sec with displaying the Navigator.

I don't see that big of a gap.

Trying the same thing with Testdokument4 (attachment 75468 [details]):

==> Too slow, LibreOffice frozen for 4+ minutes

Trying Testdokument 3 (attachment 75467 [details]):

Without Navigator: LibreOffice opens after ~ 4.8 min
With Navigator: LibreOffice opens after ~ 5 min

RESULT: the Navigator isn't speeding up anything for me.

(In reply to info from comment #14)
> I also tested
> libo-master~2014-10-16_01.04.13_LibreOfficeDev_4.4.0.0.alpha0_Win_x86.msi
> ... 
>                             loading    saving
> Bug Many from elements.odt    2s        2s
> Testdokument2.odt             4s       10s
> Testdokument3.odt             8s       10s
> Testdokument4.odt             8s       10s
> ...
> (Win7 64-bit, some i7 CPU)

Your load times are much better than mine :-)

So I *do* see slow load times, but I don't see a difference in speed related to the Navigator being open or closed.
Comment 18 Harald Koester 2015-01-23 18:34:06 UTC
I checked the bug again with version 4.3.5 and 4.4.0.2. With version 4.3.5 it still exist. But with 4.4.0.2 it WORKSFORME. With the first example document the load time is now about 1 second with and without displayed Navigator (Win7, 32 Bit). It seemed that the fix has not been backported to version 4.3.3 as stated in comment 13. But nevertheless from my point of view this bug can be closed.