Bug 58040 - FILEOPEN: Writer hangs when opening large files.
Summary: FILEOPEN: Writer hangs when opening large files.
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
4.0.0.0.beta1
Hardware: x86 (IA32) All
: high major
Assignee: Michael Stahl (CIB)
URL:
Whiteboard: target:4.2.0 target:4.1.2 target:4.0.6
Keywords: regression
Depends on:
Blocks:
 
Reported: 2012-12-09 07:00 UTC by Aibara
Modified: 2013-08-14 09:19 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
701 page test document. (22.12 KB, application/vnd.oasis.opendocument.text)
2012-12-09 07:00 UTC, Aibara
Details
3 odt test files (rtf files from bug 44736 saved as ODT) (74.60 KB, application/x-7z-compressed)
2012-12-11 16:49 UTC, s-joyemusequna
Details
ODF validator output (95.54 KB, image/png)
2012-12-21 13:17 UTC, s-joyemusequna
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Aibara 2012-12-09 07:00:20 UTC
Created attachment 71224 [details]
701 page test document.

I've had a problems with large files in Writer since the 3.5 series began. In the 3.5 and 3.6 series, big files take a long time to save, whereas in the 3.4 series and before I never had a problem and the saving process would begin instantly. 

In 4.0.0.0 beta1 the problem seems a bit different: every time a big file is opened LibreOffice Writer hangs. 

Steps to reproduce:
Open attached file.

Expected behavior:
File opening progress bars fill up and then the document can be read and edited.

Actual behavior:
After the progress bars fill, LibreOffice freezes for up to 40 seconds and is totally unresponsive.

The attached file "opens" (i.e. the progress bars fill) in about 2 seconds, but then hangs – LibreOffice goes grey – for 40 seconds. Note that the attached example is only a 700 page file, and is simply the word "why" repeated; no fancy formatting at all. Smaller files, like 300 pages ones, still hang for a bit. When I hit around 500 pages it hangs for the full 40 seconds, though oddly that's the same for files that are thousands of pages long.

I'm using Ubuntu 12.04 on a ThinkPad T60p, but I also tried this on a friend's newer Win7 Ultimate desktop. The same thing happens, though the file opens a bit faster. It still hangs for over 20 seconds (the title bar says "not responding"). 

Again, note that there was no problem with these large files in the 3.4 series and those before it. 


For the older 'saving' hang bug:
https://bugs.freedesktop.org/show_bug.cgi?id=51311

Thanks.
Comment 1 Emir Sarı (away) 2012-12-09 08:22:32 UTC
Confirmed with both 3.6.4.3 and 4.0.0.0 beta1 - OS X 10.7.5

Behaviours:
3.6.4.3- Instantly opens, fast scrolling, but inability to edit file, cursor does not even blink, consistent hanging. CPU tops 100+%

4.0.0.0 beta 1- Hanging while opening, other than that same with above. Hangs the whole system with SSD. It didn't hang like for 40 seconds, 'cause I have a SSD, but it was still pretty much.
Comment 2 s-joyemusequna 2012-12-11 16:49:28 UTC
Created attachment 71345 [details]
3 odt test files (rtf files from bug 44736 saved as ODT)

Average load time until the file is visible in editor, tested with Windows XP: 
LOdev 4.0.0 Beta1 (2012-12-07).

Tested files (see attachment): 
1. lorem421pages_unformated.odt
2. lorem421pages_format_parag.odt
3. lorem421pages_format_parag_and_text.odt
4. why_test701.odt

Results for LO 3.6.3, LOdev 4.0 in seconds:

File1: 2, 14  [2]*
File2: 2, 14  [2]* 
File3: 3, 17  [3]*
File4: 1-2, 30  [1-2]*

*) progress bar visible in seconds
Comment 3 Michael Meeks 2012-12-17 22:05:57 UTC
If you have a linux machine - it would be great to generate a callgrind profile:

export OOO_EXIT_POST_STARTUP=1
export OOO_DISABLE_RECOVERY=1
valgrind --tool=callgrind --simulate-cache=yes --dump-instr=yes ./soffice.bin -writer /path/to/document/to-load.odt

or whatever. That -should- give a callgrind.12345.txt file - that you can load in kcachegrind to poke around with to see what is going on.

Presumably some silly has crept in here that we should be able to see in a profile quite nicely; any chance of doing that ?
Comment 4 Aibara 2012-12-19 02:36:54 UTC
Here is my callgrind profile:
http://ubuntuone.com/2hO2QPQ8nOEaKroYfpi8OP

I'm not a programmer (at all), so I'm really not sure if I did this right, or even if it's useful at all. I just let it run for a while then quit. Was I supposed to manually open the file (Writer didn't open up for me when I ran the command)? Please let me know if I need to do things differently. I'm still using Ubuntu 12.04 and LO 4.0.0.0.beta1. 

Also, I didn't realize this before, but when Writer opens a big file, it not only freezes but uses 100% of the CPU during that period, but continues to do so once the file is open and editable for another a minute or two.
Comment 5 s-joyemusequna 2012-12-21 13:17:46 UTC
Created attachment 71929 [details]
ODF validator output

The file seems to be an invalid ODF file. See the attached picture from ODF validator.
Comment 6 s-joyemusequna 2012-12-21 19:41:21 UTC
Sorry, comment 5 belongs to Bug 58262.
Comment 7 s-joyemusequna 2012-12-22 09:06:44 UTC
Retested with LOdev 4.0.0 Beta2, 2012-12-22.

Average load time until the file is visible in editor, tested with Windows XP: 

Tested files (see attachment): 
1. lorem421pages_unformated.odt
2. lorem421pages_format_parag.odt
3. lorem421pages_format_parag_and_text.odt
4. why_test701.odt

Results for LO 3.6.3, LOdev 4.0 Beta1, LOdev 4.0 Beta2 in seconds:

File1: 2, 14 [2], 5 [2]
File2: 2, 14 [2], 5 [2]
File3: 3, 17 [3], 6 [2]
File4: 1-2, 30 [1-2], 15 [1]

[nn] progress bar visibility in seconds
Comment 8 Aibara 2013-01-25 17:36:17 UTC
Still a problem in version 4.0.0 RC2, though all files do open slightly faster for me now (why_test701.odt takes a full 25 seconds after the progress bar fills and disappears, whearas before it was 40; the smaller lorem421 files about 10 seconds).
Comment 9 Michael Stahl (CIB) 2013-08-13 14:27:27 UTC
Comment on attachment 71929 [details]
ODF validator output

attachment for wrong bug as per comment #6
Comment 10 Michael Stahl (CIB) 2013-08-13 15:57:46 UTC
looking at the first attachment and breaking with gdb it's going slow in the SwDoc::UpdateDocStats, counting words... 

the document has huge paragraphs that span 3 pages, perhaps some more fine tuning of the async word count is needed?
Comment 11 Michael Stahl (CIB) 2013-08-13 16:51:13 UTC
so the other 3 documents in the zip are already fast enough since commit
c138a8aec8dccb97948a7d7993e6869da4079b32 "asynchronous word-count";
it's just the why_test701.odt that is still slow because the
paragraphs are large.

have fixed that on master.
Comment 12 Commit Notification 2013-08-13 16:56:19 UTC
Michael Stahl committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=91c8008051c0bb7905a6acd822d022e144f2941f

fdo#58040: sw: fine tune async word count



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 13 Commit Notification 2013-08-14 07:04:40 UTC
Michael Stahl committed a patch related to this issue.
It has been pushed to "libreoffice-4-1":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=5837861da958eeef66c5167e75ea388cb7053ea7&h=libreoffice-4-1

fdo#58040: sw: fine tune async word count


It will be available in LibreOffice 4.1.2.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 14 Commit Notification 2013-08-14 09:19:27 UTC
Michael Stahl committed a patch related to this issue.
It has been pushed to "libreoffice-4-0":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=a0e98a12f8979286f2418606cfa4f75fcbc63b6e&h=libreoffice-4-0

fdo#58040: sw: fine tune async word count


It will be available in LibreOffice 4.0.6.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.