Created attachment 71224 [details] 701 page test document. I've had a problems with large files in Writer since the 3.5 series began. In the 3.5 and 3.6 series, big files take a long time to save, whereas in the 3.4 series and before I never had a problem and the saving process would begin instantly. In 4.0.0.0 beta1 the problem seems a bit different: every time a big file is opened LibreOffice Writer hangs. Steps to reproduce: Open attached file. Expected behavior: File opening progress bars fill up and then the document can be read and edited. Actual behavior: After the progress bars fill, LibreOffice freezes for up to 40 seconds and is totally unresponsive. The attached file "opens" (i.e. the progress bars fill) in about 2 seconds, but then hangs – LibreOffice goes grey – for 40 seconds. Note that the attached example is only a 700 page file, and is simply the word "why" repeated; no fancy formatting at all. Smaller files, like 300 pages ones, still hang for a bit. When I hit around 500 pages it hangs for the full 40 seconds, though oddly that's the same for files that are thousands of pages long. I'm using Ubuntu 12.04 on a ThinkPad T60p, but I also tried this on a friend's newer Win7 Ultimate desktop. The same thing happens, though the file opens a bit faster. It still hangs for over 20 seconds (the title bar says "not responding"). Again, note that there was no problem with these large files in the 3.4 series and those before it. For the older 'saving' hang bug: https://bugs.freedesktop.org/show_bug.cgi?id=51311 Thanks.
Confirmed with both 3.6.4.3 and 4.0.0.0 beta1 - OS X 10.7.5 Behaviours: 3.6.4.3- Instantly opens, fast scrolling, but inability to edit file, cursor does not even blink, consistent hanging. CPU tops 100+% 4.0.0.0 beta 1- Hanging while opening, other than that same with above. Hangs the whole system with SSD. It didn't hang like for 40 seconds, 'cause I have a SSD, but it was still pretty much.
Created attachment 71345 [details] 3 odt test files (rtf files from bug 44736 saved as ODT) Average load time until the file is visible in editor, tested with Windows XP: LOdev 4.0.0 Beta1 (2012-12-07). Tested files (see attachment): 1. lorem421pages_unformated.odt 2. lorem421pages_format_parag.odt 3. lorem421pages_format_parag_and_text.odt 4. why_test701.odt Results for LO 3.6.3, LOdev 4.0 in seconds: File1: 2, 14 [2]* File2: 2, 14 [2]* File3: 3, 17 [3]* File4: 1-2, 30 [1-2]* *) progress bar visible in seconds
If you have a linux machine - it would be great to generate a callgrind profile: export OOO_EXIT_POST_STARTUP=1 export OOO_DISABLE_RECOVERY=1 valgrind --tool=callgrind --simulate-cache=yes --dump-instr=yes ./soffice.bin -writer /path/to/document/to-load.odt or whatever. That -should- give a callgrind.12345.txt file - that you can load in kcachegrind to poke around with to see what is going on. Presumably some silly has crept in here that we should be able to see in a profile quite nicely; any chance of doing that ?
Here is my callgrind profile: http://ubuntuone.com/2hO2QPQ8nOEaKroYfpi8OP I'm not a programmer (at all), so I'm really not sure if I did this right, or even if it's useful at all. I just let it run for a while then quit. Was I supposed to manually open the file (Writer didn't open up for me when I ran the command)? Please let me know if I need to do things differently. I'm still using Ubuntu 12.04 and LO 4.0.0.0.beta1. Also, I didn't realize this before, but when Writer opens a big file, it not only freezes but uses 100% of the CPU during that period, but continues to do so once the file is open and editable for another a minute or two.
Created attachment 71929 [details] ODF validator output The file seems to be an invalid ODF file. See the attached picture from ODF validator.
Sorry, comment 5 belongs to Bug 58262.
Retested with LOdev 4.0.0 Beta2, 2012-12-22. Average load time until the file is visible in editor, tested with Windows XP: Tested files (see attachment): 1. lorem421pages_unformated.odt 2. lorem421pages_format_parag.odt 3. lorem421pages_format_parag_and_text.odt 4. why_test701.odt Results for LO 3.6.3, LOdev 4.0 Beta1, LOdev 4.0 Beta2 in seconds: File1: 2, 14 [2], 5 [2] File2: 2, 14 [2], 5 [2] File3: 3, 17 [3], 6 [2] File4: 1-2, 30 [1-2], 15 [1] [nn] progress bar visibility in seconds
Still a problem in version 4.0.0 RC2, though all files do open slightly faster for me now (why_test701.odt takes a full 25 seconds after the progress bar fills and disappears, whearas before it was 40; the smaller lorem421 files about 10 seconds).
Comment on attachment 71929 [details] ODF validator output attachment for wrong bug as per comment #6
looking at the first attachment and breaking with gdb it's going slow in the SwDoc::UpdateDocStats, counting words... the document has huge paragraphs that span 3 pages, perhaps some more fine tuning of the async word count is needed?
so the other 3 documents in the zip are already fast enough since commit c138a8aec8dccb97948a7d7993e6869da4079b32 "asynchronous word-count"; it's just the why_test701.odt that is still slow because the paragraphs are large. have fixed that on master.
Michael Stahl committed a patch related to this issue. It has been pushed to "master": http://cgit.freedesktop.org/libreoffice/core/commit/?id=91c8008051c0bb7905a6acd822d022e144f2941f fdo#58040: sw: fine tune async word count The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Michael Stahl committed a patch related to this issue. It has been pushed to "libreoffice-4-1": http://cgit.freedesktop.org/libreoffice/core/commit/?id=5837861da958eeef66c5167e75ea388cb7053ea7&h=libreoffice-4-1 fdo#58040: sw: fine tune async word count It will be available in LibreOffice 4.1.2. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Michael Stahl committed a patch related to this issue. It has been pushed to "libreoffice-4-0": http://cgit.freedesktop.org/libreoffice/core/commit/?id=a0e98a12f8979286f2418606cfa4f75fcbc63b6e&h=libreoffice-4-0 fdo#58040: sw: fine tune async word count It will be available in LibreOffice 4.0.6. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.