Bug 89401 - FILEOPEN: Loading 200MB+ .doc at first took extremely long (30min+) and is now obviously trashing
Summary: FILEOPEN: Loading 200MB+ .doc at first took extremely long (30min+) and is no...
Status: RESOLVED NOTABUG
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
unspecified
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-02-15 22:28 UTC by Jouni Järvinen
Modified: 2015-12-06 20:08 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
The table backup (2.32 MB, application/x-xz)
2015-02-15 22:32 UTC, Jouni Järvinen
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jouni Järvinen 2015-02-15 22:28:20 UTC
A .doc backup of one of our Minecraft maps' LogBlock tables is 255MB large and takes extremely long to load on LO (4.4.0.3 (hash=de093506bcdc5fafd9023ee680b8c60e3e0645d7) tested) on Win7 Ultimate x64, loading around  at once and maximizing a core.

However, on 3rd try, both soffice.bin and soffice.exe are mostly idling without CPU usage, .bin giving spikes of a single core usage and also loading 100 million bytes (100MB) at once or more, but 100MB at once makes me wonder how cuz a single core can't do that much and the drive can't read that much and nothing is happening on LO's end graphically.

For comparison,
- Notepad opens the file as quickly as this computer should, but it doesn't load the whole file like always.
- Wordpad takes long too, but not so slowly, instead around 20kB-50kB a second while maximizing a single core.
Comment 1 Jouni Järvinen 2015-02-15 22:32:49 UTC
Created attachment 113410 [details]
The table backup

My bad, accidently hit 'Submit bug'. This is the file tightly compressed with LZMA2. The data has nothing I need to protect, so I don't mind giving it.

Requirements: 7-Zip 9.22 on Window$, xz-utils/liblzma 5.1.0alpha.
Comment 2 Jouni Järvinen 2015-02-15 22:36:42 UTC
Now that it has loaded for long, it has loaded over 14 505 842 500 bytes which equals 14505.8MB and using a core almost completely, which means complete trashing to me. LO still is to show anything.

Wordpad has loaded way over 400 million bytes (about 400MB) while using nearly whole core, so Wordpad can't handle the file either.
Comment 3 Jouni Järvinen 2015-02-15 22:54:28 UTC
(In reply to Jouni Järvinen from comment #0)
> ..., loading around  at once ...
> 

should be 'loading around 100KB at once'. Sometimes more, sometimes less.
Comment 4 Joel Madero 2015-02-15 23:42:53 UTC
So - I haven't confirmed this bug yet (my system is occupied) but I can say that this is an extreme corner case and unlikely to ever be fixed.

How do you monitor how many megs are loaded? Also how long does the file take to open - not enough to say "extremely long time"

That being said - I'll try to find time to confirm it so at least it gets to NEW status.
Comment 5 Jouni Järvinen 2015-02-15 23:56:24 UTC
(In reply to Joel Madero from comment #4)
> So - I haven't confirmed this bug yet (my system is occupied) but I can say
> that this is an extreme corner case and unlikely to ever be fixed.
> 
> How do you monitor how many megs are loaded? Also how long does the file
> take to open - not enough to say "extremely long time"
> 
> That being said - I'll try to find time to confirm it so at least it gets to
> NEW status.

30min+ for every tested program. I monitor with Task Manager.

First time I tried, .bin used between 100-320MB RAM, shifting between until it stopped to like 220MB and obviously started trashing minutes later.

Wordpad shifted too, but nowhere as much before settling to around 100MB.

Granted, I have no clue, not even guesses, what Office can open, but if it can't handle this either, even if it was real text instead of an unusual database backup, LO would get a foothold by supporting up until to 2GB on 32-bit Win, or even multiple 2GB processes. It's not a matter of a corner case, it's a matter of program not allowing to do what it should.
Comment 6 Buovjaga 2015-02-22 13:24:38 UTC
Just a thought: would you like to try the new 64-bit Windows version to compare: http://dev-builds.libreoffice.org/daily/master/Win-x86_64@42/

Most problems with it seem to be fixed: https://wiki.documentfoundation.org/Development/msvc-x86_64
Comment 7 Jouni Järvinen 2015-02-22 14:22:27 UTC
(In reply to Beluga from comment #6)
> Just a thought: would you like to try the new 64-bit Windows version to
> compare: http://dev-builds.libreoffice.org/daily/master/Win-x86_64@42/

Will installing that, even when the installation folder is different, replace existing LO ?
Comment 8 Buovjaga 2015-02-22 15:25:26 UTC
(In reply to Jouni Järvinen from comment #7)
> (In reply to Beluga from comment #6)
> > Just a thought: would you like to try the new 64-bit Windows version to
> > compare: http://dev-builds.libreoffice.org/daily/master/Win-x86_64@42/
> 
> Will installing that, even when the installation folder is different,
> replace existing LO ?

It's a master build so it will install to a different folder and even use a different user profile, so it will not mess up anything in your current setup. Well, 64-bit programs install to "Program files" vs. "Program files (x86)" for 32-bit ones in any case.

You can determine the file associations in the end by selecting to customize the install. I normally uncheck the file associations when installing master builds.
Comment 9 Robinson Tryon (qubit) 2015-02-25 22:02:31 UTC
(In reply to Beluga from comment #6)
> Just a thought: would you like to try the new 64-bit Windows version to
> compare: http://dev-builds.libreoffice.org/daily/master/Win-x86_64@42/
> 
> Most problems with it seem to be fixed:
> https://wiki.documentfoundation.org/Development/msvc-x86_64

I'll toss this bug into NEEDINFO for now, until OP can test with the 64bit build.

Jouni: Please test and let us know your results! Thanks!
Comment 10 Jouni Järvinen 2015-02-25 23:06:50 UTC
Looking at the custom (In reply to Beluga from comment #6)
> Just a thought: would you like to try the new 64-bit Windows version to
> compare: http://dev-builds.libreoffice.org/daily/master/Win-x86_64@42/

libo-master64~2015-02-22_01.02.55_LibreOfficeDev_4.5.0.0.alpha0_Win_x64.msi:
- Shown twice in a row but installation goes just fine although I chose to have no shortcuts: Warning 1946. Property 'System.AppUserModel.ID' for shortcut 'LibreOfficeDev.lnk' could not be set.
- the size of the installer and only optional components were available for customization, but EXE files were found in expected place.
- It took about 2 minutes to do almost anything, few percents of CPU at times and no data read.
- Loading 5-25KB/s at first.
- Trashing now:
-- a core is mostly fully used
-- RAM usage 321-330MB
-- Task Manager says 7 billion bytes (7GB) read and rising, 6.7 billion bytes (around 6.69GB) written. Nothing impossible to achieve since it's taken many minutes now, but completely pointless.
Comment 11 Pedro 2015-03-18 17:13:15 UTC
I can't load this file completely on any program: MS Word 2010 or LibreOffice (x86 or x64)

Even though it has a .doc extension, this is not a Word binary file. Loading on Notepad (partially?) shows that this is an uncompressed XML file but it ends with tag <tr> which is odd.

Is the OP able to open this file on any program? Maybe the file is incomplete?

I believe this is NOTABUG
Comment 12 Joel Madero 2015-03-18 17:16:09 UTC
Agreed - if you tested with MS Office and it's not opening the file then the file is wrong. Closing it.

@OP - if you can open this with Microsoft Office - please let us know what version and post a pdf of the document. Thanks
Comment 13 Jouni Järvinen 2015-12-06 20:05:28 UTC
Somehow this bug's slipped past me altogether. Anyway ...

LO 5.0.4 RC1 x64, 16GB of read data, like 10 hours of maxing out a core and §soffice.bin§'s RAM usage ranging from around 260MB to around 280MB, LO still haven't gotten through.

Word 2016 x64 was given 4-6 hours. It had read many gigabytes too, grown to around 3GB RAM usage and maxing a core whole time, but nothing happened either.
Comment 14 Joel Madero 2015-12-06 20:08:41 UTC
Yes as Pedro said, the file is bad. Closing it again as NOTABUG. If you can't even open it right in MSO (and it is supposedly a MSO extension) then why would it be our bug? Please don't reopen the bug. Thanks