Bug 88314 - saving big file crashes writer (and LibreOffice)
Summary: saving big file crashes writer (and LibreOffice)
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: filters and storage (show other bugs)
Version:
(earliest affected)
4.4.0.2 rc
Hardware: x86-64 (AMD64) Linux (All)
: high critical
Assignee: Caolán McNamara
URL:
Whiteboard: target:5.1.0 target:5.0.3
Keywords: bibisected, bisected, haveBacktrace, regression
: 89882 (view as bug list)
Depends on:
Blocks:
 
Reported: 2015-01-12 08:20 UTC by Yury
Modified: 2016-10-25 19:20 UTC (History)
7 users (show)

See Also:
Crash report or crash signature:


Attachments
big file (obfuscated) triggering issue on my system (3.77 MB, application/vnd.oasis.opendocument.text)
2015-04-04 16:26 UTC, Yury
Details
gdbtrace (4.50 KB, application/zip)
2015-04-16 19:55 UTC, raal
Details
screen with I/O dialog in 5.0 daily build 2015-08-12 (28.19 KB, image/jpeg)
2015-08-13 10:26 UTC, Yury
Details
part of screen with 'corrupt file' dialog in 4.3 local build (102.95 KB, image/jpeg)
2015-08-13 10:27 UTC, Yury
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Yury 2015-01-12 08:20:58 UTC
I have a fairly big writer file here, 200 pages, hundreds of formulas. Size of .odt is ~4M.

From some moment it stopped opening in 3.6.7.2 (and in all variants up to and including LO 4.2 / AOO 4.1 series) -- https://bugs.freedesktop.org/show_bug.cgi?id=88140 (I can open it and work with it in 4.3 series, albeit experiencing some inconveniences -- https://bugs.freedesktop.org/show_bug.cgi?id=88139)

The problem with 4.4.0 series is I can open (and work with) this big odt but cannot reliably save it. In 4.4.0.1 writer crashes some time (few seconds) after progress indicator reaches 100% and is removed from status bar. In 4.4.0.2 writer crashes in the same manner, but not every time I save the file.

The diagnostics is, sometimes, this:

terminate called after throwing an instance of 'com::sun::star::packages::zip::ZipIOException'

and sometimes it's something about 'not connected'

I can't publish this big file at the moment.
Comment 1 retired 2015-01-12 08:26:47 UTC
Can you sanitize your file or replace sensitive images with placeholder dummys? Without a test file this is close to impossible to reproduce.

After providing a test file please set your bug back to UNCONFIRMED. Thanks
Comment 2 Yury 2015-01-12 09:06:27 UTC
Sorry, the whole file is sensitive (unpublished) at the moment.

Possibly, somebody could just look into the save subsystem extreme loads handling capability or something. Maybe auto-generate 200+ pages of 14 pt Lorem ipsum and 500+ formulas?.. There are macros for that I believe. I myself can't do this at the moment, too.
Comment 3 Yury 2015-01-28 09:21:24 UTC
The issue's still there on 4.4.0.3.
The error message seems to be always "terminate called recursively".
Comment 4 Yury 2015-01-28 09:24:08 UTC
As for providing the proofdoc, I'll have yet to xx-obfuscate the formulas. But I assure you the problem is real, replicates 100% here.
Comment 5 Yury 2015-04-04 16:26:24 UTC
Created attachment 114621 [details]
big file (obfuscated) triggering issue on my system

Happens in 4.4.2.2 , too. Not every save ends in crash, mind you, but overwhelming majority of those do.

Here're two typical console messages on crash:

bash-4.2$ ./soffice
terminate called after throwing an instance of 'com::sun::star::io::NotConnectedException'

bash-4.2$ ./soffice
terminate called recursively
terminate called after throwing an instance of 'com::sun::star::packages::zip::ZipIOException'
Comment 6 Gordo 2015-04-04 21:16:20 UTC
Something I noticed in the xml and if you Edit Index/Table on the Bibliography is the Entries tab shows the Structure tooltip as 5f_5f repeated.

Also, there are quite a few objects that appear as hidden (greyed out) in the Navigator.  When I update the bibliography, they change from grey to normal in the list.

Perhaps it is to do with the structure of the objects in the xml:
<draw:frame draw:style-name="fr4" draw:name="Object1" text:anchor-type="as-char" svg:y="-1.154cm" svg:width="4.09cm" svg:height="1.931cm" draw:z-index="275"><draw:object xlink:href="./Object 10" xlink:type="simple" xlink:show="embed" xlink:actuate="onLoad"/>
<draw:image xlink:href="./ObjectReplacements/Object 10" xlink:type="simple" xlink:show="embed" xlink:actuate="onLoad"/></draw:frame>

<draw:frame draw:style-name="fr4" draw:name="Object16" text:anchor-type="as-char" svg:y="-0.522cm" svg:width="5.136cm" svg:height="0.746cm" draw:z-index="404">
<draw:object xlink:href="./Object 1" xlink:type="simple" xlink:show="embed" xlink:actuate="onLoad"/>
<draw:image xlink:href="./ObjectReplacements/Object 1" xlink:type="simple" xlink:show="embed" xlink:actuate="onLoad"/></draw:frame>

I tried to get a backtrace but did not get anything meaningful.

Hopefully somebody more versed in these matters will be able to help.

Version: 4.4.2.2
Build ID: c4c7d32d0d49397cad38d62472b0bc8acff48dd6
Comment 7 Yury 2015-04-05 06:51:17 UTC
There are three draw objects all told there and I don't think it's those that're at fault.
This behaviour is obviously set off by processed doc (being saved) structure size and complexity. Just a text of similar size (randomly generated) doesn't trigger this (I tried).
Comment 8 Gordo 2015-04-05 13:07:00 UTC
Object 1 links to Object 10.  Object 16 links to Object 1.  Maybe there is recursion when it is trying to save these.  I picked those as an example.  There are many more.

I think draw:frame, draw:object, and draw:image should all reference the same object.

There is also a Configurations2 folder under each of the objects and the contents of Configurations2 under some of the objects.
Comment 9 Yury 2015-04-05 14:02:36 UTC
This doc was in the works for rather long time, maybe more than a year, maybe two years. Maybe the cruft got accumulated. I surely didn't introduce any of those by hand. Anyway, I understand even you do not know if those are what's causing the issue.
What do you suggest I do now? Are there any structure sanitisers, or similar?
Comment 10 Yury 2015-04-07 15:36:15 UTC
I should reformulate this as follows: am I under a threat of this doc developing some new issues on the 4.3.6.2 (which I use currently), and how (with what) do I repair such subtle issues in doc structure?
Comment 11 rcohen.lists 2015-04-16 00:16:57 UTC
This sounds similar to bug #89882

terminating with uncaught exception of type com::sun::star::io::NotConnectedException
Comment 12 raal 2015-04-16 19:55:03 UTC
I can confirm with  Version: 4.5.0.0.alpha0+
Build ID: 51e0d789c344547956764c3b5f0ef5a304f4e0aa
TinderBox: Linux-rpm_deb-x86_64@46-TDF, Branch:master, Time: 2015-04-13_16:58:45

Steps:
 - open file
 - save as
 -  crash
Comment 13 raal 2015-04-16 19:55:45 UTC
Created attachment 114841 [details]
gdbtrace
Comment 14 Matthew Francis 2015-04-24 09:53:02 UTC
This is hard to reproduce reliably, but I'm fairly sure it started (on master) at the below commit.

The error I usually see on a regular build is:
terminate called after throwing an instance of 'com::sun::star::io::NotConnectedException'

Current 5.0 dbgutil gives:
soffice.bin: /home/vmiklos/git/libreoffice/master/tools/source/stream/stream.cxx:1826: virtual sal_Size SvMemoryStream::PutData(const void*, sal_Size): Assertion `pBuf && "Possibly Reallocate failed"' failed.


    commit 3fcd2ccb443653740d114b3e4dc371c6b0b6525b
    Author:     Kohei Yoshida <kohei.yoshida@collabora.com>
    AuthorDate: Mon Dec 15 15:03:51 2014 -0500
    Commit:     Kohei Yoshida <kohei.yoshida@collabora.com>
    CommitDate: Mon Dec 15 15:13:44 2014 -0500
    
        fdo#87210: Re-enable parallel deflate, which was not the root cause.
    
        With f92183833fa569006602ac7e93c906d2094e0d4d, export no longer crashes,
        and there is no reason to leave this piece disabled any more.  Let's
        re-enable this.
    
        Change-Id: Ibeca8869f152cbcd80f1dcb55f8199110125741d
Comment 15 Caolán McNamara 2015-07-22 13:21:43 UTC
potential solution at https://gerrit.libreoffice.org/#/c/17289/
Comment 16 Yury 2015-07-22 15:15:45 UTC
Do we get anything to test? I'm not sure I'm up to building from git tree.
Comment 17 Commit Notification 2015-07-24 13:27:06 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=738cf411e9315d17c7eb8be47ded643a00dfe5c5

Resolves: tdf#88314 close temp file after each thread completes

It will be available in 5.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 18 Commit Notification 2015-07-26 05:27:10 UTC
Norbert Thiebaud committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=33a21d37f376abaaabdaceaa09160cab962038fc

Revert "Resolves: tdf#88314 close temp file after each thread completes"

It will be available in 5.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 19 Yury 2015-07-26 06:29:54 UTC
Re patch and its prompt revertion: what's happening?
Comment 20 Commit Notification 2015-08-05 22:16:40 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=458bf0812570f517dd4b80efbcfb7e0fca9479f7

Resolves: tdf#88314 close temp file after each thread completed

It will be available in 5.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 21 Cor Nouws 2015-08-06 08:16:42 UTC
(In reply to Yury from comment #16)
> Do we get anything to test? I'm not sure I'm up to building from git tree.

Hi Yuri,

Rather soon (half a day .. day..) after the commit, a daily version with the patch can be found here
  http://dev-builds.libreoffice.org/daily/master/

HTH,
Cor
Comment 22 Commit Notification 2015-08-07 07:53:32 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=ef698035aa2aa64fc4c4455b394e6782772fef4f

Related: tdf#88314 delete temp files

It will be available in 5.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 23 Matúš Kukan 2015-08-07 08:13:55 UTC
*** Bug 89882 has been marked as a duplicate of this bug. ***
Comment 24 Yury 2015-08-13 10:23:43 UTC
That RESOLVED status might be in error.

I've taken this one from daily builds:

libreoffice-5-0~2015-08-12_15.44.50_LibreOfficeDev_5.0.2.0.0_Linux_x86-64_rpm.tar.gz

and tested it with bigfile_obf.odt from attachments.

On save, file seems to be formed, but with problems -- zip structure okay, open in 4.3 series shows info on 'corrupt file' (snapshot #1 forthcoming), repair finishes okay.

Also, although LO doesn't bail out, I'm getting an I/O error in popup window  after every (half-successful) save (snapshot #2 forthcoming). Filename in dialog is truncated on the 2nd underscore char -- see snapshot #2.
Comment 25 Yury 2015-08-13 10:26:20 UTC
Created attachment 117883 [details]
screen with I/O dialog in 5.0 daily build 2015-08-12
Comment 26 Yury 2015-08-13 10:27:28 UTC
Created attachment 117884 [details]
part of screen with 'corrupt file' dialog in 4.3 local build
Comment 27 Timur 2015-08-13 10:49:59 UTC
I didn't test myself, but this fix is for 5.1.0. and you tested with 5.0+.
Please try the current development version from http://dev-builds.libreoffice.org/daily/master/Win-x86@39/current/.
Also, I added  backportRequest:5.0.
Comment 29 Yury 2015-08-13 12:57:36 UTC
Right, thank you, it wasn't obvious. Yes, it seems the 5.1 series has that issue corrected. Thank you very much to all concerned!

(and good job all these 7+ months I had the 4.3 series to fall back to, eh? :)

On a still lighter note: 15 years ago I've been using StarOffice 5.1 (OS/2). :)
Comment 30 Michael Stahl (CIB) 2015-09-15 17:34:38 UTC
backported now
Comment 31 Commit Notification 2015-09-15 17:35:19 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "libreoffice-5-0":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=d827ff9679548cdbc8520eb4e67d64ae2f0137ec&h=libreoffice-5-0

Resolves: tdf#88314 close temp file after each thread completed

It will be available in 5.0.3.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 32 Robinson Tryon (qubit) 2015-12-17 08:43:40 UTC Comment hidden (obsolete)