Bug 94212 - Saving corrupts particular Writer document containing many images and equations
Summary: Saving corrupts particular Writer document containing many images and equations
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: filters and storage (show other bugs)
Version:
(earliest affected)
4.4.5.2 release
Hardware: x86-64 (AMD64) macOS (All)
: medium critical
Assignee: Not Assigned
URL:
Whiteboard: odf
Keywords: dataLoss, regression
Depends on:
Blocks:
 
Reported: 2015-09-14 16:47 UTC by Balazs Fekete
Modified: 2016-10-27 15:33 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
The attached file becomes corrupt, when saved as copy (5.76 MB, application/vnd.oasis.opendocument.text)
2015-09-14 16:47 UTC, Balazs Fekete
Details
A smaller corrupt file from LibreOffice 4.4.5.2 (132.00 KB, application/vnd.oasis.opendocument.text)
2015-09-29 16:59 UTC, Balazs Fekete
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Balazs Fekete 2015-09-14 16:47:41 UTC
Created attachment 118711 [details]
The attached file becomes corrupt, when saved as copy

Saving the attached file corrupts the document. I tried both "Save as" and "Save Copy" and the result was the same. The saved file is 2MByte smaller than the original and when I try to open, LibreOffice tells, it is corrupted and needs fixing.
I had problems with saving documents since version 4, but normally only after editing and it was unpredictable. I suspect that the large number of equations in my documents are part of the problem.

In the past LibreOffice was able to repair the corrupt file, but not this time. After repair, the repaired document is empty. 

Best regards,

Balazs
Comment 1 Alex Thurgood 2015-09-15 11:44:15 UTC
Version: 5.0.1.2
Build ID: 81898c9f5c0d43f3473ba111d7b351050be20261
Locale : fr-FR (fr.UTF-8)

Confirming
Comment 2 Alex Thurgood 2015-09-15 11:48:43 UTC
Version: 5.1.0.0.alpha1+
Build ID: af433b6fff17e4e835043797063061055643b614
Locale : fr-FR (fr.UTF-8)

No corruption of file on save after edit
Comment 3 Alex Thurgood 2015-09-15 11:50:09 UTC
@Balazs : please try with a latest nightly build and report back your findings
Comment 4 Alex Thurgood 2015-09-15 11:54:55 UTC
No corruption in

Version: 4.3.5.2
Build ID: 3a87456aaa6a95c63eea1c1b3201acedf0751bd5

==>> regression
Comment 5 Balazs Fekete 2015-09-15 16:22:23 UTC
Hi,

I downloaded "libreoffice-5-0~2015-09-15_08.55.34_LibreOfficeDev_5.0.3.0.0_MacOS_x86-64.dmg" and I a was able to write a copy of the file and read it in. I will keep using the nightly build to see if it indeed solved the problem.

I would like to correct my original post that the latest version, which safely saved my document was 4.3.7. Yesterday, I went back to that version and I was able to modify and save the same documents repeatedly. I also added additional figures and equations.

I run into this problem with 4.4, so I went back to 4.3.7. When 5.0 came out, it seemed to work, but after a while, I run into similar problems (errors during saving or loosing equations unpredictably). Corrupted equations leave a broken object sign in the document.

5.0.1 seemed to work better, but at some point, it started to fail me again.

Best regards,
Comment 6 Balazs Fekete 2015-09-15 16:40:06 UTC
I just added an extra figure and some equations to the document and the save failed again using the nightly build.
Comment 7 Balazs Fekete 2015-09-29 16:59:40 UTC
Created attachment 119121 [details]
A smaller corrupt file from LibreOffice 4.4.5.2

I have the same problem with saving using 4.4.5.2 and 5.0.2. I attached a smaller file with the same issue. The problem is clearly with saving mathematical formulas. In the recovered files some of the formulas are missing. While the mathematical formulas seem to cause the problem.
I am back to 4.3.7, which is the last stable version.
Comment 8 Balazs Fekete 2015-11-08 19:50:48 UTC
The problem is still unresolved in 4.4.6 and 5.0.3. I am stuck at version 4.3.7, which is the last one that can handle equations without unpredictably corrupting the file while writing.
Comment 9 Michael Stahl (allotropia) 2016-06-16 21:16:21 UTC
this sounds pretty bad, but on Linux x86_64 i can't reproduce it with the attachment, tried multiple versions...

unzip -v /tmp/tdf94212-2.odt
says "End-of-central-directory signature not found."

possibly a truncated file?

that's astonishing - our framework code should write into a temp file and move that over the document file only when it's successfully written.
Comment 10 steve 2016-06-17 10:12:21 UTC
LO crashes for me on latest OSX and master when "save as" the first test file.

Version: 5.3.0.0.alpha0+
Build ID: 75ca2defe8edba8d198a05e7f64674418171536d
CPU Threads: 4; OS Version: Mac OS X 10.11.5; UI Render: default; 
TinderBox: MacOSX-x86_64@49-TDF, Branch:master, Time: 2016-06-15_23:42:04
Locale: de-DE (de.UTF-8)

Let me know if you want the crash log
Comment 11 Alex Thurgood 2016-06-17 10:21:18 UTC
Version: 4.4.5.2
Build ID: a22f674fd25a3b6f45bdebf25400ed2adff0ff99
Locale: fr.UTF-8
OSX 10.11.5

No repro

Opened first test file.
Saved as different file name
Close file and LO
Re-opened file in LO - no corruption, same file size as original
Comment 12 Alex Thurgood 2016-06-17 10:22:57 UTC
@Balazs : which version of OSX ?
Comment 13 Alex Thurgood 2016-06-17 10:37:03 UTC
Version: 5.1.3.2
Build ID: 644e4637d1d8544fd9f56425bd6cec110e49301b
CPU Threads: 2; OS Version: Mac OS X 10.11.5; UI Render: default; 
Locale: fr-FR (fr.UTF-8)

No repro either
Comment 14 Alex Thurgood 2016-06-17 10:37:37 UTC
Setting needinfo pending response to question in comment 12
Comment 15 Michael Stahl (allotropia) 2016-06-17 16:08:11 UTC
so i found that LO 4.4 added an implementation of multi-threaded zip deflate, which sounds like it has a significant risk of causing issues like this.

commit fbf714b45625c50bb1c736ef231b5dbbab0016a1
AuthorDate: Tue Oct 21 15:17:13 2014 +0200

    package: Finally implement parallel zip entries deflating


of course the hundreds of embedded objects in the attachment exercise that quite well.

i've had a crash on Windows when exporting, which is fixed on master by
commit debe788bcf3ec258b6b95df3db1f7bfeba881be1
"cppuhelper: WeakReference isn't thread-safe" - quite possibly same
crash as comment #10

there were numerous fixes to the multi-threaded zipping, maybe one of those
had an effect on this bug?
 60e75fb276778459f6055360646d879b8c615d83 "tdf#98955 hardware_concurrency not ideal for thread pools"
 7e2ea27e5d56f5cf767a6718a0f5edc28e24af14 "tdf#93553 limit parallelism at zip save time to useful amount"
 eaed822c9cf6b3762f727f1281003dafd300df6d "tdf#89236: Don't deflate encrypted document in parallel"
 458bf0812570f517dd4b80efbcfb7e0fca9479f7 "Resolves: tdf#88314 close temp file after each thread completed"
 ef698035aa2aa64fc4c4455b394e6782772fef4f "Related: tdf#88314 delete temp files"
 05661623c7e328c536814354bafd8294de0e9ee0 "uncaught exceptions in parallel deflate threads on re-export of HTB23106 odgs"
 ca5ff4559726455d0c8fbdd08cb35af3308492bf "tdf#90010 Partially revert "package: Do not bother with deflating jpeg image
s""
Comment 16 Michael Stahl (allotropia) 2016-06-17 20:27:19 UTC
... and there was another crash fixed on master, with commit 131e604073f89e6c1dd54be88b94b7befd881f2e "cppuhelper: fix use-after-free race in OWeakConnectionPoint"
Comment 17 steve 2016-06-18 06:45:49 UTC
Resolved, Fixed in my test.

1. open first test file in master from 2016-06-18
2. save as
3. works fine on 10.11.5 w master including michaels fix.

Thx to Mr. Stahl (:

Dear OP, please retest with latest master from here:
http://dev-builds.libreoffice.org/daily/master/MacOSX-x86_64@49-TDF/

Version: 5.3.0.0.alpha0+
Build ID: e6c004dd9f24c32f5e7468182a5e8d42293ec7b6
CPU Threads: 4; OS Version: Mac OS X 10.11.5; UI Render: default; 
TinderBox: MacOSX-x86_64@49-TDF, Branch:master, Time: 2016-06-17_06:40:25
Locale: de-DE (de.UTF-8)