Bug 49848 - FILEOPEN: Worse-than Linear Performance Degradation Opening Change-Tracked ODTs
Summary: FILEOPEN: Worse-than Linear Performance Degradation Opening Change-Tracked ODTs
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
3.5.3 release
Hardware: Other All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: BSA
Keywords: perf
Depends on:
Blocks: Track-Changes
  Show dependency treegraph
 
Reported: 2012-05-12 11:45 UTC by orcmid
Modified: 2019-10-31 08:48 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
Spreadsheet of Timing Tests showing degradation with document growth (15.32 KB, application/vnd.oasis.opendocument.spreadsheet)
2012-05-12 11:45 UTC, orcmid
Details
Additional Test Result (31.50 KB, application/vnd.oasis.opendocument.spreadsheet)
2015-01-05 22:16 UTC, orcmid
Details

Note You need to log in before you can comment on or make changes to this bug.
Description orcmid 2012-05-12 11:45:00 UTC
Created attachment 61518 [details]
Spreadsheet of Timing Tests showing degradation with document growth

Problem description: 
SUMMARY

There is a worse-than linear decrease in document opening and
saving performance when additional change tracking is added in 
a progression of draft changes to an original document.

At some point, the degradation is so bad that an user is 
likely to assume that the software has hung and is failing to
open the document.  On slower machines than the one the 
documents were created on, this delay can be hours, not just
too many minutes.

TEST DOCUMENTS

There are five test documents, WD03a, WD03b, WD03c, WD04a, and 
WD03x.

They are all available here:
<http://tools.oasis-open.org/version-control/svn/oic/TestSuite/trunk/odf12/ChangeTrackingResilience/>.

If you want to know what WD03c is supposed to look like, there 
is a PDF available here:
<http://www.oasis-open.org/apps/org/workgroup/office/document.php?document_id=45946>.
It is a large file, but it opens quickly in Acrobat.

To know what the 223 tracked changes are, you can also check
Section 2 of the smaller file available here:
<http://www.oasis-open.org/apps/org/workgroup/office/document.php?document_id=45936>.
It is an ODF Text (.ODT) file.

DEMONSTRATION OF THE WORSE-THAN LINEAR DEGRADATION

The defect is demonstrated by timed opening of 4 documents 
that have an increasing number of tracked changes.

 * WD03a is 476kB and has 169 changes.  It opens in around 15
   seconds on a fast system.

 * WD03b is 746kB and the number of changes is raised to 207.  
   It takes a few minutes to open the document (roughly 16x as 
   long as for WD03a on a fast system).

 * WD03c is 1,132kB and it has 223 changes.  It takes roughtly
   4x more than WD03b. On a slower Windows XP SP3 x86 system, 
   it takes more than an hour to open the document.

 * WD04a is 1,343kB although it has no more tracked changes   
   and was only updated enough to start a new working draft 
   set.  Yet it is 200kb larger    and it takes almost double 
   the time over that for WD03c.  On the slowest system 
   used, it takes 2.5 hours.

 THE SPREADSHEET (attached) will provide timing 
statistics and the the different configurations and software releases on which measurements were captured.
Comment 1 orcmid 2012-05-12 11:47:29 UTC
SPREADSHEET DETAILS 

THIS ODF SPREADSHEET provides more data points with regard
 to the timings and the different configurations and software releases tested.

 Different releases of LibreOffice are employed, depending on what was
 handy for testing with different platforms.

 When available, I provide timing tests with OpenOffice.org 3.3.0 as well.
 This is to provide a baseline and confimr that the problem has existed since
 at least that releast of OpenOffice.org.

NOTE: THE MEASURED TIMINGS ARE NOT SUITABLE FOR COMPARISONS BETWEEN PRODUCTS.
  These timings were determined manually with a stop watch.  The conditions
  were not carefully controlled and the typical variances related to
  configuration differences, background activity, and system state are too
  high.
     The sole purpose of the timings is to demonstrate that the degradation
  of performance is consistent and predictable across all OpenOffice-
  lineage software.  The variance between releases is negligible compared to
  the major source of degradation.

CONFIGURATIONS

Astraendo is a Dell XPS 9100 with Windows 7 en_US x64, 18GB RAM, and an
Intel i7-980x 3.33GHz 6-core processor.

Quadro is a Toshiba Satellite Tablet PC with Windows XP SP3, 1.5GM RAM, and
an Intel Pentium M (Celeron) 1.7GHz processor.

VVM is Vista Ultimate en_US x86 running in Virtual PC on Astraendo

Win8CP64 is Windows 8 Customer Preview en_US x64 running in VirtualBox on
Astraendo

Zorin Core is Zorin OS (Core, Debian/Ubuntu) x64 running in VirtualBox on
Astraendo

Zorin Edu is Zorin OS (Edu, Debian/Ubunto) x86 running in VirtualBox on
Astraendo

SPECIAL CASES

Package releases are those provided in a distribution.  These are not from
LibreOffice download sites and apparently there are odd failure cases with
those.

Open Fail means that the document went through all of the slow opening
process and when it appeared to be ending, the application simply closed
without the document ever being shown.

Close crash means that there was a crash on closing the document in the
application, with a report that the software had not closed properly and
work might have been lost.  The document was fine (it had not been touched)
but the lock file was still present in the file system.
Comment 2 orcmid 2012-05-12 12:55:37 UTC
This defect is apparently in common code inherited from OpenOffice.org in both LibreOffice and Apache OpenOffice: 
https://issues.apache.org/ooo/show_bug.cgi?id=119341

It appears that symptoms of this problem have been identified as far back as OpenOffice.org 1.0:
https://issues.apache.org/ooo/show_bug.cgi?id=29842
There are potentially multiple defects behind these, ones related to change-tracking on open (as well as autosave and save but not quite as slow) and to bloating of the .ODT file for no apparent reason.

All of the ODF Text documents, WD03a, ..., WD04a were produced with LibreOffice 3.3.2, the software being used for an ODF maintenance activity (no changes to the software are made during the project to avoid regressions).

The original document with which editing began is the ODF Text for the OASIS OpenDocument Format 1.1 Standard.  This document was produced by generator 
"StarOffice/8$Solaris_Sparc OpenOffice.org_project/680m5$Build-9114".
Comment 3 Michael Stahl (CIB) 2012-09-13 13:09:09 UTC
the change tracking implementation in Writer is and always has been
embarrassing, nothing new here.  SwDoc::AppendRedline is at least
O(n^2), perhaps worse (fortunately i haven't looked at it lately).

file import isn't actually the worst performance problem;
finding that is easy, just grep for

  //JP 06.01.98: MUSS noch optimiert werden!!!
Comment 4 QA Administrators 2015-01-05 17:51:37 UTC Comment hidden (obsolete)
Comment 5 orcmid 2015-01-05 22:16:55 UTC
Created attachment 111796 [details]
Additional Test Result

(In reply to QA Administrators from comment #4)
[ ... ]
> If the bug is present, please leave a comment that includes the version of
> LibreOffice and your operating system, and any changes you see in the bug
> behavior
[ ... ]

The bug remains present.  

Machine: Dell XPS with Windows 8.1 Professional
LibreOffice: 4.3.5.2

In this comparison, the times seem longer than previously but the key is that the non-linearity has not changed.  The different in timings may be because of differences in configuration and the absence of controlled timing, so these should not be viewed as absolute comparisons.

The new attachment is an ODS file that adds the test runs just made.
Comment 6 QA Administrators 2016-11-08 10:34:31 UTC
** Please read this message in its entirety before responding **

To make sure we're focusing on the bugs that affect our users today, LibreOffice QA is asking bug reporters and confirmers to retest open, confirmed bugs which have not been touched for over a year.

There have been thousands of bug fixes and commits since anyone checked on this bug report. During that time, it's possible that the bug has been fixed, or the details of the problem have changed. We'd really appreciate your help in getting confirmation that the bug is still present.

If you have time, please do the following:

Test to see if the bug is still present on a currently supported version of LibreOffice 
(5.1.6 or 5.2.3  https://www.libreoffice.org/download/

If the bug is present, please leave a comment that includes the version of LibreOffice and 
your operating system, and any changes you see in the bug behavior
 
If the bug is NOT present, please set the bug's Status field to RESOLVED-WORKSFORME and leave 
a short comment that includes your version of LibreOffice and Operating System

Please DO NOT

Update the version field
Reply via email (please reply directly on the bug tracker)
Set the bug's Status field to RESOLVED - FIXED (this status has a particular meaning that is not 
appropriate in this case)


If you want to do more to help you can test to see if your issue is a REGRESSION. To do so:
1. Download and install oldest version of LibreOffice (usually 3.3 unless your bug pertains to a feature added after 3.3)

http://downloadarchive.documentfoundation.org/libreoffice/old/

2. Test your bug
3. Leave a comment with your results.
4a. If the bug was present with 3.3 - set version to "inherited from OOo";
4b. If the bug was not present in 3.3 - add "regression" to keyword


Feel free to come ask questions or to say hello in our QA chat: http://webchat.freenode.net/?channels=libreoffice-qa

Thank you for helping us make LibreOffice even better for everyone!

Warm Regards,
QA Team

MassPing-UntouchedBug-20161108