Bug 105844 - FILESAVE: Very slow saving with password compared to 5.2.5
Summary: FILESAVE: Very slow saving with password compared to 5.2.5
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: filters and storage (show other bugs)
Version:
(earliest affected)
5.3.0.3 release
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: odf
Keywords: bibisected, bisected, haveBacktrace, perf, regression
: 112881 (view as bug list)
Depends on:
Blocks: Password-Protected
  Show dependency treegraph
 
Reported: 2017-02-07 21:57 UTC by jhertel
Modified: 2019-04-20 09:40 UTC (History)
8 users (show)

See Also:
Crash report or crash signature:


Attachments
Empty document exhibiting the behaviour, password is p (12.53 KB, application/vnd.oasis.opendocument.text)
2017-08-30 22:04 UTC, jhertel
Details
Example file (12.53 KB, application/vnd.oasis.opendocument.text)
2017-09-04 15:02 UTC, Telesto
Details
Bibisect log (3.23 KB, text/plain)
2017-11-15 16:07 UTC, Telesto
Details
Perf flamegraph (351.37 KB, image/svg+xml)
2019-04-20 09:39 UTC, Buovjaga
Details

Note You need to log in before you can comment on or make changes to this bug.
Description jhertel 2017-02-07 21:57:30 UTC
Description:
After updating from 5.2.5.1 to 5.3.0.3, saving became much, much slower. 

So slow that I had to downgrade to 5.2.5.1 again, which solved the problem.

In 5.2.5.1, saving is almost instantaneous, i.e. a few hundred milliseconds (the document is small, 20 KB, 7 pages, 1716 characters). In 5.3.0.3, saving takes several seconds, around 5 to 7 seconds before you can actually type again. And as I save frequently while typing (often after each sentence), this stops me in my writing.

My computer is a relatively fast i5, with 8 GB RAM and a fast SSD, so the computer or drive is not the culprit. This is also indicated by the fact that downgrading to 5.2.5.1 solved the problem.

The document is password protected, if that matters.

The problem is so annoying that I had to downgrade to 5.2.5 and will have to stay at that version until the bug has been solved. 

Steps to Reproduce:
1. Open the document.
2. Save it (press Ctrl+S).


Actual Results:  
It takes several seconds before I can type again. Around 5-7 seconds.

Expected Results:
Saving should be almost instantaneous, 2-300 ms.


Reproducible: Always

User Profile Reset: No

Additional Info:


User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36
Comment 1 Xisco Faulí 2017-02-07 23:15:47 UTC Comment hidden (obsolete)
Comment 2 QA Administrators 2017-08-30 19:27:12 UTC Comment hidden (obsolete)
Comment 3 jhertel 2017-08-30 22:04:44 UTC
Created attachment 135887 [details]
Empty document exhibiting the behaviour, password is p

It took a while to add more information, as I had to spend some time installing a new version of LibreOffice again, and also had to figure out how to clean the original document.

But instead of trying to clean the original, I am simply attaching an empty, new document saved with a password ("p") from LibreOffice 5.4.1.2. Opening that still takes 5 to 7 seconds after entering the password before I am free to type with no lag. Saving it also takes way too long time; it takes several seconds (at least 2 or 3) from the save is initiated until what I type is reflected immediately on the screen. That's too long and ruins the user experience.

It's so slow that I cannot upgrade from LibreOffice 5.2 but have to stay on the 5.2 branch until this bug has been fixed. Opening and saving password protected documents like the supplied one was super fast in version 5.2.6.2. I'll probably try to go back to 5.2.7.2 and see if that will do.

In LibreOffice 5.4.1.2, saving and opening documents *without* password is super fast with no problems.

If you have problems reproducing the bug, maybe the computer you are using is too fast. In that case try with a slower, more normal computer. Not that I believe my computer is slow in any way, but it's not a gaming computer, just an ordinary i5 laptop.
Comment 4 Telesto 2017-09-04 15:02:54 UTC
Created attachment 136012 [details]
Example file

1. Open the attached file (notice the fast opening speed)
2. File -> Save as -> Enter password 'p' -> Notice saving is slow
3. Open the saved file -> slower as initial file

Repro with:
Version: 6.0.0.0.alpha0+
Build ID: ec67943f59e09f0f933182f52e582e28b3f258bc
CPU threads: 4; OS: Windows 6.19; UI render: default; 
TinderBox: Win-x86@42, Branch:master, Time: 2017-08-31_07:23:27
Locale: nl-NL (nl_NL); Calc: CL

and with:
Version: 5.3.0.0.beta2+
Build ID: f4b7650ecd46e5404b35dccfb8b7d3b0a385d633
CPU Threads: 4; OS Version: Windows 6.2; UI Render: default; Layout Engine: new; 
TinderBox: Win-x86@62-merge-TDF, Branch:libreoffice-5-3, Time: 2016-12-16_12:19:09
Locale: nl-NL (nl_NL); Calc: CL

but not with:
Version: 5.2.5.0.0+
Build ID: a4d4fbeb623013f6377b30711ceedb38ea4b49f8
CPU Threads: 4; OS Version: Windows 6.2; UI Render: default; 
TinderBox: Win-x86@62-merge-TDF, Branch:libreoffice-5-2, Time: 2016-12-24_14:43:55
Locale: nl-NL (nl_NL); Calc: CL
Comment 5 Telesto 2017-10-08 15:06:01 UTC
*** Bug 112881 has been marked as a duplicate of this bug. ***
Comment 6 jhertel 2017-10-08 16:34:50 UTC
A follow-up (I am the original reporter, not a LibreOffice developer): 

I am now using version 5.2.7.2 (currently the latest in the 5.2 series), and the problem is not there; saving password encrypted files is super fast. I will have to stay with this version until the problem has been solved.

For others with the same problem, version 5.2.7.2 can be found at https://downloadarchive.documentfoundation.org/libreoffice/old/5.2.7.2/ .

All old versions can be found at https://downloadarchive.documentfoundation.org/libreoffice/old/ .
Comment 7 Patrick T. 2017-10-08 19:08:08 UTC Comment hidden (no-value)
Comment 8 Telesto 2017-10-08 19:45:33 UTC
Setting to All. Linux builds are also affected
Comment 9 Telesto 2017-11-15 16:07:12 UTC
Created attachment 137784 [details]
Bibisect log

Regression introduced by:
author	Michael Stahl <mstahl@redhat.com>	2016-11-09 17:42:46 (GMT)
committer	Michael Stahl <mstahl@redhat.com>	2016-11-09 18:07:20 (GMT)
commit	25205d5b29d0aade0ebd7c6405a91995d02a3a7c (patch)
tree	b59667a739670467f2a8a4ad524cac8640fdb100
parent	2a5bb08a2c84470a7a33547ee478d3c26f7ae159 (diff)
package: ODF: bump PBKDF2 iteration counts
Given recent elections we need to build a higher wall to keep the
government out of our documents, and we will make the government
pay for it.

These iteration counts were considered appropriate a decade ago.

http://security.stackexchange.com/questions/3959/recommended-of-iterations-when-using-pkbdf2-sha256

We get similar numbers on SandyBridge-E desktop and Haswell i7-4600U laptop:
* with 10k iterations ~20 msec per derivation
* with 100k iterations ~195 msec per derivation
* with 150k iterations ~290 msec per derivation

We can't go too high though because in ODF every package stream gets
its own derived key with a different salt, so a document with embedded
images may need a lot of these.
Comment 10 Telesto 2017-11-15 16:08:10 UTC Comment hidden (obsolete)
Comment 11 jhertel 2017-11-15 17:44:55 UTC
If the regression is caused by a sudden extreme increase in the number of iterations, then first of all I feel pissed off by the arrogance of just changing that out of the blue (if it was actually known by Michael that it would make saving noticably slower), and second of all I find it interesting how Michael referred to https://security.stackexchange.com/questions/3959/recommended-of-iterations-when-using-pkbdf2-sha256, when the most popular and accepted answer (https://security.stackexchange.com/questions/3959/recommended-of-iterations-when-using-pkbdf2-sha256#answer-3993) starts by clearly stating:

"You should use the maximum number of rounds which is tolerable, performance-wise, in your application. The number of rounds is a slowdown factor, which you use on the basis that under normal usage conditions, such a slowdown has *negligible impact* for you (*the user will not see it*, the extra CPU cost does not imply buying a bigger server, and so on)."

5-7 seconds is *not* a negligible impact that the user will not see.

There could perhaps be a special way of encrypting documents for people that are so scared about democracy (= government in all democratic countries) as Michael seems to be, but then let that be a special option for those people to specifially turn on, and let the rest of us have a useful program. Right now, the current version of LibreOffice is completely useless if you want to use password protected documents even once in a while.
Comment 12 Michael Stahl (CIB) 2017-11-24 13:03:56 UTC
i'm afraid this is going to be expensive to fix properly;
i was planning to redo ODF encryption so it stores
a single ODF package as a zip-in-a-zip which should fix it,
but have had no time to work on it, not to mention
it will be incompatible of course.

btw Word 2010 uses 100k KDF iterations as well for OOXML;
i haven't checked what the current version does.
Comment 13 Thorsten Behrens (CIB) 2018-03-30 20:40:13 UTC
(In reply to jhertel from comment #11)
> 5-7 seconds is *not* a negligible impact that the user will not see.

I wonder if perhaps using the OpenPGP encryption available from 6.0 on is an option then - it's still experimental, but uses a fixed public/private key pair, that at least does not need to run KDFs for every sub-stream:

https://wiki.documentfoundation.org/Development/gpg4libre
Comment 14 Rich 2018-07-15 00:59:26 UTC
I have the same issue. 5.2.4.2 saves quickly with password. 5.4.6.2 and 6.0.5.2 save slow with password and quickly without password.
Comment 15 mycomments2017 2019-04-04 07:34:12 UTC
Intolerably slow. Please make it an option in Options->load/save->General tab (or some other (even some no-ui hidden special config flag (just like firefox has in with it's 'about:config') or a registry/xml/json setting): iteration count: 1024 or 100000.
Comment 16 Buovjaga 2019-04-18 20:35:31 UTC
Saving with the example files is now nearly instant. Can everyone please test again?

For Calc files there is still bug 122060

Arch Linux 64-bit
Version: 6.3.0.0.alpha0+
Build ID: 1fee3f1da6291bfbcd75455512726215d41b3e83
CPU threads: 8; OS: Linux 5.0; UI render: default; VCL: gtk3; 
Locale: fi-FI (fi_FI.UTF-8); UI-Language: en-US
Calc: threaded
Built on 18 April 2019
Comment 17 Telesto 2019-04-19 06:53:21 UTC
(In reply to Buovjaga from comment #16)
> Saving with the example files is now nearly instant. Can everyone please
> test again?

Still repro
Version: 6.3.0.0.alpha0+
Build ID: 3a5d78365dd172881c16c03e67f2d170ffc6d7d4
CPU threads: 4; OS: Windows 6.3; UI render: GL; VCL: win; 
TinderBox: Win-x86@42, Branch:master, Time: 2019-04-09_22:53:59
Locale: nl-NL (nl_NL); UI-Language: en-US
Calc: CL

Which attachment did you use? My example file is working properly at the initial state (even with password set). Slow behavior will be triggered by saving the file with a recent LibO & reopening it (see comment 4)
Comment 18 Buovjaga 2019-04-19 07:11:11 UTC
(In reply to Telesto from comment #17)
> (In reply to Buovjaga from comment #16)
> > Saving with the example files is now nearly instant. Can everyone please
> > test again?
> 
> Still repro
> Version: 6.3.0.0.alpha0+
> Build ID: 3a5d78365dd172881c16c03e67f2d170ffc6d7d4
> CPU threads: 4; OS: Windows 6.3; UI render: GL; VCL: win; 
> TinderBox: Win-x86@42, Branch:master, Time: 2019-04-09_22:53:59
> Locale: nl-NL (nl_NL); UI-Language: en-US
> Calc: CL
> 
> Which attachment did you use? My example file is working properly at the
> initial state (even with password set). Slow behavior will be triggered by
> saving the file with a recent LibO & reopening it (see comment 4)

I used both attachments. The summary is "Very slow saving". The step 3 in your comment 4 seems to be something else. Do you really repro the slow saving?
Comment 19 Xisco Faulí 2019-04-19 12:43:19 UTC
attachment 135887 [details] takes less than 5 seconds to get saved

Version: 6.3.0.0.alpha0+
Build ID: c9956772ec0678498515fb60dca41e9a77457f86
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); UI-Language: en-US
Calc: threaded
Comment 20 Telesto 2019-04-20 07:20:05 UTC
Nothing changed compared to the initial bug report comment 0 
I takes +/-5 seconds to save (without progress bar, only locked screen with disappearing cursor) -> Expected 1 second (4.4.7.2)

It takes +/-7 seconds to open. Expected 3 seconds

So this can be quite annoying; especially with a short auto-save interval.
Comment 21 Buovjaga 2019-04-20 09:39:45 UTC
Created attachment 150894 [details]
Perf flamegraph

From saving attachment 136012 [details]

Arch Linux 64-bit
Version: 6.3.0.0.alpha0+
Build ID: bcb0c9b4bee1d943d9c60f9d4512dba901f85f54
CPU threads: 8; OS: Linux 5.0; UI render: default; VCL: gtk3; 
Locale: fi-FI (fi_FI.UTF-8); UI-Language: en-US
Calc: threaded
Built on 20 April 2019