Bug 122530 - FILESAVE: Auto save timing logic problem: can save too frequently
Summary: FILESAVE: Auto save timing logic problem: can save too frequently
Status: RESOLVED MOVED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
6.1.3.2 release
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: AutoSave-AutoRecovery-Backup
  Show dependency treegraph
 
Reported: 2019-01-07 14:13 UTC by Luke Kendall
Modified: 2021-09-15 08:06 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Two longish documents (1.34 MB, application/zip)
2019-05-23 08:05 UTC, Luke Kendall
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Luke Kendall 2019-01-07 14:13:53 UTC
My document (novel with editorial comments) takes about 30 secs to save.

My auto-save interval is set to 30 mins. 
 
Despite that, I have noticed several times (and again just now) that auto-save can trigger less than two minutes after manually saving the document.

The necessary preconditions seem to be:

1) Have multiple large documents open
2) Edit all of them
3) Wait for an auto save to trigger on one.

If I then save my main document, it can complete that Save but then get a fresh auto-save only a minute or two later.

I think what is happening is that the check of the time since last save is being made for all documents open; or if it is a per-document property, the 'decision to save' a document is being made in advance of executing the auto-save somehow.  As if the decision to save is made when any document currently open starts to auto-save, and the actual last saved time of each document is disregarded for the auto save which has been decided is due.
Comment 1 Xisco Faulí 2019-05-22 14:58:28 UTC
Thank you for reporting the bug. Please attach a sample document, as this makes it easier for us to verify the bug. 
(Please note that the attachment will be public, remove any sensitive information before attaching it. 
See https://wiki.documentfoundation.org/QA/FAQ#How_can_I_eliminate_confidential_data_from_a_sample_document.3F for help on how to do so.)

I have set the bug's status to 'NEEDINFO'. Please change it back to 'UNCONFIRMED' once the requested document is provided.
Comment 2 Luke Kendall 2019-05-23 07:38:50 UTC
I'm trying to do this.
Currently Writer is hanging when I try to load a good sample document.
Comment 3 Luke Kendall 2019-05-23 08:05:41 UTC
Created attachment 151613 [details]
Two longish documents

The documents I wanted to include have a lot of comments, but I can't get Writer to display the document(s).  6.2.3.2 just goes to 90-100% CPU and stays there.
I'll give it an hour and see if it loads within that time, but I'm not optimistic.
I can't provide that document without obfuscating it, however.
All the same, here are two obfuscated documents that take a reasonable length of time to save.
If you're willing to take a suggestion, I think it may be easier to find the bug by looking at the logic of the existing code and asking "How is it possible for an auto-save to start, for a document, only minutes after it has been manually saved and the auto-save period is set to 30 mins?"

If the logic for auto-save for the current document includes the test
"if Now - current_doc.save_instant < autosave_period then {don't save}",
it simply should never be possible.

(PS: 15mins later, other .odt file still not displaying, soffice 80-100% CPU)
Comment 4 QA Administrators 2019-05-24 02:59:14 UTC Comment hidden (obsolete)
Comment 5 Buovjaga 2019-08-25 07:52:07 UTC
I think implementing the ideas in bug 94042 would achieve what you want:
"...a smart autorecovery algorithm that can work within the set autorecovery time, so that if a user for example stops working on a document for a given amount of time (e.g. 5 minutes), that it smart autosaves."

"There are many possible scenarios when the smart autorecovery can kick and i think it should possibly take into consideration the amount of time it last took to save and autorecovery save. So if a user is working on a small document and save/autorecovery last took 3 seconds, we could smart autorecovery save if a user hasnt done anything in 1 minute, while if a user is working on a large spreadsheet that takes 30 seconds to save/autorecovery, we could smart autorevoery save if a user hasnt done anything in 5 minutes."

Do you agree?
Comment 6 Luke Kendall 2019-08-25 09:20:53 UTC
I'm really unsure, because it would then be explicitly doing what the user asked it NOT to.

If the new feature was enabled by a separate checkbox (smart autosave), then I suppose it wouldn't hurt, because you could fairly argue that it means the user allows the program to override their stated auto-save period.

The thing that confuses me most is that autosave can kick in minutes or even seconds after an explicit manual save.

if the user has set the autosave period to X minutes, surely the period that is compared to X, is the time since last save?  Where that save could be an autosave or a manual save.  So X is measured from the most recent save.

Surely it is not measuring the autosave from the last time autosave was triggered?!  If so, that seems to me to be the problem, and the fix is simple.
Comment 7 Buovjaga 2019-08-25 09:42:30 UTC
I tested with the documents, setting autorecovery info save time to 2 minutes. Using a stopwatch, after 2 mins I saved one of the documents (savings takes about 5 seconds for me). Then I watched what happens for the next 2 mins. Nothing happened. Apparently there is no visible effect for me upon saving autorecovery info, at least with these documents.

Arch Linux 64-bit
Version: 6.4.0.0.alpha0+
Build ID: b9a776837462eeb6d50d0decc42604c0c3008eb1
CPU threads: 8; OS: Linux 5.2; UI render: default; VCL: gtk3; 
Locale: fi-FI (fi_FI.UTF-8); UI-Language: en-US
Calc: threaded
Built on 11 August 2019
Comment 8 Luke Kendall 2019-08-26 06:10:56 UTC
Can I check my understanding of what you described?

"after 2 mins I saved one of the documents (savings takes about 5 seconds for me)."

2 mins after doing something, you saved one document.
It took about 5 secs.
You started a timer.
For the next 2 mins, nothing happened.

That is not what I did, or observed.

What I did:

I had all documents open.
I manually saved each document, with a short period between each.
  (so, I think each auto-save should start with the same short period between each document)
I edited each document.
I observed auto-save trigger on one.
As soon as that auto-save finished, I manually saved another doc (let's call it doc2)
Within 2 mins of saving doc2 (much less than auto-save period), doc2 auto-saved.

I also just confirmed that for me, the documents VC-obfusc-5x8.odt and VC-obfusc-4x7.odt take about 30 secs to save.
Comment 9 Buovjaga 2019-08-27 17:09:46 UTC
I tried it again, following your directions in comment 8 closely. I am not able to observe the auto recovery info saving process no matter what I do.
Comment 10 Luke Kendall 2020-02-15 11:30:45 UTC
I just noticed it happen again.  File last saved at 22:12, autosave period set to 30 mins, and the auto-save kicked in at 22:18.

Four documents open, the active one (370,000 characters), and three others saved and unchanged for many hours.  The other three are 3600 chars, 25,000 chars, and 193,000 chars. 

Surely the logic of the auto-save should be, the time is measured from the last time the document was saved, regardless of whether that was a manual save or an auto save?  That time should be a per-document property.

Can you simply read the code to check that this is the logic that's implemented? If it isn't, then that is the problem.

Or perhaps there are multiple places where saving can occur, and the logic used is different for one or more of them?

I observed this just now with LO Version: 6.3.4.2
Build ID: 60da17e045e08f1793c57c00ba83cdfce946d0aa
CPU threads: 4; OS: Linux 4.4; UI render: default; VCL: x11; 
Locale: en-GB (en_AU.UTF-8); UI-Language: en-US
Calc: threaded
Comment 11 Dieter 2020-05-11 11:56:31 UTC
(In reply to Luke Kendall from comment #10)
> Surely the logic of the auto-save should be, the time is measured from the
> last time the document was saved, regardless of whether that was a manual
> save or an auto save?  That time should be a per-document property.
> 
> Can you simply read the code to check that this is the logic that's
> implemented? If it isn't, then that is the problem.
> 
> Or perhaps there are multiple places where saving can occur, and the logic
> used is different for one or more of them?

Informations from LO Help: "Save AutoRecovery information [...] specifies that LibreOfficeDev saves the information needed to restore all open documents in case of a crash. You can specify the time interval between saves." https://help.libreoffice.org/7.0/en-GB/text/shared/optionen/01010200.html?&DbPAR=WRITER&System=WIN

So I think, it's different from your expectations. Does this help in any way?
Comment 12 Luke Kendall 2020-05-11 12:23:27 UTC
It helps a little, in that it clarifies that the behaviour is not fully documented.

"Specifies that LibreOfficeDev saves the information needed to restore all open documents in case of a crash. You can specify the time interval between saves."

1) The documentation doesn't specify what is used as the starting time for the time measurement.

2) The documentation doesn't specify whether the starting time is per document or for all open documents.

I would argue that for best usability, the time should be measured per document.
It should be relative to each document's last save time.

Otherwise if you are working on many large documents, one or another will always be triggering a general autosave.

Another problem if the autosave time measurement is not per-document, how can the user know when the time is measured from.  Is it the (unknown) time you started running LO?  The last time you saved one of the open documents?  The first time? The last time you saved the first document you opened (and even knowing which doc that might be)?

I'm also confident that this unexpected saving occurs even when all other open documents have been saved (so, don't require an auto save), and you're only working on one document.  The reason I'm sure is that for while I believed it was triggered by having other documents modified but not saved, but when I worked with that assumption (ensured I had other documents saved), it still happened.
Comment 13 Dieter 2020-05-11 12:32:15 UTC
So perhaps we have two issues here:

1. Improve documentation (should be reported in a new bug)
2. Enhancement request: AutoSave per document (perhaps this bug or a new one)
Comment 14 Telesto 2020-05-11 19:11:35 UTC
(In reply to Luke Kendall from comment #12)
> It helps a little, in that it clarifies that the behaviour is not fully
> documented.
> 
> "Specifies that LibreOfficeDev saves the information needed to restore all
> open documents in case of a crash. You can specify the time interval between
> saves."
> 
> 1) The documentation doesn't specify what is used as the starting time for
> the time measurement.
> 
> 2) The documentation doesn't specify whether the starting time is per
> document or for all open documents.
> 
> I would argue that for best usability, the time should be measured per
> document.
> It should be relative to each document's last save time.
> 
> Otherwise if you are working on many large documents, one or another will
> always be triggering a general autosave.

Never had any thoughts this deep about autosave ;-). I would say interval after each open document, starting from opening. 

However this will be an issue: otherwise if you are working on many large documents, one or another will always be triggering a general autosave

This is more or less a a design flaw of LibreOffice. I would prefer this solution: bug 63554 However not gonna happen any time soon.

An alternative: running multiple instances of Writer using multiple AppImages/ Snap packs (not sure what is what). This way you wouldn't into the have to wait before auto save is finished issue of all the documents.. with a relative normal saving interval (instead of setting in 20 minutes) to be able to continue work. However might be slightly confusing to differentiate between all those instances..
Comment 15 Telesto 2020-07-01 09:15:34 UTC
The who logic including the code is explained here.. Pretty well documented.. Especially interesting are of course all variables defining the start/stop of interval timer of autosave/autorecovery. And the rules defining the timing of  autosave/autorecovery (autosave & autorecovery can't be executed at the same time can't of stuff)

So if you want to investigate or some impression how the autosave/autorecovery works etc.

https://opengrok.libreoffice.org/xref/core/framework/source/services/autorecovery.cxx

Might make it easier to pinpoint the flaw
Comment 16 Luke Kendall 2020-07-01 13:46:54 UTC
I assume this comment is a little misleading:
"// force save of all currently open documents"
I assume it really means "force save of all currently open documents that have been modified".

Even so, if the design really is that autosave triggers and saves all documents at the same time, that seems wrong to me if it really means the more documents you have open and are working on, the more likely autosave is to trigger.

I'm not a C++ programmer, and I'm afraid I couldn't follow the logic.  I couldn't even work out the start point from which the time period is measured.

I get the idea it's measured from a fixed time (e.g., when Writer started running?), and is not related to the elapsed time since a document was last saved? That seems wrong.

From this user's perspective, the autosave period seems like it's a way of choosing "How many minutes of work on a document do I want to risk losing if LO crashes?"

If the time isn't measured from a document's last 'safe point' (so, the maximum of last save time, and when the doc was opened), then I don't have any idea what it means.  If it isn't measured from each doc's last 'safe time', then the user can't avoid being interrupted even if they're diligent about saving regularly.

I think implementing the request of bug 63554 would solve the problem.  The person asked for each document to be a separate process, not a separate thread.  Separate processes don't share memory as far as I understand things. At least, not memory used for data.
Comment 17 Telesto 2020-09-02 21:08:09 UTC
@Luke 
@Off topic mode
If i'm remember I asked this before, but how do you do they obfuscating thing. 
It's different (& better) compared to what's described here
https://wiki.documentfoundation.org/QA/Bugzilla/Sanitizing_Files_Before_Submission

https://bugs.documentfoundation.org/attachment.cgi?id=148801 (which produces this kind of results).. without spaces

Indirectly also directed to @Buovjaga :-). He did the sanitizing for bug 122792. I dislike sanitizing spaces the matter for how pages split etc. They sanitized version tends to sanitize bugs too.
Comment 18 Luke Kendall 2020-09-03 01:19:03 UTC
I generally sanitise by throwing away most but not all the information.
I keep capitals, vowels, punctuation.
So the change is done with a regexp and match case enabled, then
Find: ([bcdfghjklmnpqrstvwyz]) <-- Sometimes I consider 'y' a vowel and omit it
Replace: x

Typically I need to sit and wait until the question about "this is taking a long time; do you want to continue" pops up, so I can tell it to continue.

HTH
Comment 19 Luke Kendall 2020-09-18 13:41:11 UTC
Just a note:
At 11:37 local time, auto-save kicked in and interrupted my editing.
Last save had been 2 mins earlier:
-rw-rw----  1 luke kendall   801718 Sep 18 23:35 LostGirl-5x8.odt

This is with version 6.4.4.2.

The auto-save time measurement logic is clearly broken.
Comment 20 Luke Kendall 2020-09-24 08:48:09 UTC
This is happening quite frequently. Often the auto save (which I have set at 30 mins), triggers a mere two minutes after I last saved the file.

It seems to especially happen if I save the file when I see auto-save trigger in another file I'm working on.  (I sometimes save the other file a few minutes before the half hour mark, as a reminder to save my main file. But due to this bug, that practice doesn't necessarily save me from being held up by an auto-save.)

This behaviour suggests to me the code makes a check of all open files to decide if they need an auto-save.  However, since many minutes can pass between computing that check and then executing each auto save, it appears the code does not check that a manual save has been performed in the meantime.

Can you at least add a check to avoid an auto-save of a document it it has been saved more recently than the auto-save period?  That should fix the problem.
Comment 21 Telesto 2020-09-24 09:01:39 UTC
@Mike
This bit of technical topic. Which needs someone with DEV expertise. I estimate you might you might now bit more.  

(In reply to Luke Kendall from comment #20)
> This is happening quite frequently. Often the auto save (which I have set at
> 30 mins), triggers a mere two minutes after I last saved the file.
> 
> It seems to especially happen if I save the file when I see auto-save
> trigger in another file I'm working on.  (I sometimes save the other file a
> few minutes before the half hour mark, as a reminder to save my main file.
> But due to this bug, that practice doesn't necessarily save me from being
> held up by an auto-save.)
> 
> This behaviour suggests to me the code makes a check of all open files to
> decide if they need an auto-save.  However, since many minutes can pass
> between computing that check and then executing each auto save, it appears
> the code does not check that a manual save has been performed in the
> meantime.
> 
> Can you at least add a check to avoid an auto-save of a document it it has
> been saved more recently than the auto-save period?  That should fix the
> problem.
Comment 22 Luke Kendall 2020-09-30 14:08:29 UTC
A new record: it kicked in four minutes after auto-save of one edited document, and manual save of the main document.
The whole time, Writer was quite responsive and normal.
Comment 23 Dieter 2021-09-14 20:06:16 UTC
(In reply to Dieter from comment #13)
> So perhaps we have two issues here:
> 1. Improve documentation (should be reported in a new bug)
> 2. Enhancement request: AutoSave per document (perhaps this bug or a new one)

I come back to my suggestion. Do you agree, Luke? If yes, could you open two new bug reports (I think this report is very complex now). If no, please give a short reasoning.
Comment 24 Mike Kaganski 2021-09-14 20:50:16 UTC
(In reply to Luke Kendall from comment #10)
> Surely the logic of the auto-save should be, the time is measured from the
> last time the document was saved, regardless of whether that was a manual
> save or an auto save?  That time should be a per-document property.

Well, I would possibly agree that it *could* look reasonable to count per document.

However, the measure then should not be "measured from the last time the document was saved". This is flawed idea: the correct starting point should be, *given the spirit of this request*, the moment when the file got dirty (i.e., the first edit was made).

Think about it. If you count since file open, you would have the same unexpected result:

0. Set interval to 30 minutes.
1. Open a file. Don't edit it yet.
2. Open other files, and edit them ...
3. After 29 minutes and 30 seconds, switch to the first document, and start editing it.
4. And suddenly, when you have just started, the 30 minutes timeout elapses, and it starts saving your 30-second edits.

Wrt comment 13: please file a documentation bug separately.
Comment 25 Luke Kendall 2021-09-15 06:06:15 UTC
(In reply to Dieter from comment #23)
> (In reply to Dieter from comment #13)
> > So perhaps we have two issues here:
> > 1. Improve documentation (should be reported in a new bug)
> > 2. Enhancement request: AutoSave per document (perhaps this bug or a new one)
> 
> I come back to my suggestion. Do you agree, Luke? If yes, could you open two
> new bug reports (I think this report is very complex now). If no, please
> give a short reasoning.

Yes.

My hope for 1. would be that if the documentation is clear, it will also be clear that the current design causes substantial problems for some users.

My hope for 2 would be that it could be implemented, of course. I'm puzzled that LO is a multi-threaded program, actually. In my experience using LO, it has caused me massive problems, including lost data, for minuscule benefits (slightly faster access to changed options or preferences between separate instances).

Anyway, will do.
Comment 26 Luke Kendall 2021-09-15 06:15:51 UTC
(In reply to Mike Kaganski from comment #24)
> (In reply to Luke Kendall from comment #10)
> > Surely the logic of the auto-save should be, the time is measured from the
> > last time the document was saved, regardless of whether that was a manual
> > save or an auto save?  That time should be a per-document property.
> 
> Well, I would possibly agree that it *could* look reasonable to count per
> document.
> 
> However, the measure then should not be "measured from the last time the
> document was saved". This is flawed idea: the correct starting point should
> be, *given the spirit of this request*, the moment when the file got dirty
> (i.e., the first edit was made).
> 
> Think about it. If you count since file open, you would have the same
> unexpected result:
> 
> 0. Set interval to 30 minutes.
> 1. Open a file. Don't edit it yet.
> 2. Open other files, and edit them ...
> 3. After 29 minutes and 30 seconds, switch to the first document, and start
> editing it.
> 4. And suddenly, when you have just started, the 30 minutes timeout elapses,
> and it starts saving your 30-second edits.
> 
> Wrt comment 13: please file a documentation bug separately.

Since I didn't suggest you should count since file open, I see no reason to consider the problems in such an idea. I agree it wouldn't help. So I'm glad I didn't suggest it. :-)

My suggestion is to measure, per document, from the last save time, for that document.

I also disagree that it should be measured from the first time the doc was dirtied. I think it's far more useful to measure from the first time the doc was dirtied *since the last save*. Probably that's what you meant?
Comment 27 Mike Kaganski 2021-09-15 07:04:04 UTC
(In reply to Luke Kendall from comment #26)
> I also disagree that it should be measured from the first time the doc was
> dirtied. I think it's far more useful to measure from the first time the doc
> was dirtied *since the last save*. Probably that's what you meant?

Every save makes the document "clean". The following save will make it dirty, which is the moment to detect and start counting.

For the sake of auto-save, the moment of such auto-save also should mark the document as "clean" (only internally, for the purposes of the discussed problem; indeed, it should still the "unsaved edits" in UI, and ask for save on close - but it should not try to auto-save again until more edits make it internally dirty again).
Comment 28 Mike Kaganski 2021-09-15 07:28:19 UTC
(In reply to Mike Kaganski from comment #27)
> Every save makes the document "clean". The following save will make it
> dirty,

Oh a typo indeed; meant to be "The following *edit* will make it dirty"
Comment 29 Dieter 2021-09-15 08:06:21 UTC
(In reply to Dieter from comment #13)
> So perhaps we have two issues here:
> 1. Improve documentation
this is now bug 144511

> 2. Enhancement request: AutoSave per document (perhaps this bug or a new one)
This is now bug 144512

So let's discuss everything within the new bug reports and close this report.

I changed status to RESOLVED MOVED (probably not correct, but I couldn't find correct status)