Update: appears to be happening in safe mode too (i.e., is not a problem with the user profile). After closing everything and restarting LO in safe mode, I opened everything back up and continued checking:

15:23:10
  PID CMD                         %MEM %CPU
17409 /usr/lib/firefox-esr/firefo  5.2  9.0
17501 /usr/lib/firefox-esr/firefo  3.8  0.9
 7842 /usr/lib/firefox-esr/firefo  3.6  2.6

15:38:10
  PID CMD                         %MEM %CPU
 9696 /usr/lib/libreoffice/progra  7.9 57.3
17409 /usr/lib/firefox-esr/firefo  6.9  9.3
 7842 /usr/lib/firefox-esr/firefo  4.6  2.0

15:53:10
  PID CMD                         %MEM %CPU
 9696 /usr/lib/libreoffice/progra 10.4 77.5
17409 /usr/lib/firefox-esr/firefo  8.5  9.7
17501 /usr/lib/firefox-esr/firefo  5.2  1.0

15:59:29
  PID CMD                         %MEM %CPU
 9696 /usr/lib/libreoffice/progra 11.5 80.8
17409 /usr/lib/firefox-esr/firefo  6.6  9.9
17501 /usr/lib/firefox-esr/firefo  6.6  1.0

(In reply to Julien Nabet from comment #2)
> Debian testing proposes 6.4.2
> 
> Do you MS and ODF formats or only ODF formats?
> 
> About all your files, are there some which are only for reading only?
> If yes, perhaps you may convert them into pdfs so they may consume less
> memory.
> Of course, check that PDF conversion is ok.

Only ODF formats (ODS, ODT).

Hmm, well I already converted some to pdfs. Evince takes so little memory, that would definitely help. But to do that with a bunch of the files, I'd need a better PDF editor, so I could highlight text, make comments, etc. The need for such functionality is why I've kept them in libreoffice.

But really, this increasing memory usage isn't normal, right!?

(In reply to Julien Nabet from comment #4)
> To investigate, we'd need an example of case which increases memory so we'd
> have a minimal step by step process to reproduce this.
> Indeed, as it is, the bugtracker won't help much since it just indicates
> there are memory leaks in LO, nothing else. Devs already know it but need
> more information to tackle each leak.

Hi, Julien, and thanks for the reply. I'm making some progress on this:

I monitored the memory usage while closing a couple of the files at a time. I *think* that I've found the file that must be the problem. It'd be best to be able to attach it here, but I can't, since it's a text I'm using by permission from someone who will end up publishing it later.

But, here are some specifics about the suspect file:

**Written in .doc format, saved by me into libreoffice and annotated after that. (Maybe I should have said this in response to your first comment?)

**Almost entirely polytonic Greek, with some English and other languages mixed in.

**32 pages, 90000 characters, 500+ footnotes

I searched for problems involving special characters and memory usage in libreoffice and found this: https://ask.libreoffice.org/en/question/157664/slow-typing-and-high-cpu-usage-after-a-certain-point/, which suggests that using shift-enter for a newline can cause an issue. Well, it looks like in my document everything is paragraphed correctly; at least, when I turn on view->formatting marks, I can see paragraph symbols after each paragraph.

Let me know if you need the file itself to comment further. I could probably obfuscate the contents somehow.

Content can be obfuscated by replacing each char with X.
But, we can only analyze source .ODT when saving to .DOC or .DOCX and experiencing issues. 
If first save was to .DOC, no help..
If MS .DOC was saved to .ODT in LO, I guess that qualifies for acceptable report.
Footnotes may be suspicious, as in bug 76260.
 
Note: if responding to the whole previous post, no need to quote.

(In reply to Timur from comment #7)
> Content can be obfuscated by replacing each char with X.
> But, we can only analyze source .ODT when saving to .DOC or .DOCX and
> experiencing issues. 
> If first save was to .DOC, no help..
> If MS .DOC was saved to .ODT in LO, I guess that qualifies for acceptable
> report.
> Footnotes may be suspicious, as in bug 76260.
>  
> Note: if responding to the whole previous post, no need to quote.

Hi,

Indeed: an MS .DOC was saved to .ODT in LO.

Today, I found a workaround, by selecting all and copying into a new file. I think this avoids the problem. Should I close the bug, or should we pursue it (for the good of LO)?

Only way to have this bug useful is to have source MS DOC (with content obfuscated in MSO) and steps to get ODT (like: copy all and paste).
If you can prepare it and attach, please set Unconfirmed.
Unless, I close.

Created attachment 159319 [details]
first document

Created attachment 159320 [details]
second document

(In reply to Timur from comment #10)
> Only way to have this bug useful is to have source MS DOC (with content
> obfuscated in MSO) and steps to get ODT (like: copy all and paste).
> If you can prepare it and attach, please set Unconfirmed.
> Unless, I close.

OK, I think I've set it up to be reproducible:

1. open two attached documents in libreoffice (both .docx)
2. save as .odt
3. wait and watch memory being taken up (creeps up steadily on my x220 with 8GB ram)

To fix the problem, I just copied all of the text from each file into a new .odt (instead of saving it as an .odt straight from .docx format). So for me, the problem is solved. But it might be helpful for other users who are starting with MSO and moving to ODT to know how to solve this problem.

Let me know if you need other information, and thanks.

(In reply to Bjorn Wastvedt from comment #12)

I do reproduce the issue with the second file. STR
1. Attachment 159320 [details] (and let it run CPU 25%, memory usage increasing)


Version: 7.0.0.0.alpha0+ (x64)
Build ID: 4501a0ba623ad61c5a4e0b807da2e96f0e4ce82c
CPU threads: 4; OS: Windows 6.3 Build 9600; UI render: default; VCL: win; 
Locale: nl-NL (nl_NL); UI-Language: en-US
Calc: CL

Repro with
Version: 5.4.0.2
Build ID: 2b906d450a44f2bbe506dcd22c51b3fa11dc65fd
CPU threads: 4; OS: Windows 6.2; UI render: default; 
Locale: nl-NL (nl_NL); Calc: CL

No repro with
Versie: 4.4.7.2 
Build ID: f3153a8b245191196a4b6b9abd1d0da16eead600
Locale: nl_NL

File opening is also too slow, but probably already tracked in bug 76260

Bibisected with linux 6.5 repo to https://git.libreoffice.org/core/+/2e0a32b51681fb356699b4a722f461f55a46b890%5E!/

weld FontNameBox

The used memory increases rapidly upon opening.

On Windows, the memory increase is not as rapid and what is stranger, it is also observed with the oldest commit of 6.5 bibisect repo :(

On Linux I bibisected the document mentioned in this dev list post to the same commit: https://lists.freedesktop.org/archives/libreoffice/2020-May/085014.html

I am unable to repro the problem in any older version or bibisect repo I tried. I only see it in the 6.5/7.0 line.

As the result is a bit strange, I do not want to add Caolán to Cc.

Same result is observed in safe mode.

Arch Linux 64-bit
Version: 7.0.0.0.alpha0+
Build ID: 454a3c945fdc02d706b0a5ad49ca13e0443fa8e5
CPU threads: 8; OS: Linux 5.6; UI render: default; VCL: kf5; 
Locale: fi-FI (fi_FI.UTF-8); UI-Language: en-US
Calc: threaded
Built on 3 May 2020

Ok, I bibisected with win 6.4 repo and this time the commit seems more reasonable: https://git.libreoffice.org/core/+/a991ad93dcd6807d0eacd11a50c2ae43a2cfb882%5E!/

tdf#121441 improve DOCX footnote import

Adding Cc: to Jan-Marek Glogowski

To repro, simply open attachment 159320 [details] and monitor the memory use. No need to save it or anything.

Just gave it a quick test, with my master, bibisect-6-4 and bibisect-6-5 master checkouts and I can't reproduce. I used a fresh profile and both documents using kf5.

so -env:UserInstallation=file:///tmp/mem /tmp/doc*.docx

LO starts with 100% CPU but completely settles after a few seconds with constant memory and no CPU load. I'm also on Debian Buster, FWIW.

Best bet would be to get some callgrind / perf log. See performance debugging in https://wiki.documentfoundation.org/Development/How_to_debug

While callgrind is much slower, its logs are also much smaller. So if you can't or don't want to debug this yourself, you probably want to run callgrind - instead of perf - for a while (on that HW probably 30m+).

My first guess was "just" an other Writer idle layout problem, but that should normally utilize a full core, which doesn't seem to happen for you, and the memory gets leaked very slow (3% in 15m of 7.5GB), so I'm quite probably wrong. If there was a parsing problem with the new footnote code, it would probably exhaust the memory really fast.

Still in the end this would mean blind fixing for me without a reproducer, so I doubt I can help.

Created attachment 160346 [details]
Perf flamegraph

I let the memory use increase to 1GB

Arch Linux 64-bit
Version: 7.0.0.0.alpha0+
Build ID: 454a3c945fdc02d706b0a5ad49ca13e0443fa8e5
CPU threads: 8; OS: Linux 5.6; UI render: default; VCL: kf5; 
Locale: fi-FI (fi_FI.UTF-8); UI-Language: en-US
Calc: threaded
Built on 3 May 2020

(In reply to Buovjaga from comment #17)
> Bibisected with linux 6.5 repo to
> https://git.libreoffice.org/core/+/
> 2e0a32b51681fb356699b4a722f461f55a46b890%5E!/
> 
> weld FontNameBox
> 
> The used memory increases rapidly upon opening.
> 
> On Windows, the memory increase is not as rapid and what is stranger, it is
> also observed with the oldest commit of 6.5 bibisect repo :(
> 
> On Linux I bibisected the document mentioned in this dev list post to the
> same commit:
> https://lists.freedesktop.org/archives/libreoffice/2020-May/085014.html
> 
> I am unable to repro the problem in any older version or bibisect repo I
> tried. I only see it in the 6.5/7.0 line.
> 
> As the result is a bit strange, I do not want to add Caolán to Cc.

Found another bug, bug 132536 (same commit). It surely leaks.. but can't be the root cause.. must be between 5.4 and 4.4.7.2 following my observations.. comment 15

Oh, actually, the memory leak is not infinite. Something like 1,6GB taken at the end. I seems this affects nearly all documents.

These attachments seem to open very quickly now since bug #76260 was fixed.

Checked with:

Version: 7.2.0.0.alpha0+ / LibreOffice Community
Build ID: cbcec4425e04e3614a2025b49fdc221216ac51d3
CPU threads: 8; OS: Linux 5.4; UI render: default; VCL: gtk3
Locale: hu-HU (hu_HU.UTF-8); UI: en-US
Calc: threaded

*** This bug has been marked as a duplicate of bug 76260 ***