Bug 132536 - Memory usage increases after every file-reload
Summary: Memory usage increases after every file-reload
Status: CLOSED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
6.4.4.2 release
Hardware: All All
: high normal
Assignee: Caolán McNamara
URL:
Whiteboard: target:7.0.0 target:7.1.0 target:7.0.4
Keywords: bibisected, bisected, regression
Depends on:
Blocks: Memory
  Show dependency treegraph
 
Reported: 2020-04-29 21:12 UTC by Telesto
Modified: 2020-12-01 15:23 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Example file (981.49 KB, application/vnd.oasis.opendocument.text)
2020-04-29 21:13 UTC, Telesto
Details
Bibisect log (2.74 KB, text/plain)
2020-05-08 18:39 UTC, Telesto
Details
Screencast (17.67 MB, video/mp4)
2020-05-13 15:40 UTC, Telesto
Details
Reduced Example file (8.72 KB, application/vnd.oasis.opendocument.text)
2020-05-13 16:56 UTC, Telesto
Details
Screencast (1.91 MB, video/mp4)
2020-09-29 08:48 UTC, Telesto
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Telesto 2020-04-29 21:12:25 UTC
Description:
Memory usage increases after every file-reload

Steps to Reproduce:
1.open the attached file
2. Take a process monitor and check memory usage
3. File -> Reload & File -> Reload File -> Reload.
4. Close the file -> Still substantial memory usage

Actual Results:
Memory usage increases

Expected Results:
Should be stable


Reproducible: Always


User Profile Reset: No



Additional Info:
Version: 7.0.0.0.alpha0+ (x64)
Build ID: f845f74afaf087a46c82ee4209e29caca0980b71
CPU threads: 4; OS: Windows 6.3 Build 9600; UI render: GL; VCL: win; 
Locale: nl-NL (nl_NL); UI-Language: en-US
Calc: CL
Comment 1 Telesto 2020-04-29 21:13:21 UTC
Created attachment 160094 [details]
Example file

Modified version of sample file of bug 131729
Comment 2 Telesto 2020-05-08 18:39:49 UTC
Created attachment 160547 [details]
Bibisect log

Bisected to
author	Caolán McNamara <caolanm@redhat.com>	2020-04-07 12:21:47 +0100
committer	Caolán McNamara <caolanm@redhat.com>	2020-04-21 10:19:41 +0200
commit 2e0a32b51681fb356699b4a722f461f55a46b890 (patch)
tree ac96e726d777aba5b6f57513f5b00b3d766e34d3
parent 6c559b122add7db32b06faa15854df58b30460f6 (diff)
weld FontNameBox
with custom row rendering

https://cgit.freedesktop.org/libreoffice/core/commit/?id=2e0a32b51681fb356699b4a722f461f55a46b890
Comment 3 Telesto 2020-05-13 13:42:40 UTC
Confirmed in bug 132694 comment 10
Comment 4 Telesto 2020-05-13 13:43:21 UTC
Adding CC to: Caolán McNamara
Comment 5 Xisco Faulí 2020-05-13 14:29:12 UTC
In

Version: 7.0.0.0.alpha1+
Build ID: 1ffe59ef31186e36ad0aa7bbcdd32e407ee8d26c
CPU threads: 4; OS: Linux 4.19; UI render: default; VCL: gtk3; 
Locale: en-US (en_US.UTF-8); UI: en-US
Calc: threaded

after loading the file: 400
first reload: 408
second reload: 414
third reload: 423
fourth reload: 433

@Telesto, please share your measurements.
Comment 6 Telesto 2020-05-13 15:40:52 UTC
Created attachment 160762 [details]
Screencast
Comment 7 Telesto 2020-05-13 15:43:32 UTC
Starting with 300 after 5 or 6 reloads 600 MB. See also bug 132694 comment 7 and 10 (for Linux variant bisected by Buovjaga)
Comment 8 Telesto 2020-05-13 16:56:54 UTC
Created attachment 160768 [details]
Reduced Example file
Comment 9 Buovjaga 2020-05-13 18:40:39 UTC
(In reply to Xisco Faulí from comment #5)
> @Telesto, please share your measurements.

My tests indicate a definite connection with the "weld FontNameBox". This affects Writer GUI in general, it is not tied to any single specific file.

With View - Toolbars - Formatting visible, htop shows Resident memory size growing to 1564M.

Without the toolbar the memory is at 359M.

I have a fair amount of fonts installed, mainly thanks to ttf-google-fonts package on Arch. It looks like the font name box in the Formatting toolbar is loading some information about all of them into memory.
Comment 10 Caolán McNamara 2020-05-14 14:05:15 UTC
hmm, the idea was to prerender the font previews so they were ready to show when needed. It looks like that cache itself is ok, but maybe the fonts used to render are entered into some other cache or leak and memory use ends up silly
Comment 11 Commit Notification 2020-05-19 08:24:22 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/68cd0a974c83fbcbda809a0005e5d599a6e4909f

Related: tdf#132536 limit to ~25 pre-rendered font previews for now

It will be available in 7.0.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 12 Buovjaga 2020-05-19 17:35:46 UTC
(In reply to Commit Notification from comment #11)
> Caolán McNamara committed a patch related to this issue.
> It has been pushed to "master":
> 
> https://git.libreoffice.org/core/commit/
> 68cd0a974c83fbcbda809a0005e5d599a6e4909f
> 
> Related: tdf#132536 limit to ~25 pre-rendered font previews for now

Seems to be working as intended, thanks!

Arch Linux 64-bit
Version: 7.0.0.0.alpha1+
Build ID: 809ddff82dc9a28051d8f6b0d6513b1824ba0ab9
CPU threads: 8; OS: Linux 5.6; UI render: default; VCL: kf5; 
Locale: fi-FI (fi_FI.UTF-8); UI: en-US
Calc: threaded
Built on 19 May 2020
Comment 13 Commit Notification 2020-05-22 11:29:49 UTC
Caolán McNamara committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/f73980ed151e3d04b02cc463cf9fd6432f14ba03

Related: tdf#132536 drop FreetypeManager FreetypeFont caching

It will be available in 7.0.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 14 Telesto 2020-05-24 18:01:06 UTC
Only status update.. (used different steps to reproduce)
1. Open LibreOffice start center
2. Open Writer
3. Close writer (back to start center)
4. Open Writer again
5. Close writer again
6. Repeat 2/3 -> memory increases every time.. (started initially with the identified commit.. and is still present without any change)

Version: 7.0.0.0.alpha1+ (x64)
Build ID: b3c8859cd20bd02adea5d2b026001f67feacc754
CPU threads: 4; OS: Windows 6.3 Build 9600; UI render: Skia/Raster; VCL: win
Locale: nl-NL (nl_NL); UI: en-US
Calc: CL
Comment 15 Buovjaga 2020-05-24 18:07:30 UTC
(In reply to Telesto from comment #14)
> Only status update.. (used different steps to reproduce)
> 1. Open LibreOffice start center
> 2. Open Writer
> 3. Close writer (back to start center)
> 4. Open Writer again
> 5. Close writer again
> 6. Repeat 2/3 -> memory increases every time.. (started initially with the
> identified commit.. and is still present without any change)

As you are now using different steps, did you also check with a bibisect repository?
Comment 16 Telesto 2020-05-24 18:51:52 UTC
(In reply to Buovjaga from comment #15)
> (In reply to Telesto from comment #14)
> > Only status update.. (used different steps to reproduce)
> > 1. Open LibreOffice start center
> > 2. Open Writer
> > 3. Close writer (back to start center)
> > 4. Open Writer again
> > 5. Close writer again
> > 6. Repeat 2/3 -> memory increases every time.. (started initially with the
> > identified commit.. and is still present without any change)
> 
> As you are now using different steps, did you also check with a bibisect
> repository?

Yes, I bisected it.. Have another file.. where deleting images in Writer ends up in memory bumps.. also bisected to this commit.. not saying the original isn't working.. was intended as an addition.. Even to remind myself to look at it :-)
Comment 17 Telesto 2020-05-29 13:31:52 UTC
Is this still work in progress? I assume so, but there is an awful silence :-). Or can I start looking for possible issues?
Comment 18 Caolán McNamara 2020-05-29 14:29:14 UTC
My belief is that its fixed, but there's a counter claim that it isn't, so that'll have to be investigated when I have some time. Non obviously leaked memory use and performance issues are time intensive to look into and tend not to have a conclusive outcome acceptable to all.
Comment 19 Buovjaga 2020-05-29 14:54:34 UTC
(In reply to Telesto from comment #14)
> Only status update.. (used different steps to reproduce)
> 1. Open LibreOffice start center
> 2. Open Writer
> 3. Close writer (back to start center)
> 4. Open Writer again
> 5. Close writer again
> 6. Repeat 2/3 -> memory increases every time.. (started initially with the
> identified commit.. and is still present without any change)

I will just add that I do repro this.
Comment 20 Telesto 2020-06-11 07:53:42 UTC
Another 'push'. Prefer a solution before RC1. The fontcaching is a hairy matter. Developers tend to walk around it. Especially cross platform. And have seen enough to now developers struggle with it.. Kahled only touched the margins. Jmux has done some work..

The font management design is awfully bad, I assume. And nothing does exactly what it supposed to do or you might expect it to do.

Every change will cause regression. And the bad ones need to be ironed out. And the most problematic ones before 7.0, ideally

And there are more bugs about caching which might be related/ will be affected by any change. bug 133645 and bug 133646
Comment 21 dante19031999 2020-06-17 03:05:16 UTC
Changed earlist version to LO 6.4 (at least on linux).
There could be a new duplicate: bug 134026.
Comment 22 Telesto 2020-09-12 10:18:00 UTC
Bump for one of my beloved pet bugs :-)
Comment 23 Telesto 2020-09-29 06:01:44 UTC
@Jan Marek
There is a leak somewhere related to font handling at minimum on Windows. You're working on this area - fonts - slightly more often, any change to help Caolán on this? This is lingering around for a while already
Comment 24 Caolán McNamara 2020-09-29 08:28:28 UTC
Is there any concrete proof of this ?
Comment 25 Telesto 2020-09-29 08:48:06 UTC
Created attachment 165941 [details]
Screencast
Comment 26 Caolán McNamara 2020-09-29 08:57:27 UTC
But why so sure its font related ?
Comment 27 Telesto 2020-09-29 09:07:31 UTC
(In reply to Caolán McNamara from comment #26)
> But why so sure its font related ?
As everything started around comment 2

See:
https://bugs.documentfoundation.org/show_bug.cgi?id=134026#c4
https://bugs.documentfoundation.org/show_bug.cgi?id=131651#c17

-> Sorry for being so persistent :-)
Comment 28 Jan-Marek Glogowski 2020-09-29 23:11:46 UTC
If the Windows task manager can be trusted, there seems to be quite some leak, also when just opening a new document from the start center and closing it (going back to the start center); doesn't matter which LO application. My problem: DrMemory is crashing for me with Windows LO x64 and nothing free I tried could get me any usable result...

IMHO reverting that patch is also no option. And even if the patch is to blame, it just uncovered some bug on Windows, as it doesn't has any system specific parts, except for Gtk. And while I may see a little leak on Linux too, it's much less. Probably just some caching of some sort.
Comment 29 Commit Notification 2020-10-29 18:52:56 UTC
Jan-Marek Glogowski committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/d11e6edec3116f647fa085a297e29c677f5b8662

tdf#132536 free the cached VirtualDevices

It will be available in 7.1.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 30 Commit Notification 2020-10-29 20:51:30 UTC
Jan-Marek Glogowski committed a patch related to this issue.
It has been pushed to "libreoffice-7-0":

https://git.libreoffice.org/core/commit/25f8e4c43d8a083199077256ee6d7430d29d0ae3

tdf#132536 free the cached VirtualDevices

It will be available in 7.0.4.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 31 Xisco Faulí 2020-10-30 10:43:25 UTC
@Telesto, could you please verify the issue is fixed for your with a master build ?
Comment 32 Telesto 2020-10-30 10:46:23 UTC
(In reply to Xisco Faulí from comment #31)
> @Telesto, could you please verify the issue is fixed for your with a master
> build ?

There is no TB77 build for today, yet
Comment 33 Buovjaga 2020-10-30 13:07:38 UTC
(In reply to Telesto from comment #14)
> Only status update.. (used different steps to reproduce)
> 1. Open LibreOffice start center
> 2. Open Writer
> 3. Close writer (back to start center)
> 4. Open Writer again
> 5. Close writer again
> 6. Repeat 2/3 -> memory increases every time.. (started initially with the

I verify this is fixed

Arch Linux 64-bit
Version: 7.1.0.0.alpha1+
Build ID: 1a4ae360d06ae300a8fd5482b3b3a86dc021750d
CPU threads: 8; OS: Linux 5.9; UI render: default; VCL: kf5
Locale: fi-FI (fi_FI.UTF-8); UI: en-US
Calc: threaded
Built on 30 October 2020