Bug 159529 - Excessive memory consumption in v24.2
Summary: Excessive memory consumption in v24.2
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
24.2.0.3 release
Hardware: x86-64 (AMD64) macOS (All)
: high major
Assignee: Patrick Luby (volunteer)
URL:
Whiteboard: target:24.8.0 target:24.2.1.2 target:...
Keywords: bibisected, bisected, regression
Depends on:
Blocks:
 
Reported: 2024-02-02 17:37 UTC by bintoro
Modified: 2024-04-11 17:20 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Snapshot of biggest leak's stack after opening an empty Writer document (716.49 KB, image/png)
2024-02-16 13:45 UTC, Patrick Luby (volunteer)
Details
Snapshot of biggest allocation's stack after opening an empty Writer document (779.81 KB, image/png)
2024-02-16 13:45 UTC, Patrick Luby (volunteer)
Details
Debug patch that deletes the hb_font immediately after use (1.58 KB, patch)
2024-02-19 22:05 UTC, Patrick Luby (volunteer)
Details
FontNameBox on macOS in Dark Mode (462.58 KB, image/png)
2024-02-21 00:29 UTC, Patrick Luby (volunteer)
Details
Instruments output after rejecting changes several times (1.80 MB, image/png)
2024-02-21 19:30 UTC, Patrick Luby (volunteer)
Details
Hacky debug patch to eliminate the Reject Change button false positive (1.69 KB, patch)
2024-02-22 01:01 UTC, Patrick Luby (volunteer)
Details
Instruments run after applying https://gerrit.libreoffice.org/c/core/+/163774 (1.28 MB, image/png)
2024-02-22 15:04 UTC, Patrick Luby (volunteer)
Details
Instruments running steps in comment #42 (1.17 MB, image/png)
2024-03-23 13:44 UTC, Patrick Luby (volunteer)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description bintoro 2024-02-02 17:37:52 UTC
Up until v7.6, I haven’t had noticeable issues with memory use. The typical memory consumption with my usage habits has been on the order of a few hundred MBs. Even at the extreme, it can’t have been much more than 1 GB, or I would have started paying attention.

With v24.2, after a few hours of light use, I noticed LO occupying 4.5 GB of memory, so I investigated a bit.

1. Simply launching LO and creating an empty Writer document brings memory usage up to 570 MB.

2. Interacting with modal dialogs (e.g., Edit Paragraph Style) continually increases memory consumption as I cycle through the tabs in the dialog. The memory is never released, even when I close the dialog. Very quickly I get at 1 GB.

3. Opening the font selector dropdown in the toolbar and scrolling down through the entire list causes memory usage to balloon from 1 GB to 2.25 GB.

I performed the same steps with Skia disabled. Steps 1 & 3 had the same outcome as before, but in step 2 the memory leak was no longer there. I downgraded back to v7.6.4 and could no longer reproduce any of the three.
Comment 1 Telesto 2024-02-02 18:25:42 UTC
I have noticed couple of reports about this in bug tracker, however the cause was a mystery until now..

Confirm
Version: 24.8.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: ba8f4bff6015013013df652efbfaf4d9ae10c881
CPU threads: 8; OS: macOS 14.3; UI render: Skia/Metal; VCL: osx
Locale: nl-NL (nl_NL.UTF-8); UI: en-US
Calc: threaded
Comment 2 Telesto 2024-02-10 19:10:26 UTC
Easy way to reproduce 
1. Open attachment https://bz.apache.org/ooo/attachment.cgi?id=12592
2. Edit -> Track Changes -> Manage -> Keep Pressing the reject button
Comment 3 Xisco Faulí 2024-02-15 12:18:36 UTC
Regression introduced by:

author	Khaled Hosny <khaled@libreoffice.org>	2023-07-16 07:37:55 +0300
committer	خالد حسني <khaled@libreoffice.org>	2023-07-23 06:00:30 +0200
commit dfe42e756b37642355356a6035081f4473609832 (patch)
tree 77a42535ddef1f76802fa4221f33fd97755bb2ff
parent 32144b5dc617d5a375eeadf25b5b62befb529785 (diff)
vcl: Use HarfBuzz to get glyph bounding rectangle

Bisected with: mac64-24.2
Comment 4 Caolán McNamara 2024-02-15 15:45:24 UTC
Ideally we could get a massif profile to get a better view of what is eating the memory there, but I guess that's challenging on macOS.

It is quite a small patch really, hb_font_get_glyph_extents. I wonder if its a leak, or if there happens to be some specific (maybe emoji or color) font commonly found on macOS which triggers excessive memory use and all platforms would have the same problem if they happened to have the same fonts.

In a pinch, in the absence of a better idea, it doesn't look too hard to restore a ImplGetGlyphBoundRect with the default impl as the preferred harfbuzz approach and the macOS one different, if that is the root problem
Comment 5 Telesto 2024-02-15 19:20:12 UTC
@Patrick
(In reply to Caolán McNamara from comment #4)
> Ideally we could get a massif profile to get a better view of what is eating
> the memory there, but I guess that's challenging on macOS.

Any advice here? Valgrind and macOS is not easy to setup (if it currently works at all). The Apple Instruments tools surely has heap/leak detection, but unsure if it matches Valgrind kind of profiling
Comment 6 Patrick Luby (volunteer) 2024-02-15 19:34:52 UTC
(In reply to Telesto from comment #5)
> @Patrick
> (In reply to Caolán McNamara from comment #4)
> > Ideally we could get a massif profile to get a better view of what is eating
> > the memory there, but I guess that's challenging on macOS.
> 
> Any advice here? Valgrind and macOS is not easy to setup (if it currently
> works at all). The Apple Instruments tools surely has heap/leak detection,
> but unsure if it matches Valgrind kind of profiling

The nightly master Mac Silicon builds have the necessary settings for Xcode's Instruments appplication to perform leak analysis and other performance analysis. My shortcut to open Instruments from the Terminal since it is hidden in Xcode's application bundle:

open /Applications/Xcode.app/Contents/Applications/Instruments.app

IIRC, the Mac Intel nightly builds don't have the necessary settings due to cross-compiling. But, if you do a local Mac Intel build, Instruements should be able to attach to it.

Unfortunately, I am unable to look at this bug this week as I am buried in non-coding work this month. But when I get time, I see if Instruments shows anything.

I did run Intruments when I first saw this bug and when I started leak detection before opening a Calc document and then stopping before closing Calc, IIRC the "heaviest stack" was in Skia retaining CGFont instances.

Interestingly, if I didn't stop the leak detection until after I closed the Calc document, potential leaks dropped to only a few K of memory (likely global variables).

I am writing from memory so all this may not be accurate, but maybe one area to look at is if Skia is holding onto CGFonts longer than necessary after drawing text.
Comment 7 Patrick Luby (volunteer) 2024-02-15 22:52:16 UTC
(In reply to Patrick Luby from comment #6)
> IIRC, the Mac Intel nightly builds don't have the necessary settings due to
> cross-compiling. But, if you do a local Mac Intel build, Instruements should
> be able to attach to it. 

Correction: this no longer works in my Mac Silicon local master build anymore after I upgraded to Xcode 15.2. Does Apple now require an application bundle to be signed before you can add the "debug" entitlement? I don't know.

For now, Instruments can attach to the Mac Silicon nightly master builds since the TDF builds are done with a slightly older version of Xcode.
Comment 8 Commit Notification 2024-02-16 02:29:19 UTC
Patrick Luby committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/c2ebc77e636e9cdefad6cef446a2f8acec1580f1

Related: tdf#159529 fix increasing failures when setting entitlements

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 9 Xisco Faulí 2024-02-16 11:16:35 UTC
I upgraded the harfbuzz library to the latest 8.3.0 in e808e4643002aa6d2e749afca75fe923c8b2bf8e but the issue is still happening with that version
Comment 10 Patrick Luby (volunteer) 2024-02-16 12:56:41 UTC
(In reply to Xisco Faulí from comment #9)
> I upgraded the harfbuzz library to the latest 8.3.0 in
> e808e4643002aa6d2e749afca75fe923c8b2bf8e but the issue is still happening
> with that version

OK. I think I have fixed the build failure that was causing Instruments to not be able to attach to a local LibreOffice build. It didn't make it into last nights nightly build, but maybe tomorrow's (17 February 2024) nightly builds has the fix. I am hoping that the fix will also enable using Instruments on Mac Intel.
Comment 11 Patrick Luby (volunteer) 2024-02-16 13:45:17 UTC
Created attachment 192604 [details]
Snapshot of biggest leak's stack after opening an empty Writer document
Comment 12 Patrick Luby (volunteer) 2024-02-16 13:45:58 UTC
Created attachment 192605 [details]
Snapshot of biggest allocation's stack after opening an empty Writer document
Comment 13 Patrick Luby (volunteer) 2024-02-16 13:55:23 UTC
I did a quick memory leak profile of my local master build. I launched LibreOffice and then attached Instruments after the Start Center appears. Open an empty Writer document and then stop the profiling.

Interestingly, Instruments only identifies less than 30K of leaks (see https://bugs.documentfoundation.org/attachment.cgi?id=192604). So, it appears to me that Intruments cannot identify excessive caching.

Instruments does show that HarfBuzz allocates 180 MB of CoreTextFontFace instances (see https://bugs.documentfoundation.org/attachment.cgi?id=192605) when fetching glyph bounding rectangles. Maybe those CoreTextFontFace fonts are getting cached somewhere by either LibreOffice or HarfBuzz?

Version: 24.8.0.0.alpha0+ (AARCH64) / LibreOffice Community
Build ID: f846efa507252b0584d2753a251f2dd99c34541a
CPU threads: 8; OS: macOS 14.2.1; UI render: Skia/Metal; VCL: osx
Locale: en-US (en_US.UTF-8); UI: en-US
Calc: threaded
Comment 14 Patrick Luby (volunteer) 2024-02-18 16:44:58 UTC
(In reply to Patrick Luby from comment #10)
> OK. I think I have fixed the build failure that was causing Instruments to
> not be able to attach to a local LibreOffice build. It didn't make it into
> last nights nightly build, but maybe tomorrow's (17 February 2024) nightly
> builds has the fix. I am hoping that the fix will also enable using
> Instruments on Mac Intel.

FYI. The latest macOS nightly master builds (both Silicon and Intel builds) now have the proper entitlement for profiling. So, anyone with Xcode installed should (crossing my fingers) be able to attach Instruments to a LibreOffice nightly master build.
Comment 15 Patrick Luby (volunteer) 2024-02-19 19:25:42 UTC
So I now know what is going on. This looks like a Collabora/LibreOffice Online optimization in the FontNameBox. Sometime in the past LibreOffice decided to cache a bitmap for every font available. But to fit the font name into a standard size, there is a loop in shrinkFontToFit() in svtools/source/control/ctrlbox.cxx that fetches the glyph bounding rectangle in each pass through the loop.

Fetching the glyph bounding rectangle causes HarfBuzz to copy one of the font's internal tables. The fetched table is a full copy in memory and it appears to not be deleted until the enclosing LibreOffice PhysicalFont is deteled.

Problem is that Apple bundles a number of fonts that cover all supported languages. Some of these fonts e.g. CJK fonts are very large and so creating the images for each font in the FontNameBox not only loads every installed font, it also basically makes a copy of each in memory.

That is why you see the 180 MB of allocations in attachment #192605 [details] in CoreTextFontFace::GetHbTable(). Also, since the array of PhysicalFont instances that represent all installed fonts don't get deleted because the FontNameBox has a reference to them, Instruments doesn't see a leak.

So this bug really is case of excessive caching. I don't know anything about HarfBuzz or where this change occurred so I haven't been able to figure about where the copy can be deleted. But clearly making a copy of every installed font file seems uses a very, very large amount of memory. So, I think we should stop caching the font table in a PhysicalFont and delete the font table immediately after use.

Anyone know what commit this change was made in?
Comment 16 Patrick Luby (volunteer) 2024-02-19 22:05:21 UTC
Created attachment 192647 [details]
Debug patch that deletes the hb_font immediately after use
Comment 17 Patrick Luby (volunteer) 2024-02-19 22:15:09 UTC
(In reply to Patrick Luby from comment #16)
> Created attachment 192647 [details]
> Debug patch that deletes the hb_font immediately after use

I think that I have eliminated LibreOffice caching the font table data.

Working from the top of the stack in attachment #192605 [details], the lowest point that before we call HarfBuzz is in LogicalFontInstance::GetGlyphBoundRect(). So, I created the debug patch in attachment #192647 [details] which destroys the LogicalFontInstance's cached hb_font immediately after calling hb_font_get_glyph_extents().

No change with the debug patch which means the font table data isn't being cached in LogicalFontInstance.

Next step is to look in the HarfBuzz portion of the stack and see if there is any caching of font table data. hb_face_reference_table() looks like a good place to start.
Comment 18 Patrick Luby (volunteer) 2024-02-20 13:26:04 UTC
(In reply to Patrick Luby from comment #17)
> Next step is to look in the HarfBuzz portion of the stack and see if there
> is any caching of font table data. hb_face_reference_table() looks like a
> good place to start.

Bad news. It appears to me that the copied font table data is cached by hb_face_t instances. Don't yet know exactly how we can remove the data after it is cached.

The font tables that use a massive amount of memory is the 'sbix' font table. I assume the 'sbix' font table holds all of the bitmaps for emoji fonts.

I see the following memory usage for the 'glyf' and 'sbix' font tables for all fonts loaded by the FontNameBox. The 'sbix' tables are more than 70% of the total for all font tables:

Total font tables: 260887001
 glyf font tables: 64345349
 sbix font tables: 189145592

Note: I used the following debug patch to get the above data:

diff --git a/vcl/quartz/CoreTextFont.cxx b/vcl/quartz/CoreTextFont.cxx
index 6248a255c64e..8a29ae846f45 100644
--- a/vcl/quartz/CoreTextFont.cxx
+++ b/vcl/quartz/CoreTextFont.cxx
@@ -216,6 +216,24 @@ hb_blob_t* CoreTextFontFace::GetHbTable(hb_tag_t nTag) const
         const CFIndex nLength = pData ? CFDataGetLength(pData) : 0;
         if (nLength > 0)
         {
+            static uint32 nAllLength = 0;
+            static uint32 nGlyfLength = 0;
+            static uint32 nSbixLength = 0;
+
+            char aCode[5];
+            uint32_t nSwappedTag = CFSwapInt32(nTag);
+            memcpy(aCode, &nSwappedTag, 4);
+            aCode[4] = '\0';
+
+            nAllLength += nLength;
+            if (!strncmp("glyf", aCode, 4))
+                nGlyfLength += nLength;
+            else if (!strncmp("sbix", aCode, 4))
+                nSbixLength += nLength;
+            fprintf(stderr, "Total font tables: %u\n", nAllLength);
+            fprintf(stderr, " glyf font tables: %u\n", nGlyfLength);
+            fprintf(stderr, " sbix font tables: %u\n", nSbixLength);
+
             auto pBuffer = new UInt8[nLength];
             const CFRange aRange = CFRangeMake(0, nLength);
             CFDataGetBytes(pData, aRange, pBuffer);
Comment 19 Patrick Luby (volunteer) 2024-02-20 15:05:52 UTC
OK. I found where the huge 'sbix' font table is being cached: hb_face_t->sbix in HarfBuzz's src/OT/Color/sbix/sbix.hh file. Note: hb_face_t->sbix is an instance of accelerator_t held by a hb_face_lazy_loader:

    accelerator_t (hb_face_t *face)
    {
      table = hb_sanitize_context_t ().reference_table<sbix> (face);
      num_glyphs = face->get_num_glyphs ();
    }
    ~accelerator_t () { table.destroy (); }

In theory, maybe the above class can be restructured to only fetch the 'sbix' font table when needed and immediately destroy it after use.

Anyone have any better ideas? Rewriting most of the HarfBuzz file sounds like a lot of work and may introduce a bunch of new bugs.
Comment 20 Noel Grandin 2024-02-20 16:57:53 UTC
Noting that it is worth logging an issue over at
   https://github.com/harfbuzz/harfbuzz
and asking for help (and linking back to this bug)

Behdad is pretty friendly and willing to help.

Could be something we could do on our side (maybe a harfbuzz setting), or maybe we are using the API wrong
Comment 21 Patrick Luby (volunteer) 2024-02-21 00:29:42 UTC
Created attachment 192674 [details]
FontNameBox on macOS in Dark Mode
Comment 22 Patrick Luby (volunteer) 2024-02-21 00:57:57 UTC
(In reply to Noel Grandin from comment #20)
> Could be something we could do on our side (maybe a harfbuzz setting), or
> maybe we are using the API wrong

OK. I've got a potential fix at https://gerrit.libreoffice.org/c/core/+/163674.

One (hopefully minor) inefficiency in the above patch is that if we copy the 'sbix' and 'glyf' font tables' memory every time LogicalFontInstance::GetGlyphBoundRect() is called. When running the Instruments application on macOS attached to my local build with the patch, I can see a very brief spike in memory usage when opening a new Writer document. Presumably that is the loading and releasing of the color emoji fonts' 'sbix' and 'glyf' tables.
Comment 23 Caolán McNamara 2024-02-21 08:43:21 UTC
(In reply to Patrick Luby from comment #15)
> So I now know what is going on. This looks like a Collabora/LibreOffice
> Online optimization in the FontNameBox. Sometime in the past LibreOffice
> decided to cache a bitmap for every font available.

That was me I think, but it wasn't specifically for online. Those previews are the  things shown in the font dropdown for the fontname and associated sample text IIRC.
Comment 24 Commit Notification 2024-02-21 09:54:24 UTC
Patrick Luby committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/af47a78e11d4819bb4b1ae1a1277babd10d815b7

tdf#159529 clear hb_ot_face_t data after fetching 'sbix' and 'glyf' tables

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 25 Patrick Luby (volunteer) 2024-02-21 13:43:08 UTC
(In reply to Caolán McNamara from comment #23)
> That was me I think, but it wasn't specifically for online. Those previews
> are the  things shown in the font dropdown for the fontname and associated
> sample text IIRC.

I was just grabbing at straws at that point. I had initially assumed that creating offscreen bitmaps was to send server font rendering over to the client browser.

Fortunately, walking through the preview code helped me find the real cause of the leak.
Comment 26 Patrick Luby (volunteer) 2024-02-21 13:44:59 UTC
I have a fix for this bug and the fix should be in tomorrow's (22 February 2024) nightly master builds:

https://dev-builds.libreoffice.org/daily/master/current.html

Note for testers: the nightly master builds install in /Applications/LibreOfficeDev.app. These builds are not codesigned like regular LibreOffice releases so you will need to execute the following Terminal command after installation but before you launch /Applications/LibreOfficeDev:

xattr -d com.apple.quarantine /Applications/LibreOfficeDev.app
Comment 27 Telesto 2024-02-21 19:08:06 UTC
@Patrick
(In reply to Telesto from comment #2)
> Easy way to reproduce 
> 1. Open attachment https://bz.apache.org/ooo/attachment.cgi?id=12592
> 2. Edit -> Track Changes -> Manage -> Keep Pressing the reject button

Should this be move to a fresh bug? It's still occurring based on Xisco's comment on https://gerrit.libreoffice.org/c/core/+/163674

It's surely nice that that overall memory usage being reduced, though :-)
Comment 28 Patrick Luby (volunteer) 2024-02-21 19:30:23 UTC
Created attachment 192696 [details]
Instruments output after rejecting changes several times
Comment 29 Patrick Luby (volunteer) 2024-02-21 19:34:21 UTC
(In reply to Telesto from comment #27)
> Should this be move to a fresh bug? It's still occurring based on Xisco's
> comment on https://gerrit.libreoffice.org/c/core/+/163674

Looks like it is a different memory leak.

I just attached Instruments to LibreOffice (note: anyone should be able to do this now with either the Mac Silicon or Intel nightly master builds) and Intruments shows a memory leak when drawing native buttons, etc. with Skia enabled (see attachment #192696 [details]).

I'll submit a patch for that today. Looks like we are just missing a delete[] for a temporary array.
Comment 30 Patrick Luby (volunteer) 2024-02-22 01:01:58 UTC
Created attachment 192700 [details]
Hacky debug patch to eliminate the Reject Change button false positive
Comment 31 Patrick Luby (volunteer) 2024-02-22 01:07:06 UTC
(In reply to Patrick Luby from comment #29)
> I'll submit a patch for that today. Looks like we are just missing a
> delete[] for a temporary array.

Nevermind. I think what I see is a false positive in Instruments. I have verified that delete[] is called on every matching new[], replaced new[] and delete[] with malloc() and free() and still Instruments shows a leak at least in Xcode 15.2.

I was able to eliminate the false positives using the hacky debug patch in attachment #192700 [details]. Nothing jumped out at me but when I have some time, I'll try the hacky debug patch and see what Instruments says is the next biggest leak.
Comment 32 Patrick Luby (volunteer) 2024-02-22 15:04:50 UTC
Created attachment 192716 [details]
Instruments run after applying https://gerrit.libreoffice.org/c/core/+/163774
Comment 33 Patrick Luby (volunteer) 2024-02-22 15:11:10 UTC
(In reply to Patrick Luby from comment #31)
> Nevermind. I think what I see is a false positive in Instruments. I have
> verified that delete[] is called on every matching new[], replaced new[] and
> delete[] with malloc() and free() and still Instruments shows a leak at
> least in Xcode 15.2.
> 
> I was able to eliminate the false positives using the hacky debug patch in
> attachment #192700 [details]. Nothing jumped out at me but when I have some
> time, I'll try the hacky debug patch and see what Instruments says is the
> next biggest leak.

Correction: the hacky debug patch in attachment #192700 [details] is for verifying that all of the new[] calls have a matching delete[] call.

The patch that stops Skia's "draw a native button, etc." macOS code from showing as a leak in Instruments is here:

https://gerrit.libreoffice.org/c/core/+/163774

With that patch, Instruments shows less than a 100K of leaks and most of that appears to be the CTFonts in LibreOffice static font list (see attachment #192716 [details]).
Comment 34 Commit Notification 2024-02-22 16:54:16 UTC
Patrick Luby committed a patch related to this issue.
It has been pushed to "libreoffice-24-2":

https://git.libreoffice.org/core/commit/618e977e7f6973bf36bf83c04f7802cb83f8064a

tdf#159529 clear hb_ot_face_t data after fetching 'sbix' and 'glyf' tables

It will be available in 24.2.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 35 Commit Notification 2024-02-22 18:51:34 UTC
Patrick Luby committed a patch related to this issue.
It has been pushed to "libreoffice-24-2-1":

https://git.libreoffice.org/core/commit/e0fd2489d5dd4b8b2bd41f6e42cfe64817a578da

tdf#159529 clear hb_ot_face_t data after fetching 'sbix' and 'glyf' tables

It will be available in 24.2.1.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 36 ⁨خالد حسني⁩ 2024-02-24 07:19:14 UTC
On Linux we mmap font files, so such change is probably unnecessary. But also, I don’t think HarfBuzz is supposed to be copying font tables under normal circumstances, or is it us who are copying the tables wgen getting them from the CTFont? I think CTFont does not have an API to get font data without copying them (CTFontCopyTable), but I may be mistaken.

Definitely bring it on https://github.com/harfbuzz/harfbuzz/discussions to get more meaningful input on HarfBuzz side of things.
Comment 37 Commit Notification 2024-02-24 14:14:41 UTC
Patrick Luby committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/3e2dde97bc15f156837d10bf4456deb81f7f554a

Related: tdf#159529 eliminate possible memory leak

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 38 bintoro 2024-03-01 16:42:36 UTC
Version: 24.2.1.2 (X86_64) / LibreOffice Community
Build ID: db4def46b0453cc22e2d0305797cf981b68ef5ac
CPU threads: 16; OS: macOS 14.3; UI render: Skia/Metal; VCL: osx
Locale: fi-FI (en_US.UTF-8); UI: en-US
Calc: threaded

1. Launching LO and creating an empty Writer document: 222 MB of memory used.

2. Interacting with the Paragraph Style dialog: memory leak still there.

3. Opening the font selector dropdown and scrolling to the end of the list: memory consumption goes up by 450 MB. Most of this seems to be accounted for by Noto CJK fonts. But overall I don’t have a lot of fonts installed.

So, while the situation has certainly improved, the issue is not completely resolved.
Comment 39 Patrick Luby (volunteer) 2024-03-02 15:02:40 UTC
(In reply to bintoro from comment #38)
> 2. Interacting with the Paragraph Style dialog: memory leak still there.

My patch for this leak is not included in LibreOffice 24.2.1. It is only available in the master nightly builds:

https://gerrit.libreoffice.org/c/core/+/163774

I am just a volunteer and, since I really can't figure out why the above patch is needed to stop the leaking I saw in the Instruments application, I am a bit nervous about backporting that patch into a production release before anyone has tested it.
Comment 40 Patrick Luby (volunteer) 2024-03-02 16:20:46 UTC
(In reply to Patrick Luby from comment #39)
> (In reply to bintoro from comment #38)
> > 2. Interacting with the Paragraph Style dialog: memory leak still there.
> 
> My patch for this leak is not included in LibreOffice 24.2.1. It is only
> available in the master nightly builds:
> 
> https://gerrit.libreoffice.org/c/core/+/163774

I forgot to add where you can get the master nightly builds:

https://dev-builds.libreoffice.org/daily/master/current.html

Note for testers: the nightly master builds install in /Applications/LibreOfficeDev.app. These builds are not codesigned like regular LibreOffice releases so you will need to execute the following Terminal command after installation but before you launch /Applications/LibreOfficeDev:

xattr -d com.apple.quarantine /Applications/LibreOfficeDev.app
Comment 41 Patrick Luby (volunteer) 2024-03-21 20:32:25 UTC
(In reply to Patrick Luby (volunteer) from comment #39)
> (In reply to bintoro from comment #38)
> > 2. Interacting with the Paragraph Style dialog: memory leak still there.
> 
> My patch for this leak is not included in LibreOffice 24.2.1. It is only
> available in the master nightly builds:
> 
> https://gerrit.libreoffice.org/c/core/+/163774
> 
> I am just a volunteer and, since I really can't figure out why the above
> patch is needed to stop the leaking I saw in the Instruments application, I
> am a bit nervous about backporting that patch into a production release
> before anyone has tested it.

@Telesto: I have not heard any feedback on the above fix for the memory leak you described in comment #2.

It is probably too late to get it into the upcoming LibreOffice 24.2.2 release, but if you can test my fix in the nightly master build, I can submit it for the LibreOffice 24.2.3 release.
Comment 42 Telesto 2024-03-23 09:22:13 UTC
Sorry for the delay..

Comment 38 is correct. The parts you addressed are fixed / improved. Ram usage on launch has come down massively.

However there a still issues indeed.

1. Open Writer
2. Format -> Paragraph area tab. Clicking on various tabs. From 335 to 600 mb ram
3. close the dialog. Still 600 mb

---

1. Open empty writer document 335 mb or so
2. Open https://bz.apache.org/ooo/attachment.cgi?id=12592 (600 mb)
3. Close the document -> still around 600 mb being used


These should probably go to fresh bug report. To many issues addressed in a single report already..  

Version: 24.8.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: b38974391e8d4bf0d450abfaa86bbccbe1022995
CPU threads: 8; OS: macOS 14.3; UI render: Skia/Metal; VCL: osx
Locale: nl-NL (nl_NL.UTF-8); UI: en-US
Calc: threaded
Comment 43 Patrick Luby (volunteer) 2024-03-23 13:36:36 UTC
(In reply to Telesto from comment #42)
> Comment 38 is correct. The parts you addressed are fixed / improved. Ram
> usage on launch has come down massively.

Thank you for confirming that my fix for the leak in comment #2 works. I have submitted it for inclusion in LibreOffice 24.2.3.
Comment 44 Patrick Luby (volunteer) 2024-03-23 13:44:11 UTC
Created attachment 193256 [details]
Instruments running steps in comment #42
Comment 45 Patrick Luby (volunteer) 2024-03-23 13:52:07 UTC
(In reply to Patrick Luby (volunteer) from comment #44)
> Created attachment 193256 [details]
> Instruments running steps in comment #42

I attached the Instruments application to my local build and Instruments doesn't report any leaks other than 30K for the cached list of fonts.

I also see in the Allocations graph at the top of Instruments that memory was around 340 MB after opening a new Writer document grew to a peak of around 550 MB in several short bursts as a clicked on every tab in the Format > Paragraph dialog and clicked on every subtab in the Area tab. But memory went back down to   307 MB after I closed the dialog and closed the Writer document without saving.

I would assume the Allocations tracking in Instruments is more precise than other tools like the Activity Monitor application?
Comment 46 Telesto 2024-03-23 16:45:18 UTC
(In reply to Patrick Luby (volunteer) from comment #45)
I will check it again with Instruments running to be sure. Speculative part: Maybe it's some difference with local build versus the Jenkins build bot.
Comment 47 Telesto 2024-03-25 19:03:21 UTC
(In reply to Patrick Luby (volunteer) from comment #45)
You're right. Memory usage appears to be fine when using Instruments Allocations profile. There is a big discrepancy between the results of Instruments compared to Activity Monitor, odd.
Comment 48 Patrick Luby (volunteer) 2024-03-25 21:41:06 UTC
(In reply to Telesto from comment #47)
> (In reply to Patrick Luby (volunteer) from comment #45)
> You're right. Memory usage appears to be fine when using Instruments
> Allocations profile. There is a big discrepancy between the results of
> Instruments compared to Activity Monitor, odd.

I believe the difference is likely "inactive memory". macOS does not immediately reclaim memory immediately when an application calls "free", "delete", or "release" (depending on the programming language). Instead, macOS has its own internal memory manager that decides if it should reclaim unused memory previous allocated to an application as it is faster for applications to reuse memory from a block that was previously allocated to the same application.

I am guessing that the Instruments application is either counting allocations and releases of memory directly or has access to the "actually in use" portion of an application's memory.

Although Activity Monitor includes unreclaimed, unused memory, it can still be a useful tool for detecting possible leaks. If you see memory steadily climb like in the case of comment #2 or see a really large jump in memory usage, then there *might* be a leak. Instruments can usually help identify if there is a leak or if the large memory usage is due to a temporary spike in memory usage that macOS has not reclaimed.
Comment 49 Commit Notification 2024-04-11 17:20:40 UTC
Patrick Luby committed a patch related to this issue.
It has been pushed to "libreoffice-24-2":

https://git.libreoffice.org/core/commit/dd769dd028dad7b67709e6c4268c595d1916a9e7

Related: tdf#159529 eliminate possible memory leak

It will be available in 24.2.3.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.