Bibisected with win 6.2 repo to https://git.libreoffice.org/core/+/aeff83240c88435d11590f5e9c6fe9927a508c6a%5E!/
sw: save more vcl layout calls in SwFntObj

Adding Cc: to Miklos Vajna

Seems to me this could just be working as intended as it is about using a cache. Miklos will enlighten us.

(In reply to Buovjaga from comment #1)
> Bibisected with win 6.2 repo to
> https://git.libreoffice.org/core/+/
> aeff83240c88435d11590f5e9c6fe9927a508c6a%5E!/
> sw: save more vcl layout calls in SwFntObj
> 
> Adding Cc: to Miklos Vajna
> 
> Seems to me this could just be working as intended as it is about using a
> cache. Miklos will enlighten us.

There is surely a by design part.. And I'm not objecting again some memory increase.. And it how useful/relevant the caching is depends on the different area's of LibreOffice.. Caching of the main document/ caching when using multi page view (separate?)/ Print preview/ Print/ PDF export 

However it's more the big picture: memory usage of PDF export is up from 88MB to 261 MB while exporting and the 261 MB isn't released after finishing for example (bug 133652). The layout cache is not the only cause; image handling has a part in it too (starting from increasing to memory cache in 4.4.7.2 (?) and 5.2?)  I'm not seeing any really noticeable benefit to it (except maybe preventing the slowdown by Harfbuzz)

So for the record, i'm complaining about this specific commit. And not saying it's should be called a "regression" in the technical developer sense, but it is a regression compared to the past. So, in advance.. feel free to remove the keyword (including see also's)

And yes, every memory bump can be explained by itself, but I don't like the pattern/trend. Tablets/Ipads don't have endless memory available. 

Anyhow. Purging the unused part of the cache for an active document once in a while wouldn't hurt (if possible,).. And not sure how the caching works.. PDF export seems to trigger (separate?) round of catching? Should it not be shared with main document or if not purged? And a purge on document close would be nice too

For the record: not knowing anything about how everything is build together or the inner workings of LibreOffice. I don't want to proof brokenness.. or point my finger to to an certain developer..

I'm surely aware of the need of saving vcl layout calls. And yes, this is like not something to fix straight away, I fear; so we end up at the madding $$$ topic

So where to shave, what to shave, how to shave some memory usage(cheese)off, .. I don't care.. But needs a cut, IMHO. [Awful expression of the Dutch figure of speech: 'de kaasschaafmethode gebruiken'. Essence: Cuts across the board]

(In reply to Telesto from comment #2)
> And yes, every memory bump can be explained by itself, but I don't like the
> pattern/trend. Tablets/Ipads don't have endless memory available. 

Keep in mind that for tablets and laptops, using more memory is better, because it uses much less power and thus requires much less battery compared to using CPU.

We have a speed vs memory usage trade-off here. As far as I see, the code we have today has a short-time cache, cleared at the end of each layout run. That is quite similar to what e.g. browsers do.

> So for the record, i'm complaining about this specific commit. And not saying it's should be called a "regression"

Let's call this an implementation error: if somebody is interested, we could look at if it's possible to use less memory while maintaining the same performance, I didn't investigate this direction so far.

(In reply to Miklos Vajna from comment #4)
> We have a speed vs memory usage trade-off here. As far as I see, the code we
> have today has a short-time cache, cleared at the end of each layout run.

Is possible explain this a bit more in detail, what the 'cache' does. What a layout run triggers. And when the layout run ends.. 

I would expect it for
-> File open
-> Export to PDF
-> Change to multi page view 
-> Going to PrintPreview
-> Export EPUB

Does this also happen when?
-> Saving odt/docx/doc?

Is the layout cache also used while editing a large document? New paragraphs added, pages move down etc.. I would expect lots of layout runs (but maybe i'm wrong). Or has this nothing to do with the cache. 

---
And more for me understand memory management a bit better. If more memory is needed new memory is allocated (on the heap or how is called). However the memory is never - properly - de-allocated (when it's not needed anymore).. at least that my impression. So PDF export means allocated memory to go up (more memory being allocated, fine). However it does not come down (de-allocated) after export is finished. Run the PDF export again.. memory usage stays the same (not up/or down)

Same holds for opening/closing documents. Opening a document, memory goes up, closing a document.. memory doesn't get fully de-allocated - it will drop with 75% or more.. but not all. Opening a new document.. and memory usage stays the same.

There are of course more variables at work.. loading of the dictionary/ fonts etc..). And maybe timer based flushes (at least Calc does this sometimes), but still have the impression something being off. This not a recent phenomena. Even seen in 4.3.7.2. However increased (it's or font cache handling (currently bug 132536) or layout caching or image handling (bumping the cache size in 5.2))/ changes in (for example) PDF export code)

However, sometimes I question.. is the cache size of 5.2 still there? After 6.1, image handling rework? I think assume it is.. And if so, is it still needed in the same way it was needed for 5.2. The image handling has improved with 6.1 and Skia buffers large images to some extend (instead of scaling it on every scroll; causing a bad performance). So cache can maybe reduced (now or if Skia is more 'common)? Question I don't have the answer to. Increasing the cache (for images) in 4.3/4.4 and in 5.2 was an easy/cheap work around for more complex underlaying issues. Maybe solved over the years and not needed anymore in the same extend.. [or I'm I breaking large GIF rendering with this proposal ;-)]. And/or I'm masking a more technical release image from memory issue with the new system build into 6.1. Or missing the point how it works; who knows.. 

I'm certainly aware this isn't easy to solve; it's even easy to track down.. or report.. with all those incremental changes over the last 5 years or so. I'm having a list of say 6-10 bugs which actually are one package.. the are tied and knotted together; and cumulative a problem, but separately not really so.. And appears in different shapes and figures.. Central topics around memory usage are IMHO: Image handling/font handling/ sw layout cache (but probably not a real surprise). So image handling and PDF export; Image handling and ODT save (a clear cache image call for ODT save has been removed in 4.4; with the assumption of being fixed by better function cache; not so). And maybe with some issues being actually inherited.. but always masked by the way it was used.. so maybe the image handling cache before 6.1 had some issues.. which never been detected with the image handling cache being set to 'low'. 

So, released my insights. Cleared my head. Hope being useful somehow

> Is possible explain this a bit more in detail, what the 'cache' does.

Sure. If you don't cache vcl-level text layout calls, then let's say you want to draw "foo bar" on the screen. First you measure the text if "foo bar" fits into the current line, or perhaps you need to break it down to "foo" and "bar" (for that, you need to find a break point which is earlier than the end of "bar"), and finally you want to draw the text. Without a cache, this means we lay out the text (turn an unicode string into a list of positioned glyphs) at least 3 times, sometimes even more.

The cache means that in case we already know the positioned glyph list of e.g. "foo", we don't call into harfbuzz to lay it out again.

>What a layout run triggers. And when the layout run ends.. 

If you change the document model in any way (like press a key), you need to (incrementally) update the layout, then the rendering can draw the layout. So the layout run happens every time anything changes in the document that affects rendering. Pressing a key, inserting a new paragraph, etc.

Export to an editable file format typically doesn't change the document (ODT, DOCX, etc), so should not trigger a layout run.

Dear Telesto,

To make sure we're focusing on the bugs that affect our users today, LibreOffice QA is asking bug reporters and confirmers to retest open, confirmed bugs which have not been touched for over a year.

There have been thousands of bug fixes and commits since anyone checked on this bug report. During that time, it's possible that the bug has been fixed, or the details of the problem have changed. We'd really appreciate your help in getting confirmation that the bug is still present.

If you have time, please do the following:

Test to see if the bug is still present with the latest version of LibreOffice from https://www.libreoffice.org/download/

If the bug is present, please leave a comment that includes the information from Help - About LibreOffice.
 
If the bug is NOT present, please set the bug's Status field to RESOLVED-WORKSFORME and leave a comment that includes the information from Help - About LibreOffice.

Please DO NOT

Update the version field
Reply via email (please reply directly on the bug tracker)
Set the bug's Status field to RESOLVED - FIXED (this status has a particular meaning that is not 
appropriate in this case)


If you want to do more to help you can test to see if your issue is a REGRESSION. To do so:
1. Download and install oldest version of LibreOffice (usually 3.3 unless your bug pertains to a feature added after 3.3) from https://downloadarchive.documentfoundation.org/libreoffice/old/

2. Test your bug
3. Leave a comment with your results.
4a. If the bug was present with 3.3 - set version to 'inherited from OOo';
4b. If the bug was not present in 3.3 - add 'regression' to keyword


Feel free to come ask questions or to say hello in our QA chat: https://web.libera.chat/?settings=#libreoffice-qa

Thank you for helping us make LibreOffice even better for everyone!

Warm Regards,
QA Team

MassPing-UntouchedBug