Bug 108037 - CRASH: Huge memory usage (2.800 MB in 64-bit), additionally crash (on 1.400 MB in 32-bit) when exporting 58-pages ODP to PDF
Summary: CRASH: Huge memory usage (2.800 MB in 64-bit), additionally crash (on 1.400 M...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Impress (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All Windows (All)
: medium critical
Assignee: Matt K
URL:
Whiteboard: target:24.8.0
Keywords:
: 108463 (view as bug list)
Depends on:
Blocks: PDF-Export Memory
  Show dependency treegraph
 
Reported: 2017-05-23 20:19 UTC by Telesto
Modified: 2024-02-07 09:32 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Example file (1.30 MB, application/vnd.oasis.opendocument.presentation)
2017-05-23 20:19 UTC, Telesto
Details
export in pdf jpeg90 300dpi in 6.1.0.3-x64 win10-64 ok (16.43 MB, application/pdf)
2018-08-25 12:40 UTC, paulystefan
Details
Valgrind trace (6.07 KB, application/x-bzip)
2019-06-09 11:42 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Telesto 2017-05-23 20:19:15 UTC
Description:
LibreOffice crashes while exporting to PDF

Steps to Reproduce:
1. Open the attached file (based on bug 104479#c26)
2. Export it to PDF with default settings (300 dpi 90% compression)

Actual Results:  
Crash while exporting

Expected Results:
No crash


Reproducible: Always

User Profile Reset: No

Additional Info:
Probleemhandtekening:
  Gebeurtenisnaam van probleem:	APPCRASH
  Naam van de toepassing:	soffice.bin
  Versie van toepassing:	5.4.0.0
  Tijdstempel van toepassing:	591e05e6
  Naam van foutmodule:	VCRUNTIME140.dll
  Versie van foutmodule:	14.0.24215.1
  Tijdstempel van foutmodule:	57bfd587
  Uitzonderingscode:	c0000005
  Uitzonderingsmarge:	0000d3f9


User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0
Comment 1 Telesto 2017-05-23 20:19:30 UTC
Created attachment 133491 [details]
Example file
Comment 2 Xisco Faulí 2017-05-23 21:22:11 UTC
I can reproduce the crash with a 'bad allocation' error in

Version: 5.5.0.0.alpha0+
Build ID: 7662b11cad6105d56fb9acc9c431c89d3b68dc89
CPU threads: 1; OS: Windows 6.1; UI render: default; 
TinderBox: Win-x86@39, Branch:master, Time: 2017-05-20_10:09:09
Locale: es-ES (es_ES); Calc: group

however it doesn't crash in with 

Version: 5.4.0.0.alpha1+
Build ID: 74d2e606fd3605fe0a585f596eaa215ae4e20d18
CPU Threads: 4; OS Version: Linux 4.8; UI Render: default; VCL: gtk2; 
Locale: en-US (ca_ES.UTF-8); Calc: group

Which OS and build are you using?
Comment 3 Telesto 2017-05-23 22:04:56 UTC
Crash occurred with
Versie: 5.4.0.0.beta1 
Build ID: 8672113ead4e403c55e31b1d9a3d1e0f3b299577
CPU-threads: 4; Besturingssysteem:Windows 6.2; UI-render: standaard; 
Locale: nl-NL (nl_NL); Calc: CL

also with (used closed without any warning)
Version: 5.5.0.0.alpha0+
Build ID: d57e6cd9dcc96112994ca2b14ac45896e86b26e5
CPU threads: 4; OS: Windows 6.19; UI render: default; 
TinderBox: Win-x86@42, Branch:master, Time: 2017-05-18_22:43:07
Locale: nl-NL (nl_NL); Calc: CL

and in
Version: 5.2.5.0.0+
Build ID: a4d4fbeb623013f6377b30711ceedb38ea4b49f8
CPU Threads: 4; OS Version: Windows 6.2; UI Render: GL; 
TinderBox: Win-x86@62-merge-TDF, Branch:libreoffice-5-2, Time: 2016-12-24_14:43:55
Locale: nl-NL (nl_NL); Calc: CL

also found in
Lib5.0.6.3

No crash with
Version: 5.0.0.5
Build ID: 1b1a90865e348b492231e1c451437d7a15bb262b
Locale: nl-NL (nl_NL)

The issue seems to be a memory leak while exporting to PDF combined with the fact that (recent?) WinX86 builds tend to crash when the memory usage increases over 1,2 GB mark (mostly with bad alloc) (this time with a app crash report for me) See also https://bugs.documentfoundation.org/show_bug.cgi?id=104479#c27
Comment 4 Aron Budea 2017-06-04 22:40:36 UTC
The memory usage is almost the same since 3.3.0. Since the fix would be to reduce memory usage to normal levels anyway, I'm removing regression-related keywords.
Comment 5 Telesto 2017-06-11 13:08:11 UTC
*** Bug 108463 has been marked as a duplicate of this bug. ***
Comment 6 paulystefan 2018-08-25 12:40:40 UTC
Created attachment 144426 [details]
export in pdf jpeg90 300dpi in 6.1.0.3-x64 win10-64 ok

export in pdf jpeg90 300dpi in 6.1.0.3-x64 win10-64 ok

solved for me
Comment 7 paulystefan 2018-08-25 13:03:08 UTC
in 6.0.6.2-x64 and win10-x64  memory about 2500 MB at the end of export
in 6.1.0.3-x64 and win10-x64  memory about 2800 MB at the end of export

you must have more than 4 GB better 8GB in memory.

the programm added side per side memory and needs about 40 or 50 MB per side.

this is a basic problem also in pdf import.


but crash is solved
Comment 8 Timur 2018-08-28 12:43:35 UTC
Repro 6.2+, both huge memory usage in 64-bit and bad allocation in 32-bit.
Comment 9 Julien Nabet 2019-06-09 11:42:54 UTC
Created attachment 152062 [details]
Valgrind trace

On pc Debian x86-64 with master sources updated today + enable-symbols (not enable-dbgutil), I retrieved a Valgrind trace.
Comment 10 Telesto 2020-01-03 22:05:19 UTC Comment hidden (me-too)
Comment 11 Telesto 2020-01-03 22:14:35 UTC Comment hidden (obsolete)
Comment 12 Xavier Van Wijmeersch 2020-01-04 19:51:19 UTC
I have 4.852 MB memory at the end of export with fresh build master

Version: 6.5.0.0.alpha0+
Build ID: b0a468d75ff96955e9e53027d35b248235cb68d0
CPU threads: 8; OS: Linux 5.4; UI render: default; VCL: gtk3; 
Locale: nl-BE (en_US.UTF-8); UI-Language: en-US
Calc: threaded
Comment 13 Telesto 2020-05-31 12:39:04 UTC
Same huge memory usage
Version: 7.1.0.0.alpha0+ (x64)
Build ID: 83c4f86f22dc37269ac6a038fe7de053c42aad6e
CPU threads: 4; OS: Windows 6.3 Build 9600; UI render: Skia/Raster; VCL: win
Locale: en-US (nl_NL); UI: en-US
Calc: CL
Comment 14 QA Administrators 2022-06-01 03:36:16 UTC Comment hidden (obsolete)
Comment 15 Matt K 2024-01-15 01:33:37 UTC
I'm able to repro the large memory usage on export, and it looks like the memory is not deallocated even after the PDF export is finished.  Closing the presentation file deallocates a large portion of the memory.

Version: 24.8.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: ef6ff2df2e1286974da2f344aa3b8e3ae9093a79
CPU threads: 16; OS: Windows 10.0 Build 22631; UI render: Skia/Vulkan; VCL: win
Locale: en-US (en_US); UI: en-US
Calc: CL threaded
Comment 16 Matt K 2024-01-31 02:28:55 UTC
A significant fix is available at: https://gerrit.libreoffice.org/c/core/+/162785.  Tested on x64 system and no crashes were observed.  Memory usage reduced from 2.8G to 1.9G and time to export reduced from ~1 minute to 7 seconds.  There may still be room for improvement in reducing memory usage.
Comment 17 Matt K 2024-02-02 01:14:08 UTC
(In reply to Matt K from comment #16)
It turns out that some unit tests fail for any try I've done to implement this fix.  I was able to pass the unit tests locally for the first version, so it seems like this bug is fixable, just not quite sure exactly how.
Comment 18 Commit Notification 2024-02-05 11:11:15 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/1ae7a81e542c9b072e519d0ea0d4773ed26ca251

tdf#108037 Reduce time and memory consumed exporting to PDF

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 19 Commit Notification 2024-02-05 14:41:50 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/ffb6133176d5bc1824b15893b2cf1a80aea6aa02

tdf#108037 speed up exporting large pdf (2)

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 20 Noel Grandin 2024-02-07 09:32:24 UTC
So I have reduced the memory usage of this dramatically.

The remaining problem, is, essentially, that this ODP file has very small graphics that are tiled across very large numbers of tiles.

In code terms, we have one or more FillGraphicPrimitive2D objects, which are being decomposed into a bazillion child objects.

A possible path to future improvements would be to teach 

(*) the VclMetafileProcessor2D code to convert this in a more efficient manner to meta-file actions i.e. instead of converting to a bazillon transform and fillgraphic objects, instead create a new MetaAction sub-class and push a single FillGraphicAction

(*) the vcl::PDFWriter::PlayMetafile code to convert the new FillGraphicAction object to a PDF fill/tile operation.

But I am not going to do that.