Bug 99519 - Extremely high memory usage, sometimes crash when animated GIF is inserted
Summary: Extremely high memory usage, sometimes crash when animated GIF is inserted
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: graphics stack (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: Other All
: medium major
Assignee: Armin Le Grand (CIB)
QA Contact:
URL:
Whiteboard: target:5.3.0 target:5.2.0.1
Keywords: perf
Depends on:
Blocks:
 
Reported: 2016-04-26 19:39 UTC by turboboost17
Modified: 2016-10-25 18:54 UTC (History)
6 users (show)

See Also:
Crash report or crash signature:


Attachments
Large gif (6.25 MB, image/gif)
2016-05-21 06:57 UTC, turboboost17
Details
Error screen (16.87 KB, image/png)
2016-05-26 18:43 UTC, turboboost17
Details

Note You need to log in before you can comment on or make changes to this bug.
Description turboboost17 2016-04-26 19:39:48 UTC
When any GIF larger than a few MB is added to a presentation the program crashes.

This a new bug as of the last 2 years or so because it did not happen 2 summers ago.  

I've updated the last few versions and it has not been fixed so far.

FYI
Comment 1 raal 2016-04-26 20:41:26 UTC
Please attach GIF image which crashes Impress. Thank you
Comment 2 turboboost17 2016-05-21 06:57:39 UTC
Created attachment 125206 [details]
Large gif

Large motion gif that crashes Impress when added to Presentation with a thread error "Fatal Error", osl::Thread::create failed.

Gif's larger than 2MB may crash Impress.

Much greater than 1MB may freeze or lag especially during dual screen presentation.

This file works flawlessly in powerpoint 2016
Comment 3 turboboost17 2016-05-21 07:01:41 UTC
Large motion gif that crashes Impress when added to Presentation with a thread error "Fatal Error", osl::Thread::create failed.

Gif's larger than 2MB may crash Impress.

Much greater than 1MB may freeze or lag especially during dual screen presentation.

This file works flawlessly in powerpoint 2016
Comment 4 Buovjaga 2016-05-26 12:17:44 UTC
No repro.

Arch Linux 64-bit, KDE Plasma 5
Version: 5.3.0.0.alpha0+
Build ID: 60041cb237ea73c2c1885dd6afd99d88780c2dfc
CPU Threads: 8; OS Version: Linux 4.5; UI Render: default; 
Locale: fi-FI (fi_FI.UTF-8)
Built on May 26th 2016

64-bit, KDE Plasma 5
Build ID: 5.1.3.2 Arch Linux build-1
CPU Threads: 8; OS Version: Linux 4.5; UI Render: default; 
Locale: fi-FI (fi_FI.UTF-8)
Comment 5 turboboost17 2016-05-26 18:43:46 UTC
Created attachment 125301 [details]
Error screen

This is the error screen that appears when Impress fails
Comment 6 turboboost17 2016-05-26 18:45:35 UTC
Reproduced the error with different Win7 machine and Impress 5.0.0.5
Comment 7 Aron Budea 2016-05-27 00:11:16 UTC
I could reproduce this in LO 5.1.3.2/Windows 7, and with earlier ones as well, back to 3.3. Also with master build.

When the image is inserted, the memory consumed by soffice.bin rapidly starts to grow, on my machine the error screen attached in Comment 5 appears in less than 30 seconds at ~1.5 GB memory usage.
In the early LO versions the growing stops at ~1.3-1.4 GBs, the animation plays for a bit, and then disappears.

I'd say the high resource consumption with this particular gif is not a regression, and can also be observed in Writer => adjusting component to graphics stack.

Just noting that trying to reproduce it crashed my Ubuntu VM (with a modest 2 GB memory limit), does that count as reproducible? :)
Buovjaga, what's the resource usage in your system after inserting the image?
Comment 8 Buovjaga 2016-05-29 18:50:12 UTC
(In reply to Aron Budea from comment #7)
> Buovjaga, what's the resource usage in your system after inserting the image?

htop says 11.3% when the memory use climbing stops. I have 32GB of memory.

Arch Linux 64-bit, KDE Plasma 5
Version: 5.3.0.0.alpha0+
Build ID: ab0189433c1593c3c3ccf6a947aa7ba84e806d91
CPU Threads: 8; OS Version: Linux 4.5; UI Render: default; 
Locale: fi-FI (fi_FI.UTF-8)
Built on May 28th 2016
Comment 9 Aron Budea 2016-05-30 05:41:00 UTC
(In reply to Buovjaga from comment #8)
> htop says 11.3% when the memory use climbing stops. I have 32GB of memory.

11% of 32GB sounds like a lot to me... do you think this warrants Linux being included among the affected systems, with a reworked title? (something like: Extremely high memory usage, sometimes crash when animated GIF is inserted)
Comment 10 Caolán McNamara 2016-05-30 15:02:12 UTC
This .gif is an 8 bit 1920x1080 animated gif with 382 frames in it, and indeed on my debugging master LibreOffice when it is rendering "top" reports...

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
16063 caolan    20   0 4843764 3.580g 174440 S  30.3 11.4   0:27.83 soffice.bin

some debugging shows that in drawinglayer/source/primitive2d/graphicprimitivehelper2d.cxx when we have an animated gif we render each frame into a separate BitmapEx and create a BitmapPrimitive2D for each one and the size balloons once rendering starts and generates 382 of those.

With my proposed fix of https://gerrit.libreoffice.org/25671 to add a custom AnimationFrameBitmapPrimitive2D which shares a virtual device among the frames of a animated gif and moves the BitmapEx generation around to become on-demand I then get those numbers down to...

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
15438 caolan    20   0 1733572 569528 174276 R  82.2  1.7   0:24.75 soffice.bin
Comment 11 Armin Le Grand (CIB) 2016-06-03 13:13:14 UTC
Working on a solution to
- have stuff isdolated ina single primitive
- have a primitive directly specialized/derived from AnaimatedSwitchPrimitive
- have no shared VirtualDevices over multiple primitives
- have a central place to manage buffering/no buffering
Problem is that when not buffering (but creating frames on-demand) performance is bad - a single GIF uses 100% CPU time (debug, but bad). That is clear - with BitmapEx and two VirtualDevices involved too many graphic format conversions take place (CPU pixel OP's). That is exactly why the frames *are* completely prepared as BitmapEx frames/Primitives.
Target now is to
- always buffer 1st frame (Caolan already found out that it is requested out-of-order sometimes)
- decide to buffer or not based on animated GIF size
- have the future option to have ths as 'recreatable, flushable ressource'
Comment 12 Armin Le Grand (CIB) 2016-06-09 15:18:18 UTC
1st version on gerrit, added flag for very big AnimatedGIFs to handle special, commented in commit how to possibly do better in the future
Comment 13 Commit Notification 2016-06-10 13:39:55 UTC
Armin Le Grand committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=285744fef87f4ca0278834b97d7f618bdba5f4c0

tdf#99519 Added more intelligent handling of animated GIFs

It will be available in 5.3.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 14 Commit Notification 2016-06-10 14:01:59 UTC
Armin Le Grand committed a patch related to this issue.
It has been pushed to "libreoffice-5-2":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=702b58945b51e738cdcc3d383ca38881f9f1338d&h=libreoffice-5-2

tdf#99519 Added more intelligent handling of animated GIFs

It will be available in 5.2.0.1.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.