Bug Hunting Session
Bug 47148 (Image-Caching) - [META] Image handling problems
Summary: [META] Image handling problems
Status: NEW
Alias: Image-Caching
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All All
: high critical
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
: 80352 (view as bug list)
Depends on: 42550 67583 92433 100442 119100 127670 41006 63433 67624 73270 73300 74435 80659 86649 98686 99612 104716 104882
Blocks: 78067 Images 33393 46447 52226 55312 82953
  Show dependency treegraph
 
Reported: 2012-03-09 04:49 UTC by Michael Meeks
Modified: 2019-09-20 14:29 UTC (History)
32 users (show)

See Also:
Crash report or crash signature:


Attachments
extension with simple macro for importing whole series of images in one run in Writer document (6.46 KB, application/vnd.openofficeorg.extension)
2014-07-11 12:47 UTC, Cor Nouws
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Meeks 2012-03-09 04:49:42 UTC
The code we've inherited that deals with image caching, swapping in, out, lifecycle management of images via strings, swapping in and out to documents etc. is broken beyond belief.

This is a tracker bug to start aggregating these horrors.
Comment 1 zeitlinie 2012-03-13 09:27:58 UTC
Please consider https://bugs.freedesktop.org/show_bug.cgi?id=46447
It damages presentations containing images irreversibly.
It happens for more than 80% of the presentation I do.
It has been around for me for all 3.x versions (and maybe earlier ones).
Comment 2 Michael Meeks 2012-03-13 10:26:22 UTC
Of course - any chance of some hunting through bugzilla to find any more instances of this sort of image-loss meta-problem ?
Comment 3 Björn Michaelsen 2012-04-06 18:08:15 UTC
Is this metabug really a 3.5 MAB? E.g. are we expecting to kill it on the release branch or is this work that touches too much code to be really safely backported?
Without deeper knowledge of the topic I would assume the second, and thus demote this from 3.5 MAB state.
Comment 4 Bernard SIAUD 2012-05-04 06:46:31 UTC
Now with LibO 3.5.3 I just lost a frame and an image embedded in the page.
Comment 5 Bernard SIAUD 2012-05-04 06:57:28 UTC
I have a Index entry in my frame (cadre in french) and I have note lost this entry. But I can't see this entry in this frame. I see this only in the table/index off entry.
Comment 6 Michael Meeks 2012-10-22 12:53:58 UTC
Another instance of this may be: https://issues.apache.org/ooo/show_bug.cgi?id=105191
Comment 7 Michael Meeks 2012-11-08 16:10:01 UTC
The many-megabyte leakage going on in bug#52433 are a particularly nice instance of this - where we have a leakage in place of a crash.
Comment 8 stfhell 2012-11-14 01:03:07 UTC
Bug 55312 seems to indicate that images end up as read errors when inserted before Writer has completed its document layout. Adding that to the blocks-list.
Comment 9 Michael Meeks 2013-06-13 20:02:37 UTC
The code at:
https://bugs.freedesktop.org/show_bug.cgi?id=33393#c37

Is almost certainly related to this.
Comment 10 Joel Madero 2013-08-01 16:17:41 UTC
*** Bug 67624 has been marked as a duplicate of this bug. ***
Comment 11 Michael Meeks 2013-09-27 22:51:22 UTC
Work is ongoing on this in the feature/graphicobject branch ... thanks to Norbert for some great progress there during the conference: it now builds ;-) [ still a v. long way to go ].
Comment 12 Norbert Thiebaud 2013-09-28 15:12:01 UTC
it now build _and_ seems to pass the basic subsequent checks...
Comment 13 Cor Nouws 2013-10-05 23:20:59 UTC
(In reply to comment #12)
> it now build _and_ seems to pass the basic subsequent checks...

Hmm :)

If there is any build (or way to build) that we can test, pls let it know!
thanks Norbert,
Cor
Comment 14 Norbert Thiebaud 2013-10-06 10:03:23 UTC
(In reply to comment #13)
> (In reply to comment #12)
> > it now build _and_ seems to pass the basic subsequent checks...
> 
> Hmm :)
> 
> If there is any build (or way to build) that we can test, pls let it know!
> thanks Norbert,
> Cor

Don't thanks me yet... make check still blow in few places...
and more importantly graphics seems not to be displayed at all at this point
so.... not quite there yet....
Comment 15 Michael Meeks 2014-03-18 15:45:26 UTC
I have a document for which I can reproduce the image loss under 4.1.x - though it takes a longish series of:

load .odt
save .odt
save .docx
re-load
save .odt
save .docx
reload (and the image is now lost)

Hopefully that will help us get to a better understanding here of the writer issues.
Comment 16 Michael Meeks 2014-04-24 16:01:59 UTC
The writer issue is/was supposedly fixed; and is only one of the issues. Long after that fix I'm still loosing images in impress - when loading and trying to edit together large slide decks. Nothing reproducible but ... horrible.
Comment 17 Cor Nouws 2014-04-25 10:39:10 UTC
(In reply to comment #16)
> The writer issue is/was supposedly fixed; and is only one of the issues.
> Long after that fix I'm still loosing images in impress - when loading and
> trying to edit together large slide decks. Nothing reproducible but ...
> horrible.

FWIW: someone contacted me with that very same problem.
I could do the job by:
 - fully loading all slides in both applications 
   (so in Normal/Edit view pgd through all the slides)
 - then in Sort view copy some 5, 6 slides per time and 
   paste in the other presentation
 - and save after each paste
So this sort of helps to ensure that all in memory copies of the presentations on screen have all the graphics ..
Comment 18 Michael Meeks 2014-04-25 16:21:55 UTC
Cor - it sounds like your instructions are to avoid loosing the images ;-) but we're badly trying to find a repeatable example / set of slides where we can loose them =) does anyone have any repeatable steps from a fresh exit/re-start of LibreOffice ? we tend to only see the aftermath when the suite has got into a busted state after some (ill defined) usage.
Comment 19 Michael Meeks 2014-04-29 08:46:52 UTC
More stupid stuff (from a performance perspective) from Impress - why does it take forever to save big slide-decks ? - it's -reading- and de-compressing all the images (but thankfully not re-compressing them all ;-).
Comment 20 Cor Nouws 2014-06-22 19:24:17 UTC
*** Bug 80352 has been marked as a duplicate of this bug. ***
Comment 21 Rupert Kolb 2014-07-11 10:10:59 UTC
This seems to be a time bomb.

see bug 73270.
I was hit by this when switching from 4.2.x to 4.3.0  (For the moment I switched back).

Not a good feeling to know that at your long documents (manuals for students, thesis, ...) suddenly some images might just vanish.
Comment 22 Michael Meeks 2014-07-11 10:19:30 UTC
Wow - thanks for the link. Apparently, from:

https://bugs.freedesktop.org/show_bug.cgi?id=73270#c10

We have a reproducer for a similar issue ! =)
Comment 23 Cor Nouws 2014-07-11 12:47:39 UTC
Created attachment 102617 [details]
extension with simple macro for importing whole series of images in one run in Writer document

The attached adds a simple macro library

1) Start Writer document; then the macro dialog (macro 'StartInsertPictures')
2) Choose a folder with a lot of images .... OK
  The pictures will be placed one after another in the document.  
  Might also stop working after 20, 30, ... images.  
  See what this triggers in traces, debugging etc

Good luck,
Cor
Comment 24 Cor Nouws 2014-07-11 13:14:23 UTC
Simple to reproduce case
 https://bugs.freedesktop.org/show_bug.cgi?id=52226#c41
Comment 25 Cor Nouws 2014-07-11 13:39:26 UTC
Useful comment on role of autosave
  https://bugs.freedesktop.org/show_bug.cgi?id=46447#c85

Comment with summary of some of the cases
  https://bugs.freedesktop.org/show_bug.cgi?id=52226#c31


I've seen the problem recently with a writer document and a presentation. Not too complicated, both losing one of the images while editing.
I've spend time and time to try to reproduce. Not..

As I've written somewhere else in one of the many occurrences of this problem:
Knowing that there are various in memory copies of a document, and in Impress maybe related to various views (side panel, normal view, slide sorter, ..) it may be well possible that there is a problem in synchronising those copies.
This is supported by some other comments too.

This will help to _not_ reproduce the problem ;) ... but alas neither is a guaranty that it won't happen, nor does it help as a reliable solution to do reproduce it..
Comment 26 Pierre C 2014-07-12 21:50:02 UTC
I've just been working on a odp existing file. Adding two pictures, working on the slide, and back to the pictures, they were gone !?

It happens many times, but I can't reproduce.

Problem starts with 4.2.5.1, and these evening with 4.2.6.1
Comment 27 tommy27 2014-07-23 12:22:16 UTC
@choffardet
that looks like Bug 46447 which is already in the "Depends on" of this meta-bug.
read the discussion over there.
Comment 28 Pierre C 2014-07-23 14:10:20 UTC
@tommy27, yes, I've just read the bug description but yes, that is the same behaviour. 
But I'm working with OpenOffice since 1.9 beta and I'm regularly working with impress, and more, I never link my images.
I've never see this bug since LO 4.2.5.1
Comment 29 V Stuart Foote 2014-08-20 17:30:19 UTC
linking a couple of the closed issues with commits to core/graphic/ndgrf.cxx and svdraw/svdograf.cxx, and svdpagv.cxx
Comment 30 Cor Nouws 2014-10-06 13:07:33 UTC
(In reply to Cor Nouws from comment #23)
> Created attachment 102617 [details]
> extension with simple macro for importing whole series of images in one run
> in Writer document

Adapted for use in Impress and Writer
https://bugs.freedesktop.org/show_bug.cgi?id=46447#c150
Comment 31 V Stuart Foote 2015-05-26 12:09:22 UTC
added a couple of relevant issues (Windows specific) writer and impress

Bug 82465 - Permanent crashes in writer with images
Bug 81378 - Permanent Crashes and Content Loss in Presentations ("Bad allocation" warning and crash)
Comment 32 Michael Meeks 2015-11-05 18:13:33 UTC
I believe I wrote up some E-mail rant about using strings to reference images across the code left & right, instead of UNO references - which is spectacularly unhelpful (but I lost my rant). An image is a leaf-node, there is no real reason not to have a nice hard, rtl::Reference<> lifecycle for whatever wrapper we put around it (to lifecycle manage it even when it's not swapped in etc.) that I can see. I suspect we try to be too clever - rather than having a nice central image cache we copy images to in the file-system, and that something simpler and more robust with some very hard lifecycle around it would be rather better ;-)
Comment 33 Felix Stadler 2016-06-08 11:53:30 UTC
Problem persists in Writer 5.1.1.3. on Windows. I just lost half the images from a 30 page document. They simply disappeared. This makes Writer unusable for creating documents with images.
Comment 34 mace 2016-09-05 08:58:05 UTC
Image handling is by far the biggest bummer in LO for me. I went over to write my stuff and add images only at the very end of the process. Image handling urgently needs some attention
Comment 35 Dimitri Bouron 2016-12-01 15:28:59 UTC
If someone reproduced the bug and built LO from source, could you please tell us the tag/commit ID and autogen flags?

I can't reproduce it on Fedora 24 x86_64 with LO 5.2.2.2 and these flags:
--enable-icecream
--with-parallelism=64
--with-gcc-home=/usr/libexec/icecc/
--enable-werror
--enable-debug
--with-jdk-home=/mnt/jvm-ramdisk/java-1.8.0-openjdk-1.8.0.111-1.b16.fc24.x86_64/
Comment 36 Paul Menzel 2017-08-07 12:16:45 UTC
(In reply to Dimitri Bouron from comment #35)
> If someone reproduced the bug and built LO from source, could you please
> tell us the tag/commit ID and autogen flags?
> 
> I can't reproduce it on Fedora 24 x86_64 with LO 5.2.2.2 and these flags:
> --enable-icecream
> --with-parallelism=64
> --with-gcc-home=/usr/libexec/icecc/
> --enable-werror
> --enable-debug
> --with-jdk-home=/mnt/jvm-ramdisk/java-1.8.0-openjdk-1.8.0.111-1.b16.fc24.
> x86_64/

Here is the configure line used for LibreOffice 5.4.0.3.

```
$ configure --prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --libexecdir=/usr/libexec --sysconfdir=/etc --sharedstatedir=/var --localstatedir=/var --libdir=/usr/lib --includedir=/usr/include --datarootdir=/usr/share --datadir=/usr/share --infodir=/usr/share/info --localedir=/usr/share/locale --mandir=/usr/share/man --docdir=/usr/share/doc/libreoffice --exec-prefix=/usr --with-lang=de --with-help --with-jdk-home=/usr/local/java --with-ant-home=/package/mariux64/ant --without-doxygen --without-junit --disable-dconf --disable-epm --disable-odk --enable-release-build --enable-python=internal --with-system-apr --with-system-cairo --with-system-curl --with-system-expat --with-system-harfbuzz --with-system-icu --with-system-jpeg --with-system-lcms2 --with-system-libatomic_ops --with-system-libpng --with-system-libxml --with-system-neon --with-system-nss --with-system-odbc --with-system-openldap --with-system-openssl --with-system-poppler --with-system-postgresql --with-system-serf --with-system-zlib
```
Comment 37 Michael Meeks 2018-01-30 09:17:49 UTC
Tomaz - best to CC you here if you're working on this =)
Comment 38 Yury 2018-06-16 18:50:41 UTC
'Images disappearing' might be connected with contents flow and wrapping management. Few days ago I had to reposition/resize images in my big doc (there weren't even that much of them). I touched something, and bang! 'half of the images were gone', just like in the cases guys here describe.
 
However they weren't actually removed from the doc. Some of them somehow got scaled to the height of several points, and some of them got that wrapping option switched on, which allows other content (images) to be placed 'behind' them (in this doc I have the images placed in pairs in page-sized frames).

Images which are anchored so that they 'contend' for the place on the same page, on which there's not enough place for both of them, may get placed/get their wrapping set weirdly.

Maybe #118200 has some clues?...
Comment 39 Michael Meeks 2018-06-16 19:15:46 UTC
Tomaz - seems to me that the image caching / management should no longer be utterly shambolic - just in need of various improvements =)

I'd like to rename this tracker bug, and it would be good to poke the open tickets that we fixed to say: Fixed in 6.1 - please re-test there, or somesuch - at least the ones that shouldn't be there anymore.

Can you do that in some spare cycles ? =) Thanks - Miklos - I think the load-time perf issue you fixed is also a child item here ...
Comment 40 Miklos Vajna 2018-06-25 07:57:05 UTC
There is bug 116280 already for the problems caused by the image handling rework. I've just added 2 problems (1 fixed, 1 in progress) to that tracker regarding performance.
Comment 41 Miklos Vajna 2018-08-13 07:50:44 UTC
Renaming, the majority of the problems are now solved with the work from Tomaz, see <https://blog.documentfoundation.org/blog/2018/06/19/image-handling-rework-for-libreoffice-collaboras-tender-results/>. Not closing yet, there are still a few problems remaining, see the unresolved dependent bugs.
Comment 42 Knut 2019-01-29 11:10:56 UTC
The problem is still there in Draw and Impress (Debian Stretch)

Version: 5.2.7.2
Build-ID: 1:5.2.7-1+deb9u4
CPU-Threads: 4; BS-Version: Linux 4.9; UI-Render: Standard; VCL: kde4; 
Gebietsschema: de-DE (de_DE.UTF-8); Calc: group
Comment 43 Knut 2019-01-29 11:39:58 UTC
Additional hint:
I was using a clipart (png) and did some  copy and paste of this clipart
Maybe this was affecting the missing images.

The images are embedded not linked. 
Not all of the copied images are lost (about 60% are lost)

The problem occoured during creating the presentation.