Bug 79721 - writer slow on shapes, locks up if huge amount of shapes (svg)
Summary: writer slow on shapes, locks up if huge amount of shapes (svg)
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: x86-64 (AMD64) All
: high major
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: filter:odt, perf
Depends on:
Blocks: SVG-Import Writer-Images
  Show dependency treegraph
 
Reported: 2014-06-06 11:50 UTC by wabik
Modified: 2024-01-24 10:05 UTC (History)
11 users (show)

See Also:
Crash report or crash signature:


Attachments
Database test with reports one good and one bad (214.72 KB, application/vnd.sun.xml.base)
2014-06-13 09:37 UTC, wabik
Details
AOO-generated 100 entries odt (218.36 KB, application/vnd.oasis.opendocument.text)
2014-07-19 07:09 UTC, Lionel Elie Mamane
Details
LO 3.5-generated 100 entries odt (218.30 KB, application/vnd.oasis.opendocument.formula)
2014-07-19 07:10 UTC, Lionel Elie Mamane
Details
backtrace of seeming infinite loop (15.40 KB, text/plain)
2014-07-22 12:11 UTC, Lionel Elie Mamane
Details
LO 4.2-generated 174 entries odt. LibreOffice seemingly in infinite loop (360.64 KB, application/vnd.oasis.opendocument.text)
2014-07-22 12:13 UTC, Lionel Elie Mamane
Details
perf flamegraph (344.09 KB, application/x-bzip)
2020-02-13 21:11 UTC, Julien Nabet
Details
Windows Performance Analyzer output (211.97 KB, image/png)
2024-01-18 02:22 UTC, Matt K
Details

Note You need to log in before you can comment on or make changes to this bug.
Description wabik 2014-06-06 11:50:52 UTC
The report with (finally) 5 pages is being generated in Writer but ... 
if the report is going to have more pages (like >10), it is being generated (progress bar is working) but no content is displayed in Writer :( it stops with no error ... just blank report screen (either no data nor graphical objects)

Still some problems with reports
Comment 1 Alex Thurgood 2014-06-06 12:25:50 UTC
wabik : can you provide a test case setup for us please ?
Comment 2 Alex Thurgood 2014-06-06 12:27:54 UTC
wabik:

Are you using the included ORB (bundled extension) or a separately installed extension ?

Which version of Java ?
Comment 3 wabik 2014-06-06 12:33:33 UTC
Alex, I will try to prepare an example next week if I find some time ....
ORB already provided inside LO and version of JRE I use is 1.7.0_55 ...
Comment 4 Alex Thurgood 2014-06-06 12:48:00 UTC
wabik: ok thanks
Comment 5 wabik 2014-06-13 09:35:06 UTC
OK ... It is not about number of pages generated!
What I tested is attached already to the bug.

The test case is (Raport 1):
1. Table with 3500 (or so) entries.
2. Generated report which consisted of only table entries --> everything ok 493 pages

Second test case (Raport 2):
1. The same table ...
2. Report prepared with some shapes --> Generated ... but at the end stops and halts with no result :( FAILED

So it is number of pages or number of shapes generated inside the report which causes error?!
Comment 6 wabik 2014-06-13 09:37:32 UTC
Created attachment 100969 [details]
Database test with reports one good and one bad
Comment 7 wabik 2014-06-13 09:55:03 UTC
Just to say it was 
LiO: 4.3.0.0.beta2
Build ID: a06aa316117a6ff0f05c697c82831c227812d810
Comment 8 Robert Großkopf 2014-06-13 15:38:34 UTC
Have tried it with the attached database. I have set Raport2 to limit 100 for a test. Works, but has the CPU (1 core) runs at 100% for a longer time. Then tried it with limit 200 and get a grey background without content, while the CPU is running at 100% - have stopped this process after some minutes.

Then I tried to save the writer-file (limit 100). It needs nearly the same time as creating the file.

It is the same behavior with LO 4.1.6.2.

I don't know if we could say this is a Base-bug. I recognize also a problem to open the saved file. Note there are nearly 6000 shapes inside such a file and the whole report is created with frames in Writer. We could really say: The limit is reached for such a construction.
Comment 9 wabik 2014-06-16 05:47:16 UTC
Robert: OpenOffice 4.1 can handle that with no problem but I can agree it takes a long time. I did that to see the immediate result (hanging) as you said: a grey background without content.
Comment 10 Alex Thurgood 2014-06-18 11:58:07 UTC
Report 1 opens on Mac OSX 10.9.3 and LO 4242, but I see no shapes, no lines in the table (these appear to be invisible), only letters/numbers (which I assume are the field data). The report has 367 pages according to LO.

The Navigator shows no other objects in the document than 3 tables. For me this is a bug.

If I save the report as a copy (Save as copy...) and re-open the ODT document, I see the table markings, but still no shape objects.


Report 2 does not open on OSX 10.9.3 and LO 4242. It leaves the app in a strange condition, I can access the menus and other parts of the app, but the mouse cursor remains permanently on the hourglass icon, instead of the normal pointer.


Setting as new, and platform all, since OSX is also affected.
Comment 11 Alex Thurgood 2014-06-18 11:59:50 UTC
Adding Lionel to CC, but methinks this is more of a Writer shape import/draw/rendering problem than a database one
Comment 12 Alex Thurgood 2014-06-18 12:00:42 UTC
Altered title to reflect findings
Comment 13 wabik 2014-06-18 12:10:56 UTC
Alex: You cannot see shapes in Raport 1 because there are no shapes :)
It is just to visualise the problem with the shapes :(
Comment 14 Alex Thurgood 2014-06-18 14:11:25 UTC
wabik : Attempting to open Raport 2 in LO 4132 causes LO to continuously consume memory until the application is deemed "not responding" on OSX and the spinning beachball status is attained.
Comment 15 Alex Thurgood 2014-06-18 14:12:42 UTC
Confirming on OSX.
Changing OS to ALL.
Comment 16 wabik 2014-07-17 11:28:04 UTC
Bumping ... Anything on that bug?
For now Base reporting is useless at least for me :(
Comment 17 Lionel Elie Mamane 2014-07-19 07:07:57 UTC
I confirm that the problem is the time that Writer takes to open the odt file; report builder takes a rather short time to generate it.

Here are my tests comparing LibreOffice with Apache OpenOffice:

 - AOO 4.1.0 opening AOO-generated 100 entries odt: 1min20
 - AOO 4.1.0 opening LO-generated 100 entries odt: 1min20
 - LO 3.5.4.2 opening LO-generated 100 entries odt: 2min25
 - LO 3.5.4.2 opening AOO-generated 100 entries odt: 2min25
 - LO 4.2.5.2 opening LO-generated 100 entries odt: 1min35
 - LO 4.2.5.2 opening AOO-generated 100 entries odt: 1min35
Comment 18 Lionel Elie Mamane 2014-07-19 07:09:12 UTC
Created attachment 103082 [details]
AOO-generated 100 entries odt
Comment 19 Lionel Elie Mamane 2014-07-19 07:10:38 UTC
Created attachment 103083 [details]
LO 3.5-generated 100 entries odt
Comment 20 Lionel Elie Mamane 2014-07-19 07:40:06 UTC
No significant difference with LO 4.2-generaged 100 entries odt.
Comment 21 wabik 2014-07-21 10:56:33 UTC
(In reply to comment #20)
> No significant difference with LO 4.2-generaged 100 entries odt.

Lionel,
It is not only slow but it does not do anything ... it just stops with no result.
Reports do not work with shapes :(
Comment 22 Lionel Elie Mamane 2014-07-21 11:20:19 UTC
(In reply to comment #21)
> (In reply to comment #20)
> > No significant difference with LO 4.2-generaged 100 entries odt.
> 
> Lionel,
> It is not only slow but it does not do anything ... it just stops with no
> result.
> Reports do not work with shapes :(

The tests of other people on smaller datasets suggest that it is "just slow", albeit I admit too slow to be usable in your particular usage scenario of a myriad shapes per entry). Since 100 entries take about 1:30 minutes on my machine, by extrapolation 3200 entries would take on the order of magnitude of one hour (assuming linear algorithms, which is not obvious... I'd bet we have at least some n*log(n) algorithms, and possibly some quadratic or worse...).

I launched a test on the full dataset, for me LibreOffice 4.2.5.2 is "busy", but not locked up: I can open a "new" Raport 1, I can open a new document, I can open the table, etc. I haven't waited the full hour yet to see if something eventually comes out of it.

Are you *sure* your LibreOffice is completely stopped, and not just slower than your patience?
Comment 23 wabik 2014-07-22 11:06:58 UTC
(In reply to comment #22)
> (In reply to comment #21)
> > (In reply to comment #20)
[CUT]
> 
> Are you *sure* your LibreOffice is completely stopped, and not just slower
> than your patience?

Lionel, 

My patience is done ... after 6 hours of waiting Report 2 is not done and whole LO is blocked ... memory in windows process has reached 256 MB so maximum that can be set in LO Options/Memory and whole LO was blocked ... Nothing can be done except killing LO process.

I am using Windows version of LO 4.3.0.3 on Windows 7 64-bit.

Something more is broken ... As I suppose ... But maybe I am wrong ...

Can you test Report 2? :)
Comment 24 Lionel Elie Mamane 2014-07-22 11:53:55 UTC
(In reply to comment #23)

>> Are you *sure* your LibreOffice is completely stopped, and not just slower
>> than your patience?

> after 6 hours of waiting Report 2 is not done and
> whole LO is blocked ... memory in windows process has reached 256 MB so
> maximum that can be set in LO Options/Memory and whole LO was blocked ...

If you mean the "Graphics cache" setting, I don't think this is related per se. I have 50MB there, and my LibreOffice memory usage happily goes into gigabytes.

> Can you test Report 2? :)

I've tested report 2 (on a faster machine) limited to 100, 130, 150, 170 records; it gets ever slower as I add more records, but "in the end" LibreOffice wakes up. Getting to 174 records, indeed it seems to go in an infinite loop: 100%CPU utilisation for much longer than what is needed for 174 records...
Comment 25 Lionel Elie Mamane 2014-07-22 12:11:56 UTC
Created attachment 103277 [details]
backtrace of seeming infinite loop

Here's the backtrace after it seems to spin infinitely. It does not exit SwTabFrm::MakeAll for a long time; I assume that's where the ~infinite loop is.
Comment 26 Lionel Elie Mamane 2014-07-22 12:13:51 UTC
Created attachment 103278 [details]
LO 4.2-generated 174 entries odt. LibreOffice seemingly in infinite loop
Comment 27 wabik 2014-07-23 07:06:19 UTC
Lionel,

Can we do sth with the bug? :)

I can wait 1 min to get result but it should come finally :)

Brgds,
wabik
Comment 28 Lionel Elie Mamane 2014-07-23 07:26:58 UTC
(In reply to comment #27)

> Can we do sth with the bug? :)

Sure, I'd be glad to review and commit a patch that fixes it :) Else, I hope a Writer developer will pick it up and fix it.
Comment 29 wabik 2014-07-30 10:12:26 UTC Comment hidden (no-value)
Comment 30 wabik 2014-08-26 07:20:34 UTC Comment hidden (no-value)
Comment 31 wabik 2014-09-26 06:34:34 UTC Comment hidden (no-value)
Comment 32 wabik 2014-10-28 11:36:43 UTC Comment hidden (no-value)
Comment 33 Julien Nabet 2014-10-28 13:40:47 UTC
wabik: please, don't ping here without bringing new information (for example, you reproduce the problem or not with last LO version), it's quite useless.
Comment 34 Alex Thurgood 2015-01-03 17:39:37 UTC Comment hidden (no-value)
Comment 35 wabik 2015-02-11 10:44:56 UTC Comment hidden (no-value)
Comment 36 wabik 2015-02-11 10:46:40 UTC
@up ... Added to LO Base :)
Comment 37 QA Administrators 2016-02-21 08:37:57 UTC Comment hidden (obsolete)
Comment 38 Robert Großkopf 2016-02-21 09:47:57 UTC
Bug still exists with LO 5.1.1.1, OpenSUSE 42.1 Leap, 64bit rpm Linux.
Comment 39 Telesto 2016-12-28 11:36:27 UTC
File-opening and handling is slow, because of 174 SVG images (attachment 103278 [details])

Conforming with:
Version: 5.4.0.0.alpha0+
Build ID: d0288a482a3dc0f50f535565e4c66a95bb140942
CPU Threads: 4; OS Version: Windows 6.19; UI Render: default; 
TinderBox: Win-x86@42, Branch:master, Time: 2016-12-26_23:25:18
Locale: nl-NL (nl_NL); Calc: CL
Comment 40 karolbienkowski 2020-02-12 20:32:32 UTC
The bug still exists in 6.4

Version: 6.4.0.3
Build ID: 6.4.0-2
CPU threads: 4; OS:Linux 5.5; UI render:GL; VCL: kf5;
Comment 41 Julien Nabet 2020-02-13 21:11:32 UTC
Created attachment 157849 [details]
perf flamegraph

Here's a Flamegraph retrieved on pc Debian x86-64 with master sources updated today.
Comment 42 Stéphane Guillou (stragu) 2023-11-05 23:07:20 UTC
Still extremely slow at opening attachment 103278 [details] in recent trunk build:

Version: 24.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 31fb3045dabdb27d913712f3abcade315e3ea9bd
CPU threads: 8; OS: Linux 5.15; UI render: default; VCL: gtk3
Locale: en-AU (en_AU.UTF-8); UI: en-US
Calc: threaded

While 6.4 eventually displayed the contents (with a frozen UI, after more than a minute), 24.2 doesn't even get to displaying the contents (I stopped waiting after a few minutes).
Comment 43 Matt K 2024-01-18 02:22:32 UTC
Created attachment 192023 [details]
Windows Performance Analyzer output

I took an ETW trace on Windows and the attached output shows it doing ~80 secs worth of work doing a vector insert with the shapes.  I would think that inserting shapes would be a quick operation, but someone with more knowledge of the code may be able to tell why it's doing so much work there.

Here is the callstack:

swlo.dll!std::vector<SwRootFrame * ptr64,std::allocator<SwRootFrame * ptr64> >::insert
swlo.dll!SwDoc::GetUniqueShapeName
swlo.dll!SwFmDrawPage::add
swlo.dll!SwXShape::attach
swlo.dll!SwXText::insertTextContent
xolo.dll!XMLTextImportHelper::InsertTextContent
xolo.dll!XMLTextShapeImportHelper::addShape
xolo.dll!SdXMLShapeContext::AddShape
xolo.dll!SdXMLShapeContext::AddShape
xolo.dll!SdXMLCustomShapeContext::startFastElement
xolo.dll!SvXMLImport::startFastElement
saxlo.dll!sax_fastparser::FastAttributeList::size
saxlo.dll!rtl::OUString::compareToAscii
saxlo.dll!sax_fastparser::FastSaxParserImpl::parseStream
saxlo.dll!sax_fastparser::FastSaxParser::parseStream
xolo.dll!SvXMLImport::parseStream
swlo.dll!XMLReader::Read
swlo.dll!XMLReader::Read
swlo.dll!XMLReader::Read
swlo.dll!SwReader::Read
swlo.dll!SwDocShell::Load
sfxlo.dll!SfxObjectShell::LoadOwnFormat
sfxlo.dll!SfxObjectShell::DoLoad
sfxlo.dll!SfxBaseModel::load
sfxlo.dll!rtl::ToStringHelper<rtl::StringConcat<char16_t,rtl::StringConcatMarker<char16_t>,char16_t const [40],0> >::length
fwklo.dll!framework::LoadEnv::impl_loadContent
fwklo.dll!framework::LoadEnv::start
fwklo.dll!framework::LoadEnv::startLoading
fwklo.dll!framework::LoadDispatcher::impl_dispatch
fwklo.dll!framework::LoadDispatcher::dispatchWithReturnValue
comphelper.dll!comphelper::SynchronousDispatch::dispatch
sfxlo.dll!SfxApplication::OpenDocExec_Impl
sfxlo.dll!SfxStubSfxApplicationOpenDocExec_Impl
sfxlo.dll!SfxDispatcher::Call_Impl
sfxlo.dll!SfxDispatcher::Execute_
sfxlo.dll!SfxDispatcher::Execute
sfxlo.dll!SfxApplication::OpenDocExec_Impl
sfxlo.dll!SfxStubSfxApplicationOpenDocExec_Impl
sfxlo.dll!SfxDispatcher::Call_Impl
sfxlo.dll!SfxDispatcher::PostMsgHandler
sfxlo.dll!std::invoke<void (cdecl SfxDispatcher::*& ptr64)(std::unique_ptr<SfxRequest,std::default_delete<SfxRequest> >) ptr64,SfxDispatcher * ptr64 & ptr64,std::unique_ptr<SfxRequest,std::default_delete<SfxRequest> > >
sfxlo.dll!std::_Invoker_ret<std::_Unforced>::_Call<void (cdecl SfxDispatcher::*& ptr64)(std::unique_ptr<SfxRequest,std::default_delete<SfxRequest> >) ptr64,SfxDispatcher * ptr64 & ptr64,std::unique_ptr<SfxRequest,std::default_delete<SfxRequest> > >
sfxlo.dll!std::_Call_binder<std::_Unforced,0,1,void (cdecl SfxDispatcher::*)(std::unique_ptr<SfxRequest,std::default_delete<SfxRequest> >) ptr64,std::tuple<SfxDispatcher * ptr64,std::_Ph<1> >,std::tuple<std::unique_ptr<SfxRequest,std::default_delete<SfxRequest> > && ptr64> >
sfxlo.dll!std::_Binder<std::_Unforced,void (cdecl SfxDispatcher::*)(std::unique_ptr<SfxRequest,std::default_delete<SfxRequest> >) ptr64,SfxDispatcher * ptr64,std::_Ph<1> const & ptr64>::operator()<std::unique_ptr<SfxRequest,std::default_delete<SfxRequest> > >
sfxlo.dll!std::invoke<std::_Binder<std::_Unforced,void (cdecl SfxDispatcher::*)(std::unique_ptr<SfxRequest,std::default_delete<SfxRequest> >) ptr64,SfxDispatcher * ptr64,std::_Ph<1> const & ptr64> & ptr64,std::unique_ptr<SfxRequest,std::default_delete<SfxRequest> > >
sfxlo.dll!std::_Func_impl_no_alloc<std::_Binder<std::_Unforced,void (cdecl SfxDispatcher::*)(std::unique_ptr<SfxRequest,std::default_delete<SfxRequest> >) ptr64,SfxDispatcher * ptr64,std::_Ph<1> const & ptr64>,void,std::unique_ptr<SfxRequest,std::default_delete<SfxRequest> > >::_Do_call
sfxlo.dll!std::_Func_class<void,std::unique_ptr<SfxRequest,std::default_delete<SfxRequest> > >::operator()
sfxlo.dll!SfxHintPoster::DoEvent_Impl
sfxlo.dll!SfxHintPoster::LinkStubDoEvent_Impl
vcllo.dll!Link<void * ptr64,void>::Call
vcllo.dll!ImplHandleResize
vcllo.dll!ImplWindowFrameProc
vcllo.dll!SalFrame::CallCallback
vclplug_winlo.dll!ImplHandleSalObjSysCharMsg
vclplug_winlo.dll!WinSalFrame::ResetClipRegion
vclplug_winlo.dll!SalFrameWndProcW
user32.dll!UserCallWinProcCheckWow
user32.dll!DispatchMessageWorker
vclplug_winlo.dll!WinSalInstance::ImplCreateDropTarget
vclplug_winlo.dll!ImplSalYield
vclplug_winlo.dll!WinSalInstance::DoYield
vcllo.dll!ImplSVAppData::ImplQuitMsg
vcllo.dll!Application::Yield
vcllo.dll!Application::Execute
sofficeapp.dll!desktop::Desktop::Main
vcllo.dll!ImplSVMain
vcllo.dll!SVMain
sofficeapp.dll!soffice_main
soffice.bin!?
soffice.bin!main
soffice.bin!__scrt_narrow_environment_policy::initialize_environment
soffice.bin!_alloca_probe
soffice.bin!_alloca_probe
soffice.bin!mainCRTStartup
kernel32.dll!BaseThreadInitThunk
ntdll.dll!RtlUserThreadStart
[Root]
Comment 44 CarlosGleech 2024-01-24 07:43:54 UTC Comment hidden (spam)
Comment 45 MaxWill 2024-01-24 10:05:59 UTC Comment hidden (spam)