Bug 129101 - CTRL+A & Cut very slow (see comment 38)
Summary: CTRL+A & Cut very slow (see comment 38)
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
6.0.7.3 release
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: perf
Depends on:
Blocks: Image-Caching Memory
  Show dependency treegraph
 
Reported: 2019-11-29 15:38 UTC by Roland Hughes
Modified: 2021-10-02 17:57 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
LibreOffice image bug (116.39 KB, image/png)
2019-11-29 19:40 UTC, Roland Hughes
Details
book document for testing (18.45 MB, application/vnd.oasis.opendocument.text)
2020-01-06 18:25 UTC, Roland Hughes
Details
performance bug document (2.44 MB, application/vnd.oasis.opendocument.text)
2020-01-13 18:45 UTC, Roland Hughes
Details
perf flamegraph (145.38 KB, application/x-bzip)
2020-04-18 08:43 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Roland Hughes 2019-11-29 15:38:08 UTC
I have seen this on both Windows 10 and

Version: 6.0.7.3
Build ID: 1:6.0.7-0ubuntu0.18.04.10
CPU threads: 8; OS: Linux 5.0; UI render: default; VCL: gtk3; 
Locale: en-US (en_US.UTF-8); Calc: group


I'm currently writing "The Minimum You Need to Know About the Phallus of AGILE" which is tipping the scales around 600 pages. There are many images. I have tried just about every tweak I can find online for improving performance. Nothing works. 

When making a change high up in the document file, one which causes a page break type shuffle, something where LO has to move pictures around found later in the document file, machine drops to a crawl. It can take close to 10 minutes before LO responds to keyboard. The only work around is to exit LO and reopen the document.

My gut combined with 30 years of IT tells me this is a Java problem. 

Operating System: KDE neon 5.17
KDE Plasma Version: 5.17.3
KDE Frameworks Version: 5.64.0
Qt Version: 5.13.2
Kernel Version: 5.0.0-36-generic
OS Type: 64-bit
Processors: 8 × Intel® Core™ i7-6700 CPU @ 3.40GHz
Memory: 23.3 GiB of RAM

As you can see this is a 6th gen i7 having 24 Gig of physical RAM. When I pull up Kinfocenter, I see there is 18 Gig free while the document is desperately attempting to re-adjust pagination. Word processors using natively compiled 3GLs don't have this problem because they can consume as much RAM as they need. Java and other VM based interpreted languages are at the mercy of what the VM allows. Adding insult to injury garbage collection doesn't happen until the VM decides it is idle. It can't get idle when it is desperately trying to reallocate things.

The only "fix" I can see is to purge 100% of Java from the project. The VM has had over a decade and it still cannot grow to consume as much RAM as is available. It's like it is still restricted to a 32-bit address space.

roland@roland-HP-EliteDesk-800-G2-SFF:~$ java -XX:+PrintFlagsFinal -version | grep -iE 'HeapSize|PermSize|ThreadStackSize'
     intx CompilerThreadStackSize                  = 1024                                   {pd product} {default}
   size_t ErgoHeapSizeLimit                        = 0                                         {product} {default}
   size_t HeapSizePerGCThread                      = 43620760                                  {product} {default}
   size_t InitialHeapSize                          = 392167424                                 {product} {ergonomic}
   size_t LargePageHeapSizeThreshold               = 134217728                                 {product} {default}
   size_t MaxHeapSize                              = 6264193024                                {product} {ergonomic}
    uintx NonNMethodCodeHeapSize                   = 5835340                                {pd product} {ergonomic}
    uintx NonProfiledCodeHeapSize                  = 122911450                              {pd product} {ergonomic}
    uintx ProfiledCodeHeapSize                     = 122911450                              {pd product} {ergonomic}
     intx ThreadStackSize                          = 1024                                   {pd product} {default}
     intx VMThreadStackSize                        = 1024                                   {pd product} {default}
openjdk version "11.0.4" 2019-07-16
OpenJDK Runtime Environment (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3)
OpenJDK 64-Bit Server VM (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3, mixed mode, sharing)

I did find a pretty good post on determining Java settings.
https://www.mkyong.com/java/find-out-your-java-heap-memory-size/

It is where the above command came from.
Comment 1 Julien Nabet 2019-11-29 15:57:05 UTC Comment hidden (obsolete)
Comment 2 Roland Hughes 2019-11-29 18:06:14 UTC
(In reply to Julien Nabet from comment #1)
> I don't know which version you used in Win 10 but about LO 6.0.7 on Ubuntu
> is quite old.
> 6.0, 6.1 and 6.2 branches are EOL.
> Last stable LO version is 6.3.3
> Could you give a try with it?
> 
> About Ubuntu, you can find recent LO versions in PPA.

Windows 10 version I downloaded directly from the LO site just a few months ago. I got whatever was listed as "stable".

Technically I'm on KDE Neon distro which does still use some Ubuntu repos. There would need to be a PPA to keep things updated if the "maintainer" isn't maintaining the version in the LTS repos.
Comment 3 Julien Nabet 2019-11-29 18:33:17 UTC
(In reply to roland from comment #2)
> (In reply to Julien Nabet from comment #1)
> > I don't know which version you used in Win 10 but about LO 6.0.7 on Ubuntu
> > is quite old.
> > 6.0, 6.1 and 6.2 branches are EOL.
> > Last stable LO version is 6.3.3
> > Could you give a try with it?
> > 
> > About Ubuntu, you can find recent LO versions in PPA.
> 
> Windows 10 version I downloaded directly from the LO site just a few months
> ago. I got whatever was listed as "stable".
ok but before submitting a bug, you should give a try to last non dev version.
I know there are memory management pbs in LO but perhaps some have been already fixed.

> 
> Technically I'm on KDE Neon distro which does still use some Ubuntu repos.
> There would need to be a PPA to keep things updated if the "maintainer"
> isn't maintaining the version in the LTS repos.
In this case, it's not LO pb but distribution problem.

I put back UNCONFIRMED since I don't have more questions and it needs to be confirmed.

Does your book contain images/graphs or only text? If only text, the pb may be the spell or grammar checker. For the test, could you disable it?
Comment 4 Roland Hughes 2019-11-29 19:37:36 UTC
(In reply to Julien Nabet from comment #3)
> (In reply to roland from comment #2)
> > (In reply to Julien Nabet from comment #1)
> > > I don't know which version you used in Win 10 but about LO 6.0.7 on Ubuntu
> > > is quite old.
> > > 6.0, 6.1 and 6.2 branches are EOL.
> > > Last stable LO version is 6.3.3
> > > Could you give a try with it?
> > > 
> > > About Ubuntu, you can find recent LO versions in PPA.
> > 
> > Windows 10 version I downloaded directly from the LO site just a few months
> > ago. I got whatever was listed as "stable".
> ok but before submitting a bug, you should give a try to last non dev
> version.
> I know there are memory management pbs in LO but perhaps some have been
> already fixed.

I stick with LTS. I'm not about to trash a machine I'm using for a legitimate task just to satisfy curiosity.

> 
> > 
> > Technically I'm on KDE Neon distro which does still use some Ubuntu repos.
> > There would need to be a PPA to keep things updated if the "maintainer"
> > isn't maintaining the version in the LTS repos.
> In this case, it's not LO pb but distribution problem.
> 
> I put back UNCONFIRMED since I don't have more questions and it needs to be
> confirmed.
> 
> Does your book contain images/graphs or only text? If only text, the pb may
> be the spell or grammar checker. For the test, could you disable it?

As the bug report clearly stated.

<blockquote>I'm currently writing "The Minimum You Need to Know About the Phallus of AGILE" which is tipping the scales around 600 pages. <b>There are many images.</b></blockquote>

I went here.

https://launchpad.net/~libreoffice/+archive/ubuntu/ppa

Did this:
sudo add-apt-repository ppa:libreoffice/ppa
sudo pkcon update

rebooted

Opened a scrap copy of the file with this.
Version: 6.3.3.2
Build ID: 1:6.3.3-0ubuntu0.18.04.1~lo1
CPU threads: 8; OS: Linux 5.0; UI render: default; VCL: gtk3; 
Locale: en-US (en_US.UTF-8); UI-Language: en-US
Calc: threaded


I went to around page 14 and manually typed roughly 1.5 new pages of text, forcing 2 new page breaks, then I deleted a pre-existing page break.

While it no longer takes 10 minutes to re-shuffle, it's still hoping to one day get good. I would not call it fixed. This version introduced a rather annoying "feature" to an almost complete book. The main reason I didn't want to change this software. AGILE always introduces numerous new points of failure. Now, in exchange for a tiny speed increase, it looks like every image gets a margin inside the top of the frame which I cannot fix. See attachment.
Comment 5 Roland Hughes 2019-11-29 19:40:41 UTC
Created attachment 156189 [details]
LibreOffice image bug

Bug 6.3.3.2 introduced when I was told to install it to verify the speed bug due to massive memory problems. Previously images in frames with captions didn't have a forced top margin. Now there is no way to get rid of it.
Comment 6 Julien Nabet 2019-11-29 19:55:09 UTC
> ...
> As the bug report clearly stated.
> 
> <blockquote>I'm currently writing "The Minimum You Need to Know About the
> Phallus of AGILE" which is tipping the scales around 600 pages. <b>There are
> many images.</b></blockquote>
Indeed, sorry for this.

> ...
> While it no longer takes 10 minutes to re-shuffle, it's still hoping to one
> day get good. I would not call it fixed. This version introduced a rather
> annoying "feature" to an almost complete book. The main reason I didn't want
> to change this software. AGILE always introduces numerous new points of
> failure. Now, in exchange for a tiny speed increase, it looks like every
> image gets a margin inside the top of the frame which I cannot fix. See
> attachment.
Ok but that's another bug (perhaps it's been already declared, I don't know).
The main point here was memory management. Is it better with 6.3.3?
Rethinking about spell/grammar check, it can influence too even if it's not text only so the test could be relevant.
Comment 7 Roland Hughes 2019-11-29 20:05:12 UTC
(In reply to Julien Nabet from comment #6)
> > ...
> > As the bug report clearly stated.
> > 
> > <blockquote>I'm currently writing "The Minimum You Need to Know About the
> > Phallus of AGILE" which is tipping the scales around 600 pages. <b>There are
> > many images.</b></blockquote>
> Indeed, sorry for this.
> 
> > ...
> > While it no longer takes 10 minutes to re-shuffle, it's still hoping to one
> > day get good. I would not call it fixed. This version introduced a rather
> > annoying "feature" to an almost complete book. The main reason I didn't want
> > to change this software. AGILE always introduces numerous new points of
> > failure. Now, in exchange for a tiny speed increase, it looks like every
> > image gets a margin inside the top of the frame which I cannot fix. See
> > attachment.
> Ok but that's another bug (perhaps it's been already declared, I don't know).
> The main point here was memory management. Is it better with 6.3.3?
> Rethinking about spell/grammar check, it can influence too even if it's not
> text only so the test could be relevant.

It's not the spell checker. I wrote the bulk of the book before adding the images. This problem surfaced with the image handling. You can also tell by watching the page count flux around as page break shuffling is happening. Besides, all of the other word processors are using the same spell checkers on this system and when I open other large (400-850 pages) documents to edit there are no issues.

The speed problem is somewhat improved. I would not use the word "better" because "better" would be interpreted as good-to-close which I would not classify this as.
Comment 8 V Stuart Foote 2019-11-30 01:40:38 UTC
So, what behavior if you show only the wire frames of the inserted images?

Tools -> Options -> LibreOffice Writer -> View: Display 'Images and Objects'

That is the intended means of reducing overhead/memory foot print of positioning graphics and draw objects on document canvas. The wireframe place holders are negligible in size and allow manipulation of much larger texts without having to work in Master Documents.
Comment 9 Roland Hughes 2019-11-30 13:40:45 UTC
(In reply to V Stuart Foote from comment #8)
> So, what behavior if you show only the wire frames of the inserted images?
> 
> Tools -> Options -> LibreOffice Writer -> View: Display 'Images and Objects'
> 
> That is the intended means of reducing overhead/memory foot print of
> positioning graphics and draw objects on document canvas. The wireframe
> place holders are negligible in size and allow manipulation of much larger
> texts without having to work in Master Documents.

I don't know who came up with that, but they've obviously never written a book or documentation which could pass FDA audit. You physically cannot work from a wireframe. The image must be viewable by the author in its final size, not full sized in a graphics program on another monitor. Scaling removes detail which may well be discussed in the text. Talk about something in an image in your FDA filing which nobody reading the document can see and get ready to fail the audit.

Seriously, AGILE is a criminally fraudulent methodology. This sounds exactly like a kid, over his head, reached up his back side to the elbow and pulled this "solution" out so they could call the user story done.

I realize, you don't know me from Adam so please indulge me for a moment. I've spent over 30 years in IT and written a lot of books. Some of them award winning. One of them on the Dr. Dobb's list of recommended reading for all developers.

http://theminimumyouneedtoknow.com/

I also belong to a group of authors for community marketing purposes.
https://www.interestingauthors.com/
Between us, I think we have well over 30 books.

I also belong to the writers-editors network.

I'm telling you this because nobody that I have encountered in my professional careers moves wireframes around in a work. You cannot take the risk when writing prose.

Do I use wireframes when I'm laying out a touchscreen interface for a medical device? You betcha. That's what I did for this one. 
https://www.welchallyn.com/en/products/categories/patient-monitoring/vital-signs-devices/connex-spot-monitor.html

We didn't have access to the graphics in the designer. All I needed was positioning anyway. __Completely__ different concept than a word processor.
Comment 10 V Stuart Foote 2019-11-30 18:53:49 UTC
(In reply to roland from comment #9)
> ... You physically cannot work from a wireframe. The image must be viewable
>  by the author in its final size, not full sized in a graphics program on
>  another monitor.
> ...

Well of course, and was one of the reasons EPS provides a bitmap preview. 

Wireframe is done so you can manipulate your layout and scroll without the memory and graphic processing overhead of rendering document pages. Using the function to toggle images/graphics to wireframe lets you get your editing done.

Toggle it off to move about the document, and on to work on a specific page. The projects Navigator deck in the Sidebar makes this easy if you title your tables and frames and the images they contain. 

Just verified the Tools -> Options -> Writer -> View "Images and Objects" is the same .unoShowGraphc control provided from the main menu View -> 'Images and Charts'. So no need to dig into the Options dialog.

And, for convenience it can be assigned to Context menu customizations as the 'View Images and Charts' action.

So the toggle is very convenient for use, and I'd suggest essential for working with abusively long manuscripts.

But your 6.0.7.3 build is past EOL and this BZ issue is no longer actionable nor really relevant. 

As noted, Image/Graphics object handling was refactored at the 6.1.0 release, improving actual memory management and lifecycle for images within an edit session [1] (related issues against the META bug 116280)

So, please upgrade to current build when your distro allows, or PPA provides--or maybe better integrate a TDF build. The 6.3.4 release, and the 6.4.0 release candidates will be rolling soon. Issues against those would be appropriate and appreciated.

=-ref-=
[1] https://blog.documentfoundation.org/blog/2018/06/19/image-handling-rework-for-libreoffice-collaboras-tender-results/
Comment 11 Roland Hughes 2019-11-30 19:26:34 UTC
(In reply to V Stuart Foote from comment #10)
> 
> So the toggle is very convenient for use, and I'd suggest essential for
> working with abusively long manuscripts.

No. Purging of all things Java to get out from under artificial restrictions imposed by a VM is essential to allow this word processor to work with standard business length documentation.

According to KinfoCenter there was over 10 Gig of RAM free. An application written with a legitimate natively compiled development language would have been able to easily utilize all of the RAM it needed. Instead we have Java code causing the application to gasp and flail around for many minutes.

> 
> So, please upgrade to current build when your distro allows, or PPA
> provides--or maybe better integrate a TDF build. The 6.3.4 release, and the
> 6.4.0 release candidates will be rolling soon. Issues against those would be
> appropriate and appreciated.

Please actually read a message thread before leaping to a baseless conclusion. I did install a PPA. True to AGILE development it busted other things which had been working perfectly. This is why people doing actual work don't install "new" stuff until they are all done with actual work. This is why LTS versions are important.

While the image handling is not as horrible in the current release, it still cannot mail a letter to good.

Btw, there needs to be a blank entry on the second combo box so when things are erroneously closed they can be reset properly.
Comment 12 V Stuart Foote 2019-11-30 22:36:50 UTC
(In reply to roland from comment #11)
> 
> No. Purging of all things Java to get out from under artificial restrictions
> imposed by a VM is essential to allow this word processor to work with
> standard business length documentation.
> ...

Huh? Please make time to read the source before spouting such nonsense.

Resolved 'Works For Me' is the correct resolution--you are welcome to verify that when you upgrade to a current release build.
Comment 13 Roland Hughes 2019-11-30 22:40:20 UTC
(In reply to V Stuart Foote from comment #12)
> (In reply to roland from comment #11)
> > 
> 
> Resolved 'Works For Me' is the correct resolution--you are welcome to verify
> that when you upgrade to a current release build.

Huh? Please take the time to actually read the conversation before spouting such nonsense. The problem is not as bad, yet still exists in the current release build. 

Verified 'Still Open' is the correct setting.
Comment 14 Julien Nabet 2019-11-30 22:48:50 UTC
Just taking a look about what part is in Java in LO:
- reports builder in Base (with Pentaho)
- connector for HSQLDB (related to Base)
- Solver tool in Calc
- bridge for script in Java
- some QA tests
- xmerge "For converting documents among from and into formats and also for merging them"
- script/sdk/odk part
- accessibility toolkit test
- some UNO framework

Stephan: any idea about what Java part in LO I could miss which could explain this high memory consumption indicated by the reporter?
Comment 15 V Stuart Foote 2019-11-30 22:59:36 UTC
Back to UNCONFIRMED

Note for OP regard image frame spacing -- watch out during conversion from 6.0 <= as new content created at  6.2 >= the Image caption is reworked from Illustration -> Figure labeling. Don't beleive there was any adjustment in the padding/space to contents.

/me washing my hands, no time for Luddites...
Comment 16 Roland Hughes 2019-11-30 23:10:34 UTC
(In reply to Julien Nabet from comment #14)
> 
> Stephan: any idea about what Java part in LO I could miss which could
> explain this high memory consumption indicated by the reporter?

Not high memory consumption. Poor memory utilization. There was over 10Gig of free RAM on the box and the program was just flailing.
Comment 17 Luke 2019-11-30 23:13:44 UTC
Roland,
In order to confirm this bug, we're going to need a document to reproduce the behavior. If you document contains confidential text, you can strip out all content by following these steps:
https://wiki.documentfoundation.org/QA/BugReport/Attachments

Then you can use a free cloud storage service to share your bug doc. Here are some examples: Google Drive, Mega, Dropbox, pCloud, OneDrive,  or iCloud
Comment 18 Roland Hughes 2019-11-30 23:14:44 UTC
(In reply to V Stuart Foote from comment #15)
> Back to UNCONFIRMED
> 
> Note for OP regard image frame spacing -- watch out during conversion from
> 6.0 <= as new content created at  6.2 >= the Image caption is reworked from
> Illustration -> Figure labeling. Don't beleive there was any adjustment in
> the padding/space to contents.
> 
> /me washing my hands, no time for Luddites...

Thank you for the note.

I assume by Luddite you mean this:

One who fears technology (or new technology, as they seem pleased with how things currently are...why can't everything just be the same?)

found here: https://www.urbandictionary.com/define.php?term=luddite

That's not the case. Those of us using tools to do something need those tools to stay stable until the task is done. I upgrade often, between projects. I'm not between writing projects right now, hence my fear of the knee-jerk response "upgrade to the latest and then test." My fear was justified. Other things broke. This book has missed the Christmas market now. The two others behind it are delayed.
Comment 19 Jean-Baptiste Faure 2019-11-30 23:22:18 UTC
(In reply to Julien Nabet from comment #14)
[...]
> Stephan: any idea about what Java part in LO I could miss which could
> explain this high memory consumption indicated by the reporter?

Perhaps LanguageTools which isn't part of LO but only an extension. LT is written in Java.
@Roland: do you use LanguageTool ?

Best regards. JBF
Comment 20 Roland Hughes 2019-11-30 23:23:19 UTC
(In reply to Luke from comment #17)
> Roland,
> In order to confirm this bug, we're going to need a document to reproduce
> the behavior. If you document contains confidential text, you can strip out
> all content by following these steps:
> https://wiki.documentfoundation.org/QA/BugReport/Attachments
> 
> Then you can use a free cloud storage service to share your bug doc. Here
> are some examples: Google Drive, Mega, Dropbox, pCloud, OneDrive,  or iCloud

Sadly, there is no way to do that. I cannot release the source file for a book going into production or which is currently in print. I've had significant problems with book piracy.

At some point I can attempt to reproduce the problem with a large book I did not and will not be releasing. I do have a second edition of this book

https://theminimumyouneedtoknow.com/app_book.html

which I didn't put out due to a vendor participation (lack there-of) issue. The first edition weighed in around 800 pages. It didn't have as many images as the current work, but I could stick some of the images from this book in it to find the tipping point. The problem "feels like" there is a tipping point. One image over the limit and LO jumps off a cliff.
Comment 21 Roland Hughes 2019-11-30 23:42:39 UTC
(In reply to Jean-Baptiste Faure from comment #19)
> (In reply to Julien Nabet from comment #14)
> [...]
> > Stephan: any idea about what Java part in LO I could miss which could
> > explain this high memory consumption indicated by the reporter?
> 
> Perhaps LanguageTools which isn't part of LO but only an extension. LT is
> written in Java.
> @Roland: do you use LanguageTool ?
> 
> Best regards. JBF

No. This machine was a fresh install of KDE Neon for the project.

http://www.logikalsolutions.com/wordpress/wp-content/uploads/2019/11/LO-extensions.png

Once these 3 books are written the machine will most likely be wiped and setup for other projects.

If KinfoCenter hadn't been showing over 10Gig of free RAM I wouldn't even have reported this. Not using all of the RAM it could get access to points to either a VM issue (i.e. Java) or some internal thread limit. By the last part I mean that something limits either the number of worker threads or the amount of RAM they are allowed to consume. Kind of tired now, might be rambling. Long day at keyboard completing other writing projects.

Not expecting this to be fixed today. Just wanted to file the bug and have someone thinking about LO suitability for business class documentation. I routinely have documents well north of 600 pages and routinely run into what appears to be resource constraints on machines which are in no way resource constrained. That causes me to have to port the documents to other word processors. Sometimes I even have to jump to Windows using either Word Perfect or Word.

Honestly, I wonder if someone didn't put a 32-bit build of LO in the repos and PPA. That would explain a lot. LO couldn't ever get more than 4Gig no matter how hard it tried. That's just a thought from a tired brain though. Can't think of how to check off top of head even.
Comment 22 Luke 2019-12-01 01:35:02 UTC
(In reply to roland from comment #20)
If you document contains confidential text, you can strip out
> > all content by following these steps:
> > https://wiki.documentfoundation.org/QA/BugReport/Attachments
> Sadly, there is no way to do that. I cannot release the source file for a
> book going into production 

Why can't you use the macro for Confidential Attachments?
Comment 23 Julien Nabet 2019-12-01 08:35:01 UTC
Do you use any accessibility tool? If yes could you disable them for the test?
(some accessibility tools are created with Java)

Also, in LO, could you disable "Use Java" in Tools/options/advanced", restart LO and give a new try? (the goal is to force LO not to use Java and to know what requires Java in your case)
Comment 24 Roland Hughes 2019-12-01 13:44:00 UTC
(In reply to Luke from comment #22)
> (In reply to roland from comment #20)
> If you document contains confidential text, you can strip out
> > > all content by following these steps:
> > > https://wiki.documentfoundation.org/QA/BugReport/Attachments
> > Sadly, there is no way to do that. I cannot release the source file for a
> > book going into production 
> 
> Why can't you use the macro for Confidential Attachments?

There is absolutely nothing secure on the Internet. Anything uploaded for testing would need to be passed around volunteer to volunteer for testing. This is a book for commercial release that I've put roughly a year into and am now at the stage of putting around $10K in editing services into.

Yes, with every book there is always the chance it will sell just 10 copies, but with an electronic version out there anywhere, that chance is 10,000%. 

According to the Indian technical recruiters who call me via VOIP from India, this book,

https://theminimumyouneedtoknow.com/app_book.html

is one of the most pirated books in India.

How did that happen? It was the book which was going to fund my retirement. Written so well it got onto the Dr. Dobb's list of recommended reading for all developers on all platforms. It was selling well. Every year a new batch of Interns brought a new batch of orders.

A situation much like this happened and I shared an electronic copy via "secure" channels for "testing." A few weeks later sales went to zero and I started hearing from recruiters how it was being printed in India and sold for $5.

I _have_ a book I'm willing to risk. It was the second edition of that book which I chose to never release after the piracy issue. What I don't have right now is the time to load images to duplicate the issue. The Holidays are here and I have 3 books to finish writing before the end of January. I can write the other two with OnlyOffice.https://www.onlyoffice.com/

Indeed I started them with OnlyOffice when I was waiting entire minutes between keystrokes for LO to stop flailing around like a gut shot animal.

When I have time I will attempt to replicate the problem with a work I can risk. I will not risk a new book which has consumed more than a year of my "free" time.
Comment 25 Roland Hughes 2019-12-01 13:47:54 UTC
(In reply to Julien Nabet from comment #23)
> Do you use any accessibility tool? If yes could you disable them for the
> test?
> (some accessibility tools are created with Java)
> 
> Also, in LO, could you disable "Use Java" in Tools/options/advanced",
> restart LO and give a new try? (the goal is to force LO not to use Java and
> to know what requires Java in your case)

I turned it off now so I wouldn't forget. Hopefully that setting is global and not just for the currently opened file.

Have to spend the morning with my father in the nursing home. Will leave myself a note to experiment when I get home. The other two books I'm actually working on do not have this problem as they are both < 400 pages and currently without images.
Comment 26 Luke 2019-12-01 20:12:43 UTC
Since you are unwilling to run the macro that strips out the text of your document, we cannot help you. If you change your mind and provide us with a document for testing then we can reopen this bug and try to get the problem fixed.
Comment 27 Julien Nabet 2019-12-01 20:18:28 UTC
roland: I understand your documents are confidential but if you don't provide a minimal sanitized doc so people may reproduce this, it is indeed INSUFFICIENTDATA.
Comment 28 Roland Hughes 2019-12-01 20:42:52 UTC
No. I said I would work on one but I'm not taking time to do it immediately. Closing the bug now is not INSUFFICIENTDATA, it is WON'TFIX.

I have 3 titles to get ready for publication by end of January. Then and only then will I spend the weeks it takes to create 800+ pages of gibberish with embedded photos to replicate the problem.

If someone here wants to "make the stats look better" by closing this instead of leaving it in it's true state, OPEN INSUFFICIENTDATA, then you will close it honestly. WON'TFIX.
Comment 29 Julien Nabet 2019-12-01 20:45:38 UTC
Don't push too much.
Comment 30 Roland Hughes 2019-12-01 20:55:06 UTC
(In reply to Julien Nabet from comment #29)
> Don't push too much.

I'm not.

I have to manufacture _from scratch_ ~800 page document which has nothing to do with any client or any book project past or present. I have to legally obtain (not just thieve from the Web) unrelated images. Make the page size and alternating page styles with running headers exactly match, including all font usage. Duplicate the problem. Write step by step instructions to recreate the problem, THEN I can release it to you. That is many weeks of work, full time. It used to take me a year to create an 800 page work. It will definitely take many weeks, if not two months, to create useful gibberish meeting the criteria.

There is no way in God's green earth I'm sending valid content macro-morphed or otherwise out into the insecure Internet. I made that mistake once before to "diagnose a problem" and it has been costing me north of $60K/yr in lost book sales since I did it.
Comment 31 Julien Nabet 2019-12-01 20:59:13 UTC
Sorry Stephan for having cc you here, don't hesitate to uncc if you feel like of course.

As for me, I'm fed up here, so uncc myself.
Comment 32 Roland Hughes 2020-01-06 18:25:05 UTC
Created attachment 156970 [details]
book document for testing

Took this morning to create a test book document which is beginning to show the sluggishness on current release. Basically kind of dies on the version included with distros based on Ubuntu 18.04.

You can get more text to insert from here: https://loremipsum.io/generator/?n=900&t=p

More random images from here: https://lorempixel.com/

The only thing I didn't take the time to do is insert a bunch of tables (the original problem document had several) as well as both bullet and numbered lists. Four hours was about all I could spend on this. Document weighs in north of 900 pages. Has left and right page styles and a plethora of images and 26 heading 1 headings
Comment 33 Luke 2020-01-06 19:29:36 UTC
Thanks for a bug doc. Now all we need are steps to reproduce. I copy and pasted multiple times a paragraph with an image. No issue observed on 
Version: 6.5.0.0.alpha0+ (x64)
Build ID: 314f15bff08b76bf96acf99141776ef64d2f1355
CPU threads: 4; OS: Windows 10.0 Build 18362; UI render: GL; VCL: win; 

I'll try Linux later.
Comment 34 Roland Hughes 2020-01-06 19:42:15 UTC
(In reply to Luke from comment #33)
> Thanks for a bug doc. Now all we need are steps to reproduce. I copy and
> pasted multiple times a paragraph with an image. No issue observed on 
> Version: 6.5.0.0.alpha0+ (x64)
> Build ID: 314f15bff08b76bf96acf99141776ef64d2f1355
> CPU threads: 4; OS: Windows 10.0 Build 18362; UI render: GL; VCL: win; 
> 
> I'll try Linux later.

Sadly, pasting generally doesn't show the problem. Editing high up in the document causing the page breaks to change, and I'm assuming navigator info to update, is what used to cause it. I was "starting" to see the problem with the linux build I have.

Version: 6.3.4.2
Build ID: 1:6.3.4-0ubuntu0.18.04.1~lo2
CPU threads: 8; OS: Linux 5.0; UI render: default; VCL: gtk3; 
Locale: en-US (en_US.UTF-8); UI-Language: en-US
Calc: threaded

I didn't have time for a full reproduction and I couldn't use images from the document which had the problem. Four hours was all I had this morning and it will be several more weeks before  have another 4 hours to devote to it.

One thing I didn't do because it would have been invalid on this machine is to use the ubuntu supplied Linux Libertine font family, just in case it has something to do with fonts being expensive to render, especially inside of the captions.

Please save the file for general testing prior to a release. To be usable by many LO needs to easily handle files of that size.

I was lazy putting most of the images in. Doing a copy from the Web site and a <shift><insert> to put them in the document. That wasn't the case with the original document that had many problems. I went through the slow path of insert->image->choose file. I don't know if there are additional mechanics involved in that path or not.
Comment 35 Luke 2020-01-06 21:09:18 UTC
Thanks, but without clear step-by-step process to reproduce this issue, I'm going to have to leave this as NEEDINFO. Once you have the steps + a bug doc that exhibits this issue we can confirm it.
Comment 36 Roland Hughes 2020-01-13 18:45:18 UTC
Created attachment 157129 [details]
performance bug document

I stumbled into this again working on second edition of a big document. Problem got really bad when using alternating page styles with running page headings. One for left page of chapter and another for right so page number would always be at outside.

The macro from here: https://wiki.documentfoundation.org/QA/BugReport/Attachments

Took forever to run and seemed to clobber some of the page styles, chapter, and page headings, but enough of it is there to begin duplicating the problem. I cannot release that book because it will be stolen so including the "confidential" version. The problem is not as severe as it was with the actual document, but you can begin to see the issue.

A wheel/scroll mouse makes this problem very obvious.

Open document
use Navigator to go to page 18.
Delete the page break which is at the beginning of the white space after the text.
Use scroll/wheel mouse to scroll down looking for next bad page break.
You should already see quite a bit of stutter/hang and no "busy" cursor.
Find next bad page break and delete it. 

The more you manage to do the worse the problem gets. If you go away for 5 minutes the problem seems to be gone but comes right back as soon as you start fixing page breaks or moving the beginning of a paragraph to be with its other half on the following page.

In truth this document is just under 1000 pages, but can appear as 13-1500 on open. Much of this problem is caused by losing the 6x9 page styles for many pages and the random page breaking problems this reflow causes.
Comment 37 QA Administrators 2020-01-14 03:39:51 UTC Comment hidden (obsolete)
Comment 38 Telesto 2020-04-17 20:56:05 UTC
Not exact problem, described.. but slow as hell
1. Open attachment 157129 [details]
2. CTRL+A
3. CTRL+X -> slow

Version: 7.0.0.0.alpha0+ (x64)
Build ID: 4475bcd83aac7e033fc5250f268eb922bd471e7b
CPU threads: 4; OS: Windows 6.3 Build 9600; UI render: default; VCL: win; 
Locale: nl-NL (nl_NL); UI-Language: en-US
Calc: CL

also in
4.4.7.2
Comment 39 Telesto 2020-04-17 20:59:58 UTC
Press CTRL+Z and wait again
Comment 40 Telesto 2020-04-17 21:01:55 UTC
@Julien
A perf graph for comment 38 and 39 would be nice to have (if you can confirm of course)
Comment 41 Julien Nabet 2020-04-18 08:43:11 UTC
Created attachment 159673 [details]
perf flamegraph

Telesto: following your last comment, here's a Flamegraph retrieved on pc Debian x86-64 with master sources updated today.