Bug 150749 - Find and replace - replace all - 100% CPU
Summary: Find and replace - replace all - 100% CPU
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
7.5.0.0 alpha0+
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:7.5.0
Keywords: perf
Depends on:
Blocks: Performance CPU-AT-100%
  Show dependency treegraph
 
Reported: 2022-09-02 04:04 UTC by Pierre Fortin
Modified: 2024-04-25 06:39 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
sheet I was working on; expands to 300MB. 1437695x101 (7.44 MB, application/zip)
2022-09-02 04:04 UTC, Pierre Fortin
Details
Flamegraph (182.75 KB, image/svg+xml)
2022-09-02 10:47 UTC, Julien Nabet
Details
Flamegraph (68.08 KB, application/x-bzip)
2022-09-05 19:24 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Pierre Fortin 2022-09-02 04:04:18 UTC
Created attachment 182168 [details]
sheet I was working on; expands to 300MB. 1437695x101

Created sheet with 2-character codes in many cells -- over 1.4M rows by 101 columns. Cells without codes were filled with a single space (so I could apply a simple conditional format). Then, needing to find last code in each column, using Find and Replace, inserted a single space in "Find:", nothing in "Replace:" and clicked Replace All. CPU pegged at 100% -- no response; killed after 10-15 minutes on this:.
Operating System: Mageia 9
KDE Plasma Version: 5.25.4
KDE Frameworks Version: 5.97.0
Qt Version: 5.15.5
Kernel Version: 5.19.4-server-2.mga9 (64-bit)
Graphics Platform: X11
Processors: 20 × 12th Gen Intel® Core™ i7-12700K
Memory: 125.5 GiB of RAM
Graphics Processor: AMD Radeon RX 6600 XT
Manufacturer: Dell Inc.
Product Name: XPS 8950
Comment 1 Julien Nabet 2022-09-02 10:46:02 UTC
What's the "map" extension in zip file?
Anyway, I renamed it to use "tsv" extension, I'm giving a try right now.
Comment 2 Julien Nabet 2022-09-02 10:47:23 UTC
Created attachment 182174 [details]
Flamegraph

Here's a Flamegraph retrieved on pc Debian x86-64 with master sources updated today (b392d0a25e229f305497ab5f9f084b71b24cc387)
Comment 3 Julien Nabet 2022-09-02 10:49:18 UTC
Noel: Flamegraph shows SharedString but also mdds, do you think we should ping Kohei (because he's the "mdds" guy) or Eike first (since he's the Calc expert) or do you have some idea here?
Comment 4 Pierre Fortin 2022-09-02 17:32:40 UTC
(In reply to Julien Nabet from comment #1)
> What's the "map" extension in zip file?
> Anyway, I renamed it to use "tsv" extension, I'm giving a try right now.

I use many different extensions when I know they don't matter to various programs. Calc doesn't care -- it opens files correctly; even renaming XX.tsv as XX.ods still opens correctly.  A rose by any other name is still a rose. :)
Comment 5 Julien Nabet 2022-09-02 17:44:56 UTC
(In reply to Pierre Fortin from comment #4)
> (In reply to Julien Nabet from comment #1)
> > What's the "map" extension in zip file?
> > Anyway, I renamed it to use "tsv" extension, I'm giving a try right now.
> 
> I use many different extensions when I know they don't matter to various
> programs. Calc doesn't care -- it opens files correctly; even renaming
> XX.tsv as XX.ods still opens correctly.  A rose by any other name is still a
> rose. :)

On pc Debian x86-64 with master sources updated today, with "map" extension, LO tries to open the file with Writer, not with Calc when doing:
./soffice ncid.map
Comment 6 Commit Notification 2022-09-05 11:46:12 UTC
Noel Grandin committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/a740176009411d21e20d7c11097af1d8812d251d

tdf#150749 Find and replace on very large sheet

It will be available in 7.5.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 7 Julien Nabet 2022-09-05 11:51:16 UTC
Thank you Noel! I'll test this tonight.

Kohei: I put you in cc since Noel's patch includes a change in mdds which could perhaps be added upstream.
Comment 8 Julien Nabet 2022-09-05 19:24:22 UTC
Created attachment 182239 [details]
Flamegraph

Here's an updated Flamegraph retrieved on pc Debian x86-64 with master sources updated today.
I don't know at all if it can still be improved or if there are too much data in the file.