Created attachment 143958 [details] Slow file Hello, I have a LibreOffice file with lot of data and lot of formulas. It can be very slow. For example, try the following procedure: 1 - Open the joined file 2 - In the "Devis Strasbourg" sheet, try to insert a new column next to the "P" one 3 - Libreoffice freezes for a while. Thanks, Jean-Sébastien
There are 90000 VLOOKUP in the file looking in a range of 10000 rows and 20000 COUNTIF. So when you insert a column all need to be evaluated. Repro with Version: 6.0.6.2 (x64) Build ID: 0c292870b25a325b5ed35f6b45599d2ea4458e77 CPU threads: 4; OS: Windows 10.0; UI render: GL; Locale: es-ES (es_ES); Calc: CL Version: 6.2.0.0.alpha0+ (x64) Build ID: 715fcaff01ed048c52c69264a7a0fb773dd57b32 CPU threads: 4; OS: Windows 10.0; UI render: default; TinderBox: Win-x86_64@42, Branch:master, Time: 2018-07-21_02:21:27 Locale: es-ES (es_ES); Calc: CL
Also reproduced in LibreOffice 3.3.0 OOO330m19 (Build:6) tag libreoffice-3.3.0.4 IMHO, this bug is too general, but let's keep it open as the document is attached and it can be useful...
On pc Debian x86-64 with master sources updated today or with LO 6.1.5.2, I don't reproduce this (But I've got a Ryzen 5 2600 + RAM 32GB and SSD Nvme)
Created attachment 154778 [details] FlameGraph
Julien, I can repro it in current master Версия: 6.4.0.0.alpha0+ ID сборки: 66e45a1ae861d50edf65fed9e39c9c9d5b15e0ac Потоков ЦП: 4; ОС:Linux 5.0; Отрисовка ИП: по умолчанию; VCL: kf5; TinderBox: Linux-rpm_deb-x86_64@86-TDF, Branch:master, Time: 2019-09-28_15:31:38 Локаль: ru-RU (ru_RU.UTF-8); UI-Language: ru-RU Calc: threaded here is Core i3 2330m, 7 year old CPU with only 2 core and 4Gb of memory And I tried to make FlameGraph (look at it in attach). It my first expirience, so may be it is not correct =( Noel, could you look at it?
Created attachment 154779 [details] perf flamegraph On pc Debian x86-64 with master sources updated today, I could reproduce this. I was on the wrong sheet. It's even worse for me since it's a complete freeze. I had to kill LO.
Comment on attachment 154778 [details] FlameGraph this graph is wrong
Dear Jean-Sebastien Bevilacqua, To make sure we're focusing on the bugs that affect our users today, LibreOffice QA is asking bug reporters and confirmers to retest open, confirmed bugs which have not been touched for over a year. There have been thousands of bug fixes and commits since anyone checked on this bug report. During that time, it's possible that the bug has been fixed, or the details of the problem have changed. We'd really appreciate your help in getting confirmation that the bug is still present. If you have time, please do the following: Test to see if the bug is still present with the latest version of LibreOffice from https://www.libreoffice.org/download/ If the bug is present, please leave a comment that includes the information from Help - About LibreOffice. If the bug is NOT present, please set the bug's Status field to RESOLVED-WORKSFORME and leave a comment that includes the information from Help - About LibreOffice. Please DO NOT Update the version field Reply via email (please reply directly on the bug tracker) Set the bug's Status field to RESOLVED - FIXED (this status has a particular meaning that is not appropriate in this case) If you want to do more to help you can test to see if your issue is a REGRESSION. To do so: 1. Download and install oldest version of LibreOffice (usually 3.3 unless your bug pertains to a feature added after 3.3) from https://downloadarchive.documentfoundation.org/libreoffice/old/ 2. Test your bug 3. Leave a comment with your results. 4a. If the bug was present with 3.3 - set version to 'inherited from OOo'; 4b. If the bug was not present in 3.3 - add 'regression' to keyword Feel free to come ask questions or to say hello in our QA chat: https://kiwiirc.com/nextclient/irc.freenode.net/#libreoffice-qa Thank you for helping us make LibreOffice even better for everyone! Warm Regards, QA Team MassPing-UntouchedBug
Column inserting takes around 2 minutes in Version: 7.3.0.0.alpha0+ (x64) / LibreOffice Community Build ID: fbfd91f2c5f4d66570c2d5a6f048b21f5d1671a4 CPU threads: 8; OS: Windows 10.0 Build 19043; UI render: Skia/Vulkan; VCL: win Locale: ru-RU (ru_RU); UI: ru-RU Calc: threaded I think it's still a problem
how long does Excel take to perform this operation?
Noel Grandin committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/c998691e22ceda15c89d55cf7005201f0392dadb tdf#119083 small improvement to large vlookup sheet insert It will be available in 7.3.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
About 4 seconds with excel, about 15 with calc. Microsoft® Excel® for Microsoft 365 MSO (Version 2109 Build 16.0.14430.20154) 32-bit Version: 7.2.2.2 (x64) / LibreOffice Community Build ID: 02b2acce88a210515b4a5bb2e46cbfb63fe97d56 CPU threads: 4; OS: Windows 10.0 Build 21390; UI render: Skia/Raster; VCL: win Locale: es-ES (es_ES); UI: en-US Calc: CL
(In reply to Roman Kuznetsov from comment #9) > Column inserting takes around 2 minutes in > > Version: 7.3.0.0.alpha0+ (x64) / LibreOffice Community > Build ID: fbfd91f2c5f4d66570c2d5a6f048b21f5d1671a4 > CPU threads: 8; OS: Windows 10.0 Build 19043; UI render: Skia/Vulkan; VCL: > win > Locale: ru-RU (ru_RU); UI: ru-RU > Calc: threaded So strange, it takes 3 min 45 sec now in Version: 7.3.0.0.alpha1+ (x64) / LibreOffice Community Build ID: c36fa9f86e54afa4e1876a9d296ebcbfcbd3a0ad CPU threads: 4; OS: Windows 6.1 Service Pack 1 Build 7601; UI render: Skia/Raster; VCL: win Locale: ru-RU (ru_RU); UI: ru-RU Calc: CL
Created attachment 176245 [details] perf flamegraph Here's a new Flamegraph retrieved on pc Debian x86-64 with master sources updated today + gen rendering. Remark: for a 600kB file, it's quite long to open too (about 30 seconds on a Ryzen 5 2600 + 32GB).
Luboš: noticing https://cgit.freedesktop.org/libreoffice/core/commit/?id=ec0edb0969c23b25576f4d1b3b2ee5d3f21990ad (optimize VLOOKUP by returning SharedString if possible), thought you might be interested in this one. (see attached Flamegraph in my previous comment). Of course, don't hesitate to uncc yourself if not interested, have no time, other.
I cannot reproduce the problem. If I right-click on column P and select to add a column after, it's almost instant. Julien: Your flamegraph is from a debug build.
(In reply to Luboš Luňák from comment #16) > I cannot reproduce the problem. If I right-click on column P and select to > add a column after, it's almost instant. Nevermind, wrong sheet, I can reproduce it.
Created attachment 176246 [details] Flamegraph
(In reply to Luboš Luňák from comment #17) > (In reply to Luboš Luňák from comment #16) > > I cannot reproduce the problem. If I right-click on column P and select to > > add a column after, it's almost instant. > > Nevermind, wrong sheet, I can reproduce it. Yes, I was writing this in a comment but you were quicker than me. Sorry for the Flamegraph retrieved on a debug build, I retrieved another one on a non-debug build.
Luboš Luňák committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/e55723c0ba4172bf6ee80fda49b069fd441a0c90 try to broadcast in bulk (tdf#119083) It will be available in 7.4.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Luboš Luňák committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/8406139062d9ffe1daed32aefe4e261c6c55d63e process broadcasts for adjacent cells together (tdf#119083) It will be available in 7.4.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Luboš Luňák committed a patch related to this issue. It has been pushed to "libreoffice-7-3": https://git.libreoffice.org/core/commit/47198583da8e67e0178466205d82835c391c8d73 try to broadcast in bulk (tdf#119083) It will be available in 7.3.0.0.beta2. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Luboš Luňák committed a patch related to this issue. It has been pushed to "libreoffice-7-3": https://git.libreoffice.org/core/commit/8579b6e39ea30048ae7763f27d77e093b26e76dc process broadcasts for adjacent cells together (tdf#119083) It will be available in 7.3.0.0.beta2. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
1 min 40 sec without Luboš's latest patches 38 sec with Luboš's latest patches CPU - Intel i7-10510U , Windows 10 It's a pity if you can't do something else here.
Created attachment 176777 [details] Flamegraph @Roman: weird, I gave a new try with master sources updated today (7e5af164b7d293dd410710bed411e1ca64bbecf7) + gen rendering, it was about 2 secs (I double checked I was on the right sheet, "Devis Strasbourg") I attached a new Flamegraph just in case.
(In reply to Julien Nabet from comment #25) > Created attachment 176777 [details] > Flamegraph > > @Roman: weird, I gave a new try with master sources updated today > (7e5af164b7d293dd410710bed411e1ca64bbecf7) + gen rendering, it was about 2 > secs (I double checked I was on the right sheet, "Devis Strasbourg") > > I attached a new Flamegraph just in case. I agree, it's really strange, because I tried it on my home PC with very old Intel Core 2 Quad 9450 right now and I really got only 2-3 sec result. I'll retest tomorrow on my work notebook
(In reply to Roman Kuznetsov from comment #26) > I'll retest tomorrow on my work notebook 18 sec >_< CPU - Intel i7-10510U , Windows 10 Version: 7.4.0.0.alpha0+ (x64) / LibreOffice Community Build ID: 3a61cce54277fd12570103a191c50d9b37ef3dd3 CPU threads: 8; OS: Windows 10.0 Build 19043; UI render: Skia/Raster; VCL: win Locale: ru-RU (ru_RU); UI: ru-RU Calc: threaded may be the problem here is the CPU is mobile and has only 1,8GHz by default and Windows can be in power save mode always...
(In reply to Roman Kuznetsov from comment #27) Are you testing using a debug build, or a non-debug build? Performance issues may appear much slower in debug builds.
45 seconds in Version: 7.3.0.0.beta1+ (x64) / LibreOffice Community Build ID: d7a6869a531d0f2f26c47714466d5d47d78ddbfd CPU threads: 4; OS: Windows 10.0 Build 19044; UI render: Skia/Raster; VCL: win Locale: ro-RO (ro_RO); UI: en-US Calc: threaded
I can actually reproduce on my Windows build that this doesn't help there. The cells to process together are in random order, possibly as a result of Windows threading acting slightly differently. Reopening, maybe this can be improved (although it's a question if and when given that I did this in my spare time).
In any way, patches gets a great time reduction. About 00:03:52 with Version: 7.2.4.1 (x64) / LibreOffice Community Build ID: 27d75539669ac387bb498e35313b970b7fe9c4f9 CPU threads: 4; OS: Windows 10.0 Build 21390; UI render: Skia/Raster; VCL: win Locale: es-ES (es_ES); UI: en-US Calc: CL About 59 seconds with master 2021-12-08. Version: 7.4.0.0.alpha0+ (x64) / LibreOffice Community Build ID: 7e5af164b7d293dd410710bed411e1ca64bbecf7 CPU threads: 4; OS: Windows 10.0 Build 21390; UI render: Skia/Raster; VCL: win Locale: es-ES (es_ES); UI: en-US Calc: CL About 3~4 seconds. Microsoft® Excel® Microsoft 365 MSO (Version 2110 Build 16.0.14527.20234) 32-bit
(In reply to BogdanB from comment #29) > 45 seconds in > Version: 7.3.0.0.beta1+ (x64) / LibreOffice Community > Build ID: d7a6869a531d0f2f26c47714466d5d47d78ddbfd > CPU threads: 4; OS: Windows 10.0 Build 19044; UI render: Skia/Raster; VCL: > win > Locale: ro-RO (ro_RO); UI: en-US > Calc: threaded Bogdan, LO 7.3 beta 1 doesn't have latest Luboš's patches
(In reply to Kevin Suo from comment #28) > (In reply to Roman Kuznetsov from comment #27) > Are you testing using a debug build, or a non-debug build? Performance > issues may appear much slower in debug builds. I tested it in windows daily build by link https://dev-builds.libreoffice.org/daily/master/current.html
I found the bug number in About dialog, so maybe appear there because were more changes in code from this bug. So you have right.
Eike Rathke committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/a1f21d7094deb6f1ae5388718f2bc28eecd5737a Resolves: tdf#147398 Test Intersects() instead of Contains(), tdf#119083 It will be available in 7.4.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Eike Rathke committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/f6b9c471cace1963e8b625ecfe2c20f5248984eb Handle the possible case of broadcasted row block, tdf#119083 follow-up It will be available in 7.4.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Eike Rathke committed a patch related to this issue. It has been pushed to "libreoffice-7-3": https://git.libreoffice.org/core/commit/963330b5605e2350a34405c4752989c43c223b78 Resolves: tdf#147398 Test Intersects() instead of Contains(), tdf#119083 It will be available in 7.3.2. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Now on my old Intel Core 2 Quad 9450 the column inserting took only 3 sec in Version: 7.4.0.0.alpha0+ (x64) / LibreOffice Community Build ID: c856f9bec12d98ed49f01578ded79f16ae7be051 CPU threads: 4; OS: Windows 6.1 Service Pack 1 Build 7601; UI render: Skia/Raster; VCL: win Locale: ru-RU (ru_RU); UI: en-US Calc: CL Jumbo Finally fixed?
Luboš Luňák committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/fdcea8ccc8236b0975ac9c85dcabf9c663e4b627 make CollectCellAction sort cells by cell address (tdf#119083) It will be available in 7.4.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Eike Rathke committed a patch related to this issue. It has been pushed to "libreoffice-7-3": https://git.libreoffice.org/core/commit/89c4b004a748457408781972c5b6c6d44360fb3e Handle the possible case of broadcasted row block, tdf#119083 follow-up It will be available in 7.3.2. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Eike Rathke committed a patch related to this issue. It has been pushed to "libreoffice-7-3-1": https://git.libreoffice.org/core/commit/eb07d94074cbc06cea4d938068161bbb25442d0a Resolves: tdf#147398 Test Intersects() instead of Contains(), tdf#119083 It will be available in 7.3.1. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
I tried the following: Inserting new column as describes works fine for me. It takes just a moment. But it takes 51.37 sek if i insert a new row. 1)insert new row e.g. over/under row 13 2)It takes 51.37 sek Version: 7.3.2.2 (x64) / LibreOffice Community Build ID: 49f2b1bff42cfccbd8f788c8dc32c1c309559be0 CPU threads: 8; OS: Windows 10.0 Build 19044; UI render: Skia/Raster; VCL: win Locale: de-DE (de_DE); UI: de-DE Calc: CL