Created attachment 124761 [details] Example file to display the issue While a filter is active in Calc, the CPU resource usage is significantly high. It seems to depend on the number of data items on the sheet, as well as their diversity, how high the CPU resources rise. On a lower-spec machine, using a filter on a large spreadsheet will be accompanied by significant delays in filter selection, activation and general LO responsiveness. Test procedure: 1. Open the attached "SA8000_Q1_2015_public_list.xlsx" file (found on the internet to illustrate the issue, which was originally observed with an even greater impact on a different XLSX file, but which can't be provided due to confidentiality reasons). 2. Activate any one of the filters it contains. (Note: Select only one data point for filtering. Because selecting two (e.g., "CHINA" and "China" under Country) seems to make the issue go away after an initial CPU resource rise.) 3. Deactivate the filter. Expected results: 2. A resource rise due to filter change, followed by the resources going back down to idle afterwards. 3. Same as point 2. Actual results: 2. A continuous high CPU resource consumption. (Switching between the CPU cores.) 3. As expected - the resource consumption goes back to idle. Linux Mint 17.1, 64 bit, on a Netbook with the dual core Intel Atom N450 mobile processor and 2GB RAM LO as provided by ppa:libreoffice/libreoffice-5-0 Version: 5.0.5.2 Build ID: 1:5.0.5~rc2-0ubuntu1~trusty1
Created attachment 124762 [details] Screenshot with filter active for a while
Created attachment 124763 [details] Screenshot after filter deactivation
No repro Version: 5.2.0.0.alpha0+ Build ID: 170a473597534cf59887b1d817538322e7039862 CPU Threads: 4; OS Version: Linux 4.2; UI Render: default; TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time: 2016-04-19_00:41:06
Created attachment 124770 [details] Screenshot with filter active, Debian 32bit (In reply to raal from comment #3) > No repro Version: 5.2.0.0.alpha0+ Thanks for testing! I can reproduce it with current master on Debian 32-bit though (same machine, liveUSB): Version: 5.2.0.0.alpha1+ Build ID: 039b75d6cdc26dcce03e37c67115405e6f2a8ebe CPU Threads: 2; OS Version: Linux 4.3; UI Render: default; TinderBox: Linux-rpm_deb-x86@71-TDF, Branch:master, Time: 2016-05-01_01:31:14 Locale: en-US (en_US.UTF-8)
I cannot reproduce this in Version: 5.2.0.0.alpha1+ Build ID: 07a641b110beee4f7c76617fcd6ed558025321a2 CPU Threads: 2; OS Version: Linux 4.4; UI Render: default; TinderBox: Linux-rpm_deb-x86@71-TDF, Branch:master, Time: 2016-05-02_02:15:45 Locale: nl-NL (nl_NL.UTF-8) and in 5.2.1.1 But I do notice a peak when starting the filter as suggested.
Thanks for testing! The impact of this issue might depend on the power of the used machine. E.g., I noticed on other, smaller files that the CPU usage was increased (e.g. to 50%), but did not go all the way to 100% like on a larger file like the one attached. Additional information: I tested on a five-year-old netbook with the Intel Atom N450 CPU, 1 core, 2 threads, 1.66GHz (see http://ark.intel.com/products/42503/Intel-Atom-Processor-N450-512K-Cache-1_66-GHz ), 2GB RAM, 1024x600 screen. The process appearing responsible in the System Monitor is "soffice.bin". Its memory consumption does not go up with the issue.
Created attachment 124843 [details] Screenshots of tests with WinXP and Linux Mint (LO 4.2.6, 4.3.5, 5.0.5) Additional tests on the same machine: - Windows XP Home SP3, 32-bit: Version: 4.3.5.2 Build-ID: 3a87456aaa6a95c63eea1c1b3201acedf0751bd5 Version: 5.0.5.2 Build-ID: 55b006a02d247b5f7215fc6ea0fde844b30035b3 - Linux Mint 17.1 (original ISO as liveUSB), 64-bit: Version: 4.2.6.3 Build ID: 420m0(Build:3) Both 4.x versions above show a smaller issue impact than the 5.x versions tested so far. But there is still a significant continuous resource increase. So, I'm going to update the affected version and OSes accordingly.
Created attachment 124879 [details] Screenshot with filter active, Windows 10 Home, 64bit The issue is also reproducible on the following: I. A new desktop PC with Intel Core i5-6400 CPU, 3.2GHz, cuadcore Intel H110 chipset 4GB RAM (3.89GB available) CPU-built-in graphics, 1920x1080 screen Windows 10 Home 64-bit LO Version: 5.0.5.2 Build-ID: 55b006a02d247b5f7215fc6ea0fde844b30035b3 II. A notebook from the year 2002 mobile AMD Athlon XP2200+ 518MHz, 512MB RAM (448MB available) Radeon IGP 320M graphics card 1024x768 screen Tested with Windows XP Professional SP3, 32-bit No Java installed LO (LibreOfficePortablePrevious_5.0.5_MultilingualAll.paf.exe): Version: 5.0.5.2 Build-ID: 55b006a02d247b5f7215fc6ea0fde844b30035b3 -------------------------------------------------- ADDITIONAL TESTS Note: I'm going to use following PC configurations on these tests: - Config 1: The Intel Atom N450 netbook from the comment 6 with Linux Mint 17.1 (original ISO as liveUSB), 64-bit - Config 2: The above AMD Athlon XP2200+ notebook from this comment (point II), with Windows XP Professional SP3, 32-bit All tested LO versions downloaded from https://downloadarchive.documentfoundation.org/libreoffice/old/ Tests: A) Issue not reproducible / different issue a) LO 3.3.0 - PC config 2 with LibreOfficePortable_3.3.0.paf.exe OOO330m19 (Build:6) tag libreoffice-3.3.0.4 => There is a different issue: CPU load goes up on filter activation, but then comes back down of itself after a few minutes b) LO 3.5.3.2 - PC config 2 with LibreOfficePortable_3.5.3.1_MultilingualNormal.paf.exe: LibreOffice 3.5.3.2 Build-ID: 235ab8a-3802056-4a8fed3-2d66ea8-e241b8 => There is a different issue: File import takes ca. 10-15min, after which the file is not displayed correctly - only column A is shown, while the rest of the used columns is a white field. Then as in the point c below. c) LO 3.6.4.3 - PC config 2 with LibreOfficePortable_3.6.4_MultilingualAll.paf.exe Version 3.6.4.3 (Build ID: 2ef5aff) => There is a different issue: Continuously elevated CPU load (around 65%) after file import. The load goes down ("soffice.bin" at 0% CPU) when one tries closing the modified (e.g. by filter activation) file and the "Save changes?" dialog appears (which is canceled). No issue with filter activation after that. d) LO 4.0.4.2 - PC config 1 with LibreOffice_4.0.4.2_Linux_x86-64_deb.tar.gz Version 4.0.4.2 (Build ID: 9e9821abd0ffdbc09cd8c52eaa574fa09eb08f2) - PC config 2 with LibreOfficePortable_4.0.4_MultilingualAll.paf.exe Version 4.0.4.2 (Build ID: 9e9821abd0ffdbc09cd8c52eaa574fa09eb08f2) => As in the point c above. e) LO 4.1.6.2 - PC config 1 with LibreOffice_4.1.6.2_Linux_x86-64_deb.tar.gz Version: 4.1.6.2 Build ID: 40ff705089295be5be0aae9b15123f687c05b0a => As in the point c above. B) Issue reproducible a) LO 4.2.0.1 - PC config 1 with LibreOffice_4.2.0.1_Linux_x86-64_deb.tar.gz Version: 4.2.0.1 Build ID: 7bf567613a536ded11709b952950c9e8f7181a4a b) LO 4.2.4.2 - PC config 1 with LibreOffice_4.2.4.2_Linux_x86-64_deb.tar.gz Version: 4.2.4.2 Build ID: 63150712c6d317d27ce2db16eb94c2f3d7b699f8 ------------------------------------------------ SUMMARY - The issue first appears in LO 4.2.0.1. The LO 4.1.6.2 is the last good one, with a different issue (see next point). - Appearance of this issue coincides with disappearance of a different issue: continuous elevated CPU usage by "soffice.bin" (ca. 20% with PC config 1, ca. 60% with PC config 2) after file import, until a file closure attempt. So that there may be a relation.
Setting the version to the 4.2.0.4 release, since the 4.2.0.1 RC is not available.
I repro in that CPU remains at 100% and only relaxes after I deactivate the filtering. Johnny: would you like to join the QA team in triaging all the unconfirmed nasties? https://wiki.documentfoundation.org/QA/Triage_For_Beginners 64-bit, KDE Plasma 5 Build ID: 5.1.2.2 Arch Linux build-1 CPU Threads: 8; OS Version: Linux 4.5; UI Render: default; Locale: fi-FI (fi_FI.UTF-8)
(In reply to Buovjaga from comment #10) > I repro in that CPU remains at 100% and only relaxes after I deactivate the > filtering. Thanks for testing! > Johnny: would you like to join the QA team in triaging all the unconfirmed > nasties? https://wiki.documentfoundation.org/QA/Triage_For_Beginners Thanks for suggesting! Its a bit hard at the moment, because that poor little Intel Atom N450 CPU netbook above is currently my best machine. I'll be glad to get more involved once I get a new machine some time in the future. In the meantime I'm mostly limiting myself to just submitting and supporting the issues I experience. Thanks to all those who do more!
(In reply to Johnny_M from comment #11) > Thanks for suggesting! Its a bit hard at the moment, because that poor > little Intel Atom N450 CPU netbook above is currently my best machine. I'll > be glad to get more involved once I get a new machine some time in the > future. In the meantime I'm mostly limiting myself to just submitting and > supporting the issues I experience. Thanks to all those who do more! Cool :) Drop by the -qa IRC channel, when you are ready: https://wiki.documentfoundation.org/Website/IRC
possibly 64 bits only?
(In reply to Cor Nouws from comment #13) > possibly 64 bits only? It doesn't seem to be. I've been able to reproduce it on following, per comments above: - 32-bit mobile AMD Athlon XP2200+ CPU (1 core, 1 thread, external graphics), 1024x768 screen, with 32-bit WinXP Pro (installed) - 64-bit Intel Atom N450 CPU (1 core, 2 threads, built-in graphics), 1024x600 screen, with 32-bit WinXP Home (installed), 32-bit Debian Jessie (liveUSB, Linux 4.3), 64-bit Linux Mint 17.1 (installed Linux 3.13 and liveUSB Linux 3.13) - 64-bit Intel Core i5-6400 CPU (4 cores, 4 threads, built-in graphics), 1920x1080 screen, with 64-bit Win10 Home (installed) Other reproduced: - @Buovjaga with CPU Threads: 8; OS Version: Linux 4.5, 64-bit Not reproduced: - @raal with CPU Threads: 4; OS Version: Linux 4.2, 64-bit - @Cor Nouws with CPU Threads: 2; OS Version: Linux 4.4, 32-bit So that there could be some kind of HW dependency, but it's not the 64-bit by itself. It's not yet clear what it depends on. The PC age on my tests is also between 14 years (mobile AMD Athlon XP2200+ CPU) and 0 years (Intel Core i5-6400 CPU).
Is this still an issue in 5.2? I think I fixed one or two performance issues around the filter popup for the 5.2 cycle.
(In reply to Markus Mohrhard from comment #15) > Is this still an issue in 5.2? I think I fixed one or two performance issues > around the filter popup for the 5.2 cycle. Thanks for looking into this! I've just checked the following and the issue is still there with all of those: 1) A year-2008 machine with AMD turion X2 processor, with 32-bit Windows Vista: Version: 5.2.0.4 Build-ID: 066b007f5ebcc236395c7d282ba488bca6720265 CPU-Threads: 2; BS-Version: Windows 6.0; UI-Render: Standard; Gebietsschema: de-DE (de_DE) 2) A year-2010 machine with Intel Atom N450, with: a) 32-bit Debian on live USB: Version: 5.3.0.0.alpha0+ Build ID: cb95159e79ee531f3908e30fefae73693672f0a6 CPU Threads: 2; OS Version: Linux 4.3; UI Render: default; TinderBox: Linux-rpm_deb-x86@71-TDF, Branch:master, Time: 2016-07-25_03:57:42 Locale: en-US (en_US.UTF-8); Calc: group b) 64-bit Linux Mint 17.1 (based on Ubuntu 14.04.1) live USB: Version: 5.3.0.0.alpha0+ Build ID: 39e300612c97b7742c8d8417d4dc6c0022cfa040 CPU Threads: 2; OS Version: Linux 3.13; UI Render: default; TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time: 2016-08-05_01:22:59 Locale: en-US (en_US.UTF-8); Calc: group I'm not sure why the issue couldn't be reproduced by @raal und @Cor Nouws. Maybe it somehow/sometime depends on the column where the filter is activated for the test, or the number of selected items in the filter. I tried several, all with the same results, but then (for reproducibility) started always using the same on the test step 2. So, here are the steps defined more specifically: 2. In the "Company" column filter, unselect "All", then select "LATECH S.R.L." (so that it's the only one active). 3. In the "Company" column filter, select "All" to deactivate the filtering. For a rough regression point, please see the "SUMMARY" in the comment 8.
(In reply to Markus Mohrhard from comment #15) > Is this still an issue in 5.2? I think I fixed one or two performance issues > around the filter popup for the 5.2 cycle. To prevent any confusion: Unlike the bug 91307, etc., this is not about the behavior of the "filter popup" (which behaves without problems for me). Rather, the issue appears/disappears after the popup gets closed using the OK button. (I missed adding the "... and click OK to confirm" part at the end of the test steps 2 and 3.)
Looks like this one is related to online spelling checking. When the range is filtered even the filtered range is considered part of the visible range. The spell check then goes through each cell and calls for each one a repaint. There are several things we can do to improve the performance. The first one and most likely the easiest one is to skip hidden rows and columns. That one should already bring the performance to acceptable levels. Finally I think it is insane to call a repaint for all the wrongly spelled cells so we might need to investigate how we can limit that to one repaint call.
Markus Mohrhard committed a patch related to this issue. It has been pushed to "master": http://cgit.freedesktop.org/libreoffice/core/commit/?id=eadd75ef212b4dd1b43aeacb34c8dd3ab40df369 skip hidden rows/columns for spellchecking, tdf#99607 It will be available in 5.3.0. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Review request for 5-2 in gerrit.
Markus Mohrhard committed a patch related to this issue. It has been pushed to "libreoffice-5-2": http://cgit.freedesktop.org/libreoffice/core/commit/?id=64c25f120cd96cb6c1c232cc86661d9927840851&h=libreoffice-5-2 skip hidden rows/columns for spellchecking, tdf#99607 It will be available in 5.2.3. The patch should be included in the daily builds available at http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: http://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.