Description: On a mac, in the current nightly build, the LibreOffice UI refreshes with a low frame rate and cannot keep up with rapid mouse movements, resulting in a laggy experience. For example, when resizing columns in Calc, or dragging to change a text selection in Writer, the LibreOffice UI only updates to reflect the mouse pointer a few times a second, resulting in a very laggy experience. This is a regression: with 24.8.3.1, the LibreOffice UI is smooth and keeps up with my mouse however fast I move it. See video that I will attach. Possibly related to https://bugs.documentfoundation.org/show_bug.cgi?id=157312#c19 ? Steps to Reproduce: 1.In Calc, drag a column header boundary backwards and forwards quickly. Actual Results: The line showing the provisional new column size updates only a few times a second, not keeping up with rapid mouse movements. Expected Results: The line smoothly follows the mouse during the drag operation. Reproducible: Always User Profile Reset: No Additional Info: Skia is enabled. Skia log: RenderMethod: metal Version: 25.2.0.0.alpha0+ (AARCH64) / LibreOffice Community Build ID: 503d7594e3ac32e26752ea294dacf255f66c061c CPU threads: 8; OS: macOS 15.1; UI render: Skia/Metal; VCL: osx Locale: en-GB (en_GB.UTF-8); UI: en-US Calc: threaded
Created attachment 197671 [details] Video demonstrating the issue Nightly build on the left; 24.8.3.2 on the irght
Only happens if skia rendering is enabled in preferences.
I can reproduce this in my local master build. What caught my eye is that in your video while using the nightly build, I see rapid drawing of thin white vertical lines in the column headers when you dragged the mouse rapidly right and left. So this looks like some drawing timer is not getting a chance to run. If I remove commit 5a38e4f9798c5ff247aa57581adf2671485627fd, the vertical lines in the cell area keeps up with the thin white vertical lines in the column headers so I am guessing that the Skia "flush to native window" timer causes this bug when that timer's priority is set to "HIGHEST". In LibreOffice 24.8.3, the Skia "flush to native window" timer is still set to "POST_PAINT" which is a much lower priority than in the nightly build. So, I will try to see if setting the Skia flush timer priority between "HIGHEST" and "POST_PAINT" will fix this bug without causing tdf#163734 to reoccur.
Patrick Luby committed a patch related to this issue. It has been pushed to "libreoffice-24-8": https://git.libreoffice.org/core/commit/a68e25937f719e4eed938969e458b126d44748e2 tdf#163945 Lower priority of Skia flush timer It will be available in 24.8.4. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Patrick Luby committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/dbe8a75f7933d3f63a6fdd54f0c562a944b53975 tdf#163945 Lower priority of Skia flush timer It will be available in 25.2.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
I have committed a fix for this bug. The fix will be in tomorrow's (19 November 2024) nightly master builds: https://dev-builds.libreoffice.org/daily/master/current.html Note for macOS testers: the nightly master build installer does not overwrite any LibreOffice official versions. Instead, it will be installed as a separate application called "LibreOfficeDev" in the /Applications folder. Because this is a "test" build, you will need to do the following steps before you launch the LibreOfficeDev application: 1. Go to the Finder and navigate to the /Applications/Utilities folder 2. Launch the "Terminal" application 3. Paste the following command in the Terminal application window and press the Return key to execute the command: xattr -d com.apple.quarantine /Applications/LibreOfficeDev.app
Created attachment 197692 [details] Issue still visible in 2024-11-19 nightly build
I tested the nightly build 2024-11-06 05:23:09 https://dev-builds.libreoffice.org/daily/master/MacOSX-aarch64@tb92-TDF/2024-11-06_05.25.21/LibreOfficeDev_25.2.0.0.alpha0_MacOS_aarch64.dmg The rendering is improved but still much less fluid than 24.8.3.2. Most noticeable when dragging a cell selection (or text selection in Writer). The blue selection does not anything like keep up with the mouse in the nightly build. I have uploaded a second video demo.
Sorry, previous comment should say "I tested the nightly build 2024-11-19 04:24:04 https://dev-builds.libreoffice.org/daily/master/MacOSX-aarch64@tb94-TDF/2024-11-19_04.26.34/LibreOfficeDev_25.2.0.0.alpha0_MacOS_aarch64.dmg " Version: 25.2.0.0.alpha0+ (AARCH64) / LibreOffice Community Build ID: 3e9658a201f60dee95bd1bd8421b18bf8905c308 CPU threads: 8; OS: macOS 15.1; UI render: Skia/Metal; VCL: osx Locale: en-GB (en_GB.UTF-8); UI: en-US Calc: threaded
Unfortunately, I can't restore the LibreOffice 24.8.3 behavior with causing tdf#163734. I suspect that this bug will have to be solved somewhere deep in the Calc code. I am not familiar with the Calc code and all of its various timers (my experience is limited to macOS event handling and drawing) so unassigning myself. We probably need a Calc expert to track down which Calc repaint timers are competing with the Skia flush timer.
(In reply to Seb from comment #8) > Most noticeable when dragging a cell selection (or text selection in > Writer). The blue selection does not anything like keep up with the mouse in > the nightly build. FWIW, I don't see any lag like in your second video when I rapidly change selected cell range by dragging on my trackpad in a circle. I tried this with both the 2024-11-19 AARCH64 build (the same as the one you tested) as well as with my local master build and, for me, the selected cell range repaints about as fast as LibreOffice 24.8.3.1: Version: 25.2.0.0.alpha0+ (AARCH64) / LibreOffice Community Build ID: 3e9658a201f60dee95bd1bd8421b18bf8905c308 CPU threads: 8; OS: macOS 15.1; UI render: Skia/Metal; VCL: osx Locale: en-CA (en_CA.UTF-8); UI: en-US Calc: threaded
Thanks Patrick. Just to note it is not Calc-specific. Will attach another video of the Writer text selection in Writer not keeping up with even quite modest mouse movements. And I just tried a Presentation. Interestingly, dragging a text box to move it around was responsive, but dragging the select tool was terrible. If the mouse is moving even moderately fast, the selection rectangle doesn't refresh at all. It waits until the mouse slows down before resizing. Yet another video to upload as it's quite dramatic to see and I think would be very undesirable to release in this state. (I confirmed it does not happen in 24.8.3.2). So seems to be something specific to updating the "provisional selection" when dragging. This is on an M2 MacBook Air so no shortage of CPU/GPU resource.
Created attachment 197693 [details] Demonstrating laggy text selection in Writer (2024-11-19 nightly build)
Created attachment 197694 [details] Demonstrating selection box in Presentation very reluctant to refresh when dragging
Just to add, in Presenter, if I drag a new rectangle using the rectangle tool, it's perfectly responsive. It is only the selection tool (and it seems 'drag to select' everywhere) that does not refresh frequently enough.
(In reply to Seb from comment #12) > So seems to be something specific to updating the "provisional selection" > when dragging. I cannot reproduce the behaviors in your Writer and Impress videos with either the 2024-11-19 nightly build or my local master build. Is anyone else seeing the behavior in that attached videos? It's just a wild guess, but in the videos LibreOffice is acting like it isn't able to keep up with the flood of mouse dragged events. So, LibreOffice skips many painting timers and just does processing of the backlog of events until the event queue is empty. Then all of the painting timers finally fire and the UI is updated. But it looks like we both have roughly the same hardware and the fix works on my machine but not yours. Maybe some other application or system service is hogging CPU, GPU, and/or some other system resource on your machine? Or maybe you are using some macOS accessibility applications (I only use Rectangle) and/or LibreOffice extensions?
(In reply to Patrick (volunteer) from comment #16) > But it looks like we both have roughly the same hardware and the fix works > on my machine but not yours. Maybe some other application or system service > is hogging CPU, GPU, and/or some other system resource on your machine? Or > maybe you are using some macOS accessibility applications (I only use > Rectangle) and/or LibreOffice extensions? Or clipboard/pasteboard management applications?
I'm not seeing any lag on my Intel Macbook Version: 25.2.0.0.alpha0+ (X86_64) / LibreOffice Community Build ID: 3746195ff989a4596afc456d53010d8860bd7b64 CPU threads: 8; OS: macOS 14.5; UI render: Skia/Metal; VCL: osx Locale: nl-NL (nl_NL.UTF-8); UI: en-US Calc: threaded Does forcing Skia/Raster help? LibreOffice -> Preferences -> View -> Check Force Skia Software rendering
Some new observations: 1. The issue goes away with 'Force Skia Software rendering' enabled. 2. It turns out I had 'Force Skia Software rendering' enabled in my non-Dev LibreOffice settings (probably left over from working around #157312). Turning that off also introduces the laggy selection refreshing in 24.8.3.2. Apologies for not noticing that sooner. So this is not in fact a regression between 24.8.3.2 and nightly, but some issue with skia hardware rendering that affects both versions. I don't have any LibreOffice extensions, and am not using any Accessibility tools/settings or clipboard management apps. I have rebooted, and confirmed the issue with nothing else running.
(In reply to Seb from comment #19) > 2. It turns out I had 'Force Skia Software rendering' enabled in my non-Dev > LibreOffice settings (probably left over from working around #157312). > Turning that off also introduces the laggy selection refreshing in 24.8.3.2. That's interesting. I would've never guessed that Skia/Raster would be more responsive than Skia/Metal. Given this new piece of data, I no longer think this is a bug in LibreOffice internal painting timers. After all, LibreOffice uses the same drawing code and paint timers for both Skia/Metal and Skia/Raster. So here's my new theory: Google's Skia/Metal code doesn't copy pixels to the screen as frequently as Skia/Raster. When I get some time, I'll try to dig down into Google's Skia code and look at how Skia/Metal copies pixels to the screen. Some background about LibreOffice and Skia: the LibreOffice paint timers do all their drawing to an offscreen image in memory. Then, LibreOffice waits for macOS to invoke a callback function that copies the pixels in the offscreen image to the native window. This two step process is necessary because it is only safe to draw to a native window when the callback function is invoked by macOS. Drawing to a native window any other time usually leads to a crash and/or erratic window behavior. I know that when using Skia/Raster or Skia disabled, macOS calls LibreOffice's -[SalFrameView drawRect:] function. But it appears that that function never gets called when using Skia/Metal. So, when I get some time, I'll dive into Google's Skia/Metal code and see if I can figure what what function macOS calls when it is safe to copy the pixels to the native window. I suspect that the callback function is somewhere in the Skia/Metal code but I don't know. Hopefully, if I can find the Skia/Metal callback function, I can then see if we can tell macOS to run this callback more frequently. Right now, LibreOffice calls -[SalFrameView setNeedsDisplayInRect:] to tell macOS that it wants to copy pixels to the native window. This seems to be enough for Skia/Raster and Skia disabled, but maybe this has no effect when using Skia/Meta.
Well, I can actually reproduce comment 1 and comment 14. No clue how I tested it yesterday... I can clearly see the label showing the column width flashing massively when following comment 1 Version: 25.2.0.0.alpha0+ (X86_64) / LibreOffice Community Build ID: 3746195ff989a4596afc456d53010d8860bd7b64 CPU threads: 8; OS: macOS 14.5; UI render: Skia/Metal; VCL: osx Locale: nl-NL (nl_NL.UTF-8); UI: en-US Calc: threaded
Patrick Luby committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/32eb29377c5bbf6a965357756d2813add2789053 tdf#163945 present drawable immediately It will be available in 25.2.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
I have committed the following possible fix for Skia/Metal. The fix will be in tomorrow's (22 November 2024) nightly master build. See comment #6 for the download link and installation note: https://gerrit.libreoffice.org/c/core/+/176879 Not sure if it will fix this bug but I was able to reproduce this bug if I set the "present drawable" time to "now + 0.2 seconds". So hopefully setting the time to "now" will do the opposite and force Metal to update the native window sooner.
I tested the nightly build of 2024-11-22 https://dev-builds.libreoffice.org/daily/master/MacOSX-aarch64@tb94-TDF/2024-11-22_04.21.38/LibreOfficeDev_25.2.0.0.alpha0_MacOS_aarch64.dmg Version: 25.2.0.0.alpha0+ (AARCH64) / LibreOffice Community Build ID: d3613d15e518c0c8cc4930c4bdf37219813fd665 CPU threads: 8; OS: macOS 15.1.1; UI render: Skia/Metal; VCL: osx Locale: en-GB (en_GB.UTF-8); UI: en-US Calc: threaded Unfortunately it does not fix the selection refresh issue. I couldn't detect any difference.
Patrick Luby committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/97c0797cf379cca6056b8ab34499b56c7f01fd9b Revert "tdf#163945 present drawable immediately" It will be available in 25.2.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Created attachment 197729 [details] Xcode Instruments Performance trace resizing a column in calc Version: 25.2.0.0.alpha0+ (X86_64) / LibreOffice Community Build ID: 3746195ff989a4596afc456d53010d8860bd7b64 CPU threads: 8; OS: macOS 14.5; UI render: Skia/Metal; VCL: osx Locale: nl-NL (nl_NL.UTF-8); UI: en-US Calc: threaded
Created attachment 197730 [details] Xcode Instruments time profile rapidly changing selected Calc cells with all toolbars, status bars, and sidebar hidden
(In reply to Telesto from comment #26) > Created attachment 197729 [details] > Xcode Instruments Performance trace resizing a column in calc Thank you for the Instruments output. Even though I cannot reproduce this bug, I was trying to use Instruments earlier today to see there were any obvious places in the LibreOffice or Skia code that is using a high percentage of the CPU time. In your output, LibreOffice's slowest stack is in its "draw a native control" code so I tried to turned off automatic spellchecking, hid every toolbar, hid the status bar, and closed the sidebar since those all contain native controls (e.g. buttons, comboboxes, etc.). I then did an Instruments Time Profile while rapidly changing the selected cells using the trackpad since resizing a column in the column header displays tooltips which appear to also draw native controls. After doing the above, I got the Instruments output in attachment #197730 [details]. In my output, LibreOffice's slowest stack is in SkiaSalGraphicsImpl::flushSurfaceToWindowContext() (see row 24). This is the function where LibreOffice copies the Skia/Metal offscreen image to the native window. So I expanded that function before I copied it from Instruments (see rows 25 through 36) and it looks like most of time in that function is spent in Skia's GrDirectContext::flushAndSubmit(GrSyncCpu) (see row 25). I would be interested to see if you get similar results as me (and is the CPU usage noticeably worse than mine) if you hide native controls and disable automatic spellchecking. Or maybe we'll find that drawing native controls when using Skia/Metal is the problem. If you
Created attachment 197733 [details] Calc configuration to avoid drawing most native controls Notes: 1. Open new Calc document 2. Uncheck the following menu items: View > Formula Bar View > Status Bar View > Sidebar 3. Unchecked the Tools > Automatic Spell Checking 4. Click and drag to select a rectangle of cells but continue dragging in a circle rapidly
(In reply to Patrick (volunteer) from comment #28) > I would be interested to see if you get similar results as me (and is the > CPU usage noticeably worse than mine) if you hide native controls and > disable automatic spellchecking. Or maybe we'll find that drawing native > controls when using Skia/Metal is the problem. > > If you I forgot to ask if anyone can try running a recent nightly build with the same Calc configuration as me. @Telesto's Instruments profile in attachment #197729 [details] shows a lot of CPU is being used to redraw the native buttons in the toolbars and tooltip windows. So, if any of you can run my steps in comment #29, does this bug still occur? Even if it appears noticeably faster or slower, that would be good to know as well.
Created attachment 197737 [details] Screencast resizing column with flickering tooltip
Created attachment 197738 [details] Instruments CPU profile with setup seen in screencast Version: 25.2.0.0.alpha0+ (X86_64) / LibreOffice Community Build ID: 3746195ff989a4596afc456d53010d8860bd7b64 CPU threads: 8; OS: macOS 14.5; UI render: Skia/Metal; VCL: osx Locale: nl-NL (nl_NL.UTF-8); UI: en-US Calc: threaded > 4. Click and drag to select a rectangle of cells but continue dragging in a > circle rapidly Nothing interesting here for me, performance is fine. Doing this causing a lag is specific for Seb his setup.. No painting of native controls either. For me it's just the 'tooltip' when resizing the column (as shown in screencast) Drawing the tooltip becomes even slower with instruments profile recording running.
I have some new information, but the various issues are getting a bit mixed up so let me clarify what I see on my machine. All of this is with skia/metal except where stated. ** Issue A: When resizing a column by dragging, the vertical line position does not keep up (left hand side of video in comment#1). - status: seems fixed in nightlies, I think by https://bugs.documentfoundation.org/show_bug.cgi?id=163945. Resolved. ** Issue B: When resizing a column by dragging, the "Width: xxx" tooltip flickers or disappears - In 24.8.3.2, my tooltip is 'dimmed' by flickering when dragging at modest speed. i.e. a good fraction of the frames are drawn without the tooltip. You can see this clearly on the video I am about to upload, by stepping through the video frames: many show the window rendered without the tooltip. - In 24.8.3.2 with skia software rendering, the tooltip is pretty stable but not perfect - every so often there is a glitch frame where it is not drawn. Visible on video. - In 2024-11-23 nightly, there are similar 'no tooltip frames' at low dragging speed. But at faster (but still reasonable) speeds, the tooltip disappears completely. - summary: The complete disappearance of the tooltip at faster drag speed is a regression. But Issue B is pretty minor TBH, I kind of expect tooltips to be glitchy :) ** Issue C: When dragging to make a cell selection in Calc, the block of cells that is highlighted does not refresh frequently enough to keep up with the pointer **when View->Headers is enabled**. - 2024-11-23 nightly: Occurs with skia/metal **but only when View->Headers** is enabled. When resizing the selection, the column/row headers that contain the selection are also highlighted. These header highlights refresh much faster than the cells themselves, but they do slightly lag behind the pointer (when compared to 24.8.3.2), as if they are only just able to keep up. My theory is that LO is spending too much time rendering the headers to have time to render the cell highlights. - 24.8.3.2: **does not occur** (even with skia/metal and Headers). So this one is in fact a regression. ** Issue D: In Impress, when dragging a selection box on an empty background, the selection box does not keep up with the pointer, and does not refresh at all if the pointer is moving quickly **when View->Status Bar is enabled**. - 2024-11-23 nightly, this only happens if the Status Bar is visible. I think the rapid redrawing of the selection size in the status bar is preventing the selection box from redrawing. Only happens with metal. - 24.8.3.2: same. ** Issue E: In Writer, when dragging to make a text selection, the selected text highlight does not keep up with the pointer. - Only occurs in Nightly (not 24.8.3.2), so this is regression - Still occurs with all toolbars and status bars and ruler disabled (only the document visible) Common themes: - seems like refreshing the Column/Row headers and/or status bar is is (a) slow/glitchy and (b) inhibiting refresh of the main selection. The exception seems to be Issue E (Writer) where it happens even with nothing visible except the document. I will upload videos of most of these cases as they are quite informative to study. Also want to say I am a developer (on other platforms) so if it would be very helpful for me to run profiling or try out speculative changes before they hit master then I could get set up to do that.
Created attachment 197745 [details] Column width tooltip flicker 24.8.3.2 metal
Created attachment 197746 [details] Column width tooltip flicker Nightly 2024-11-23 metal
Created attachment 197747 [details] Column width tooltip flicker 24.8.3.2 - software
Created attachment 197748 [details] Impress selection box - Nightly 2024-11-23 - metal
Writer video too large to upload: it's here https://www.dropbox.com/scl/fi/soz2b2087kksoogwa59j2/Writer-text-selection-both-versions-metal.mov?rlkey=c2wa2006b5uzkkopoecuezvz4&dl=0
(In reply to Seb from comment #33) > I have some new information, but the various issues are getting a bit mixed > up so let me clarify what I see on my machine. All of this is with > skia/metal except where stated. > Thank you for your detailed analysis. I cannot reproduce most of what is in your list but sometimes I see the "no tooltip appears" when resizing a Calc column via dragging. I can try reverting any recent Skia changes in my local build and see if I can make that specific bug disappear. If that doesn't uncover any specific change, we may need someone who can reproduce most of the bugs to do a bibisect to find which change caused this. I know that a newer version of Google's Skia code was recently put in master but we would need a bibisect to find the change: https://wiki.documentfoundation.org/QA/Bibisect (In reply to Seb from comment #33) > Also want to say I am a developer (on other platforms) so if it would be > very helpful for me to run profiling or try out speculative changes before > they hit master then I could get set up to do that. That is very good to hear. If you Xcode installed, you should be able to connect Xcode's Instruments application or lldb to a running nightly build. No need to install any third party packages. It won't work for release builds but for a nightly or local build, debugging and profiling are enabled by default. Here are the steps I use: 1. Launch a nightly or local build and get the application in the state you want to test (e.g. open an empty Calc document and set the selected cell) 2. In a Terminal application window, run the following command: open /Applications/Xcode.app/Contents/Applications/Instruments.app 3. In Instruments, select "Time Profiler" and in the window that appears, select "soffice" to connect to LibreOffice and press the red Start icon 4. Do your testing in LibreOffice (e.g. change a column width rapidly) 5. In Instruments, press the Stop icon If you are interested in doing a local build, you could try out possible fixes without having to go through the "commit and wait for the next nightly build" process. That might be useful for joint debugging as well. I use LibreOffice's LODE environment to build LibreOffice. The thing I really like about LODE is that it downloads all of the third party packages automatically and puts them in a LODE folder so it doesn't pollute your existing macOS configuration. With LODE, you only need to install Xcode and a Java Development Kit (JDK): https://wiki.documentfoundation.org/Development/lode
(In reply to Seb from comment #37) > Created attachment 197748 [details] > Impress selection box - Nightly 2024-11-23 - metal Good news: I can reproduce this specific case. With the status bar visible, Skia/Metal updates the rectangle significantly slower than Skia/Raster. With Skia/Raster, the rectangle is updated immediately with the status bar visible so, at least in this case, I think this confirms what @Telesto saw: high CPU when drawing native controls with Skia/Metal. I'll start debugging the "draw native control" code to see if I can narrow down where the slow code is.
Well I started off bibisecting using the mac64-25.2 repo, but discovered there are no changes to any of the issues between the earliest and latest commit it covers. So to unpick what is going on I tested almost every nightly build between 202-11-09 and 2023-11-23, plus various release builds, and discovered: === Issue E (Impress selection drag box doesn't keep up if status bar enabled): - present in every build I have tested, including 24.2.7.2, 24.8.3.2, the whole range of bibisect repo mac64-25.2, and all recent nightly builds. Seems this is a longstanding issue, not a recent regression. Happy to test older version if you want to suggest any (when was skia/metal introduced to LO?) Should we move this to a separate bug report since it has a distinct symptom and appears to have a different cause in the code? === Issue A (vertical line doesn't keep up when dragging to resize column): - First appeared in 2024-11-11 nightly, presumably due to 5a38e4f9798c5ff247aa57581adf2671485627fd "tdf#163734 Revert commit f4c2c7c79cfe4464ac596afda37b8904d06969db" - Then fixed in 2024-11-19 nightly, by dbe8a75f7933d3f63a6fdd54f0c562a944b53975 "tdf#163945 Lower priority of Skia flush timer" - Status in latest nightly: resolved. === Issues C and D (selected cells/text doesn't keep up with drag to select), and also the 'tooltip disappears' form of Issue B: - Also first appeared in 2024-11-11, presumably due to 5a38e4f9798c5ff247aa57581adf2671485627fd "tdf#163734 Revert commit f4c2c7c79cfe4464ac596afda37b8904d06969db" - Sadly not fixed in 2024-11-19 by "tdf#163945 Lower priority of Skia flush timer" - Nor in 2024-11-22 by "tdf#163945 present drawable immediately" - Still present in 2024-11-23. This is sad because it means the resolution to tdf#163734 is causing new issues which so far you have not been able to resolve.
(In reply to Seb from comment #41) > This is sad because it means the resolution to tdf#163734 is causing new > issues which so far you have not been able to resolve. Thanks for the testing! I had a feeling that commit 5a38e4f9798c5ff247aa57581adf2671485627fd was at least part of the cause. I was able to revert that commit and fix tdf#163734 elsewhere in the following patch: https://gerrit.libreoffice.org/c/core/+/177190 That patch should hopefully restore performance to the same as in LibreOffice 24.8.3. (In reply to Seb from comment #41) > Issue E (Impress selection drag box doesn't keep up if status bar enabled): No need for a separate bug. What I am seeing is that even with the above patch, Issue E is slow with the status bar displayed. So I disabled drawing of native controls drawing got significant faster. Not as fast as with Skia/Raster, but not much slower. That confirms what is in @Telesto's last Instruments sample: drawing native controls is very slow with Skia/Metal. So my next step is to see if I can find any optimizations to the native control drawing code.
(In reply to Patrick (volunteer) from comment #29) > Created attachment 197733 [details] > Calc configuration to avoid drawing most native controls > > Notes: > 1. Open new Calc document > 2. Uncheck the following menu items: > View > Formula Bar > View > Status Bar > View > Sidebar > 3. Unchecked the Tools > Automatic Spell Checking To make testing life more complicated: Something I did miss before. The above actually matters, performance resizing a column actually improves quite a bit by disabling stuff: especially standard and formatting toolbar and the sidebar. However only with an additional step 4 in between: restart of LibreOffice Disabling toolbars/ sidebar in the same session without a restart doesn't improve anything.
Patrick Luby committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/16b629cf08f93e27e52816c761eeb3b5c128647b tdf#157312 and tdf#163945 Lower Skia flush timer priority on macOS It will be available in 25.2.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
(In reply to Patrick (volunteer) from comment #42) > Thanks for the testing! I had a feeling that commit > 5a38e4f9798c5ff247aa57581adf2671485627fd was at least part of the cause. I > was able to revert that commit and fix tdf#163734 elsewhere in the following > patch: > > https://gerrit.libreoffice.org/c/core/+/177190 > > That patch should hopefully restore performance to the same as in > LibreOffice 24.8.3. I have committed the above patch and the fix should be in tomorrow's (25 November 2024) nightly master build. See comment #6 for the download link and installation note. Meanwhile, I will continue looking to see if I can do anything to speed up the slow native control drawing with Skia/Metal. That slowness is really obvious for me when testing Issue E with the status bar displayed.
(In reply to Patrick (volunteer) from comment #45) > (In reply to Patrick (volunteer) from comment #42) > > Thanks for the testing! I had a feeling that commit > > 5a38e4f9798c5ff247aa57581adf2671485627fd was at least part of the cause. I > > was able to revert that commit and fix tdf#163734 elsewhere in the following > > patch: > > > > https://gerrit.libreoffice.org/c/core/+/177190 > > > > That patch should hopefully restore performance to the same as in > > LibreOffice 24.8.3. Confirmed: in 2024-11-25 build, the regressions from 24.8.3.2 (Issues B, C, D) are gone (and A has not returned). Only E (which is not new) remains. > Meanwhile, I will continue looking to see if I can do anything to speed up > the slow native control drawing with Skia/Metal. That slowness is really > obvious for me when testing Issue E with the status bar displayed. Thank you for all your work on these issues and for being so responsive!
(In reply to Seb from comment #46) > Confirmed: in 2024-11-25 build, the regressions from 24.8.3.2 (Issues B, C, > D) are gone (and A has not returned). Only E (which is not new) remains. Glad to hear that the regressions are fixed. I just committed some performance improvements in the code that draws native controls and the commit should be in tomorrow's (26 November 2024) nightly master build. See comment #6 for the download link and installation note. This commit will hopefully make issue E less noticeable when using Skia/Metal: https://gerrit.libreoffice.org/c/core/+/177225 Issue E will still be slower with Skia/Metal than with Skia/Raster but faster than before. With my latest commit, Instruments is showing the time spent in AquaSkiaSalGraphicsImpl::drawNativeControl() cut in half with Skia/Metal.
I tested 2024-11-26, which I confirmed has your commit in it. Unfortunately there is no noticeable improvement. In Impress if I drag the select box at moderate speeds, with the Status Bar displayed, the box doesn't redraw, i.e. same as video in comment #14. I grabbed an Instruments profile made during which I was doing that kind of dragging. I will upload so you can peruse yourself, but I can see 41% of the time is in the DoPaint that is updating the 'size and position' text in the Status Bar, and another 30% in StatusBar::SetItemData. Both of these are inside ImplHandleMouseEvent. So either these are really slow or they are being called way too frequently. Can I throw in some naive thoughts (please excuse my uninformed speculation): - to reduce CPU load, could we only update the status bar text at a max rate, rather than on every mouse move event as we are perhaps doing now? - that said, it feels like with the current design (or my loose understanding of it), the skia buffer is never flushed if there is a UI (mouse move) event waiting to be handled. Which is bound to occur for some crazy systems (crazy high DPI, poor CPU/GPU, etc). Is there any way to insert periodic highest-prioirty skia flushes, even if mostly they are lower priority. So that even if the event loop is maxed out handling events, at least once in a while it does a skia flush? I realise you have to avoid regressing the other issues observed around the skia task priority though.
Created attachment 197809 [details] Instruments trace while rapidly resizing select box in Impress with Status Bar shown
A) Performance dragging columns in Calc has improved :-) Thanks! B) The performance issue described in comment 48 (and comment 14) is (still) reproducible and indeed related to the status bar combined with Skia/Metal FWIW: It would be somewhat cleaner to file separate (follow up) bug for the Impress part, IMHO. It's slightly nicer for bug fixing stats and release notes and readability of the bug report. Version: 25.2.0.0.alpha1+ (X86_64) / LibreOffice Community Build ID: 5e0c670e6534fee529ea46d520c6b442bea93aac CPU threads: 8; OS: macOS 14.5; UI render: Skia/Metal; VCL: osx Locale: nl-NL (nl_NL.UTF-8); UI: en-US Calc: threaded
(In reply to Seb from comment #48) > - to reduce CPU load, could we only update the status bar text at a max > rate, rather than on every mouse move event as we are perhaps doing now? Thanks for the Instruments trace file. My CPU usage for the drawBox() function was about half your usage. At least we got a little bit of savings in drawing time. Your idea makes sense to try. But to be honest, I get lost in the "application layers" of LibreOffice. My experience is in the "glue" layer that maps the firehose of events and drawing requests back and forth between macOS and LibreOffice's internal GUI framework (aka "vcl layer"). So I am lazy and just commented out the drawBox() function in vcl/osx/salnativewidgets.cxx. No drawing so, in theory, no CPU. But I run LibreOffice with Instruments Time Profiler connected and the shape outline updates immediately (or at least as fast as with Skia/Raster). But then I restart LibreOffice and run without Instruments connected and updating is noticeably slower and is somewhere between Skia/Raster and the nightly master build. And with drawBox() a noop, the heaviest stack in Instruments is the Skia/Metal timer flushing (i.e. pushing a snapshot of the offscreen Skia/Metal surface to the window's Metal layer). Since the Skia flushing timer is now back to POST_PAINT - a very low priority - it appears to me that the Skia flush timer is getting enough idle time to be fired as fast as the application layers posts painting timers. Maybe Issue E is due to a timer set to a very low priority in the Impress application layer? When I get some spare time, I'll look around and see if I can find the painting timer for the rectangle. Usually that, for me at least, is a tedious process because you have to try to find which code actually draws the thing you are debugging. So I usually grep for common names in C++ classes and methods and add printf's until I find the code. Then add a usleep(1) after the printf() and break in lldb to find the timer up the stack. Fun stuff. (In reply to Seb from comment #48) > - that said, it feels like with the current design (or my loose > understanding of it), the skia buffer is never flushed if there is a UI > (mouse move) event waiting to be handled. Which is bound to occur for some > crazy systems (crazy high DPI, poor CPU/GPU, etc). Is there any way to > insert periodic highest-prioirty skia flushes, even if mostly they are lower > priority. So that even if the event loop is maxed out handling events, at > least once in a while it does a skia flush? I realise you have to avoid > regressing the other issues observed around the skia task priority though. Do you see a drawing lag in the middle box of the status bar? When I drag as fast I can, I see the coordinates in the status bar rapidly updating. Not sure what the actual frame rate is but seems like the Skia timer is flushing frequently on my machine. With Skia/Metal, the entire window contents are updated by adding a new drawable (i.e. bitmap of LibreOffice's offscreen drawing buffer for the window) to macOS Metal's screen update queue. So there shouldn't be any issue with marking of dirty regions.
(In reply to Patrick (volunteer) from comment #51) > Thanks for the Instruments trace file. My CPU usage for the drawBox() > function was about half your usage. At least we got a little bit of savings > in drawing time. My trace was done with the 2024-11-26 build that has your change in it though. (If that's what you're suggesting is why your usage is half mine.) > Do you see a drawing lag in the middle box of the status bar? When I drag as > fast I can, I see the coordinates in the status bar rapidly updating. Not > sure what the actual frame rate is but seems like the Skia timer is flushing > frequently on my machine. Same - I see the numbers in the status bar updating rapidly, independently of the selection rectangle which is more or less frozen. So Impress "thinks" the rectangle is changing size fluently, it's just visually not updating. > With Skia/Metal, the entire window contents are updated by adding a new > drawable (i.e. bitmap of LibreOffice's offscreen drawing buffer for the > window) to macOS Metal's screen update queue. So there shouldn't be any > issue with marking of dirty regions. I see. If the whole window is one bitmap (and I am seeing rapid updating of the status bar) then presumably the problem is not the timing of the Skia flushes but - as you say - something to do with the frequency of updating the rectangle size. Is every element drawn every cycle (starting with a blank off-screen bitmap), or is the final bitmap composited from multiple layers/regions so things like select boxes can be updated without redrawing all widgets every cycle?
Patrick Luby committed a patch related to this issue. It has been pushed to "libreoffice-24-8": https://git.libreoffice.org/core/commit/514aef42f6d5cab539e5ba80bdc2de8fb51801ba tdf#157312 and tdf#163945 Lower Skia flush timer priority on macOS It will be available in 24.8.4. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
Created attachment 197837 [details] Debug patch that stops Issue E from occurring in Impress This isn't a fix for the bug. But what it demonstrates is that if I disable the "flush now" code that I add to fix tdf#155266, Issue E stops and dragging a rectangle in an Impress document with Skia/Metal is the same speed as with Skia/Raster. So, it appears that LibreOffice is flooding the Metal drawing queue which makes sense to me given this and that lowering the Skia flush timer's priority improved performance. Before I found that reducing flushing stops Issue E, I added a CALayerDelegate to the CAMetalLayer that Skia creates for each native window and added a displayLayer: delegate function in our NSView subclass that flushes to the screen. This seemed to double the number of flush calls and Issue E performance was noticeably worse so that is one more confirming piece of data. Interestingly, the displayLayer: delegate function only gets called after LibreOffice marks a portion or all of a native window as dirty by calling -[NSView setNeedsDisplay:] or -[NSView setNeedsDisplayInRect:] so the flooding is caused by the LibreOffice code calling those frequently. AFAICT, with Skia/Raster and Skia disabled, a rapid sequence of calls to either of the setNeedsDisplay gets coalesced into only one or a few calls to -[NSView drawRect:] which is where the dirty portion of the offscreen buffer gets copied to the screen. But Metal works differently. Instead of telling the window a region is dirty and then waiting for a callback from macOS to update the window's dirty region, with Metal you push a new snapshot of the offscreen buffer to the CAMetalLayer's drawing queue. The CAMetalLayer is designed like an animation engine: it doesn't draw directly to window like -[NSView drawRect:]. Instead, you feed it new "frames" like you are constructing a video. The CAMetalLayer then displays each frame in succession as LibreOffice pushes new snapshots of the offscreen buffer to CAMetalLayers frame drawing queue. So my current guess is that I need to throttle the number of frames (aka "drawables") that LibreOffice can push per second. Essentially, implement something like the coalescing that is done for Skia/Raster and Skia disabled. Since there is no coalescing occurring right now with Skia/Metal, I assume that Issue E pushes dozens of snapshots in a fraction of a second and CAMetalLayer gets backlogged displaying every single one of them sequentially. Then, when mouse moves stop, CAMetalLayer displays all of the backlogged frames in such quick succession that it appears to catch up by skipping a bunch of frames. Next step: add some debug code in Skia's Metal code and see if I can see any obvious place to throttle the number of flushes to the screen.
Patrick Luby committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/core/commit/c585b697b583fa0f8cdadeab594c31d270367ba7 Related: tdf#163945 don't directly flush graphics with Skia/Metal It will be available in 25.2.0. The patch should be included in the daily builds available at https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More information about daily builds can be found at: https://wiki.documentfoundation.org/Testing_Daily_Builds Affected users are encouraged to test the fix and report feedback.
I just committed a fix for the Impress selection box when using Skia/Metal and the commit should be in tomorrow's (30 November 2024) nightly master build. See comment #6 for the download link and installation note.
Tested 2024-11-30 nightly build on my machine and confirm Issue E is resolved too. Thank you Patrick!!