Bug 148864 - Kubuntu 22.04 LTS LibreOffice Impress 7.3.3 Fails to Play Embedded Videos
Summary: Kubuntu 22.04 LTS LibreOffice Impress 7.3.3 Fails to Play Embedded Videos
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Impress (show other bugs)
Version:
(earliest affected)
7.3.0.3 release
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:7.4.0 target:7.3.5
Keywords: bibisected, bisected, regression
: 149119 (view as bug list)
Depends on:
Blocks: KDE, KF5 Media
  Show dependency treegraph
 
Reported: 2022-04-30 10:13 UTC by NoWorries
Modified: 2022-06-17 07:39 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
Example of LO Impress embeded video being played when returned to it (19.15 KB, image/png)
2022-04-30 10:48 UTC, NoWorries
Details
LO Impress 7.3.3 embedded video wehn first played (139.96 KB, image/png)
2022-04-30 10:58 UTC, NoWorries
Details
LO Presentation showing it playing the first embedded video. Dual Monitor with Presentation of Rightning to it (578.77 KB, image/png)
2022-05-02 02:51 UTC, NoWorries
Details
When I go back and view the same video it is totally blank but has the correct sound (37.53 KB, image/png)
2022-05-02 02:57 UTC, NoWorries
Details

Note You need to log in before you can comment on or make changes to this bug.
Description NoWorries 2022-04-30 10:13:33 UTC
Description:
I have two Laptops with Kubuntu 22.04 LTS installed.  One I treat as my production Laptop which has LibreOffice Version 7.2.5.2 installed so that it cannot be replaced by the latest Version 7.3.3.  Both Laptops have the same multimedia packages installed.

The system with LO 7.2.5.2 plays Impress files with embeded videos perfectly.

The system with LO 7.3.3 Impress fails to play embeded Videos.  The first video is played perfectly, however when I advance to the next embeded video there is no image, ie a blank screen, but the sound associated with the video plays.

Even if I step back to the video that previously worked it fails in the same way.

I hope I can add images to illustrate this.  The first is the Video being played and the second is the video not being played when I step back to it.



Steps to Reproduce:
1.Start LO Impress 6.3.3 and select a black background for all slides and add say 4 slides with no text boxes.
2.On slide 2 select a mp4 video to insert.  Set its page size and position so that it fits within the slide window.
3.Use F5 to start the presentation. Advance so that the video is played on the presentation screen and then advance to the third slide.
4.Now step back to the 3rd slide and you will find that no video is shown but the audio associated with the video is played.

Actual Results:
No video is played after first successfully playing the first video.  All subsequent videos or the first video will not play.  Only the sound of the video is played. 

Expected Results:
I expected to see the video played again with the image and sound.  This is done correctly on LO Version 7.2.5.2.


Reproducible: Always


User Profile Reset: No


OpenGL enabled: Yes

Additional Info:
I reported this problem recently and it was associated with LO Impress 4.0.2.1 which showed a white flash before the start of a video.

This is NOT the case in this fault report, nor is it related to Windows 10 LO system.
Comment 1 NoWorries 2022-04-30 10:48:04 UTC
Created attachment 179859 [details]
Example of LO Impress embeded video being played when returned  to it
Comment 2 NoWorries 2022-04-30 10:58:57 UTC
Created attachment 179860 [details]
LO Impress 7.3.3 embedded video wehn first played

This example shows the first time the embedded video is played.  I have two screens where the projection is shown on the right side and the control is on the left side. This shows that it is slide 2, the same one shown where the video does not work.
Comment 3 NoWorries 2022-05-02 02:51:17 UTC
Created attachment 179883 [details]
LO Presentation showing it playing the first embedded video.  Dual Monitor with Presentation of Rightning to it

This image for the first video being played correctly.  I have found that if I load again it is not shown, ie blank image with correct sound.
Comment 4 NoWorries 2022-05-02 02:57:28 UTC
Created attachment 179884 [details]
When I go back and view the same video it is totally blank but has the correct sound

This has the same monitor setup as the first attachment.  As you can see the speaker is now not shown on the control monitor
Comment 5 Michael Weghorn 2022-05-03 19:11:38 UTC
I can reproduce, and on master, even the initial video playback doesn't work.

This can also be reproduced as follows:

1) open attachment 145517 [details] from bug 120452 with LO with kf5 VCL plugin
2) start slide show by pressing F5
3) advance by pressing button "right"
4) move back to the first slide by pressing the button "left"

On current master, no video is played in either step 2 nor step 4.

Bibisecting shows that the behavior changed in 3 steps/commits:

Playback works in both, 2) and 4) before

    commit 76611452da46aec18ba14954671beee4f6dacf49
    Author: Jan-Marek Glogowski
    Date:   Wed Nov 17 13:03:23 2021 +0100

        Qt refactor SystemEnvData setup
        
        Change-Id: I900d1079c9a832a9b5170e58ce4e7a8b81d7d01b
        Reviewed-on: https://gerrit.libreoffice.org/c/core/+/125393
        Reviewed-by: Michael Weghorn
        Tested-by: Jenkins

After above commit, neither 2) nor 4) work, but after the exactly next commit on master, 

    commit f3bd9ef889d1834f7b1feb49197a58e679826c00
    Author: Jan-Marek Glogowski
    Date:   Wed Nov 17 13:26:09 2021 +0100

        Qt resolve native winId() on demand
        
        As for gtk3 in commit ac9789dbb36f45dcc1caf7dd2951353b1574c8ea
        ("tdf#139609 avoid fetching unnecessary xid under gtk3").
        
        Change-Id: I82b2c22437e5ab957706c25fcc118b28abb07242
        Reviewed-on: https://gerrit.libreoffice.org/c/core/+/125395
        Reviewed-by: Michael Weghorn
        Tested-by: Jenkins

2) works again, while 4) still doesn't (which is the behavior described in this bug report), and then starting at

        commit dc886bc6de2c0061a840bea2426663c3be2ecd26
        Author: Jan-Marek Glogowski
        Date:   Tue Apr 19 16:03:56 2022 +0200

            tdf#140463 Qt handle mouse enter+leave events
            
            Currently just implemented for the QtWidget, but still as a static
            function, so it may be used for QtObject at some point too.
            
            But there is no (mouse) enter or leave event function in QWindow,
            so no way to handle these there. And since we can't modify the
            returned QWidget from QWidget::createWindowContainer, the only way
            would be to expand the static QtWidget::handleEvent used by
            QtObjectWindow::event ... if it's actually needed at some point.

neither 2) nor 4) work again.

Adding CC: to Jan-Marek Glogowski



Version: 7.4.0.0.alpha0+ / LibreOffice Community
Build ID: 02a5bd2e6c4eee50cbd8cb4ed009731c409711a5
CPU threads: 12; OS: Linux 5.17; UI render: default; VCL: kf5 (cairo+xcb)
Locale: en-GB (en_GB.UTF-8); UI: en-US
Calc: threaded
Comment 6 Jan-Marek Glogowski 2022-05-23 12:51:12 UTC
(In reply to Michael Weghorn from comment #5)
>     commit 76611452da46aec18ba14954671beee4f6dacf49
>     Author: Jan-Marek Glogowski
>     Date:   Wed Nov 17 13:03:23 2021 +0100
> 
>         Qt refactor SystemEnvData setup

Had just some time looking into this and at least I can confirm that in the bibisect repo this commit breaks playback. For master, initial playback works here, but not going back to the first slide, resulting in a black box instead of a video (but my master has a lot of changes for the per-screen scaling, but that is nothing Qt related AFAIK). Changing the QtWidget Enter/Leave code changes nothing here.

And I just spend quite a bit of time staring at the refactoring code and it's small and nothing looks remotely wrong with it. Actually adding the winId() calls to the constructors of QtFrame and QtObject after the QtFrame::FillSystemEnvData breaks playback for me...

All this smells a lot like some race condition, but anything should run in the main thread, so there can't be a race (FWIW I need to verify this later). I'm really puzzled, as nothing seems to make sense.
Comment 7 Michael Weghorn 2022-05-24 05:30:08 UTC
(In reply to Jan-Marek Glogowski from comment #6)
> Had just some time looking into this and at least I can confirm that in the
> bibisect repo this commit breaks playback. For master, initial playback
> works here, but not going back to the first slide, resulting in a black box
> instead of a video (but my master has a lot of changes for the per-screen
> scaling, but that is nothing Qt related AFAIK).

I retested: When using the 7.4 bibisect repo, I initially see a light gray rectangle in the place where the video is supposed to be. However, initial playback works fine with a self-compiled master, the behavior is the same for me as the one you described.
Comment 8 Jan-Marek Glogowski 2022-05-24 06:39:15 UTC
FWIW the playback in the Impress frame is also broken before the commits. My ~/.xsession-errors has many "qt.qpa.xcb: QXcbConnection: XCB error:..." entries with BadDrawable and BadWindow entries.

The result (working or broken overlay) already changes here, if I add just a SAL_DEBUG(std::hex << rData.GetWindowHandle(this)); to QtFrame::ResolveWindowHandle. And a RunInMain just adds to the fragility; sometimes it works with some other combination, like commenting the "pChild->setAttribute(Qt::WA_NativeWindow);" call.

This looks more like the whole GStreamer X11 overlay approach already just was working with Qt by pure chance.

At least that winId() refers to some existing Window from "xwininfo -tree -root", but interestingly not the leaf window on playback. AFAIK avmedia creates this subwindow for the overlay and there might be a problem with the returned AWT window from (X)Player::createPlayerWindow in Qt.

Oh, and gtk3 does use some gtk specific video sink, not direct X11 overlay. gtk4 has it's own player implementation in avmedia/source/gtk using the Gtk 4 multimedia functions; I had some other bug where I mentioned to use Qt multimedia instead of direct gstreamer. Both toolkit abstraction work probably better generally and it would work eventually in other platforms.

Playback in Impress is generally broken for gen and qt, because the internal playing window doesn't resize the gstreamer overlay on its resize, so plays with a 1x1 overlay per default. I already fixed that, but again it works in gen, but Qt results in brokenness, where now the position is relative to the whole Impress window, but the overlay clipping happens based on the QtObject window, so I see the lower right corner of that video in the upper left corner of the video frame. Mostly. And the video is actually not playing for Qt, just gen.

And clicking on the overly in Qt sometimes moves it and might even crash LO. Ohh, and I just hit DBG_TESTSOLARMUTEX in gdb abort in WindowOutputDevice::AcquireGraphics... so some MT stuff might be the real bug; lets see where that path leads to. That abort is definitely a fixable bug.

BTW: all this is just talking about the actual visual video. The media plays correctly, audio always works.
And the sidebar slider isn't updated and the button states are wrong while playing, but that is more of a nuisance, compared to the broken playback.
Comment 9 Michael Weghorn 2022-05-24 09:39:57 UTC
(In reply to Jan-Marek Glogowski from comment #8)
> Oh, and gtk3 does use some gtk specific video sink, not direct X11 overlay.

qt5 already uses qwidget5videosink for Wayland. In a quick testing, commenting out the 

    if (pEnvData->platform != SystemEnvData::Platform::Wayland)
        return nullptr;

check in QtInstance::CreateGStreamerSink didn't result in a working result either. (The behavior is slightly different, though: I see a single "black rectangle" when moving back to the first slide, while there are 2 without that change in place.)

IIRC, gtk3 also falls back to the x11 video sink if the gstreamer plugins containing the gtk3 sink are not installed.

> gtk4 has it's own player implementation in avmedia/source/gtk using the Gtk
> 4 multimedia functions; I had some other bug where I mentioned to use Qt
> multimedia instead of direct gstreamer. Both toolkit abstraction work
> probably better generally and it would work eventually in other platforms.

Yes, that sounds like a good way forward (at least in the long run), and was also one idea I mentioned in tdf#145735. (I also have some WIP branch for using waylandsink for qt6 on Wayland, but ran into various problems there.)
Comment 10 Commit Notification 2022-05-25 11:54:44 UTC
Jan-Marek Glogowski committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/4366e0605214260e55a937173b0c2e02225dc843

tdf#148864 Qt switch QtObjectWindow to QWidget

It will be available in 7.4.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 11 Jan-Marek Glogowski 2022-05-25 12:27:17 UTC
*** Bug 149119 has been marked as a duplicate of this bug. ***
Comment 12 Commit Notification 2022-05-25 14:26:05 UTC
Jan-Marek Glogowski committed a patch related to this issue.
It has been pushed to "libreoffice-7-3":

https://git.libreoffice.org/core/commit/70e497f786dad6bda3aa593c07c778b06381c0e4

tdf#148864 Qt switch QtObjectWindow to QWidget

It will be available in 7.3.5.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 13 NoWorries 2022-06-17 07:26:11 UTC
I downloaded the file LibreOfficeDev_7.3.5.0.0_Linux_x86-64_deb.tar.gz dated 2022-Jun-17 and installed the extracted deb files on an ASUS R501VM N56VM.  This Laptop has Kubuntu 22.10 installed.

I tested Impress 7.3.5 on files with embedded videos and it worked perfectly. I could not get it to fail and I am very pleased with its performance. 

As I currently have LibreOffice 7.2.5.2 installed on my Laptop which I use for presentations, I look forward to the day when 7.3.5 is available on the ubuntu pre-release files for installation.

Thanks very much for getting 7.5.3.0 to work so well!
Comment 14 Michael Weghorn 2022-06-17 07:39:17 UTC
Thanks for testing! Setting to VERIFIED.