Description: If you look into bean/ folder in LibreOffice core source code, there is only "unix" and "win32" in bean/native. macOS is not supported, as JNI support is not implemented. The task here is to add a comparable implementation for macOS. Complexity The native implementation for OfficeBean is very small. It consists of 66 lines of C code for Linux/Unix and 113 lines of C code for Windows (measured with cloc). A comparable code for macOS is expected to be of similar size, around 100 line of C/Objective C code. Implementation It is expected to write a vcl/osx/com_sun_star_comp_beans_LocalOfficeWindow.c or .mm for macOS, which uses Cocoa API to create NSView, get its pointer and attach it to the Java window. Make sure that you keep the Java signature of the code, as they are used in JNI: /* * Class: com_sun_star_comp_beans_LocalOfficeWindow * Method: getNativeWindowSystemType * Signature: ()I */ Java_com_sun_star_comp_beans_LocalOfficeWindow_getNativeWindowSystemType /* * Class: com_sun_star_beans_LocalOfficeWindow * Method: getNativeWindow * Signature: ()J */ Java_com_sun_star_comp_beans_LocalOfficeWindow_getNativeWindow
Created attachment 205437 [details] tdf#170415 Add support for macOS in OfficeBean (native bridge) The connection was successful of the Java Bean Native Bridge. The window was created and embedded without crashing, However the content remains grey blank which on investigating, it points on visual class library (VCL). I used a test environment and isolated the root cause of the rendering failure. It is confirmed to be a conflict between Java AWT (NsView) which lacks Core Animation layer and the libreOffice VLC backend, which needs one but crashed when forced externally. I have also attached copies of screenshots, one with the grey window that appears but draws nothing and the other one with terminal output that shows, it is working. This is what I have understood so far regarding this issue, I would really value your feedback on this. Please correct me if I am missing something here. I would appreciate, since I know little about this.
Created attachment 205438 [details] Bug 170415 Could not attach two attachments together previously. (Terminal output) when attempting to force [view setWantsLayer:YES]. The immediate NoConnectionException confirms that while we can acquire the window handle, modifying this layer state externally causes a fatal conflict with the LibreOffice VCL backend."
So my memory is fuzzy but IIRC the OfficeBean would create a native window in its process, then launch OpenOffice or LibreOffice in subprocess, and then send the native window ID (not pointer) to the subprocess. This worked on Windows and Linux as processes can share native window IDs and their window managers don't seem to put any restriction on which process can draw. I only know macOS so that is what I remember from a meeting 20+ years ago so definitely double-check if things are different now. Anyway, do you have a link to your patch on Gerrit? I am guessing that the problem on macOS is that we can't share NSViews or NSWindows between processes. I found some forums on the web suggesting that -[NSWindow setSharingType:NSWindowSharingReadWrite] might work, but Apple has deprecated NSWindowSharingReadWrite starting in macOS so, even if that worked, it looks like is planning on removing that option.
Correction for comment #3: NSWindowSharingReadWrite is deprecated starting in macOS Tahoe. Anyway, another problem on macOS is that menus are attached to native windows on Windows and Linux but not on macOS. On macOS, the menubar is a shared system resource so only the application with focus gets to control it. So, if the OfficeBean application could pass window IDs to the LibreOffice subprocess and LibreOffice could draw to OfficeBean's window ID, we'd still have the problem that only the menus added by the OfficeBean process would appear since Libreoffice would be running in the background and would never get focus. One possible solution, although it might be too ugly and/or unwieldy, would be to launch LibreOffice as its own, detached visible process. Then, LibreOffice could get focus and its menus would display. I don't know what OfficeBean applications people have implemented in the wild, so maybe a possible downside is that the OfficeBean application developers really need precise control of where LibreOffice draws stuff on the screen.
197487: tdf#48140: Remove UNO calls from AutoFilter import in XLSX (WIP) | https://gerrit.libreoffice.org/c/core/+/197487 here is the gerrit patch. Also @Patrick Thank you for the valuable insight regarding the process boundary and window sharing limitations on macOS. Your explanation about the architectural differences compared to Windows/Linux clarified why the direct window pointer method is hitting a wall, and because of that I learned something new. Following your point about the process isolation, I looked up on web, how other cross-process applications handle this on macOS just to see what others are doing. I came across the Chromium Design Documents(https://www.chromium.org/developers/design-documents/chromium-graphics/mac-delegated-rendering/), which describe a similar challenge they faced with "Ubercompositor." They explicitly moved away from legacy window sharing to Delegated Rendering using IOSurface and CALayer hosting to solve the stability and "fighting for layer" issues. I also checked the Apple Developer Documentation (https://developer.apple.com/documentation/iosurface), which confirms that IOSurface is the designated API for sharing hardware-accelerated buffers across processes. Since I do not have enough architectural knowledge, I would like to know your thoughts on this CALayer Hosting method.
Using a shared IOSurface sounds like a very interesting approach. I read your links in comment #5 but the details of how you actually draw to IOSurface are still hazy to me. From what I read, it looks a bit complicated but maybe it would work like the following?: 1. In OfficeBean process, instead of sending the NSWindow or NSView pointers to LibreOffice, create a CAMetalLayer instance and assign it to the NSView. 2. In OfficeBean process, create a shareable IOSurface and send reference to it to LibreOffice via Mach ports or XPC or some other means? 3. In LibreOffice process, use Skia/Metal to do all drawing (don't know if Skia/Raster could be made to work or not) and modify the Skia code to swap in the shared IOSurface as the backing buffer for the CAMetalLayer that Skia creates. 4. In OfficeBean process, after LibreOffice has updated the IOSurface (via drawing to its CAMetalLayer), swap in the shared IOSurface as the NSView's CAMetalLayer backing buffer and force the NSView to redraw itself. Steps 2 and 3 are probably where most of the complexity will be. It doesn't look like you can just serialize a reference to a shared IOSurface so it looks like the OfficeBean and LibreOffice processes have to connect via a Mach port or XPC service. Maybe step 3 could be done without modifying the Skia/Metal code directly and, instead, overriding the "copy to window" timer in LibreOffice's vcl/skia code. A little bit of background on how LibreOffice draws: LibreOffice creates an offscreen drawing surface for each window and draws to that. Then, a timer runs that ccopies the offscreen drawing surface to the NSView's CAMetalLayer (Skia/Metal) or CALayer (Skia/Raster). Since we can't pass NSWindow or NSView pointers between processes, LibreOffice won't have a window so the existing timer won't do anything. So maybe in that timer you could just create a CAMetalLayer unattached to any NSView, swap in the IOSurface as its backing buffer, and copy LibreOffice's offscreen drawing surface. Hope the above helps.