Bug 163664 - implement support for Apple Intelligence Writing Tools
Summary: implement support for Apple Intelligence Writing Tools
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
Inherited From OOo
Hardware: All macOS (All)
: medium enhancement
Assignee: Not Assigned
URL: https://developer.apple.com/videos/pl...
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-10-28 23:32 UTC by steve
Modified: 2024-11-12 00:34 UTC (History)
1 user (show)

See Also:
Crash report or crash signature:


Attachments
Apple Intelligence Writing Tools (115.47 KB, image/png)
2024-10-29 00:04 UTC, steve
Details
Design concept of native AI window floating above LibreOffice Writer (551.49 KB, image/png)
2024-11-12 00:07 UTC, Patrick (volunteer)
Details
Xcode project to build native window in attachment #197556 (15.45 KB, application/zip)
2024-11-12 00:09 UTC, Patrick (volunteer)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description steve 2024-10-28 23:32:44 UTC
Description:
macOS 15.1 introduces Apple Intelligence. As part of that "Writing Tools" are now available - also for third party apps. The feature is outlined in https://support.apple.com/en-lk/guide/mac-help/mchldcd6c260/mac

It would be very useful and welcome to add support for Apple Intelligence Writing Tools in LibreOffice Writer.

To see how integration can be done, CotEditor can be looked into, which added support for Apple Intelligence in v5.0: https://coteditor.com/releasenotes/5.0.1.en
https://github.com/coteditor/CotEditor

Steps to Reproduce:
Right-click text in writer.

Actual Results:
No support for Apple Intelligence Writing Tools

Expected Results:
Would be awesome to have support for Writing Tools in Writer.


Reproducible: Always


User Profile Reset: No

Additional Info:
todays main build or Version: 24.8.2.1 (AARCH64) / LibreOffice Community
Build ID: 0f794b6e29741098670a3b95d60478a65d05ef13
CPU threads: 12; OS: macOS 15.1; UI render: Skia/Metal; VCL: osx
Locale: en-US (en_US.UTF-8); UI: en-US
Calc: threaded
Comment 1 steve 2024-10-29 00:04:27 UTC
Created attachment 197286 [details]
Apple Intelligence Writing Tools
Comment 2 Eyal Rozenberg 2024-10-30 08:36:14 UTC
(In reply to steve from comment #0)
> It would be very useful and welcome to add support for Apple Intelligence
> Writing Tools in LibreOffice Writer.

I don't think it will. Correct me if I'm wrong, but this involves:

* Non-free, closed-source software
* Sending your personal content to Apple, where it is stored, analyzed, mined etc.

assuming that is the case - is not welcome, and its usefulness is far outweighed by the detriment in using it.
Comment 3 steve 2024-10-30 22:08:21 UTC
Hi Eyal, can you elaborate how you come to your conclusions?

* Non-free, closed-source software
Well, just like Apple Intelligence, macOS itself is closed source right, and yet LibreOffice runs on it. Integrating with Writing Tools is a matter of supporting 
* Sending your personal content to Apple, where it is stored, analyzed, mined etc.
Again, what brings you to this conclusion? Can you share more resources about this? From what I understand most operations are done on device. I believe this is especially true for writing tools (https://support.apple.com/en-lk/guide/mac-help/mchlfc0d4779/15.0/mac/15.0).

Apple Intelligence is optional and users who enable it, obviously want to make use of it. I don't understand the argument to not support it in LibreOffice, just because you personally don't like it. Why not leave the decision to users whether they want to use it or not? And adding support in LibreOffice adds obvious value and helps keep LibreOffice an attractive choice as a writing tool on macOS.

3:50 of the linked video explains how Writing Tools are integrated in third-party apps. Making it work is a matter of using UITextView, NSTextView WKWebView. I am not sure if this is possible in LibreOffice writer.
Comment 4 Kush Srivastava 2024-10-31 16:16:56 UTC
Disclaimer: I am an assoc. member for the FSF, and genuinely believe in libre software. I am also new to understanding more on how Libreoffice works, but I have been a long time user (since 2015, at least), and use it on all my platforms (Alpine, Windows, and Mac). I am also new to contributing here. 

I second this bug, and the opinion that Apple Intelligence features should be implemented (which really just require Libreoffice to implement view windows that can be detected by the system). I do not know how feasible this is with Libreoffice, given that it uses it's own rendering engine, but having the feature available will be very beneficial to the vast majority of people wanting to use Libre software suites on Mac. 

On top of the comments mentioned above:

* All writing tools on Apple Intelligence are on-device, and private. You get an option for when you want to send data to Apple's servers, but even then, this is the best server implementation I have seen. Refer to (https://security.apple.com/blog/private-cloud-compute/), where it can be seen that data is private, removed, and the servers themselves have zero privileged access or a shell.

Also the server architecture and code is open to all for research purposes. (https://security.apple.com/blog/pcc-security-research/)

* Writing tools do not explicitly need to be "implemented" in Libreoffice. As long as we're using a native way of rendering text, or making sure the system can detect text in our rendering engine.

Clearly, there are steps taken from Apple to actually showcase that data is not stored/mined, users explicitly send data to the server if needed (and only for complex tasks like image generation), and writing tools are 100% locally run and stored, with their back-end open for scrutiny...I would love to see why this may not be viable. Especially, if it is something regarding how LibreOffice renders the UI, or how it runs on the Mac. 

I also believe that if we were to go down this route, a good option would be to have this be an opt in feature, instead of opt out. Open to hearing new thoughts on this matter.
Comment 5 steve 2024-11-01 14:41:04 UTC
This feature is opt-in to begin with, as Apple Intelligence is opt-in. And when a user opts-in, they certainly will be happy to see support in LO should this happen.

Since this only ever gets used, when users a) opted into Apple Intelligence on a system level and b) actively use one of the Writing Tools option, I am not sure an additional opt-in setting in LO would be needed or provide any use.
Comment 6 steve 2024-11-01 14:41:21 UTC
Setting to NEW as per comment 4
Comment 7 Patrick (volunteer) 2024-11-12 00:07:27 UTC
Created attachment 197556 [details]
Design concept of native AI window floating above LibreOffice Writer
Comment 8 Patrick (volunteer) 2024-11-12 00:09:53 UTC
Created attachment 197557 [details]
Xcode project to build native window in attachment #197556 [details]
Comment 9 Patrick (volunteer) 2024-11-12 00:34:54 UTC
Unfortunately LibreOffice has its own custom C++ layout engine and each NSWindow only has a single, customize NSView subclass that the LibreOffice code uses as a drawing canvas so we can't seamlessly integrate Apple Intelligence in its current form directly into Writer.

I searched around for any non-GUI APIs like -[NSSpellChecker checkGrammarOfString:...] that we could send a chunk of text to and it would return some modified (or not) text but I didn't find anything that looked remotely promising.

So, for now, I appears that Apple Intelligence is embedded within NSTextView on macOS.