A signed document? The crash appears to happen around: _gpgme_get_program_version (of gpg, i guess)

Note: I'm not really qualified to read backtraces, so I might be wrong

Thanks for taking a peek - document is neither encrypted nor signed. So unclear why gpgme would be involved. Was surprised about it showing in the crash log so prominently.

My guesstimate, from the trace, it looks like the dispatcher launches :

19  libsfxlo.dylib                	       0x104327edf SfxObjectShell::CheckSecurityOnLoading_Impl() + 31

which causes a document e-signature check (call to DocumentSignatureManager).

which in turn creates a fork to check for the presence of gpgmeio. 

Due to bug 115538, GPGTools, if present on the OS, can not be found, and the forked process call/instantiation fails, and the fork dies without returning gracefully to main() ?

OK, so after setting the xattr to get LODev out of gatekeeper jail, I started LODev7.6, loaded a Calc document.

Waited around a bit - no crash.
Not reproducible for me so far with :

Version: 7.6.0.0.alpha0+ (AARCH64) / LibreOffice Community
Build ID: 28ff4647e9dac8eebe3a169e828bacc8dc78e363
CPU threads: 8; OS: Mac OS X 13.0.1; UI render: default; VCL: osx
Locale: fr-FR (fr_FR.UTF-8); UI: en-US
Calc: threaded

@Alex: do you still have an intel mac around? Are you planning on updating to 13.1?

(In reply to Telesto from comment #1)
> A signed document? The crash appears to happen around:
> _gpgme_get_program_version (of gpg, i guess)
> 
> Note: I'm not really qualified to read backtraces, so I might be wrong

Just noticed on this particular point that gpgme is getting pulled in as an external library when the macos build is done (after having tried and failed to build LO myself on aarch64), so that might be one explanation why we are seeing this library appear in the trace. What I don't know is whether this is very recent, or whether it was already underlying and just never showed up. Probably, one would need to look at when gpgme was introduced into the build chain.

@Steve
Is this still a thing with current master build.. I have seen some commits related to gpg in gerrit a while back. No clue if this makes a difference..

The problem with this bug here is, I have no repro steps. The crash happened while nothing special was going on, LO just launched and no specific operation I could recall was taken. Decided to report a crash, since crashing should never happen for an app.

However without repro steps, addressing this crash is complicated. Since I have not seen this crash happen again, lets close as worksforme. We can always re-open if this surfaces again.

*** Bug 154468 has been marked as a duplicate of this bug. ***

This is still happening frequently in recent main builds. Back to unconfirmed.

cc:ing Patrick. Not sure if you have capacity or are willing to maybe look into this crash here.

(In reply to steve from comment #13)
> cc:ing Patrick. Not sure if you have capacity or are willing to maybe look
> into this crash here.

Unfortunately, from your crash log it looks outside my skills.

The only thing that I see in your crash log is that the crash is occurring in the third party gpgmepp code that LibreOffice pulls into the build. The third party code appears to be trying to launch a subprocess and then macOS is crashing deep in its own code with the ominous error message:

BUG IN CLIENT OF LIBDISPATCH: Owner in ulock is unknown - possible memory corruption
Abort Cause 34825
*** multi-threaded process forked ***
crashed on child side of fork pre-exec

libdispatch.dylib is a macOS system library so my best guess is that there is a bug in macOS (at least Ventura). I presume the "ulock" in the above message is some internal macOS variable.

Still astonished no other users are seeing this. I run into this crash on a daily basis and have to look at the death beachball for ~30 seconds before I then see the crash info.

I can continue to use LO after the crash, but this is not very usable.

LibreOffice is currently using gpgme-1.18.0.tar.bz2 as per https://git.libreoffice.org/core/+/refs/heads/libreoffice-7-6/download.lst#216

https://gnupg.org/download/index.html#gpgme indicates 1.23.1 is the current stable release.

Wondering if updating gpgme may help resolve this issue. Updating it previously resolved finding OpenPGP keys https://bugs.documentfoundation.org/show_bug.cgi?id=115538#c49

We are nearing the first year of this crash, which is persisting for me with

Version: 24.2.0.0.alpha1+ (X86_64) / LibreOffice Community
Build ID: 1a74a87b442857567d20da5dc97bbbc278745afd
CPU threads: 8; OS: macOS 13.6.1; UI render: Skia/Metal; VCL: osx
Locale: en-US (en_DE.UTF-8); UI: en-US
Calc: threaded

1 year: https://bin.disroot.org/?fe0f52be36e7f9a3#8h8D9TcP43WGb3GoNbZ3UqiZLaEFgjZ1Jeb2ReWhEV6B

Created attachment 191576 [details]
Screenshot 2023-12-23 LO main build, crash after launch

Upping priority to high. Those crashes keep happening with latest main build resulting in slow startups (beachball showing for ~30 seconds until macOS crash report pops up) making LO frustrating and inefficient to use.

We are now at gpgme 1.23.2: https://git.gnupg.org/cgi-bin/gitweb.cgi?p=gpgme.git
Still wondering if updating gpgme would be worth a try.

I am beginning to think that the crash here has the same cause as https://bugs.documentfoundation.org/show_bug.cgi?id=156352 which also is reproducible and crashes since save as dialog was fixed on macOS back in July 2023.

It seems any OpenPGP operation or interaction with gpgme crashes LO. Was there some testing on macOS for this? If so, which version combination (gpgme, LO, macOS) is known to be functional.

To give an idea of what using LibreOffice currently looks like, I attached a screenshot.

Reset user profile: does not resolve issue.

Attempt at poor-mans bibisect (manually) with following results:
OK 7.0.0.3
OK 7.2.7.2
OK 7.5.3.1
OK 7.5.8.1
OK 7.5.9.2
NOT OK 7.6.0.1

(In reply to steve from comment #19)
> Attempt at poor-mans bibisect (manually) with following results:
> OK 7.0.0.3
> OK 7.2.7.2
> OK 7.5.3.1
> OK 7.5.8.1
> OK 7.5.9.2
> NOT OK 7.6.0.1

I checked the LibreOffice code commit logs and LibreOffice uses the same version - 1.8.0 - of the GPGME third party code so I think we can eliminate changes in GPGME third party code.

But the one change in LibreOffice was I added the following patch starting in LibreOffice 7.6:

https://cgit.freedesktop.org/libreoffice/core/commit/?id=28d736113317da3e6373f27d6ce5d75fd40d68e2

This patch does one thing: add the standard Homebrew and MacPorts paths onto the end of the search path. So, my first guess is that you might have an old or incompatible version of the gpg* commands in /opt/homebrew/bin or in /opt/local/bin.

Is there any files in the output of the following Terminal commands?:

ls -l /opt/homebrew/bin/gpg*
ls -l /opt/local/bin/gpg*

I only have gpgme installed via Homebrew and below is the output on my machine:

ls -l /opt/local/bin/gpg*
lrwxr-xr-x  1 pluby  staff  29 Sep 29 09:54 /opt/homebrew/bin/gpg -> ../Cellar/gnupg/2.4.3/bin/gpg
lrwxr-xr-x  1 pluby  staff  35 Sep 29 09:54 /opt/homebrew/bin/gpg-agent -> ../Cellar/gnupg/2.4.3/bin/gpg-agent
lrwxr-xr-x  1 pluby  staff  34 Sep 29 09:54 /opt/homebrew/bin/gpg-card -> ../Cellar/gnupg/2.4.3/bin/gpg-card
lrwxr-xr-x  1 pluby  staff  43 Sep 29 09:54 /opt/homebrew/bin/gpg-connect-agent -> ../Cellar/gnupg/2.4.3/bin/gpg-connect-agent
lrwxr-xr-x  1 pluby  staff  41 Sep 29 09:54 /opt/homebrew/bin/gpg-error -> ../Cellar/libgpg-error/1.47/bin/gpg-error
lrwxr-xr-x  1 pluby  staff  48 Sep 29 09:54 /opt/homebrew/bin/gpg-error-config -> ../Cellar/libgpg-error/1.47/bin/gpg-error-config
lrwxr-xr-x  1 pluby  staff  40 Sep 29 09:54 /opt/homebrew/bin/gpg-wks-client -> ../Cellar/gnupg/2.4.3/bin/gpg-wks-client
lrwxr-xr-x  1 pluby  staff  40 Sep 29 09:54 /opt/homebrew/bin/gpg-wks-server -> ../Cellar/gnupg/2.4.3/bin/gpg-wks-server
lrwxr-xr-x  1 pluby  staff  33 Sep 29 09:54 /opt/homebrew/bin/gpgconf -> ../Cellar/gnupg/2.4.3/bin/gpgconf
lrwxr-xr-x  1 pluby  staff  38 Sep 29 09:54 /opt/homebrew/bin/gpgparsemail -> ../Cellar/gnupg/2.4.3/bin/gpgparsemail
lrwxr-xr-x  1 pluby  staff  44 Sep 29 09:54 /opt/homebrew/bin/gpgrt-config -> ../Cellar/libgpg-error/1.47/bin/gpgrt-config
lrwxr-xr-x  1 pluby  staff  32 Sep 29 09:54 /opt/homebrew/bin/gpgscm -> ../Cellar/gnupg/2.4.3/bin/gpgscm
lrwxr-xr-x  1 pluby  staff  31 Sep 29 09:54 /opt/homebrew/bin/gpgsm -> ../Cellar/gnupg/2.4.3/bin/gpgsm
lrwxr-xr-x  1 pluby  staff  34 Sep 29 09:54 /opt/homebrew/bin/gpgsplit -> ../Cellar/gnupg/2.4.3/bin/gpgsplit
lrwxr-xr-x  1 pluby  staff  32 Sep 29 09:54 /opt/homebrew/bin/gpgtar -> ../Cellar/gnupg/2.4.3/bin/gpgtar
lrwxr-xr-x  1 pluby  staff  30 Sep 29 09:54 /opt/homebrew/bin/gpgv -> ../Cellar/gnupg/2.4.3/bin/gpgv
zsh: no matches found: /opt/local/bin/gpg*

➜  ~ ls -l /opt/homebrew/bin/gpg*
zsh: no matches found: /opt/homebrew/bin/gpg*
➜  ~ ls -l  *
zsh: no matches found: /opt/local/bin/gpg*

gpg installed via GPG Suite (download at https://gpgtools.com/) on this 13.6.3 macOS, so:

➜  ~ ls -l /usr/local/bin/gpg                   
lrwxr-xr-x  1 root  admin  27 Mar 27  2023 /usr/local/bin/gpg -> /usr/local/MacGPG2/bin/gpg2

What doesn't add up in regards to the commit you linked is the timeline, since I've been seeing those crashes since december 2022, probably earlier as I didn't immediately file the bug when I first saw the crashes and linked commit is from 2023-07.

(In reply to steve from comment #22)
> What doesn't add up in regards to the commit you linked is the timeline,
> since I've been seeing those crashes since december 2022, probably earlier
> as I didn't immediately file the bug when I first saw the crashes and linked
> commit is from 2023-07.

Oh well, it least we know that it isn't Homebrew or MacPorts causing this in your case since those paths don't contain any gpg* commands. /usr/local/bin is earlier in gpgme's search path both before and after my patch so I thin we can conclude that LibreOffice is trying to run the gpg* commands installed by GPGTools in /usr/local/bin and /usr/local/MacGPG2/bin.

So I installed GPGTools yesterday and I am able to do digital signing with a personal key I created in GPGTools. What is maddening about this bug is the crash is occuring in a macOS system function. After much online searching, the closest to your crash that I could find is in the following Apple link (note: the link seems to only be readable in Safari):

https://developer.apple.com/forums/thread/737464

I'll read that and see if that might give me some idea on why launching a Terminal command would crash. I have never experienced the fork() system function crashing.

Here is an interesting article that might be related to your problem:

https://www.wefearchange.org/2018/11/forkmacos.rst.html

The "Band-aid the problem" section has a debugging step. Can you quit LibreOffice and try running LibreOffice in a Terminal using the following commands?:

  export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
  /Applications/LibreOffice.app/Contents/MacOS/soffice

Does that stop the crashing? If no, can you post a new crash log?

To verify that GPG Suite is at play, uninstalled GPG Suite and indeed, crash and hang on open are gone.

Patrick, if you remove all custom (homebrew, macports, ...) gpgs and only run MacGPG from GPG Suite you are still unable to reproduce the hang / crash on your Apple Silicon mac?

(In reply to steve from comment #25)
> To verify that GPG Suite is at play, uninstalled GPG Suite and indeed, crash
> and hang on open are gone.
> 
> Patrick, if you remove all custom (homebrew, macports, ...) gpgs and only
> run MacGPG from GPG Suite you are still unable to reproduce the hang / crash
> on your Apple Silicon mac?

I've been debugging with GPGSuite for the last couple of days. No hang or crash.

But, after reading the link in comment #24 last night, I now suspect that what you are seeing is a symptom of that "Objective-C and fork()" problem.

What fits in your LibreOffice version timeline is that, IIRC, the LibreOffice build servers were upgraded to macOS Ventura in the last year and the "Objective-C and fork()" problem occurs in code built with the macOS Ventura or later APIs.

Have you had a chance to try my "run in the Terminal" steps in comment #24 yet? What those steps do is force macOS to run LibreOffice like it was compiled against a pre-Ventura version of macOS.

If crashing stops, then I have an idea of how to fix this: force execution of gpgme on the main thread. I haven't confirmed it yet, but I think LibreOffice is calling fork() on a background thread while LibreOffice is calling Objective-C code (i.e. windows and drawing) on the main thread.

It might be tricky doing this change so I'd like to know if the comment #24 workaround works or not.

(In reply to Patrick Luby from comment #26)
> What fits in your LibreOffice version timeline is that, IIRC, the
> LibreOffice build servers were upgraded to macOS Ventura in the last year
> and the "Objective-C and fork()" problem occurs in code built with the macOS
> Ventura or later APIs.

Correction: it should read "Ventura and later SDKs". IIRC, LibreOffice now builds for pre-Ventura APIs, but with the macOS 13.x SDK. This is why I think this bug is a symptom of the macOS "Objective-C and fork()" problem.

Thanks a lot for investigating.

Are you referring to command `ulimit -c 0`?

On a OCLP iMac with 14.2.1, when checking the value I got:

➜  ~ ulimit -c
0

Not sure why this would be set already. However I am not seeing the hangs on this mac. Will have to get back to another mac on macOS 13 to further test this.

Why won't you try what I asked for in comment #24?

(In reply to Patrick Luby from comment #29)
> Why won't you try what I asked for in comment #24?

Repeating the steps in comment #24 here:

The "Band-aid the problem" section has a debugging step. Can you quit LibreOffice and try running LibreOffice in a Terminal using the following commands?:

  export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
  /Applications/LibreOffice.app/Contents/MacOS/soffice

Does that stop the crashing? If no, can you post a new crash log?

Setting the OBJC_DISABLE_INITIALIZE_FORK_SAFETY environment variable to "YES" before running the LibreOffice command causes Apple's Objective-C runtime to use its pre-Ventura implementation. If crashing goes away, then that confirms that Apple's change to the Objective-C runtime is the source of the bug.

Sorry, didn't read comment 24 thoroughly enough.

Opening LO via the following command on macOS 13.6.4, where I otherwise see the lag / crash on open, resolves the crashes:

  export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
  /Applications/LibreOffice.app/Contents/MacOS/soffice

LO opens fine, beachball of death does not show and document can almost instantly be interacted with.

Amazing you found this glitch. What is the way forward here on the long run? Is this something that needs to be fixed by Apple in macOS?

(In reply to steve from comment #31)
> Sorry, didn't read comment 24 thoroughly enough.
> 
> Opening LO via the following command on macOS 13.6.4, where I otherwise see
> the lag / crash on open, resolves the crashes:
> 
>   export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
>   /Applications/LibreOffice.app/Contents/MacOS/soffice
> 
> LO opens fine, beachball of death does not show and document can almost
> instantly be interacted with.
> 
> Amazing you found this glitch. What is the way forward here on the long run?
> Is this something that needs to be fixed by Apple in macOS?

This is very good news. The way I found this was by searching on all the unknown function names in the crash log window in the screen snapshot attached to this bug combined with "fork() crash" until I bumped into the Python blog in comment 24.

That's where it all started to make sense: Apple changed the Objective-C runtime if you build with the Ventura SDK or newer. This is a common way that Apple introduces macOS changes: they enable changes based on the macOS SDK version that you build with. Build with an older SDK, you get Apple's old code. But then when you start building with a newer SDK, that is when start seeing the effects of these subtle Apple changes.

The simplest solution is to set the "__DATA,__objc_fork_ok" attribute in the LibreOffice executable during the build. That shouldn't be difficul and that should force LibreOffice to have the same behavior as setting OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES does in the Terminal.

My only worry is that Apple arbitrary decides to deprecated and ignore that attribute in the future. So, before I implement the simple solution, I want check if there is something obvious causing this crash. My first guess from your 
"lag / crash on open" is that LibreOffice is launching gpg very early stages of LibreOffice's launch process and runs gpg on a separate thread. The problem if that is the case is that if the separate gpg thread is run early enough in the launch process, the main thread will still be in the middle of loading Objective-C classes and it is the loading of such classes that crashes all of this new "fork() safety" feature in macOS.

I'll post when I have more news in the next few days. In the meantime, at least we know about the temporary workaround in comment #30.

Ignore my thread collision theory in my last post. When running in debugger, I found that the GPGME's _gpgme_io_spawn function is only called on LibreOffice's main thread.

So looks like setting the "__DATA,__objc_fork_ok" attribute in the LibreOffice executable during the build is what I will work on next.

I am still mystified as to what causes Objective-C classes to be loaded when the _gpgme_io_spawn function has called fork() but hasn't yet called one of the exec*() functions in the child process. _gpgme_io_spawn is called on the main thread and the GPGME code doesn't appear to be using Objective-C code. Weird.

@Patrick: I believe that the issue occurs in a atfork handler registered with pthread_atfork.

A quick grep for pthread_atfork in the libreoffice source code reveals the following line:

```
pthread_atfork(nullptr, nullptr, atfork_child);
```

```
static void atfork_child()
{
    if (SvpSalInstance::s_pDefaultInstance != nullptr)
    {
        SvpSalInstance::s_pDefaultInstance->CloseWakeupPipe();
    }
}
```


While I have no understanding of the libreoffice source code what so ever, I think that the AquaSalInstance is involved here, which does call out to macOS frameworks.

An Apple developer in the forums suggested replacing fork with posix_spawn since that guarantees that no code is run in-between fork and exec*, but for that gpgme would have to be patched.

So ultimately "__DATA,__objc_fork_ok" might be the easiest fix if that works.

Hope that helps tracks the error down.

(In reply to Luke Le from comment #34)
> @Patrick: I believe that the issue occurs in a atfork handler registered
> with pthread_atfork.

Unfortunately, that atfork handler is only registered on Linux headless builds. No atfork  But maybe the problem is there are other threads loading Objective-C classes during fork. Fork is called within the main thread's event loop there shouldn't be any new Objective-C classes loaded.

I looked at src/posix-io.c in the gpgme code and that is where the fork is. LibreOffice includes gpgme as a shared library and from what I could see, gpgme fork's twice, the parent process (LibreOffice) waits for the first child, then the first child immediately forks and creates a second child.

Only the second child seems to do any work between fork and exec and that is just iterating through some of the parent's data structures and closing file descriptors.

Unfortunately, the second child's iterating through parent data structures makes switching to posix_spawn not trivial.

The crash log in the following bug comment is very representative of the crash logs that @steve has posted. This particular crash log, the crash is inside a libdispatch function:

https://bugs.documentfoundation.org/show_bug.cgi?id=156352#c5

Created attachment 192180 [details]
Debug patch that tracks number of classes loaded before and after gpgme forks a new process

(In reply to Patrick Luby from comment #37)
> Created attachment 192180 [details]
> Debug patch that tracks number of classes loaded before and after gpgme
> forks a new process

So I have been running the above debug patch in my local build and I am not seeing any Objective C classes loaded during the gpgme code's forking.

I am only seeing Objective C classes loaded on the main thread (the same thread that gpgme is forked from) and that is down during drawing code (e.g. the first time a type of native control is drawn).

gpgme doesn't load any Objective C classes so, unless @steve has some extensions installed that are loading Objective C classes on a secondary thread, there isn't any secondary thread that is grabbing the Objective C runtime "initialize lock" as there are no classes loading in LibreOffice's secondary threads that I can see.

Below is a snippet of the output while signing a Writer document in the Digital Signatures dialog displayed by selecting the File > Digital Signatures > Digital Signatures... menu item:

Before: 25091
After: 25091
warn:xmlsecurity.dialogs:30495:49364427:xmlsecurity/source/dialogs/digitalsignaturesdialog.cxx:757: Could not find embedded certificate!
Before: 25091
After: 25091
Before: 25091
After: 25091
warn:xmlsecurity.dialogs:30495:49364427:xmlsecurity/source/dialogs/digitalsignaturesdialog.cxx:757: Could not find embedded certificate!
warn:xmlsecurity.dialogs:30495:49364427:xmlsecurity/source/dialogs/digitalsignaturesdialog.cxx:757: Could not find embedded certificate!
Before: 25091
After: 25091
Before: 25091
After: 25091
Before: 25091
After: 25091
Before: 25091
After: 25091
Before: 25091
After: 25091
Before: 25091
After: 25091

Based on my debugging notes in comment #38 and based on the crash log in the following comment, I really think that this is a memory corruption bug:

https://bugs.documentfoundation.org/show_bug.cgi?id=156352#c5

Conclusion: add a __DATA,__objc_fork_ok section to LibreOffice's soffice executable.

I know that the following blog says this is the worst option, but when I see a crash in Apple's code and their internal locks are garbage memory, I think is pretty clear that there is a memory corruption bug:

https://sealiesoftware.com/blog/archive/2017/6/5/Objective-C_and_fork_in_macOS_1013.html

Is this memory corruption from gpgme? Or does Apple's internal Objective C locking have a bug? Who knows and, as a volunteer, I unfortunately don't have the time to track down the source of this memory corruption.

Maybe I'm missing something, but in comment #38 I have found no evidence of secondary threads loading Objective C classes while in the gpgme forking code. So, what does Apple's Objective C locking buy us? Per the blog above:

"Before macOS 10.13, the Objective-C runtime did not support use between fork() and exec() in the child process of a multithreaded parent process. Calling any Objective-C method in that interval was not allowed...."

I don't see any Objective C code in the gpgme forking process and the child process, before exec, was forked from the main thread. AFAICT, the crash is happening in the parent's system atfork handler on LibreOffice's main thread. Since no other threads, nor the child process before calling exec, loads any classes or does any Objective C calls, the parent's system atfork handler should have no blocking issues locking its locks. But it appears that memory is corrupted and LibreOffice crashes.

Per the above blog, by using the "worst" option, we risk the following:

"Sometimes it would fail: for example, if a thread in the parent process happened to be holding one of the Objective-C runtime's locks when the fork() occurred, the child process would deadlock when it tried to take that lock."

This is actually better IMO. Debugging a deadlock is much easier (at least for me) than debugging memory corruption in third party code. And who knows, by catching deadlocks, we may actually find out it there is some extension or other infrequently used code in LibreOffice, that is loading Objective C classes on a secondary thread or in the child process.

Patrick Luby committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/c6652e280b0690497abf27380dd064898f91db32

tdf#152524 Add a __objc_fork_ok data section for Mac Intel executable

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.

OK. The fix should be in tomorrow's (27 January 2024) nightly master build.

Note: this fix is only in Mac Intel builds. I have not seen any reports of this bug on Mac Silicon. Maybe there just aren't enough Mac Silicon users who do digital signing, but I want to be cautious about using this fix since Apple could remove it at anytime in the future.

And, to be honest, this is really a very blunt fix. So, I really don't think we should be in any hurry to backport this fix. What's in tonight's nightly builds won't be in production release until August 2024 so we have some time for @steve and other testers to see if this causes different crashes or hangs.

I am going to leave this bug open as we may find that there are new, different crashes or hangs than before the fix. That's not a bad thing as it may help us find what code is really causing the original bug.

So don't hesitate to attach any crash logs or Activity Monitor samples if the nightly build crashes or hangs. If we find a better, less blunt, fix between now and August 2024, then I can revert the current fix before it is in a production release.

Note: I really hate selecting the option that an expert has judged to be the worst. I'll probably be proven wrong, but I just can't help but think that memory corruption is occurring. And given the size of LibreOffice, that could be anywhere so might as well grab the one approach that seems to stop the crashing.

(In reply to Patrick Luby from comment #41)
> OK. The fix should be in tomorrow's (27 January 2024) nightly master build.

Note for myself: I verified that today's (27 January 2024) nightly master build for Mac Intel contains my fix using the following Terminal command:

otool -s __DATA __objc_fork_ok /Applications/LibreOfficeDev.app/Contents/MacOS/soffice

and that outputs the following:

Contents of (__DATA,__objc_fork_ok) section
0000000100008018	01 00 00 00

So, with the Mac Intel build, the above bits in the LibreOffice executable should do the same thing as setting the OBJC_DISABLE_INITIALIZE_FORK_SAFETY environment variable to YES.

In other words, the workaround in comment #30 should no longer be needed and now, LibreOffice on Mac Intel automatically enables that workaround no matter how you launch LibreOffice.

Fix verified in Version: 24.8.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 2cedb1a19ad605df4e148589e9027512e4dd9265
CPU threads: 8; OS: macOS 13.6.4; UI render: Skia/Metal; VCL: osx
Locale: en-US (en_DE.UTF-8); UI: en-US
Calc: threaded

Opening LO no longer causes a 1 minute hang and or crash.

Will use this main build for a while and obviously report crashes or hangs should I encounter any.

Only intel macs seem to have been affected. Maybe the issue was Ventura only as on an intel mac running Sonoma I could no longer reproduce the crash with version prior to this patch.

I leave you the honors of setting this to verified | fixed if you think that's what we should call this case.

(In reply to steve from comment #43)
> Opening LO no longer causes a 1 minute hang and or crash.
> 
> Will use this main build for a while and obviously report crashes or hangs
> should I encounter any.
> 
> Only intel macs seem to have been affected. Maybe the issue was Ventura only
> as on an intel mac running Sonoma I could no longer reproduce the crash with
> version prior to this patch.
> 
> I leave you the honors of setting this to verified | fixed if you think
> that's what we should call this case.

Glad to hear that the fix works. For now, let's keep this bug open. I really think it needs a lot of testing before we backport it to LibreOffice 24.2.

I am really curious if any new crashes or hangs show up after running LibreOffice for hours without quitting and after repeated use of gpg. In other words, does memory corruption still happen after repeated usage of gpg in LibreOffice?

Another thing to test after a lot of gpg usage is if you can open a PDF file in Draw. LibreOffice forks and exec a command similar to gpgme when importing a PDF file into draw. If that doesn't crash or hang or fail to import with this fix, then that is good news.

It is interesting that you only see this bug when running on macOS Ventura. I tried setting some OBJ_* environment variables in main() to see if they have any effect in the hope that I could limit the fix to only Ventura. But unfortunately, even setting OBJ_* environment variables at the beginning of main() is too late. It appears that the Objective C runtime gets loaded before main() during launch so I think we're stuck with "all macOS versions on Mac Intel". Fortunately, I am able to not apply this fix in Mac Silicon builds so we can still see if the bug ever occurs Mac Silicon.

BTW, I found a way to see what Objective C classes are being loaded if you run LibreOffice in a Terminal using the following commands:

  export OBJC_PRINT_INITIALIZE_METHODS=YES
  /Applications/LibreOfficeDev.app/Contents/MacOS/soffice

So, if you see any crashing or hanging or PDF import failures, it might be helpful to use the above Terminal commands to see the last Objective C classes loaded before the crash or hang. That might give us an idea whose code is loading Objective C classes right before the memory corruption. The output of the above Terminal command looks like the following for each Objective C when it is first loaded:

objc[58562]: INITIALIZE: thread 0x30e48c000: acquiring lock for +[NSPersistentUIWindowInfo initialize]
objc[58562]: INITIALIZE: thread 0x30e48c000: finished +[NSPersistentUIWindowInfo initialize]

FYI just ran into a crash while looking at a presentation and having it sit idle for a while. soffice crashed and gpgme seems to be involved.

Crash log (1 year): https://bin.disroot.org/?3aeb4cc8a13895d9#HhCLYiyCaSmxgkt7tEmXzEsKXJiGiffrgidZQeF4Ryvw

(In reply to steve from comment #45)
> FYI just ran into a crash while looking at a presentation and having it sit
> idle for a while. soffice crashed and gpgme seems to be involved.
> 
> Crash log (1 year):
> https://bin.disroot.org/
> ?3aeb4cc8a13895d9#HhCLYiyCaSmxgkt7tEmXzEsKXJiGiffrgidZQeF4Ryvw

Unfortunately, that crash is in the same place as the whole "Objective C locks" was hopefully going to fix. So, it just looks like that fix only delays the inevitable crash.

I wonder if the fix in tdf#156352 has any affect on this bug. They are definitely different bugs, but it will be interesting if you see any change in when the crash occurs with tomorrow's (06 February 2024) nightly master build.

Patrick Luby committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/839cf255e2670fdf8e974af38432aacf63be4e90

tdf#152524 fork() and exec() in a separate libdispatch queue

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.

(In reply to Commit Notification from comment #47)
> Patrick Luby committed a patch related to this issue.
> It has been pushed to "master":
> 
> https://git.libreoffice.org/core/commit/
> 839cf255e2670fdf8e974af38432aacf63be4e90

The above commit should be in tomorrow's (07 February 2024) nightly master build.

I don't know if this will affect the crash that @steve sees on Mac Intel running Ventura, but I what I am hoping is that by having gpgme call fork() on a separate, dedicated thread that I know has no Objective C calls on it, maybe that will avoid or reduce the crashing in Apple's libdispatch code.

(In reply to Patrick Luby from comment #48)
> The above commit should be in tomorrow's (07 February 2024) nightly master
> build.

Correction: it should be in tomorrow's (08 February 2024) nightly master build.

Patrick Luby committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/b0656e6ca668a0719fbcb71b6d46c68093dda470

tdf#152524 use dispatch_async() instead of dispatch_sync()

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.

(In reply to Commit Notification from comment #50)
> https://git.libreoffice.org/core/commit/
> b0656e6ca668a0719fbcb71b6d46c68093dda470
> 
> tdf#152524 use dispatch_async() instead of dispatch_sync()

The above commit should be tomorrow's (09 February 2024) nightly master build.

According to the crash log in the following comment, gpgme still calls fork() on the main thread after my commit in comment #47. So, I have reworked my latest fix to force fork() to be called on a separate thread and verified in a debugger that it is actually running on a separate thread:

https://bugs.documentfoundation.org/show_bug.cgi?id=156352#c27

*** Bug 153626 has been marked as a duplicate of this bug. ***

(In reply to Patrick Luby from comment #51)
> The above commit should be tomorrow's (09 February 2024) nightly master
> build.

@steve: have you had a chance to test my last change? If yes, is LibreOffice still crashing?

Been running following build for past days as daily driver:

Version: 24.8.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 43d962c27b6efb04d22b05ad8dec08f6056078a0
CPU threads: 8; OS: macOS 13.6.4; UI render: Skia/Metal; VCL: osx
Locale: en-US (en_DE.UTF-8); UI: en-US
Calc: threaded

While I have just seen the beachball on opening a document for ~15 seconds. That is no longer always happening or reproducible and I have seen no crashes at all on opening. Have to admit that there has not been a lot of LO usage in past days, so I am careful to to call the case closed, but it's probably fair to set to fixed. Will report back should I see a crash around gpgme.

This is a huge improvement as this bug has been preventing efficient work for a long time. So thanks a lot for investigating and tackling the cause.

Patrick Luby committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/3e0f6665e3c9b7432404eb49efd74bff91f62935

Revert "tdf#152524 Add a __objc_fork_ok data section for Mac Intel executable"

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.

(In reply to Commit Notification from comment #55)
> https://git.libreoffice.org/core/commit/
> 3e0f6665e3c9b7432404eb49efd74bff91f62935
> 
> Revert "tdf#152524 Add a __objc_fork_ok data section for Mac Intel
> executable"

I don't think this will change anything, but IIRC this never really worked so I reverted this wildly experimental patch.

The reversion should be in tomorrow's (19 February 2024) nightly master build.

Patrick Luby committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/3b4a8977652b7e69469d0e56e12de50e07ceae2f

Revert "tdf#152524 use dispatch_async() instead of dispatch_sync()"

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.

Crash after opening settings.

I noticed that with Version: 24.8.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 3b73071f7a7fcf80547da81e5effe4ed6018bbb4
CPU threads: 8; OS: macOS 13.6.4; UI render: default; VCL: osx
Locale: en-US (en_DE.UTF-8); UI: en-US
Calc: threaded

I am seeing the beachball on launch again, although no immediate crash.

Then below crash happened after opening settings. I could imaging the fact that in settings the User Data option was selected which shows OpenPGP keys under the Cryptography section. So maybe that was enough to trigger the crash?

Crash log: https://bin.disroot.org/?59fbcb802097583e#FA7CcCpmT4Ptx1JrGXcbp6v14cp1KUdPSEfPZ2aUSKCz

Not always reproducible on settings open so maybe unrelated to that.

Updated to latest main build: Version: 24.8.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: e2473fe3a547e5a11d3b91ab8ded833bf5b74356
CPU threads: 8; OS: macOS 13.6.4; UI render: default; VCL: osx
Locale: en-US (en_DE.UTF-8); UI: en-US
Calc: threaded

and ran into another crash after opening settings: https://bin.disroot.org/?0c2f1588d4636bd0#8ifQ8yEJTsA5QGsSdpcU3kJYWHpCj5vowZvd1F9tSCwo

(In reply to steve from comment #58)
> Crash after opening settings.
> 
> I noticed that with Version: 24.8.0.0.alpha0+ (X86_64) / LibreOffice
> Community
> Build ID: 3b73071f7a7fcf80547da81e5effe4ed6018bbb4
> CPU threads: 8; OS: macOS 13.6.4; UI render: default; VCL: osx
> Locale: en-US (en_DE.UTF-8); UI: en-US
> Calc: threaded
> 
> I am seeing the beachball on launch again, although no immediate crash.
> 
> Then below crash happened after opening settings. I could imaging the fact
> that in settings the User Data option was selected which shows OpenPGP keys
> under the Cryptography section. So maybe that was enough to trigger the
> crash?588d4636bd0#8ifQ8yEJTsA5QGsSdpcU3kJYWHpCj5vowZvd1F9tSCwo

You are correct: the crash is happening in the same spot as your other recent crashes. Were you using the "export GPGME_DEBUG=3:..." trick and running LibreOffice in a Terminal when these crashes occur? If no, does that trick stop the crashing?

export GPGME_DEBUG=3:~/mygpgme.log                            
/Applications/LibreOfficeDev.app/Contents/MacOS/soffice

resolves the beachball on launch of LibreOffice.

Digitally signing a writer document worked.

Encrypt with OpenPGP key still crashes though.

Created attachment 192922 [details]
All keys disabled in GPG Keychain application

(In reply to steve from comment #60)
> Digitally signing a writer document worked.
> 
> Encrypt with OpenPGP key still crashes though.

Can you try an experiment? I am currently slowly reading through the LibreOffice and gpgme code that executes just before your crash and the crash is occurring just after the gpg command returns a list of private keys to LibreOffice. Then, LibreOffice iterates through the list of private keys and, for each private key, executes the gpg to retrieve each's matching public key. Fetching the private keys works. The crash doesn't occur until one of the subsequent "get matching public key" gpg commands.

So, can you open the GPG Keychain application and disable *all* of the listed keys like I did in attachment #192922 [details]? Then, try to encrypt a new Writer file in LibreOffice. Does the crashing stop? If yes, can you reenable one key in GPG Keychain and rerun? Repeat until there is a crash.

I am hoping that it is only certain types of public keys that are causing this crash. Fortunately, it seems that it is a public key that triggers this so if a certain key type causes this, we can try to figure out what is unique about this key. Presumably something unique uses different sections of the gpgme code?

(In reply to Patrick Luby from comment #62)
> I am hoping that it is only certain types of public keys that are causing
> this crash. Fortunately, it seems that it is a public key that triggers this
> so if a certain key type causes this, we can try to figure out what is
> unique about this key. Presumably something unique uses different sections
> of the gpgme code?

I also noticed that the Select Certificate dialog has a shorter list of keys when digital signing a document whereas when encrypting a file, I see more keys (in my case, the "more keys" are the GPGTools keys).

I may end up being wrong, but I am guessing that if one or more of the keys cause crashing when exporting, the problematic keys are likely to be one of the keys that aren't shown when digital signing.

I'll continue to look for the code that restricts the list during digital signing. Maybe LibreOffice needs to apply the same restrictions when encrypting?

So, if the crashing when encrypting stops after you disable all keys (using the steps in comment #62), I would recommend reenabling your personal key first to see if that key works. If it works, then continue reenabling third-party keys and then GPGTools keys.

(In reply to Patrick Luby from comment #63)
> I also noticed that the Select Certificate dialog has a shorter list of keys
> when digital signing a document whereas when encrypting a file, I see more
> keys (in my case, the "more keys" are the GPGTools keys).
> 
That is to be expected - for encryption, you only need the public GPG keys of the recipient(s). For digital signing, the private gpg key is required.

(In reply to Thorsten Behrens (allotropia) from comment #64)
> That is to be expected - for encryption, you only need the public GPG keys
> of the recipient(s). For digital signing, the private gpg key is required.

That is good to know. I now see where the "if signing, fetch only personal keys, otherwise fetch all keys" in the CertificateChooser class.

Maybe I am being too optimistic, but if @steve can narrow down the crash to one or more specific public keys and one of those keys are publicly published, then maybe I'll be able to finally be able reproduce this bug on my machine.

This is an interesting finding. It would match my test results (signing OK, encrypting NOT OK crash). 

Since I have many keys I continue testing in a vm running macOS 14.3.1 with a more minimized setup. Disabled all keys in GPG Keychain.
started via terminal command
Version: 24.2.1.2 (X86_64) / LibreOffice Community
Build ID: db4def46b0453cc22e2d0305797cf981b68ef5ac
CPU threads: 2; OS: macOS 14.3.1; UI render: Skia/Raster; VCL: osx
Locale: en-US (en_DE.UTF-8); UI: en-US
Calc: threaded

LO always launched via terminal command
export GPGME_DEBUG=3:~/mygpgme.log
/Applications/LibreOffice.app/Contents/MacOS/soffice

Test Round 1: all keys disabled
- As expected Digital Signatures shows an empty key list, can't test that feature in this state.
- Encrypt with GPG key on save also shows an empty key list (also expected).
- No Crash

Test Round 2: single public key enabled
- As expected Digital Signatures shows an empty key list, can't test that feature in this state.
- Encrypt with GPG key on save shows a single key > select > encrypt successful
- re-open saved file and I am asked for a password to open that file, seems unexpected, I'd expect LO to look for the corresponding secret key (which I don't have as the key in question is a public only key), no password is accepted (expected) and file cannot be opened (expected)

Test Round 3: single secret/public key enabled
- Digital Signatures > Sign Document shows a singled key (expected), sign successfully creates a signature
- Encrypt with GPG key on save shows a single key > select > encrypt successful
- re-open saved file opens fine (OpenPGP password stored in macOS keychain), when opened on a different system, pinentry mac asks for password for the secret key in question

Test Round 4: 3 sec/pub keys and 5 pub keys active
- Encrypt with GPG key and select team@gpgtools.org pub key to encrypt with. Result is an error message: "Warning: OpenPGP key not trusted, damaged, or encryption failure. Please try again.".
This is a 2048 bis DSA key with Capabilities Esc, with Subkeys in 4096 RSA for Cabability s and e.
Slightly curious since encrypting to another public key lukele@gpgtools.org works fine and that also is a 2048 DSA key ESC, with Subkeys 4096 RSA although only a single subkey Cabability e.
Both public keys can be found on the keys.openpgp.org to test against.

So there are a few surprising test results to unpack here

a) most surprisingly, not a single crash happened
b) a key that can be encrypted to via GUI or CLI tools besides LO can not be used to encrypt a document from within LO
c) while I introduced another moving part in this testing with the different macOS version, to me the test results may support your theory about a faulty key. Now I just need a way to track down the faulty key amongst 1k+ keys on the mac that is seeing the crashes. Maybe a good time to clean up that keychain.

(In reply to steve from comment #66)
> So there are a few surprising test results to unpack here
> 
> a) most surprisingly, not a single crash happened
> b) a key that can be encrypted to via GUI or CLI tools besides LO can not be
> used to encrypt a document from within LO
> c) while I introduced another moving part in this testing with the different
> macOS version, to me the test results may support your theory about a faulty
> key. Now I just need a way to track down the faulty key amongst 1k+ keys on
> the mac that is seeing the crashes. Maybe a good time to clean up that
> keychain.

I also see b) occur when I try to encrypt with either of GPGTools' two keys. I can encrypt with my personal key that I created in GPG Keychain so I am guessing that this is probably a new bug?

As for c), I think that I might have an easier way to find the key that triggers the crash using the GPGME_DEBUG trick using the following steps.

1. Run LibreOffice in a Terminal using the following commands. When LibreOffice launches, open a new Writer document, save it, check the Encrypt with GPG checkbox, and wait for LibreOffice to crash:

   export GPGME_DEBUG=3:~/mygpgme.log                            
   /Applications/LibreOfficeDev.app/Contents/MacOS/soffice

2. Immediately after the crash, run the following Terminal command:

   grep gpgme_op_export: ~/mygpgme.log

3. Look at the last line of output from the above Terminal command and you should see something similar to the following:

   2024-03-04 09:06:06 gpgme[35457.8a81]     gpgme_op_export: enter: ctx=0x000000038f718140 pattern=29FCDC03537EBEC1B062405674A660EE7A65632E, mode=0x0, keydata=0x000000038f717a80

4. Copy the text between "pattern=" and the next "," and paste it the "Filter" text field in the top right corner of the GPG Keychain application main window.

5. Step 4 should result in a list of only one key. Disable that key and repeat the above steps until LibreOffice stops crashing.

If disabling a dozen or so keys still does not stop the crashing, then there is likely a different problem.

Yes, b) seems to be another implementation or gpgme bug in LO, which needs to be filed separately and we should ignore here for now, as it seems unrelated to the crashes.

Did various rounds of testing and kept removing or disabling the keys of which the fingerprint showed in the logs last line.

Did this over 10 times

Another crash, another fingerprint. this time it is an example key with fingerprint 38FBE1E4BF6A5E1242C8F6A13BDBEDB1777FBED3 and email romeo@example.net
It is a weak 1024 bit DSA key. Attaching this public key for further testing and disabled locally in GPG Keychain
Public key: https://upload.disroot.org/r/umbqJMxb#/Gqni+ccxh8GSMg+5BpeT3lK/F2fd+BzWctroEVGBAI=

Next: another 1024 bit DSA key, removed
Next: 4096 bit RSA key, disabled
Next: 3072 DSA key, removed
Next: first time getting to the key list, which took a while until it showed. Selecting a random key then resulted in a crash and error that key could not be used, which I doubt is true.
Next round and crash happens prior to showing the key list, log lists another 1024 DSA key, disabled.
Disabled another 1024 DSA key
Removed 4096 RSA key

Since this doesn't seem to be stopping, decided to massively reduce the number of keys by ~80%

Another few test runs and this keeps going. By now I doubt, the keys are causing the crash and there is another reason or it is just random timing? Could certainly be wrong. Sorry I can't provide more straight forward results this time around.

Created attachment 192948 [details]
Crash when encrypting a file during saving with public key in comment #68

Created attachment 192949 [details]
More readable crash log when encrypting a file during saving with public key in comment #68

(In reply to steve from comment #68)
> Since this doesn't seem to be stopping, decided to massively reduce the
> number of keys by ~80%
> 
> Another few test runs and this keeps going. By now I doubt, the keys are
> causing the crash and there is another reason or it is just random timing?
> Could certainly be wrong. Sorry I can't provide more straight forward
> results this time around.

So I downloaded the public key you linked to in comment #68 and tried to save an empty Writer file with that public key and LibreOffice crashed (see crash log in attachment #192949 [details]).

However, the crash occurs *after* the Select Certificate dialog appears and I select that key and save. So, on my Mac Silicon machine running macOS Sonoma, the crash occurs in the LibreOffice code when the Select Certificate dialog closes and just before it starts saving the file.

I don't know if your crash and my crash are related, but I will see if I can fix the crash that I am seeing and we can see if anything changes on your machine.

If so many keys trigger the crash even after remvoing 80% of your public key list, I am guessing that the LibreOffice or gpgme code is mishandling certain types of keys.

(In reply to Patrick Luby from comment #71)
> If so many keys trigger the crash even after remvoing 80% of your public key
> list, I am guessing that the LibreOffice or gpgme code is mishandling
> certain types of keys.

I am definitely seeing memory corruption of the UserData shared pointers for that particular key. By the time the Select Certificate dialog is closing and the list of UserData shared pointers are deleted, the public key I download from  the link in comment #68 is filled with invalid memory addresses.

So, I believe that loading that particular key is corrupting memory.

Was also wondering if there was a specific key format that would cause the corruption. Sadly the key fingerprints in the log were pointing to old 1024 bit DSA keys but also to 4096 bit RSA keys. The log pointed to some pub only keys and also sec/pub keys. So no clear pattern could be derived from that.

For completeness sake, here's a crash log of the persisting crashes I am seeing:
https://bin.disroot.org/?afe032ae5929b0cf#DwnkHsKS2Zi8TD7NtvUWpuDu9uBrBzLW3kXSpJ1LFza5

Maybe Intel vs Apple Silicon and or macOS 13 vs 14 makes a difference after all?

(In reply to steve from comment #73)
> Was also wondering if there was a specific key format that would cause the
> corruption. Sadly the key fingerprints in the log were pointing to old 1024
> bit DSA keys but also to 4096 bit RSA keys. The log pointed to some pub only
> keys and also sec/pub keys. So no clear pattern could be derived from that.
> 
> For completeness sake, here's a crash log of the persisting crashes I am
> seeing:
> https://bin.disroot.org/
> ?afe032ae5929b0cf#DwnkHsKS2Zi8TD7NtvUWpuDu9uBrBzLW3kXSpJ1LFza5
> 
> Maybe Intel vs Apple Silicon and or macOS 13 vs 14 makes a difference after
> all?

So I fixed the crash I was seeing but it was due to a newer, more compact, build configuration that is not yet used for the nightly builds. After rebuilding without my fix for that and without the newer build configuration, I don't have any crashes when encrypting with the public key in comment #68.

Unfortunately, our crash log looks the same as your other recent ones. Do you have a similar number of keys when your are running macOS 14? If not, would it be possible to export all of the keys in GPG Keychain to a file while running macOS 13 and then import the file into GPG Keychain on macOS 14?

Does that cause LibreOffice to crash on macOS 14? If yes, I am hoping that the crash log looks different than your macOS 13 crash logs. If no, then we may be dealing with a macOS 13 bug.

Patrick Luby committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/ce8d6b6836744045264eda41bcfdc4571c8b1b26

Related: tdf#152524 eliminate UserData struct name collision

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.

Sorry for the slow response, had a few busy days.

=== macOS 14.4 testing ===
macOS 14.4 via OCLP on 2012 iMac w Version: 24.8.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: a2265e8faa099d9652efd12392c2877c2df1d1eb
CPU threads: 4; OS: macOS 14.4; UI render: Skia/Raster; VCL: osx
Locale: en-US (en_US.UTF-8); UI: en-US
Calc: threaded

While the key-ring is not identical it is very similar to the macOS 13.6.5 mac.

Created new writer file, save as and tick Encrypt with GPG key option, stalls for a while but key list does show.

Encrypted document is successfully created and opens as expected when secret key exists on mac.

Digitally signing the document also worked fine and when document is opened again the header info about the signature is shown.

Another round: again, it took some stalling until the key list shows, that goes along the rainbow beachball spinning, BUT the key list does show after ~60 seconds. Key-ring holds ~1,2 k keys so it is not small.

What I get consistency on Ventura and Sonoma with is the keys LO is complaining about as 
That info is incorrect, CLI does allow utilizing the key in question as does the GPG Services GUI tool coming with GPG Suite.
"OpenPGP key not trusted, damaged, or encryption failure. Please try again."
Funny, that is even correct, so the problematic keys all had Ownertrust "Unknown". Changing that to Ultimate allows using the sec/pub key in question.
I'd say that is unexpected. There really is no reason to not allow encryption with keys with unknown ownertrust, so I have created https://bugs.documentfoundation.org/show_bug.cgi?id=160184

On Sonoma I could not get a crash, which is interesting. Both macs are intel macs so either Apple has changed things in Sonoma so that the crash issue seen on Ventura is no longer happening or something around gpgme on the Ventura mac is very broken.

=== macOS 13.6.5 testing ===
This crash on macOS 13.6.5 happened after opening LO and not interacting with it:
https://bin.disroot.org/?0d3d0bc5bffbd5a3#GyAdzw3yeVn1s2rERVSi4tz4WRdt7S5eJvRaaHc8u2nj
Version: 24.8.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: a2265e8faa099d9652efd12392c2877c2df1d1eb
CPU threads: 8; OS: macOS 13.6.5; UI render: Skia/Metal; VCL: osx
Locale: en-US (en_US.UTF-8); UI: en-US
Calc: threaded

In the next try to encrypt a document seeing an instant crash (no key dialog) along with the General input/output error message:
https://bin.disroot.org/?5f0d53b9e3bdd27f#8X7YJWjs3u83FEbjvhHKLL7jR8oG2t82qxjsqKHYiArc

Third attempt was successful in creating an encrypted file. Adjusting title to reflect macOS 13 only.

(In reply to steve from comment #76)
> In the next try to encrypt a document seeing an instant crash (no key
> dialog) along with the General input/output error message:
> https://bin.disroot.org/
> ?5f0d53b9e3bdd27f#8X7YJWjs3u83FEbjvhHKLL7jR8oG2t82qxjsqKHYiArc
> 
> Third attempt was successful in creating an encrypted file. Adjusting title
> to reflect macOS 13 only.

Thank you for the detailed testing. Both of your Ventura crash logs are identical to your past ones: memory corruption when the gpgme code fetches one of the third-party public keys.

So what is interesting to me is that your third attempt didn't crash. This is a wild guess, but maybe LibreOffice is hitting some memory or other system resource limit while loading all of your third-party public keys. I remember you deleted something like 80% of your public keys, is that still the case?

Do you have Apple's Xcode installed on Ventura? If yes, it would be interesting to see to connect Xcode's bundled Instruments application to a LibreOffice nightly build and watch its memory usage graph between pressing the Encrypt button in the Save As dialog up until the crash.

Note: Xcode is available from Apple for free from the Mac App Store. The downside is that it requires up to 10 GB of free disk space. But if you have it installed, I can post instructions for watching LibreOffice memory usage in real time.

Xcode is installed.

Number of keys macOS Ventura ~200
Number of keys macOS Sonoma ~1,2k

Happy to provide instruments debug data if you can link or provide (ideally beginner proof) instructions.

(In reply to steve from comment #78)
> Xcode is installed.
> 
> Number of keys macOS Ventura ~200
> Number of keys macOS Sonoma ~1,2k
> 
> Happy to provide instruments debug data if you can link or provide (ideally
> beginner proof) instructions.

I can provide instructions. But I'm a bit swamped this week so, in the meantime, can you try the following steps first? These steps run LibreOffice in Xcode's debugger (i.e. the "lldb" command). My hope is that by running LibreOffice within the debugger, we might get a more detailed crash log since the debugger will literally halt LibreOffice at the moment a crash occurs.

1. Make sure you have a recent nightly master build installed in /Applications/LibreOfficeDev.app. The debugger will not work with official release builds.

2. Open the Terminal application and, in the Terminal window, execute the following command:

   lldb /Applications/LibreOfficeDev.app/Contents/MacOS/soffice

3. lldb will then show the "(lldb)" prompt. Enter "run" and press the Return key to launch LibreOffice.

4. Open a new Writer document and saves with GPG encryption. When LibreOffice crashes, you will see the "(lldb)" prompt appear in the Terminal and LibreOffice will appear hung. At this point, enter "bt all" and press the Return key at the "(lldb)" prompt.

5. Copy the output of "bt all" and post or attachment the copied output. This is lldb's crash log.

6. Enter "quit" and press the Return key at the "(lldb)" prompt to exit lldb.

"When LibreOffice crashes, you will see the "(lldb)" prompt appear in the Terminal and LibreOffice will appear hung. At this point, enter "bt all" and press the Return key at the "(lldb)" prompt."

Managed to start LO as per your steps via terminal and lldb.

The crash happened but I did not see any prompt in terminal.

This is the outpit until the crash happened (when trying to save the encrypted file, which resulted in an instant crash):
https://bin.disroot.org/?811ab07788190f9e#6qrW8mZnCZj1TqoyekzkWSeqM5MxScX8djkaBR7pCqof

I can't seem to execute any prompt at this point and see no way to close the debug session.

Is lldb misbehaving or am I doing something wrong?

(In reply to steve from comment #80)
> I can't seem to execute any prompt at this point and see no way to close the
> debug session.
> 
> Is lldb misbehaving or am I doing something wrong?

Weird. LibreOffice crashes but your output looks like lldb still thinks LibreOffice is still running.

Can you go to the Terminal that lldb is running in and press the Control-C keys (not Command-C)? That should cause lldb to pause and display the "(lldb)" prompt.

If you can get the "(lldb)" prompt. Type "bt all" (maybe there is some useful data, maybe not) and then type "quit" to end the lldb sesssion.

This is  the crash that happens on trying to save + encrypt with gpg key:
https://bin.disroot.org/?ac8b05df0778cb21#9whdhDHLdvHC2ctZeZNkW2PppPc1Y9N7FdfJ3NXL39vK

Just to be clear: the crash happens, but LibreOffice can still be used, work on the document can continue, saving the document is possible etc.

Maybe that's the reason lldb does not recognize the crash?

This is the bt all output after the crash happened and ctrl + c:
https://bin.disroot.org/?89bbc4e2cfc64b81#3xWbXUXe5tTTe6BNPqL5fhNDbTDbUwPGuhwQuputYRH8

(In reply to steve from comment #82)
> Just to be clear: the crash happens, but LibreOffice can still be used, work
> on the document can continue, saving the document is possible etc.

That is very interesting. So when you run in lldb and see a crash but LibreOffice keeps running, that tells me the crash is in the first child process created by gpgme, not in the LibreOffice (parent) process. So, my guess is that when running normally, the crash in the first process takes down the parent process but LibreOffice is someone shielded from begin taken down.

One question, when running LibreOffice in lldb and a crash occurs, does nothing happen in LibreOffice or does some sort of error dialog appear? Also, are you able to retry Save As with encrypting?

Lastly, can you quit LibreOffice normally after a crash while running in lldb? Note: if you can quit normally, just type "quit" in exit lldb.

Before the macOS Crash report shows, an error message in LO shows:

Error saving the document Untitled 1:
General Error.
General input/output error.

If I click "Ok" (only option) and repeat save as + encrypt with gpg key, the same repeats, i.e. LO error message briefly followed by macOS "soffice quit unecpectedly" crash report.

When pressing cmd + q after launching from lldb I can save (but not encrypt) the file and then exit lldb via terminal.

(In reply to steve from comment #84)
> Before the macOS Crash report shows, an error message in LO shows:
> 
> Error saving the document Untitled 1:
> General Error.
> General input/output error.
> 
> If I click "Ok" (only option) and repeat save as + encrypt with gpg key, the
> same repeats, i.e. LO error message briefly followed by macOS "soffice quit
> unecpectedly" crash report.
> 
> When pressing cmd + q after launching from lldb I can save (but not encrypt)
> the file and then exit lldb via terminal.

I think the above confirms that the crash is occuring in a child process of LibreOffice that is crashing, not LibreOffice itself. It's a minor difference but after staring at gpgme's src/posix-io.c file, I found a way to implement the suggestion in comment #34 to replace the second fork() and its matching exec() calls with a posix_spawn() call.

I have uploaded the following patch. The idea behind the patch is that since the crash is occurring in a system "atfork" handler function, we can prevent the crashing atfork handler function from running using posix_spawn(). posix_spawn() creates a child process much more cleanly than fork() and exec() so it never calls any atfork handler functions:

https://gerrit.libreoffice.org/c/core/+/165103

Patrick Luby committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/3c6c5ef5d1c4f555b465bf56cf9d99e4d67224cc

tdf#152524 fix crash by skipping second fork()

It will be available in 24.8.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.

I have committed my latest attempt to fix this bug. The fix should be in tomorrow's (22 March 2024) nightly master builds. It will be interesting to hear if it either stops the crashing. If not, I am expecting that the crash log will look different than previous ones:

https://dev-builds.libreoffice.org/daily/master/current.html

Note for macOS testers: the nightly master builds install in /Applications/LibreOfficeDev.app. These builds are not codesigned like regular LibreOffice releases so you will need to execute the following Terminal command after installation but before you launch /Applications/LibreOfficeDev:

xattr -d com.apple.quarantine /Applications/LibreOfficeDev.app

Opening LO still results in beachball showing for a minute. For that time it is unresponsive. No crash happened. This only seems to happen on the very first open and unrelated to the issue here.

- open writer, save as > encrypt with gpg key
- Select Certificate opens
- select a public key with ultimate ownertrust
- encrypted file is created

Awesome. Looks like your patch resolves the issue 🎊

(In reply to steve from comment #88)
> Opening LO still results in beachball showing for a minute. For that time it
> is unresponsive. No crash happened. This only seems to happen on the very
> first open and unrelated to the issue here.
> 
> - open writer, save as > encrypt with gpg key
> - Select Certificate opens
> - select a public key with ultimate ownertrust
> - encrypted file is created
> 
> Awesome. Looks like your patch resolves the issue 🎊

Excellent! It's too late to include the fix in the upcoming LibreOffice 24.2.2 release, but I'll backport the fix so that it will be included in LibreOffice 24.2.3.

As for the beachball at startup, can you open a new performance bug for that? I don't have any specific ideas on how to fix that yet, but I know LibreOffice loads all of your certificates sequentially so maybe I can see if we can delay loading each certificate's details until it is selected in the certificate selection dialog.

Side question for you and @Thorsten: I noticed that you can select multiple certificates in the the certificate selection dialog when encrypting with GPG. Is that expected?

I'll file that performance bug in the coming days. Maybe it will be end of next week.

As your your other question: There is nothing wrong with encrypting a file with as many public keys as you like. Whoever then has access to the matching secret key part + passphrase will be able to decrypt the file.

Patrick Luby committed a patch related to this issue.
It has been pushed to "libreoffice-24-2":

https://git.libreoffice.org/core/commit/be132c413c49175a27a55cfe3e42c748b5660a92

tdf#152524 fix crash by changing the macOS fork() and exec() process

It will be available in 24.2.3.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.

Patrick Luby committed a patch related to this issue.
It has been pushed to "libreoffice-24-2-2":

https://git.libreoffice.org/core/commit/9012b402ec25c5eb17400f93ffdc73e3db3a122a

tdf#152524 fix crash by changing the macOS fork() and exec() process

It will be available in 24.2.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.

(In reply to steve from comment #90)
> I'll file that performance bug in the coming days. Maybe it will be end of
> next week.
> 
> As your your other question: There is nothing wrong with encrypting a file
> with as many public keys as you like. Whoever then has access to the
> matching secret key part + passphrase will be able to decrypt the file.

Good to know. If I'm able to figure out how to delay loading a certificate's details until it is selected, I am hoping we can avoid the temporary hang if you have a large amount of certificates. Of course, if you select a large number of certificates, the temporary hang will occur then.

Patrick Luby committed a patch related to this issue.
It has been pushed to "libreoffice-7-6":

https://git.libreoffice.org/core/commit/74e3a9e4dce18b2acd400b5a350a31605b5cc0e2

tdf#152524 fix crash by changing the macOS fork() and exec() process

It will be available in 7.6.7.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.

(In reply to steve from comment #88)
> Opening LO still results in beachball showing for a minute. For that time it
> is unresponsive. No crash happened. This only seems to happen on the very
> first open and unrelated to the issue here.

I forgot to mention that when you file a bug for the above, can you take attach a sample while the beachball is spinning using the /Applications/Utilities/Activity Monitor application? That should help me narrow down what code is taking so much time.

Follow-up bug with process sample while LO hangs: https://bugs.documentfoundation.org/show_bug.cgi?id=160699