Description: libcurl is going to remove the backend we use at some point, so we will need to start using the libcurl that ships with macOS See thread here: https://curl.se/mail/lib-2025-01/0052.html Actual Results: Expected Results: Reproducible: Always User Profile Reset: Yes Additional Info:
Set to NEW
(In reply to Noel Grandin from comment #0) > See thread here: > https://curl.se/mail/lib-2025-01/0052.html I don't have any experience with using curl self-signed certificate so I did some searching to try and test if the curl that Apple bundles with macOS works with a certificate stored in the Keychain Access application. 1. I added a self-signed certificate using the steps in the following Apple support page and named the new certicate "My Cert": https://support.apple.com/en-ca/guide/keychain-access/kyca8916/mac 2. I then ran the following Terminal command: CURL_SSL_BACKEND=secure-transport /usr/bin/curl -v --cert 'My Cert' 'https://bugs.documentfoundation.org/' Unfortunately, the above command always fails with the following error: SSL: Can't find the certificate "My Cert" and its private key in the Keychain. Any ideas what I am doing wrong? Below is the curl version output on macOS Sequoia: /usr/bin/curl --version curl 8.7.1 (x86_64-apple-darwin24.0) libcurl/8.7.1 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.63.0 Release-Date: 2024-03-27 Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL threadsafe UnixSockets
I found the cause of the failure I saw in comment #2. I recreated a "My Cert" certificate and this time checked the "Let me override defaults" in the Keychain Access dialog. Then I made sure the "SSL client" checkbox was checked. So now I have confirmed that at least macOS Sequoia's bundled libcurl has access to the Keychain Access certificate store. Is this the use case that the switch is meant to handle? Or are there other use cases?
One more detail: the curl command bundled with macOS Sequoia only supports --tlsv1.2. --tlsv1.3 will fail. So is being stuck at TLS v1.2 a problem?
(In reply to Patrick (volunteer) from comment #4) > One more detail: the curl command bundled with macOS Sequoia only supports > --tlsv1.2. --tlsv1.3 will fail. So is being stuck at TLS v1.2 a problem? This is weird, because this message: https://curl.se/mail/lib-2025-01/0062.html indicates that the macOS supports TLS1.3 But yes, being stuck at TLS1.2 is a problem, servers will stop supporting that at some point.
(In reply to Noel Grandin from comment #5) > indicates that the macOS supports TLS1.3 I think TLS 1.3 is limited to the LibreSSL backend but that does not appear to be able to read certificates in the Keychain Access application. Using the following command (note that no backend is specified): /usr/bin/curl -v --tlsv1.3 --cert 'My Cert' 'https://bugs.documentfoundation.org/' ...I get the following error: curl: (4) A requested feature, protocol or option was not found built-in in this libcurl due to a build-time decision. Using the following command with Secure Transport: CURL_SSL_BACKEND=secure-transport /usr/bin/curl -v --tlsv1.3 --cert 'My Cert' 'https://bugs.documentfoundation.org/' ...works, but I see the following in the output: * Client certificate: My Cert * TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 > But yes, being stuck at TLS1.2 is a problem, servers will stop supporting > that at some point. So, from my limited testing, it looks like Secure Transport is stuck at TLS 1.2. Maybe switching to system libcurl may by us some time though until either the cURL developers or Apple implements some replacement for Secure Transport?
@Buovjaga do you know of any steps for using a certificate in the Keychain Access application within LibreOffice? I see Firefox and Thunderbird's certificate stores as well as GPG's store but I am not familiar with how users are using the Keychain Access certificates.
(In reply to Patrick (volunteer) from comment #7) > @Buovjaga do you know of any steps for using a certificate in the Keychain > Access application within LibreOffice? I see Firefox and Thunderbird's > certificate stores as well as GPG's store but I am not familiar with how > users are using the Keychain Access certificates. Let's ask Michael.
(In reply to Buovjaga from comment #8) > Let's ask Michael. While we're waiting for how to test this cURL functionality, what I can do is try setting up a simple Xcode project (I will upload it) that extracts the certificate for a particular name (I assume LibreOffice stores a name or hash?) and verifies that it is trusted. I think we can get an X.509 certicate from the macOS Keychain using the following: https://developer.apple.com/documentation/security/storing-a-der-encoded-x-509-certificate?language=objc ...and then verify that it is trusted using the following: https://developer.apple.com/documentation/security/sectrustevaluatewitherror(_:_:)?language=objc Here is where things get hazy for me: once we have a trusted X.509 certificate in memory, can we then pass the memory as a cURL option? I was looking at the following link and was thinking that we use it except that CURLOPT_SSL_VERIFYPEER would be set to 0 (since we already verified trust natively) and sslctx_function() wouldn't do anything and would just return CURLE_OK: https://curl.se/libcurl/c/CURLOPT_SSL_CTX_DATA.html Does that sound reasonable?
LibreOffice has a document signature feature where the user can choose X509 certificates from Mozilla products' profiles; on Windows, the system certificate store is used, and the same could be done on MacOSX in theory. this is entirely unrelated to any network connections, and the WebDAV/HTTP UCP simply relies on libcurl to get the certificates. for Linux systems, libcurl is built with the OpenSSL backend, and we configure that to read system certificates from various paths in include/systools/opensslinit.hxx. my experience with MacOSX is limited to version 10.4, so you're probably not interested in it :)
(In reply to Michael Stahl (allotropia) from comment #10) > LibreOffice has a document signature feature where the user can choose X509 > certificates from Mozilla products' profiles; on Windows, the system > certificate store is used, and the same could be done on MacOSX in theory. OK. So that would be a separate new feature for macOS and isn't related to this bug. Let me know if my understanding is incorrect. > this is entirely unrelated to any network connections, and the WebDAV/HTTP > UCP simply relies on libcurl to get the certificates. > > for Linux systems, libcurl is built with the OpenSSL backend, and we > configure that to read system certificates from various paths in > include/systools/opensslinit.hxx. I did some reading today and maybe I am missing something, but it sounds to me like we are only using cURL's "secure-transport" backend to validate the server certificates during SSL connection initialization. Let me know if there are other cases that cURL is using "secure-transport". But assuming the above, it appears that cURL can be built with the WolfSSL backend and that has support for the macOS Keychain. However, WolfSSL is GPL v2 so we can't include that in a Mac App Store version even if we put WolfSSL into a separate process due to Apple's App Store rules. So the next idea I found was to build cURL on macOS with the OpenSSL backend like Linux. But instead of looking for files and folders, maybe we can override the validation function and set CURLOPT_SSL_CTX_FUNCTION to our own custom function. Our custom function can then do validation of the server's X.509 certificates using native calls to the macOS Keychain: https://curl.se/libcurl/c/CURLOPT_SSL_CTX_FUNCTION.html I think our custom function would get the server's current certificate using SSL_CTX_get0_certificate(). Does this sound reasonable? I admit that I am far from an expert in any of this. But if this feasible, maybe it would also be a feasible replacement for "stunnel" on Windows? Any criticism of the above is welcome.
at first glance, CURLOPT_SSL_CTX_FUNCTION might work to verify the certificates.
(In reply to Michael Stahl (allotropia) from comment #12) > at first glance, CURLOPT_SSL_CTX_FUNCTION might work to verify the > certificates. Is anyone looking at cURL on Windows? I am thinking of just trying to create a hacky, ugly patch (maybe for HTTPS connections?) just to test out this concept.
curl seems to be working fine on Windows? at least i haven't heard any complaints, and the STunnel thing seems to support TLS v1.3 so it doesn't meet upstream's criteria for removing backends.
OK. I'll add this bug to my "todo" list.
Unassigning myself as I am not going to have any time for this in the next few months.
Hi Noel, Patrick, *, I'm wondering if https://gerrit.libreoffice.org/c/core/+/201850 works for you. I don't really now how to test it. it would be great if someone could provide the steps to test it. I asked in curl's IRC and this is what I got: x1sc0> hello, can --with-apple-sectrust be consider a replacement of --with-secure-transport ? <bagder> if what you want is access to the native CA store, sure
Hi Patrick, Reading your comment in comment 11, it seems secure-transport is only used for SSH connections. I tried to create a remote files connection on macOS with this settings but it doesn't work server: test.rebex.net port: 22 user: demo password: password OTOH, it works on Linux. Am I missing something ?
(In reply to Xisco Faulí from comment #18) > Hi Patrick, > Reading your comment in comment 11, it seems secure-transport is only used > for SSH connections. Unfortunately, this week is turning into a very busy week for me so I won't be able to spend any time on this until maybe next week. I can't remember much about this bug but I think I was assuming that this was used to load an HTTPS URL e.g. loading an HTML document in Writer using the URL toolbar that is usually hidden.
(In reply to Patrick (volunteer) from comment #19) > I can't remember much about this bug but I think I was assuming that this > was used to load an HTTPS URL e.g. loading an HTML document in Writer using > the URL toolbar that is usually hidden. OK. I had a few spare minutes so I rebuilt my local master build with the patch in comment #17. I then opened Writer, selected Tools > Customize menu item, selected Toolbars in the dialog, checked "Load URL", and pressed the OK button. I then pasted "https://bugs.documentfoundation.org/" into the Load URL toolbar and the URL loaded in a Writer document. Don't have time to look at the cURL code so I don't know if the https: URL loading is going through cURL or not. Hope that helps.
Hi Patrick, According to the comment from external/curl/ExternalProject_curl.mk "use --with-secure-transport on macOS >10.5 and iOS to get a native UI for SSL certs for CMIS usage", I would expect a native UI for SSL certs to pop up at some point, but that's not the case with the steps you described in comment 20 so I'm not sure those are the rights steps to make use of secure-transport
(In reply to Patrick (volunteer) from comment #20) > I then pasted "https://bugs.documentfoundation.org/" into the Load URL > toolbar and the URL loaded in a Writer document. > > Don't have time to look at the cURL code so I don't know if the https: URL > loading is going through cURL or not. don't know this toolbar, but try like this, it will give very verbose output from the most important curl client: SAL_LOG="+RELATIVETIMER+INFO.ucb.ucp.webdav+WARN" instdir/program/soffice (In reply to Xisco Faulí from comment #21) > "use --with-secure-transport on macOS >10.5 and iOS to get a native UI for > SSL certs for CMIS usage" this refers to the UI in some configuration application of the operating system, e.g. for Windows it's `certmgr.msc`, where the user can manage CA certificate trust. no certificate related UI should be shown while you use curl in LO.
(In reply to Michael Stahl from comment #22) > don't know this toolbar, but try like this, it will give very verbose output > from the most important curl client: > SAL_LOG="+RELATIVETIMER+INFO.ucb.ucp.webdav+WARN" instdir/program/soffice Thanks for the tip. Interestingly, in my debug/dbgutil local build with @Xisco's patch, I do not see any ucb.ucp.webdav messages in stdout/stderr which makes me think that the failure is occurring before we setup a cURL session. I started chasing down where in the code the failure is actually occurring. So far, I have narrowed it down to the following line. When I have time, I keep pushing further down the stack into ucbhelper and ucb code: diff --git a/fpicker/source/office/contentenumeration.cxx b/fpicker/source/office/contentenumeration.cxx index 1c35f32e2b88..d2e82c8d823d 100644 --- a/fpicker/source/office/contentenumeration.cxx +++ b/fpicker/source/office/contentenumeration.cxx @@ -171,6 +171,7 @@ namespace } if ( !aFolder.aContent.get().is() ) { +fprintf(stderr, "The following line throws an exception for sftp on macOS\n"); aFolder.aContent = ::ucbhelper::Content( aFolder.sURL, xEnvironment, comphelper::getProcessComponentContext() ); { std::unique_lock aGuard( m_aMutex );
OK. Found where sftp (i.e. SSH service in Remote Files dialog) is failing. Apparently there is no provider found in UniversalContentBroker::queryContentProvider() so I think this confirms my suspicion that we never get to the cURL code on macOS. Any suggestions where to look next?: diff --git a/ucb/source/core/ucb.cxx b/ucb/source/core/ucb.cxx index 43e5ca0ab096..feb8c879da6b 100644 --- a/ucb/source/core/ucb.cxx +++ b/ucb/source/core/ucb.cxx @@ -713,6 +713,8 @@ Reference< XContentProvider > UniversalContentBroker::queryContentProvider( osl::MutexGuard aGuard( m_aMutex ); ProviderList_Impl const * pList = m_aProviders.map( Identifier ); +if (Identifier.startsWith("sftp:")) +fprintf(stderr, "pList is a nullptr for sftp on macOS: %i %s %p\n", bResolved, Identifier.toUtf8().getStr(), pList); return pList ? bResolved ? pList->front().getResolvedProvider() : pList->front().getProvider() : Reference< XContentProvider >();
Created attachment 206323 [details] Debug patch that forces webdav code to be used for sftp URLs in the Remote Files dialog
I hack together a debug patch in attachment #206323 [details] that gets SSH entries in the Remote Files dialog to actually call the cURL code and ran LibreOffice with the following command: SAL_LOG="+RELATIVETIMER+INFO.ucb.ucp.webdav+WARN" ./instdir/LibreOfficeDev.app/Contents/MacOS/soffice With the debug patch, I now see several of the following "curl_easy_perform failed" warnings: warn:ucb.ucp.webdav.curl:6296:137288:ucb/source/ucp/webdav-curl/CurlSession.cxx:1077: curl_easy_perform failed: (1) Protocol "sftp" disabled Seems strange that I had to add registry entries and define a new enumeration value so I wonder if sftp URLs ever worked at all on macOS. Does Linux use some other code path that I should look at?
(In reply to Patrick (volunteer) from comment #26) > I hack together a debug patch in attachment #206323 [details] that gets SSH > entries in the Remote Files dialog to actually call the cURL code and ran > LibreOffice with the following command: > > SAL_LOG="+RELATIVETIMER+INFO.ucb.ucp.webdav+WARN" > ./instdir/LibreOfficeDev.app/Contents/MacOS/soffice > > With the debug patch, I now see several of the following "curl_easy_perform > failed" warnings: > > warn:ucb.ucp.webdav.curl:6296:137288:ucb/source/ucp/webdav-curl/CurlSession. > cxx:1077: curl_easy_perform failed: (1) Protocol "sftp" disabled > > Seems strange that I had to add registry entries and define a new > enumeration value so I wonder if sftp URLs ever worked at all on macOS. Does > Linux use some other code path that I should look at? So we don't use system curl on Mac by default (right?) but even if we did, Tahoe "curl -V" doesn't list sftp as a supported protocol. But even with built-in curl, I see this in config.log: $ ./configure --disable-ftp --enable-http --enable-ipv6 --without-libidn2 --without-libpsl --without-librtmp --without-libssh2 --without-nghttp2 --without-libssh --without-brotli --without-ngtcp2 --without-qu that --without-libssh2/--without-libssh means we don't built internal curl with SFTP support either, I believe.
(In reply to dcbw@libreoffice.org from comment #27) > (In reply to Patrick (volunteer) from comment #26) > > I hack together a debug patch in attachment #206323 [details] that gets SSH > > entries in the Remote Files dialog to actually call the cURL code and ran > > LibreOffice with the following command: > > > > SAL_LOG="+RELATIVETIMER+INFO.ucb.ucp.webdav+WARN" > > ./instdir/LibreOfficeDev.app/Contents/MacOS/soffice > > > > With the debug patch, I now see several of the following "curl_easy_perform > > failed" warnings: > > > > warn:ucb.ucp.webdav.curl:6296:137288:ucb/source/ucp/webdav-curl/CurlSession. > > cxx:1077: curl_easy_perform failed: (1) Protocol "sftp" disabled > > > > Seems strange that I had to add registry entries and define a new > > enumeration value so I wonder if sftp URLs ever worked at all on macOS. Does > > Linux use some other code path that I should look at? > > So we don't use system curl on Mac by default (right?) but even if we did, > Tahoe "curl -V" doesn't list sftp as a supported protocol. > > But even with built-in curl, I see this in config.log: > > $ ./configure --disable-ftp --enable-http --enable-ipv6 --without-libidn2 > --without-libpsl --without-librtmp --without-libssh2 --without-nghttp2 > --without-libssh --without-brotli --without-ngtcp2 --without-qu > > that --without-libssh2/--without-libssh means we don't built internal curl > with SFTP support either, I believe. Does any system using the LO-provided external curl have SFTP support? Because it doesn't look like it to me...
sorry, i missed that you were trying `sftp` scheme - that is not supported in by the `webdav` UCP at all, it only supports `http`/`https`/`webdav`/`webdavs`. on Linux systems you can use `gvfs` UCP to access `sftp` URLs. also, depending on the system, it may or may not be possible to input `http` URLs in the file picker dialog; what should always work is to invoke `soffice` with a `http` URL on the command line.
(In reply to Michael Stahl from comment #29) > on Linux systems you can use `gvfs` UCP to access `sftp` URLs. gvfs appears to have been ported to macOS. It's not bundled with macOS but MacPorts (https://ports.macports.org/port/gvfs/) provides binaries. It's not ideal, but if installing MacPorts and gvfs is something the LibreOffice users who would use this feature could live with, then maybe we could use the same approach as Linux? > also, depending on the system, it may or may not be possible to input `http` > URLs in the file picker dialog; what should always work is to invoke > `soffice` with a `http` URL on the command line. So I think I have confirmed that loading https using libcurl plus the patch in comment #17 does use the macOS native certificate store. Here's what I did: Running in lldb, break on SecPolicyCreateSSL (macOS function), open an empty Writer document, enabled the "Load URL" toolbar item, and enter https://bugs.documentfoundation.org/show_bug.cgi?id=164781 in the Load URL's combo box. Once lldb hit the break, I next'd through the calling libcurl function Curl_vtls_apple_verify() until it returned. It returned CURLE_OK for me and I saw it loop through some certs so I think libcurl is using the native certificate store.
Hi Patrick, What happens if you remove line 55 from external/curl/ExternalProject_curl.mk from my patch $(if $(filter iOS MACOSX,$(OS)),--with-apple-sectrust) \ Does it fail to use macOS native certificate store ?
(In reply to Xisco Faulí from comment #31) > Hi Patrick, > What happens if you remove line 55 from > external/curl/ExternalProject_curl.mk from my patch > > $(if $(filter iOS MACOSX,$(OS)),--with-apple-sectrust) \ > > Does it fail to use macOS native certificate store ? Interestingly, after a "make clean" in external/curl and then "make" at the top of the tree, the "Load URL" toolbar button still loads https://bugs.documentfoundation.org/show_bug.cgi?id=164781. But removing the --with-apple-sectrust definitely stops linking of the macOS Security framework so I assume cURL is using its own internal certificate store. Here's the differences I see: 1. otool -L ./instdir/LibreOfficeDev.app/Contents/Frameworks/libcurl.4.dylib: - with --with-apple-sectrust: macOS Security framework in link list - without --with-apple-sectrust: Security framework *not* in link list 2. nm -u ./instdir/LibreOfficeDev.app/Contents/Frameworks/libcurl.4.dylib: - with --with-apple-sectrust: includes the following Security framework unctions: _SecCertificateCreateWithData _SecPolicyCreateRevocation _SecPolicyCreateSSL _SecTrustCreateWithCertificates _SecTrustEvaluateWithError _SecTrustSetOCSPResponse - without --with-apple-sectrust: does not include any Security framework functions. Hope that helps.
After the discussion here, I pushed 8a9e4e8631f8b80d9d7013dbcd7e4fe3693bb2f6 to master. I guess we can close this issue now