Bug 128220 - Fatal Python error: Python memory allocator called without holding the GIL
Summary: Fatal Python error: Python memory allocator called without holding the GIL
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
6.3.2.2 release
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-10-17 20:28 UTC by julien
Modified: 2020-05-12 14:36 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description julien 2019-10-17 20:28:29 UTC
Hi!

On Debian Bullseye, with

$ libreoffice --version
LibreOffice 6.3.2.2 30(Build:2)

    $ apt-cache policy libreoffice-common
    libreoffice-common:
      Installed: 1:6.3.2-1

If I start libreoffice and click "Writer Document" to create a document I'm getting a crash:

    $ libreoffice
    Fatal Python error: Python memory allocator called without holding the GIL

It uses /lib/x86_64-linux-gnu/libpython3.7m.so.1.0 from Debian package libpython3.7 3.7.5~rc1-2.
Comment 1 Julien Nabet 2019-10-18 08:12:17 UTC
Bullseye seems to correspond to testing version.
On pc Debian testing (updated yesterday) x86-64, I don't reproduce this.

Did you install packages only from official Debian testing repo using apt-get or did you external packages with dpkg or some packages from unstable?
Also, for the test, could you rename your LO directory profile (see https://wiki.documentfoundation.org/UserProfile#GNU.2FLinux)?
Comment 2 julien 2019-10-19 15:07:48 UTC
Hi!

> Did you install packages only from official Debian testing repo using apt-get or did you external packages with dpkg or some packages from unstable?

TL;DR: You should install libreoffice-voikko to reproduce the issue.

I'm having only two sources: Debian testing (Bulleye) and Debian sid. Almost all my packages are from testing, see:

    $ apt list --installed | grep -v 'testing\|firmware\|microcode'
    ecryptfs-utils/unstable,now 111-4 amd64 [installed]
    firefox/unstable,now 69.0.2-1 amd64 [installed]
    iucode-tool/now 2.3.1-1 amd64 [installed,local]
    libecryptfs1/unstable,now 111-4 amd64 [installed,automatic]
    libjpeg-progs/unstable,now 1:9c-2 amd64 [installed,automatic]
    libjpeg9/unstable,now 1:9c-2 amd64 [installed,automatic]

Yesterday I uninstalled libreoffice, to test your idea I re-installed it today and I'm no longer having the issue.

Let's investigate:

- I had the issue.
- I uninstalled using `aptitude purge ~ilibreoffice`.
- I reinstalled using `apt install libreoffice` (probably a smaller subset)
- I'm no longer having the issue.

I parsed my /var/log/apt/history using a bit of Python like:

>>> installed = {s.strip() for s in purged.replace(", automatic", "").split("\n")[2].split(":", 1)[1].split(",")}
>>> uninstalled = {s.strip() for s in purged.split("\n")[0].split(":", 1)[1].split(",")}

The purge uninstalled 110 packages, the install installed 185.

This give me the list of packaged I had but I miss (99 of them), composed of:
- libreoffice-gnome:amd64 (1:6.3.2-1+b1)
- libreoffice-gtk3:amd64 (1:6.3.2-1+b1)
- libreoffice-help-*
- libreoffice-l10n-*
- libreoffice-style-elementary:amd64 (1:6.3.2-1)
- libreoffice-voikko:amd64 (5.0-3+b1)


libreoffice-gtk3 and libreoffice-style-elementary are dependencies of libreoffice-gnome, installing the 3 of them don't bring the bug back.

help-* and l10n-* are not suspected.

I install libreoffice-voikko, doing so I spot this has python3-libvoikko as a dependency, red flag.

BOOM I can reproduce the bug!

$ apt-cache policy libreoffice-voikko
libreoffice-voikko:
  Installed: 5.0-3+b1
Comment 3 Julien Nabet 2019-10-19 16:16:46 UTC
Timo/Rene: thought you might be interested in this one

On pc Debian testing x86-64 updated today, I installed libreoffice-voikko
root@debianamd:~# apt-cache show libreoffice-voikko
Package: libreoffice-voikko
Source: libreoffice-voikko (5.0-3)
Version: 5.0-3+b1

which installed:
root@debianamd:~# apt-cache show python3-libvoikko
Package: python3-libvoikko
Source: libvoikko
Version: 4.2-2

I started LO and had startcenter, then I click on Writer icon, no crash here.
which python:
/usr/bin/python
lrwxrwxrwx 1 root root 7 oct.  10 13:32 /usr/bin/python -> python2

julien@debianamd:~$ which python3
/usr/bin/python3
lrwxrwxrwx 1 root root 9 oct.   2 14:31 /usr/bin/python3 -> python3.7

julien@debianamd:~$ apt-cache show python2
Package: python2
Source: python-defaults
Version: 2.7.17-1

Package: python3
Source: python3-defaults
Version: 3.7.5-1
Comment 4 Julien Nabet 2019-10-19 16:20:05 UTC
Julien: I type some text and indicated it was finnish. Then I call spell checking, still no crash
Comment 5 julien 2019-10-19 19:58:11 UTC
If it can help:

here's my apt list --installed | grep libreoffice:

libreoffice-base-core/testing,now 1:6.3.2-1+b1 amd64 [installed,automatic]
libreoffice-base-drivers/testing,now 1:6.3.2-1+b1 amd64 [installed,automatic]
libreoffice-base/testing,now 1:6.3.2-1+b1 amd64 [installed,automatic]
libreoffice-calc/testing,now 1:6.3.2-1+b1 amd64 [installed,automatic]
libreoffice-common/testing,now 1:6.3.2-1 all [installed,automatic]
libreoffice-core/testing,now 1:6.3.2-1+b1 amd64 [installed,automatic]
libreoffice-draw/testing,now 1:6.3.2-1+b1 amd64 [installed,automatic]
libreoffice-gnome/testing,now 1:6.3.2-1+b1 amd64 [installed]
libreoffice-gtk3/testing,now 1:6.3.2-1+b1 amd64 [installed]
libreoffice-impress/testing,now 1:6.3.2-1+b1 amd64 [installed,automatic]
libreoffice-java-common/testing,now 1:6.3.2-1 all [installed,automatic]
libreoffice-math/testing,now 1:6.3.2-1+b1 amd64 [installed,automatic]
libreoffice-nlpsolver/testing,now 0.9+LibO6.3.2-1 all [installed,automatic]
libreoffice-report-builder-bin/testing,now 1:6.3.2-1+b1 amd64 [installed,automatic]
libreoffice-report-builder/testing,now 1:6.3.2-1 all [installed,automatic]
libreoffice-script-provider-bsh/testing,now 1:6.3.2-1 all [installed,automatic]
libreoffice-script-provider-js/testing,now 1:6.3.2-1 all [installed,automatic]
libreoffice-script-provider-python/testing,now 1:6.3.2-1 all [installed,automatic]
libreoffice-sdbc-firebird/testing,now 1:6.3.2-1+b1 amd64 [installed,automatic]
libreoffice-sdbc-mysql/testing,now 1:6.3.2-1+b1 amd64 [installed,automatic]
libreoffice-sdbc-postgresql/testing,now 1:6.3.2-1+b1 amd64 [installed,automatic]
libreoffice-style-colibre/testing,now 1:6.3.2-1 all [installed,automatic]
libreoffice-style-elementary/testing,now 1:6.3.2-1 all [installed]
libreoffice-style-tango/testing,now 1:6.3.2-1 all [installed,automatic]
libreoffice-voikko/testing,unstable,now 5.0-3+b1 amd64 [installed]
libreoffice-wiki-publisher/testing,now 1.2.0+LibO6.3.2-1 all [installed,automatic]
libreoffice-writer/testing,now 1:6.3.2-1+b1 amd64 [installed,automatic]
libreoffice/testing,now 1:6.3.2-1+b1 amd64 [installed]

    mdk@seraph$ which python3
    /usr/bin/python3
    mdk@seraph$ dpkg -S $(which python3)
    python3-minimal: /usr/bin/python3
    mdk@seraph$ apt-cache policy python3-minimal
    python3-minimal:
      Installed: 3.7.5-1
      Candidate: 3.7.5-1
      Version table:
     *** 3.7.5-1 990
            990 https://deb.debian.org/debian bullseye/main amd64 Packages
            500 https://deb.debian.org/debian sid/main amd64 Packages
            100 /var/lib/dpkg/status

According to strace, it uses this Python, not Python 2, I see:

    $ grep -i python --color /tmp/libreoffice.strace 
    14771 openat(AT_FDCWD, "/usr/lib/libreoffice/program/services/scriptproviderforpython.rdb", O_RDONLY) = 7
    14771 openat(AT_FDCWD, "/usr/lib/libreoffice/program/libpythonloaderlo.so", O_RDONLY|O_CLOEXEC) = 22
    14771 openat(AT_FDCWD, "/usr/lib/libreoffice/program/libpython3.7m.so.1.0", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
    14771 openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libpython3.7m.so.1.0", O_RDONLY|O_CLOEXEC) = 22
    14771 access("/usr/lib/libreoffice/program/pythonloader.unorc", F_OK) = 0
    14771 openat(AT_FDCWD, "/usr/lib/libreoffice/program/pythonloader.unorc", O_RDONLY) = 22
    [...]
    14771 stat("/usr/lib/python3/dist-packages/libvoikko.py", {st_mode=S_IFREG|0644, st_size=33294, ...}) = 0

Also:

    $ apt list --installed | grep voikko
    libenchant-voikko/testing,unstable,now 1.6.0-11.3 amd64 [installed,automatic]
    libreoffice-voikko/testing,unstable,now 5.0-3+b1 amd64 [installed]
    libvoikko1/testing,unstable,now 4.2-2 amd64 [installed,automatic]
    python3-libvoikko/testing,unstable,now 4.2-2 all [installed,automatic]
    voikko-fi/testing,unstable,now 2.3-1 amd64 [installed,automatic]
Comment 6 julien 2019-10-19 19:58:38 UTC
In my case, opening LO and just clicking "Writer Document" is enough to crash.
Comment 7 Julien Nabet 2019-10-19 20:16:53 UTC
I installed libenchant-voikko which wasn't present, still no crash.

What's your localization? (UI + local parameters)
Comment 8 Julien Nabet 2019-10-19 20:19:59 UTC
Also when you indicated you uninstalled LO, did you include the profile in ~/.config/libreoffice ?
Did you install any specific extensions?

what desktop do you use? Personally, I use gtk3.
Comment 9 julien 2019-10-19 20:42:56 UTC
So my locales: 

    LANG=en_US.UTF-8
    LANGUAGE=
    LC_CTYPE="en_US.utf8"
    LC_NUMERIC="en_US.utf8"
    LC_TIME="en_US.utf8"
    LC_COLLATE="en_US.utf8"
    LC_MONETARY="en_US.utf8"
    LC_MESSAGES="en_US.utf8"
    LC_PAPER="en_US.utf8"
    LC_NAME="en_US.utf8"
    LC_ADDRESS="en_US.utf8"
    LC_TELEPHONE="en_US.utf8"
    LC_MEASUREMENT="en_US.utf8"
    LC_IDENTIFICATION="en_US.utf8"
    LC_ALL=en_US.utf8

> Did you install any specific extensions?

No, it's a brand new laptop, freshly installed.

> Also when you indicated you uninstalled LO, did you include the profile in ~/.config/libreoffice ?

No but rm -fr it does not help (still crashes).

Tried:

$ sudo apt purge python3-libvoikko
[sudo] password for mdk: 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be REMOVED:
  libreoffice-voikko* python3-libvoikko*
0 upgraded, 0 newly installed, 2 to remove and 0 not upgraded.
After this operation, 168 kB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 328933 files and directories currently installed.)
Removing libreoffice-voikko (5.0-3+b1) ...
Removing python3-libvoikko (4.2-2) ...
Processing triggers for libreoffice-common (1:6.3.2-1) ...

Then no crash.

Then:

$ sudo apt install libreoffice-voikko
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  python3-libvoikko
The following NEW packages will be installed:
  libreoffice-voikko python3-libvoikko
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 50.5 kB of archives.
After this operation, 168 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 https://cdn-aws.deb.debian.org/debian bullseye/main amd64 python3-libvoikko all 4.2-2 [27.1 kB]
Get:2 https://cdn-aws.deb.debian.org/debian bullseye/main amd64 libreoffice-voikko amd64 5.0-3+b1 [23.4 kB]
Fetched 50.5 kB in 2s (31.8 kB/s)              
Selecting previously unselected package python3-libvoikko.
(Reading database ... 328897 files and directories currently installed.)
Preparing to unpack .../python3-libvoikko_4.2-2_all.deb ...
Unpacking python3-libvoikko (4.2-2) ...
Selecting previously unselected package libreoffice-voikko.
Preparing to unpack .../libreoffice-voikko_5.0-3+b1_amd64.deb ...
Unpacking libreoffice-voikko (5.0-3+b1) ...
Setting up python3-libvoikko (4.2-2) ...
Setting up libreoffice-voikko (5.0-3+b1) ...
Processing triggers for libreoffice-common (1:6.3.2-1) ...
mdk@seraph$ rm -fr ~/.config/libreoffice/
mdk@seraph$ libreoffice  # Then click on "Writer Document"
Fatal Python error: Python memory allocator called without holding the GIL

> what desktop do you use? Personally, I use gtk3.

I don't really understand this question, I'll try to answer, tell me if I'm out of scope: I'm using i3wm, started by lightdm, on top of Xorg.
Comment 10 julien 2019-10-19 20:53:57 UTC
Looks like the error happen really near some:

    > /usr/lib/python3/dist-packages/uno.py(562)_uno_extract_printable_stacktrace()
    -> return ''.join(traceback.format_tb(trace))

probably right after. This looks called directly from Java (I have no stacktrace), the function runs well, and right after the return I have the crash.
Comment 11 Julien Nabet 2019-10-19 20:59:25 UTC
Could you check these libs:
unoconv
uno-libs3
ure
?

Here what I got, I expect you got the same but since you also got unstable.

root@debianamd:~# apt-cache policy unoconv
unoconv:
  Installé : (aucun)
  Candidat : 0.7-1.1
 Table de version :
     0.7-1.1 500
        500 http://ftp.fr.debian.org/debian stable/main amd64 Packages
        500 http://ftp.fr.debian.org/debian testing/main amd64 Packages
root@debianamd:~# apt-cache policy uno-libs3
uno-libs3:
  Installé : 6.3.2-1+b1
  Candidat : 6.3.2-1+b1
 Table de version :
 *** 6.3.2-1+b1 500
        500 http://ftp.fr.debian.org/debian testing/main amd64 Packages
        100 /var/lib/dpkg/status
     6.1.5-3+deb10u3 500
        500 http://ftp.fr.debian.org/debian stable/main amd64 Packages
root@debianamd:~# apt-cache policy ure
ure:
  Installé : 6.3.2-1+b1
  Candidat : 6.3.2-1+b1
 Table de version :
 *** 6.3.2-1+b1 500
        500 http://ftp.fr.debian.org/debian testing/main amd64 Packages
        100 /var/lib/dpkg/status
     6.1.5-3+deb10u3 500
        500 http://ftp.fr.debian.org/debian stable/main amd64 Packages
Comment 12 julien 2019-10-19 21:02:24 UTC
> Could you check these libs:

Yes:

$ apt-cache policy unoconv uno-libs3 ure
unoconv:
  Installed: (none)
  Candidate: 0.7-1.1
  Version table:
     0.7-1.1 990
        990 https://deb.debian.org/debian bullseye/main amd64 Packages
        500 https://deb.debian.org/debian sid/main amd64 Packages
uno-libs3:
  Installed: 6.3.2-1+b1
  Candidate: 6.3.2-1+b1
  Version table:
     6.3.3~rc1-1 500
        500 https://deb.debian.org/debian sid/main amd64 Packages
 *** 6.3.2-1+b1 990
        990 https://deb.debian.org/debian bullseye/main amd64 Packages
        100 /var/lib/dpkg/status
ure:
  Installed: 6.3.2-1+b1
  Candidate: 6.3.2-1+b1
  Version table:
     6.3.3~rc1-1 500
        500 https://deb.debian.org/debian sid/main amd64 Packages
 *** 6.3.2-1+b1 990
        990 https://deb.debian.org/debian bullseye/main amd64 Packages
        100 /var/lib/dpkg/status


Oh, while I'm in this function, I can see the traceback being pretty printed, if it helps:

  File "/usr/lib/libreoffice/program/pythonloader.py", line 136, in activate
    mod = self.getModuleFromUrl( locationUrl )
  File "/usr/lib/libreoffice/program/pythonloader.py", line 104, in getModuleFromUrl
    exec(codeobject, mod.__dict__)
  File "/usr/lib/libreoffice/share/extensions/voikko/lovoikko.py", line 15, in <module>
    from SettingsEventHandler import SettingsEventHandler
  File "/usr/lib/python3/dist-packages/uno.py", line 359, in _uno_import
    return _builtin_import(name, *optargs, **kwargs)
  File "/usr/lib/libreoffice/share/extensions/voikko/pythonpath/SettingsEventHandler.py", line 18, in <module>
    from PropertyManager import PropertyManager
  File "/usr/lib/python3/dist-packages/uno.py", line 359, in _uno_import
    return _builtin_import(name, *optargs, **kwargs)
  File "/usr/lib/libreoffice/share/extensions/voikko/pythonpath/PropertyManager.py", line 18, in <module>
    from VoikkoHandlePool import VoikkoHandlePool
  File "/usr/lib/python3/dist-packages/uno.py", line 359, in _uno_import
    return _builtin_import(name, *optargs, **kwargs)
  File "/usr/lib/libreoffice/share/extensions/voikko/pythonpath/VoikkoHandlePool.py", line 15, in <module>
    from libvoikko import Voikko, VoikkoException
  File "/usr/lib/python3/dist-packages/uno.py", line 359, in _uno_import
    return _builtin_import(name, *optargs, **kwargs)

sadly I don't see the exception being handled, the only parameter is the traceback. I don't know if I can gather information about the running Java process at the moment of the exception (I'm more fluent in debugging Python than debugging Java).
Comment 13 julien 2019-10-19 21:09:35 UTC
Hum no forget the last one about the traceback, it looks like I introduced it myself while trying to debug ...
Comment 14 julien 2019-10-19 21:17:43 UTC
OK, I may have something, but yet I'm not fluent with this Java/Python interaction.

Right before the crash I see:


> /usr/lib/python3/dist-packages/unohelper.py(252)getTypes()->(<Type instanc...INTERFACE')>)>, <Type instanc...INTERFACE')>)>, <Type instanc...INTERFACE')>)>)
-> return _unohelper_getHandle( self )
(Pdb) l
247  	        _g_typeTable[self.__class__] = ret
248  	    return ret
249  	
250  	class Base(XTypeProvider):
251  	    def getTypes( self ):
252  ->	        return _unohelper_getHandle( self )
253  	    def getImplementationId(self):
254  	        return ()
255  	
256  	class CurrentContext(XCurrentContext, Base ):
257  	    """a current context implementation, which first does a lookup in the given
(Pdb) p _unohelper_getHandle( self )
(<Type instance com.sun.star.lang.XTypeProvider (<Enum instance com.sun.star.uno.TypeClass ('INTERFACE')>)>, <Type instance com.sun.star.lang.XServiceInfo (<Enum instance com.sun.star.uno.TypeClass ('INTERFACE')>)>, <Type instance com.sun.star.lang.XSingleComponentFactory (<Enum instance com.sun.star.uno.TypeClass ('INTERFACE')>)>)
(Pdb) n
Fatal Python error: Python memory allocator called without holding the GIL
Comment 15 Julien Nabet 2019-10-19 21:33:47 UTC
I noticed the use of python3-uno
Package: python3-uno
Source: libreoffice (1:6.3.2-1)
Version: 1:6.3.2-1+b1
but I suppose you've got the same version.

I don't have more questions, so I'll put this back to UNCONFIRMED.
I must recognize I'm stuck.
Comment 16 julien 2019-10-19 21:36:30 UTC
Yes same version of python3-uno:

  Installed: 1:6.3.2-1+b1

thanks for trying hard to reproduce, I appreciate it.
Comment 17 julien 2019-10-19 21:45:01 UTC
Still investigating slowly, I bet this is near:

/usr/lib/libreoffice/share/extensions/voikko/pythonpath/PropertyManager.py(83)initialize()

in the call of:

self.__linguPropSet.addPropertyChangeListener("IsSpellWithDigits", self)

Which I bet is a Java function as __linguPropSet is a pyuno object (com.sun.star.uno.XInterface)0x561ace7484c0{implementationName=com.sun.star.lingu2.LinguProps, supportedServices={com.sun.star.linguistic2.LinguProperties}, supportedInterfaces={com.sun.star.linguistic2.XLinguProperties,com.sun.star.beans.XFastPropertySet,com.sun.star.beans.XPropertyAccess,com.sun.star.lang.XComponent,com.sun.star.lang.XServiceInfo,com.sun.star.lang.XTypeProvider,com.sun.star.uno.XWeak}}
Comment 18 Julien Nabet 2019-10-19 22:23:48 UTC
We checked Python but what about Java?
On my pc:
julien@debianamd:~$ java --version
openjdk 11.0.5 2019-10-15
OpenJDK Runtime Environment (build 11.0.5+10-post-Debian-1)
OpenJDK 64-Bit Server VM (build 11.0.5+10-post-Debian-1, mixed mode)
Comment 19 Timo Jyrinki 2019-10-20 17:41:01 UTC
Right now I can add nothing interesting, I can't reproduce the crash on up-to-date Debian sid, but interesting bug regardless.

I'll try to find time to ponder about this a bit more at some point too.
Comment 20 julien 2019-10-20 20:32:35 UTC
$ java --version
openjdk 11.0.5-ea 2019-10-15
OpenJDK Runtime Environment (build 11.0.5-ea+9-post-Debian-1)
OpenJDK 64-Bit Server VM (build 11.0.5-ea+9-post-Debian-1, mixed mode, sharing)
Comment 21 Xisco Faulí 2020-05-11 07:37:11 UTC
Hello Julien,
A new major release of LibreOffice is available since this bug was reported.
Could you please try to reproduce it with the latest version of LibreOffice
from https://www.libreoffice.org/download/libreoffice-fresh/ ?
I have set the bug's status to 'NEEDINFO'. Please change it back to
'UNCONFIRMED' if the bug is still present in the latest version.
Comment 22 julien 2020-05-12 14:16:35 UTC
Hi Xisco Faulí!

That's true, I can't reproduce the issue, tried with libreoffice 6.4.4~rc1-1 from Debian testing.

I think this issue can be closed, thanks for everybody trying to help me!