Bug 143980 - Add numpy to LibreOffice\program\python
Summary: Add numpy to LibreOffice\program\python
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
7.2.0.4 release
Hardware: All All
: medium enhancement
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: Macro-Python
  Show dependency treegraph
 
Reported: 2021-08-20 17:23 UTC by edil
Modified: 2021-09-01 13:18 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description edil 2021-08-20 17:23:42 UTC
Description:
Calc is a container of Tables, and from DataArray, is a container of Arrays.
I noticed that numpy is a very powerful and great manager of Arrays, and useful for many math activites.
Why don't add numpy python library to LibreOffice? 

Actual Results:
Now the numpy python library is missing

Expected Results:
Add the numpy python library to LibreOffice


Reproducible: Always


User Profile Reset: No



Additional Info:
Add the numpy python library to LibreOffice
Comment 1 Roman Kuznetsov 2021-08-22 20:13:09 UTC
Andreas, Alain, what do you think?
Comment 2 Andreas Heinisch 2021-08-22 20:40:11 UTC
Imho numpy is a very popular library for python, but I don't know if it requires something special in order to work with the uno arrays. We could add the library and make some experiments.
Comment 3 Roman Kuznetsov 2021-08-22 21:09:53 UTC
Set to NEW by Comment 2

Thanks Andreas
Comment 4 edil 2021-08-22 21:27:17 UTC
Thanks,
I use Range.DataArray in Basic, so I have Variant Array of Arrays, and this is one parameter to deliver to Python def, that take this as tuple of tuples.
Will be Python to transform this in numpy array, and then work with.
At end, must return a tuple of tuples for copy this to a Calc Range.
Comment 5 Ming Hua 2021-08-22 23:17:44 UTC
While I would welcome addition of Numpy to LibreOffice's bundled Python, I don't see it happening without a dramatic change of the current practice of python module packaging.

We don't even ship the full core library of Python.  To use the Windows installer as an example (I assume TDF's RPM and DEB packages are similar), I think we build python without Tcl/Tk and the accompanying tkinter graphic UI module, and we strip a lot of modules LO doesn't use when bundling python into the installer.

As a result, the whole program\python-core-3.8.10\lib directory (with .pyc cache files generated) for installed 7.2.0 on my system is 28 MB with 1,225 files.  As a comparison, the official python.org installation's python38\Lib directory here (excluding subdirectory site-packages\) is 115 MB with 7,183 files, also with .pyc files generated.

And Numpy alone on my system is 56 MB.  It also links to BLAS/LAPACK and maybe many other C/C++ libraries, so building it correctly is going to be tricky, too.  And by adding it we are more and more like shipping a custom python distribution, instead of an office suite bundling python.

Without discouraging anyone who want to work on this, I personally feel the developers' efforts would be better put into making installing external modules from PyPI and building virtual environments easier with LO's bundled python.  Then we can avoid frustrated users like in bug 143867.  Maybe starting with adding "ensurepip" module into our bundle, and writing a good tutorial about setting up PIP and installing packages from PyPI with LO-bundled python?
Comment 6 edil 2021-08-23 09:22:45 UTC
Thank  Ming Hua
I am a poor developer and don't know the problems of build python.
For me is a very good idea to make something easy like setting up PIP and installing packages from PyPI with LO-bundled python.
Every developer will be free to load what he need for enforce LibreOffice.
Now I use PyCharm and have added new modules from his PIP environment, bat I can use this only in  PyCharm debug.
Comment 7 Rafael Lima 2021-08-31 17:43:31 UTC
I would like to share some findings that might help us come up with a solution to install additional Python packages into LO.

On Ubuntu-based operating systems, if you install LO from their PPA, LibreOffice will not be installed with the bundled Python. Instead LO will use the same python used by the operating system. The PPA installation does not have the "program/python.bin" that comes with the DEB file downloaded from TDF's website.

For Ubuntu users that install LO from the PPA the problem reported here does not exist. Packages as numpy come preinstalled in Ubuntu and any packages installed using system-wide pip will be available in Python scripts launched by LO.

After Alain drew my attention to this bug I did some testing on Ubuntu and I was able to create a script using numpy, pyqt5 and ScriptForge and launched the script from withing LO Calc. I created a PyQt5 dialog and had it interact with the Calc file. Although this only works in Ubuntu using the PPA installation, it shows that it is possible to use anything from Python from within LO. And IMO this is very powerful specially for those working with data science, data mining, and so forth.

(In reply to Ming Hua from comment #5)
> Maybe starting with adding "ensurepip" module into our bundle, and writing a good
> tutorial about setting up PIP and installing packages from PyPI with LO-bundled python?

I liked the idea of having the ability to install additional packages using PIP.

However, another idea would be to have a Python setup dialog as the one we currently have for JRE (in Tools - Options - Advanced). The user could use such dialog to choose which Python interpreter to use.
Comment 8 Roman Kuznetsov 2021-08-31 17:47:54 UTC
(In reply to Rafael Lima from comment #7)
> 
> However, another idea would be to have a Python setup dialog as the one we
> currently have for JRE (in Tools - Options - Advanced). The user could use
> such dialog to choose which Python interpreter to use.

WOW! An excellent idea! Rafael, can you file a different enhancement about it?
Comment 9 Rafael Lima 2021-08-31 18:04:49 UTC
Hi Roman, I created a new enhancement request (see 144224).
Comment 10 edil 2021-08-31 19:47:47 UTC
May be only a problem of PYTHONPATH if we have more than one installation of python? We need uno.py unohelper.py ..with python.exe.. under LibreOffice>Program?.
In PyCharm one can have more  Python interpreter to choose which to use... but we need a bridge LO-Python.
Comment 11 flywire 2021-09-01 13:18:35 UTC
(In reply to edil from comment #0)
> Add the numpy python library to LibreOffice

You can add numpy yourself but it isn't easy, remember to add the dll too if some other install hasn't put it in your path - see https://ask.libreoffice.org/t/install-python-package-for-libre-office/66934/15 - Installs scipy and numpy. Fiddle around and ad the dll too.

The big issue is Python needs pip to install packages but it is not implemented in LO and the community is fairly clueless how to install it - see Bug #143867

Clearly, the community has little idea how to do it - https://ask.libreoffice.org/t/install-python-package-for-libre-office/66934 Even there the dll is missing. Someone muddling through the process and installing pip and packages doesn't mean everything is fine.

(In reply to Ming Hua from comment #5)
> ... I personally feel the
> developers' efforts would be better put into making installing external
> modules from PyPI and building virtual environments easier with LO's bundled
> python.  Then we can avoid frustrated users like in bug 143867.  Maybe
> starting with adding "ensurepip" module into our bundle, and writing a good
> tutorial about setting up PIP and installing packages from PyPI with
> LO-bundled python?

Unless users express concern with poor package support bugs like bug 143867 will keep getting closed.