Bug 100850 - libreoffice base use too much memory
Summary: libreoffice base use too much memory
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Base (show other bugs)
Version:
(earliest affected)
5.1.4.2 release
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: Memory
  Show dependency treegraph
 
Reported: 2016-07-11 13:26 UTC by mug896
Modified: 2023-03-22 03:23 UTC (History)
6 users (show)

See Also:
Crash report or crash signature:


Attachments
memory usage screenshot (44.82 KB, image/png)
2016-07-11 13:26 UTC, mug896
Details
.odb file (4.18 KB, application/vnd.oasis.opendocument.database)
2016-07-18 00:22 UTC, mug896
Details

Note You need to log in before you can comment on or make changes to this bug.
Description mug896 2016-07-11 13:26:18 UTC
Created attachment 126164 [details]
memory usage screenshot

When i open  writer, calc, impress, draw 
memory usage is under 100M on average.
but when i use base, "open an existing database file"( that is very small ) and open a table then
memory usage is suddenly over 800M
Comment 1 Buovjaga 2016-07-17 09:18:41 UTC
Can you attach an example .odb file?

Set to NEEDINFO.
Change back to UNCONFIRMED after you have provided the document.
Comment 2 mug896 2016-07-18 00:22:33 UTC
Created attachment 126259 [details]
.odb file
Comment 3 Alex Thurgood 2016-07-18 11:15:21 UTC
No repro with 

Version: 5.1.3.2
Build ID: 644e4637d1d8544fd9f56425bd6cec110e49301b
CPU Threads: 2; OS Version: Mac OS X 10.11.5; UI Render: default; 
Locale: fr-FR (fr.UTF-8)

and test file provided by OP.

Empty Writer document : 77 Mb

Open Hello.odb : 83 Mb
Open table Hello : 149 Mb
Comment 4 Alex Thurgood 2016-07-18 11:29:21 UTC
No repro with 

Version: 5.1.4.2
Build ID: f99d75f39f1c57ebdd7ffc5f42867c12031db97a
CPU Threads: 2; OS Version: Mac OS X 10.11.5; UI Render: default; 
Locale: fr-FR (fr.UTF-8)


or with 

Version: 5.3.0.0.alpha0+
Build ID: 33d58de8c05d7a3591ac81164c1786d183a09991
CPU Threads: 2; OS Version: Mac OS X 10.11.5; UI Render: default; 
Locale: fr-FR (fr.UTF-8)
Comment 5 Alex Thurgood 2016-07-18 11:31:19 UTC
@mug896 : we need more information.

OS : version, provider
LO : distrib-provided or TDF download ?
Java : JRE or JDK version
Comment 6 mug896 2016-07-18 14:30:53 UTC
1. OS : version, provider

Ubuntu 16.04 xenial (x86_64)

2. LO : distrib-provided or TDF download ?

I have installed libreoffice with following command
"apt install libreoffice"

3. Java : JRE or JDK version

/usr/lib/jvm/java-8-openjdk-amd64/jre
Comment 7 Alex Thurgood 2016-07-19 13:23:30 UTC
Tested your ODB file against 

Version: 5.1.4.2
Build ID: 1:5.1.4-0ubuntu1
Threads CPU : 4; Version de l'OS :Linux 4.4; UI Render : par défaut; 
Locale : fr-FR (fr_FR.UTF-8)

Linux Mint 18 (Sarah)

Compare with empty Writer document : 117 Mb

ODB file on open (default) : 116 Mb
ODB file click on Table icon : 153 Mb
ODB file select Hello table : 678 Mb !!
ODB file open Hello table : 695 Mb !!


Memory usage tops out at about 710 Mb

Confirming - could be that there is stuff leaking memory
Comment 8 Alex Thurgood 2016-07-19 13:24:33 UTC
Oops, sorry, rescinding confirmation until tests can be made with a TDF provided build, otherwise this remains Ubuntu-specific.
Comment 9 Alex Thurgood 2016-07-19 13:26:07 UTC
@CCs : would welcome comparative testing on other Linux OSes
Comment 10 Alex Thurgood 2016-07-19 13:28:23 UTC
Hmm, the varchar field length might be a possible indication, I seem to remember we had a bug report about something similar already
Comment 11 Alex Thurgood 2016-07-19 13:38:38 UTC
(In reply to Alex Thurgood from comment #10)
> Hmm, the varchar field length might be a possible indication, I seem to
> remember we had a bug report about something similar already

Nope, don't seem to be able to find it, there is a similar report about performance drop with identity PKs on mysql (bug 90780) perhaps related.
Comment 12 Terrence Enger 2016-07-21 14:53:34 UTC
I am setting status NEW, based on daily Linux dbgutil version
2016-07-18, for example, running on debian-stretch.

In that version, I see the following numbers in output from `top` as I
progress through the program ...

                                          virt     res     shr     CPU time
                                       -------  ------  ------     --------
    to Writer document "Untitled 1" :  1219104  238064  179868     0:07.98
    Open File dialog                :  1227592  243096  184748     0:12.26
    Hello.odb                       :  1265568  276312  205776     0:14.54
    display list of tables          :  4346684  327660  221588     0:18.14
    close Hello.odb                 :  4363088  327884  221588     0:19.48
    close "Untitled 1"              :  4365156  332368  221128     0:21.55

I see similarly dramatic increases in a local build of commit ff4ce07,
pulled 2016-07-18.

For comparison, in bibisect 43all version oldest, the growth in
virtual memory used happens upon opening the .odb ...

                                          virt     res     shr     CPU time
                                       -------  ------  ------     --------
    to Start Center                 :   504604   75548   53428     0:01.96
    Open File dialog                :   563188   86856   59800     0:05.35
    Hello.odb                       :  3011836  140292   94836     0:06.30
    display list of tables          :  3035828  170544   97576     0:08.29
    open table hello                :  3048124  202816  104592     0:11.03
    close hello                     :  3048148  204096  104820     0:11.40
    close Hello.odb                 :  3048148  207004  105156     0:11.81

In the daily dbgutil version 2016-07-18, comparison of output from gdb
`info shared` before and and after displaying the list of tables shows
the following libraries added (whitespace added) ...

    From                To                  Syms Read   
        Shared Object Library
    0x00007fffb1c9cef0  0x00007fffb1ced19b  Yes (*)     
        /home/terry/lo_hacking/bibisect/lo-linux-dbgutil-daily/opt/program/
        ../program/libdbaxmllo.so
    0x00007fffaf5519c0  0x00007fffaf77f8f7  Yes (*)     
        /home/terry/lo_hacking/bibisect/lo-linux-dbgutil-daily/opt/program/
        ../program/libdbalo.so
    0x00007fffaf111020  0x00007fffaf122278  Yes (*)     
        /home/terry/lo_hacking/bibisect/lo-linux-dbgutil-daily/opt/program/
        liblocaledata_others.so
    0x00007fffae657490  0x00007fffae9e8b73  Yes (*)     
        /home/terry/lo_hacking/bibisect/lo-linux-dbgutil-daily/opt/program/
        ../program/libdbulo.so
    0x00007fffae2a98c0  0x00007fffae2d20e1  Yes (*)     
        /home/terry/lo_hacking/bibisect/lo-linux-dbgutil-daily/opt/program/
        ../program/libdbpool2.so
    0x00007fffae06e7f0  0x00007fffae08a327  Yes (*)     
        /home/terry/lo_hacking/bibisect/lo-linux-dbgutil-daily/opt/program/
        ../program/libsdbc2.so
    0x00007fffade32bf0  0x00007fffade4907f  Yes (*)     
        /home/terry/lo_hacking/bibisect/lo-linux-dbgutil-daily/opt/program/
        ../program/libloglo.so
    0x00007fffadb962d0  0x00007fffadbd79b7  Yes (*)     
        /home/terry/lo_hacking/bibisect/lo-linux-dbgutil-daily/opt/program/
        ../program/libhsqldb.so
    0x00007fffad8d51d0  0x00007fffad91f2a3  Yes (*)     
        /home/terry/lo_hacking/bibisect/lo-linux-dbgutil-daily/opt/program/
        ../program/libjdbclo.so
    0x00007fffad6ac940  0x00007fffad6adeb6  Yes (*)     
        /home/terry/lo_hacking/bibisect/lo-linux-dbgutil-daily/opt/program/
        libaffine_uno_uno.so
    0x00007fffad4a4550  0x00007fffad4a8202  Yes (*)     
        /home/terry/lo_hacking/bibisect/lo-linux-dbgutil-daily/opt/program/
        libuno_purpenvhelpergcc3.so.3
    0x00007fffad266e50  0x00007fffad286a06  Yes (*)     
        /home/terry/lo_hacking/bibisect/lo-linux-dbgutil-daily/opt/program/
        libjavavmlo.so
    0x00007fffac516680  0x00007fffacda7ae9  Yes (*)     
        /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
    0x00007fffac11d490  0x00007fffac12613c  Yes (*)     
        /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libverify.so
    0x00007fffabef7300  0x00007fffabf0f4d3  Yes (*)     
        /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libjava.so
    0x00007fffabce3870  0x00007fffabce7470  Yes (*)     
        /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libzip.so
Comment 13 QA Administrators 2017-09-01 11:16:15 UTC Comment hidden (obsolete)
Comment 14 Robert Großkopf 2019-01-11 18:38:02 UTC
Have tried the testfile. What language should it be? Seems I could only see rectangles, no fieldnames and also only one field with readable content. There is much memory used - but for what?

So I tested with my own databases:
1. database connected to internal HSQLDB
140 MB before opening the database, 250 MB after opening the "Table"-pane and 490 MB after opening a table with only 20 rows/5 columns (names, birthdays, gender).
2. same database, created by the migration-wizard, works now with internal Firebird
140 MB before, then 160 MB after opening the database-file and switching to "Tables" and 175 MB after opening the table.

Seems the memory is all needed for HSQLDB to work in the memory. Don't know if the bug could be fixed and if we should invest in this bug, because we want to migrate to Firebird and this bug seems to be a good argument for doing this step.
Comment 15 Robert Großkopf 2019-01-11 18:41:05 UTC
Have forgotten:

All tests with LO 6.2.0.2 OpenSUSE 15 64bit rpm Linux
Comment 16 QA Administrators 2021-01-11 03:54:26 UTC Comment hidden (obsolete)
Comment 17 Terrence Enger 2021-03-21 19:05:23 UTC
A local build (--enable-dbgutil) of commit feb6fd1f (2021-00-17) built
and running on debian-buster shows the following numbers from `top`:

                                          virt     res     shr     CPU time
                                       -------  ------  ------     --------
    to Start Center                 :   437264  182496  137528     0:02.70
    Open File dialog                :   451052  190648  140892     0:08.21
    Hello.odb                       :   481920  217980  165588     0:12.94
    display list of tables          :  3098460  304912  193964     0:15.79
    open table hello                :  3144664  339844  206908     0:19.11
    close hello                     :  3148016  341392  207404     0:20.34
    close Hello.odb                 :  3156212  343856  208536     0:21.97

Comparing these numbers to comment 12, I see:
  - "virt" is closer to numbers from bibisect 43all version oldest
  - "res" and "shr" are close to numbers from daily dbgutil version
    2016-07-18.

I conclude that the problem--if it is a problem--persists.
Comment 18 QA Administrators 2023-03-22 03:23:55 UTC
Dear mug896,

To make sure we're focusing on the bugs that affect our users today, LibreOffice QA is asking bug reporters and confirmers to retest open, confirmed bugs which have not been touched for over a year.

There have been thousands of bug fixes and commits since anyone checked on this bug report. During that time, it's possible that the bug has been fixed, or the details of the problem have changed. We'd really appreciate your help in getting confirmation that the bug is still present.

If you have time, please do the following:

Test to see if the bug is still present with the latest version of LibreOffice from https://www.libreoffice.org/download/

If the bug is present, please leave a comment that includes the information from Help - About LibreOffice.
 
If the bug is NOT present, please set the bug's Status field to RESOLVED-WORKSFORME and leave a comment that includes the information from Help - About LibreOffice.

Please DO NOT

Update the version field
Reply via email (please reply directly on the bug tracker)
Set the bug's Status field to RESOLVED - FIXED (this status has a particular meaning that is not 
appropriate in this case)


If you want to do more to help you can test to see if your issue is a REGRESSION. To do so:
1. Download and install oldest version of LibreOffice (usually 3.3 unless your bug pertains to a feature added after 3.3) from https://downloadarchive.documentfoundation.org/libreoffice/old/

2. Test your bug
3. Leave a comment with your results.
4a. If the bug was present with 3.3 - set version to 'inherited from OOo';
4b. If the bug was not present in 3.3 - add 'regression' to keyword


Feel free to come ask questions or to say hello in our QA chat: https://web.libera.chat/?settings=#libreoffice-qa

Thank you for helping us make LibreOffice even better for everyone!

Warm Regards,
QA Team

MassPing-UntouchedBug