Bug 148778 - Librecalc memory leakage - it cannot empty memory
Summary: Librecalc memory leakage - it cannot empty memory
Status: RESOLVED WONTFIX
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
7.4.0.0 alpha0+
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-04-25 18:22 UTC by elias estatistics
Modified: 2022-09-12 19:02 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
image showing >5gb consumed by a simple workbook. (1.01 MB, image/jpeg)
2022-04-25 18:22 UTC, elias estatistics
Details
workbench with simple tables. (62.63 KB, application/vnd.oasis.opendocument.spreadsheet)
2022-04-25 18:26 UTC, elias estatistics
Details

Note You need to log in before you can comment on or make changes to this bug.
Description elias estatistics 2022-04-25 18:22:47 UTC
Created attachment 179764 [details]
image showing >5gb consumed by a simple workbook.

Librecalc memory likage - it cannot empty memory 

I have done a lot resets working only two calc workbooks.
My computer almost freezes. 

When i close all other workbooks except one which contains
simple tables (no formulas), memory is almost >5GBs 
while when shutting down libreoffice, and opening only that workbook, 
then total memory is about ~1GB.

(total ~8GBs mem)

Reproduce
0) restrict memory to libre about 8GBs
a) open many sheets with formula in order to memory goes almost full (~7GBs in total)
b) close all "complicated sheets".
c) except the simplest one without formulas. 
d) computer cannot empty memory after 1 to 5 minutes... 
e) freezes... 


Version: 7.4.0.0.alpha0+ / LibreOffice Community
Build ID: 566ee17c2709f9939145ddc7fbb1d61c35049321
CPU threads: 4; OS: Linux 5.11; UI render: default; VCL: kf5 (cairo+xcb)
Locale: el-GR (el_GR.UTF-8); UI: en-US
Calc: threaded Jumbo

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   39 bits physical, 48 bits virtual
CPU(s):                          4
On-line CPU(s) list:             0-3
Thread(s) per core:              1
Core(s) per socket:              4
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           60
Model name:                      Intel(R) Core(TM) i5-4570S CPU @ 2.90GHz
Stepping:                        3
CPU MHz:                         2178.963
CPU max MHz:                     3600.0000
CPU min MHz:                     800.0000
BogoMIPS:                        5786.55
Virtualization:                  VT-x
L1d cache:                       128 KiB
L1i cache:                       128 KiB
L2 cache:                        1 MiB
L3 cache:                        6 MiB
NUMA node0 CPU(s):               0-3
Vulnerability Itlb multihit:     KVM: Mitigation: VMX disabled
Vulnerability L1tf:              Mitigation; PTE Inversion; VMX conditional cach
                                 e flushes, SMT disabled
Vulnerability Mds:               Mitigation; Clear CPU buffers; SMT disabled
Vulnerability Meltdown:          Mitigation; PTI
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled v
                                 ia prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user
                                  pointer sanitization
Vulnerability Spectre v2:        Mitigation; Full generic retpoline, IBPB condit
                                 ional, IBRS_FW, STIBP disabled, RSB filling
Vulnerability Srbds:             Mitigation; Microcode
Vulnerability Tsx async abort:   Not affected
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtr
                                 r pge mca cmov pat pse36 clflush dts acpi mmx f
                                 xsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rd
                                 tscp lm constant_tsc arch_perfmon pebs bts rep_
                                 good nopl xtopology nonstop_tsc cpuid aperfmper
                                 f pni pclmulqdq dtes64 monitor ds_cpl vmx smx e
                                 st tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_
                                 1 sse4_2 x2apic movbe popcnt tsc_deadline_timer
                                  aes xsave avx f16c rdrand lahf_lm abm cpuid_fa
                                 ult invpcid_single pti ssbd ibrs ibpb stibp tpr
                                 _shadow vnmi flexpriority ept vpid ept_ad fsgsb
                                 ase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid
                                  xsaveopt dtherm ida arat pln pts md_clear flus
                                 h_l1d


$$$$$$$$$$P.       ------------ 
  ,g$$P"     """Y$$.".        OS: Debian GNU/Linux 11 (bullseye) x86_64 
 ,$$P'              `$$$.     Host: B85M-HD3 R4 
',$$P       ,ggs.     `$$b:   Kernel: 5.11.0-18-generic 
`d$$'     ,$P"'   .    $$$    Uptime: 1 hour, 28 mins 
 $$P      d$'     ,    $$P    Packages: 6371 (dpkg), 4 (flatpak) 
 $$:      $$.   -    ,d$$'    Shell: bash 5.1.4 
  Y$$.    `.`"Y$$$$P"'        Resolution: 1920x1080 
     `$$b.                    DE: Plasma 5.21.4 
       `Y$$b.                 WM: KWin 
                              Terminal: lxterminal 
                              Terminal Font: Monospace 10 
                              CPU: Intel i5-4570S (4) @ 3.600GHz 
                              GPU: NVIDIA GeForce GT 1030 
                              Memory: 2733MiB / 7909MiB
Comment 1 elias estatistics 2022-04-25 18:26:36 UTC
Created attachment 179765 [details]
workbench with simple tables.
Comment 2 elias estatistics 2022-04-25 18:33:28 UTC
In my machine, i can reproduce, freeze, very well.

I think that it has to do with "adapt-row" topic.

Cause, when i am copying some tables over formula sheet, it may need 1 minute (!!) in order to copy them....
Comment 3 Telesto 2022-04-26 09:48:16 UTC
(In reply to elias estatistics from comment #1)
> Created attachment 179765 [details]
> workbench with simple tables.

What I'm I supposed to with this file to reproduce excessive memory usage?
Comment 4 m_a_riosv 2022-04-26 16:45:03 UTC
Please test with a clean profile, Menu/Help/Restart in Safe Mode
Comment 5 elias estatistics 2022-04-26 21:08:12 UTC
Please watch video fully. (i was not very organized at first minutes). 
https://easyupload.io/ss75q1


a) Calc opens with clean profile in safe mode - ~1GB mem (out of 8gb)  
b) I open my formula workbook (A)
c) I open simple html file from calc (B)
d) copy html tables to formula sheet
e) then copy the new produced tables to a 3rd workbook (C)
f) close (B) and save (C)


1) I repeat this process about 9 times or less.
2) It is clearly seen that when open html file and copy tables to formula sheet about 500mb memory are charged each time (1 time 1GB !)
3) When finished, I closed ALL workbooks except the simplest one (C)
4) C workbook had only 1 sheet with tables without formulas... mem at ~5.5GB. 
5) I even delete entirely that sheet with tables, leaving open only an empty sheet... 
6) mem at 5.5GB for about ~3 whole minutes without even later to goes down to ~1Gb... 


My questions:

a) Even when i copied tables and formulas were calculated, why 500mbs of mem data were produced? The formula workbook was loaded already... in these 500mbs of mem, they are saving also every recalculation that is processed previously from formulas or what? 
---I repeat - i copied about 20-30 text tables ONLY....this is a load of 500mb of mem?

b) Even when i am closing ALL except the simplest one, memory is not emptied from CALC.... why??? Why is keeping past useless data in memory even when only an empty sheet is open???



Reproduce:
I repeat:
a) make a complicated large formula sheet that produce something from tables...
b) repeat many times this process... and watch memory...
c) then close all workbooks except the simplest one... 
d) memory is not emptied

I think this is a very serious bug cause causing to my computer to hung.... cause memory is getting full...
Comment 6 elias estatistics 2022-04-26 21:09:13 UTC
* video for 30 days

* i can give the workbooks if u want, but you may better make something more meaningful.
Comment 7 QA Administrators 2022-04-27 04:02:14 UTC Comment hidden (obsolete)
Comment 8 Michael Warner 2022-04-29 14:07:29 UTC
I think this is just another example of the memory fragmentation problem, which has been reported before. See, for example, here: 
https://bugs.documentfoundation.org/show_bug.cgi?id=140505#c3

Unfortunately, as it says in that comment, this is a problem which will likely never be fixed. Therefore I recommend this be closed as WF. 

Workaround is just to restart LibO when memory usage gets too high.
Comment 9 elias estatistics 2022-04-29 15:47:36 UTC
I like open source programs.
and i thank you for your time for creating a such wonderful product.

However, i think this problem is very serious (low mem systems almost will hung as it happened previously to me often).

You must have a very big RED sign when someone is downloading and using this program cause this may cause computer problems. And he/she must be aware of that problem in order to watch his/her memory when is reaching full... in order to shutting down librecalc 

(note: why when i terminate libre calc also libre writer and all the other libre apps terminate too? this is bug too?) 

I ate many hours to find out what was causing my system to hung up. 

Until now, the 2 most serious bugs of Libre calc are not solved:
a) memory problem 
b) adapt height row
 
Both causing a lot of work hours lost for me and possibly for many other people that are formula-calc oriented with large tables. 

Hopefully, by upgrading memory solved in part the mem problem for now. 
For (b) problem, no solution.... 

i know that you offer your free time, and this is suburb. however, If it is hard to fix this bug, put a warning sign please... (or tell them to have mem >12GB otherwise hungs may occur)

sincerely,
Elias the vegan, for animals