Created attachment 111705 [details] File with external links, which are automatically updated every 10 secs -> eats all memory A file with external links to websites eats all memory until libreoffice crashes or the OS kills it, because of low memory. A simple file is attached to this case. Open the attached file and wait until the system runs out of memory.
Created attachment 111712 [details] Valgrind trace with master sources On pc Debian x86-64 with master sources, I retrieve a Valgrind trace.
Memory increases slowly but surely, so put it at NEW.
The example file is a very simple file. With a much more complex file, libreoffice calc uses more than 3 GB of memory within 4-5 hours.
This problem is getting worse in version 4.8.2.8 (OS: Ubuntu 14.04). It seems that more memory leaks exist than in older versions.
Ralf: version 4.8 doesn't exist. There are these: - 4.3.7 - 4.4.3 - 5.0.0 in beta
Thank you for your reply. Maybe the Ubuntu team did something on its own? The following text is copied out of the about-box: Version: 4.2.8.2 Build-ID: 420m0(Build:2) Then, I will have a look at ubuntu launchpad for this issue. The performance is getting worse and the memory usage rises faster than before. I recently upgraded my notebook ram to 12GB ... 4GB were too few (for my file). The file attached to this case is a simplified file to demonstrate the issue. Best regards, Ralf
4.2.8 is EOL (see https://wiki.documentfoundation.org/ReleasePlan) Please try a newer version from ppa (see https://launchpad.net/~libreoffice/+archive/ubuntu/ppa)
Thnk you very much for your reply. I will try it out. Best regards, Ralf
I tested the newest version, but the memory consumption does not change. VIRT RES SHR S %CPU %MEM TIME+ 1953808 701352 112180 S 10,3 18,5 1:40.77 2963636 1,515g 65356 S 29,5 41,9 6:50.41 3644456 2,131g 58548 S 16,2 59,0 10:39.36 Withhin one hour, the memory consumption increases by 1,4GB. Best regards, Ralf
Created attachment 116531 [details] bt with debug symbols from first message I noticed this kind of messages several times: warn:legacy.tools:13423:1:editeng/source/editeng/eehtml.cxx:54: EditHTMLParser::EditHTMLParser: Where does the encoding come from? (pkix_CacheCert_Add: PKIX_PL_HashTable_Add for Certs skipped: entry existed I attached a bt from first message
I noticed too that ImpEditEngine::ReadHTML (see http://opengrok.libreoffice.org/xref/core/editeng/source/editeng/impedit4.cxx#175) creates a new EditHTMLParser but doesn't delete it.
Eike: the file is an ods but the pb isn't in Calc part. Any thoughts?
Michael: In addition of the memory leak (see comment 11 for a lead), I don't even succeed in opening the file now (pc Debian x86-64 with master sources updated yesterday). Looking at the bt retrieved, I wonder if it could be due to timer work since I can open it with LO Debian package (4.4.4.1) Any thought?
Hi Julien - thanks for looking into this ! =) I -think- the: EditHTMLParserRef xPrsr = new EditHTMLParser( rInput, rBaseURL, pHTTPHeaderAttrs ); is likely to be a smart reference that frees that thing. Thanks for the valgrind log - it woudl be great to get a longer log of the main process with: "Rerun with --leak-check=full to see details of leaked memory" So we get stack traces of where that memory was allocated from (and not freed) =) Many thanks !
Thank you Michael for your feedback, I'll try to find some time to retrieve a new Valgrind trace (Valgrind is still very slow :-)) About timer part, any thoughts? Indeed, as I indicated, I can't even open the file either I click OK or I click No to update links. Would you prefer, I open a new tracker about this to distinguish the 2 problems? (Of course, it doesn't prevent me from retrieving a Valgrind trace :-))
Created attachment 116606 [details] Valgrind trace Here's a new Valgrind trace. It seems LO exits (crashes?) before asking if I want to update links.
Created attachment 118054 [details] A smaller and slower update rate file for running on valgrind which produces leak Attached an ods file which pulls data from http://kurse.boerse.ard.de/ard/indizes_einzelkurs_uebersicht.htn?i=159096 on a single sheet with 10 minutes update interval. This also causes memory leak and can be run in valgrind more easily.
Created attachment 118055 [details] Valgrind log where calc loads the smaller file previously attached. Attached valgrind log where calc loads the smaller file (previously attached).
If you think the last traces answer Michael Meeks' question set it back to NEW. Is this still a problem with newer versions of LibO?
Dear Bug Submitter, This bug has been in NEEDINFO status with no change for at least 6 months. Please provide the requested information as soon as possible and mark the bug as UNCONFIRMED. Due to regular bug tracker maintenance, if the bug is still in NEEDINFO status with no change in 30 days the QA team will close the bug as INSUFFICIENTDATA due to lack of needed information. For more information about our NEEDINFO policy please read the wiki located here: https://wiki.documentfoundation.org/QA/Bugzilla/Fields/Status/NEEDINFO If you have already provided the requested information, please mark the bug as UNCONFIRMED so that the QA team knows that the bug is ready to be confirmed. Thank you for helping us make LibreOffice even better for everyone! Warm Regards, QA Team MassPing-NeedInfo-Ping-20161108
The problem is solved in the latest version, which is shipped with UBUNTU 16.04. Memory usage is nearly stable over several hours (performance could be better). Thank you for your support! Best regards, Ralf
Thank you for your feedback. Let's put this one WFM then.
One year later ... the same problem occured. Libreoffice with Ubuntu 16.04 with all updates installed. LO calc has been killed by the operating system, because it uses all available memory. Memory consumption reaches about 15GiB within max. 5 minutes. My system has 16GiB installed.
ooops, I forgot to mention ... the libreoffice version is Version: 5.4.3.2 Build-ID: 1:5.4.3~rc2-0ubuntu0.16.04.1~lo1 and the os version is: lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.3 LTS Release: 16.04 Codename: xenial
the problem exists on all my systems: Another system with 32GiB memory installed: top - 16:28:25 up 1:56, 2 users, load average: 0,96, 0,71, 0,37 Tasks: 313 gesamt, 2 laufend, 311 schlafend, 0 gestoppt, 0 Zombie %CPU(s): 8,1 be, 0,3 sy, 0,0 ni, 91,6 un, 0,0 wa, 0,0 hi, 0,0 si, 0,0 st KiB Spch : 31919772 gesamt, 9765204 frei, 21096172 belegt, 1058396 Puff/Cache KiB Swap : 20971452 gesamt, 20971452 frei, 0 belegt. 10424484 verfü Spch PID USER PR NI VIRT RES SHR S %CPU %MEM ZEIT+ BEFEHL 26083 ralf 20 0 20,719g 0,019t 124168 R 49,2 64,9 3:38.40 soffice.bin 20GiB ... This seems to be the maximum. Don't know why. While experimenting, I also deleted the user settings. But there is no change. Thank you in advance. Best regards, Ralf
Hmm; I guess running thus under 'valgrind --tool=massif' will show where the huge memory use is (I guess) if it is a fast leak it'll find the problem fast too ;-) Thanks.
Ralf: just to be sure, do you: 1) Just open the file without even responding to the dialog asking about updating links 2) open the file + respond Yes to update 3) open the file + respond No to update ? I tried option 2 with Instruments/memory leaks on MacOs and Valgrind but found nothing useful
Hi Julien and Michael, sorry, sorry, sorry. I could not reproduce the situation. The only thing I have seen, was an update of libc. Maybe, this library was responsible for the bug. The memory consumption seem to be ok now. Thank you very much for your quick answers. Best regards, Ralf ps: I changed the status of the bug. ok?
Thank you for your feedback, ok for the status.
Just to add; the problem is not only limited to file with external references. Recreation: Open but any excel file. simple make changes in the file like , adding filter etc. then leave it for some time and do some other work in another application. when you refocus on the libreoffice calc file again. it would have by then consumed all the available memory space.
My System Info. Ubuntu 17.04 64bit LibreOffice Version: 6.0.4.2
I have about 45 user files that are consolidated into a single workbook, using Links to External Files. Data for one range (about 3000 lines) is pulled in from the user files into separate sheets. It is not possible to update all the links at the same time or even "update links" when opening the file. Once 4-5 links are updated, the memory usage goes up by around 2-3 GB. This memory is not released after the update is completed. Thus it is not possible to complete the full update process. Saving the file also does not release memory. OS: Linux Mint 19.1 Libreoffice Version: 6.0.7.3 Build ID: 1:6.0.7-0ubuntu0.18.04.5 CPU threads: 2; OS: Linux 4.15; UI render: default; VCL: gtk3; I had to revise my consolidation file due to the non-resolution of this bug: [Bug 117563] LibreOffice Calc: Links to External Files - Linked Ranges are saved incorrectly - Update Links overwrites data and removes links Would appreciate any help that I can get. Thanks KSN
(In reply to KSN from comment #32) >... > Once 4-5 links are updated, the memory usage goes up by around 2-3 GB. This > memory is not released after the update is completed. Thus it is not > possible to complete the full update process. Saving the file also does not > release memory. >... Please don't hijack a bugtracker but submit a new one. Before this, give a try to last LO version 6.2.4 since 6.0.X versions are EOL and 6.1.X will be soon.