LOOL, at least in its incarnation as Collabora CODE 3.4 as well as 4.0rc2, fills the directory /tmp/convert-to with gigabytes of stuff, until storage space runs out. This already has been reported some time ago by someone in the Nextcloud forums and on Twitter, but apparently this wasn't the correct location - is it here? Or is it just a packaging problem of the Collabora CODE Docker container? So where should this problem be reported? * https://help.nextcloud.com/t/loolwsd-not-cleaning-up-files-in-tmp-convert-to/40653 * https://twitter.com/jsxc_org/status/1059477288528633862
Yes, this is the correct place. Confirmed, and thanks for reporting!
LOOL in a Collabora CODE docker-container also leaves much cruft behind in /var/cache/loolwsd and /tmp/lu*.tmp it's just much smaller in size so it isn't noticed as quickly by the disk running out of space. The files in those directories are often weeks old, and /var/cache/loolwsd already occupies about half a gigabyte on my system at them moment, while the directories under /tmp/lu*.tmp take only few space, but there are more than *480* (!) of them... Does this fit into this bug, shall I file further bugs, or isn't it a problem of LOOL at all and I should file it somewhere else (and if so, where)?
The /var/cache stuff -should- be cleaned up by a chron job periodically I believe. At least the spec file has: mkdir -p %{buildroot}/etc/cron.d echo "#Remove old tiles once every 10 days at midnight" > %{buildroot}/etc/cron.d/loolwsd.cron echo "0 0 */1 * * root find /var/cache/loolwsd -type f -a -atime +10 -exec rm {} \;" >> %{buildroot}/etc/cron.d/loolwsd.cron The latest code has that in-memory instead of disk-backed too - but that's still a WIP. I'm looking at the convert-to cache leakage - I'm curious if you're using 4.0.0 what the content of those files is: run 'file' on a few; and/or attach one that is ~content free here. I can only assume that the conversion fails for them.
Hopefully now fixed - with some simpler code too; will need some back-porting action though.
Michael Meeks committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/online/+/2d473222e4cad399b131345395d6506b26e0e134%5E%21 tdf#123482 - cleanup convert-to folder more reliably.
(In reply to Michael Meeks from comment #3) > I'm looking at the convert-to cache leakage - I'm curious if you're using > 4.0.0 what the content of those files is: run 'file' on a few; and/or attach > one that is ~content free here. I can only assume that the conversion fails > for them. root@7f5fcf32a9ca:/tmp/convert-to# file */* tmp29aaeaaa/callback:: OpenDocument Spreadsheet tmp29abeaaa/callback:: OpenDocument Spreadsheet tmp29aceaaa/callback:: OpenDocument Spreadsheet tmp29adeaaa/callback:: OpenDocument Spreadsheet tmp29aeeaaa/callback:: OpenDocument Spreadsheet tmp29atdaaa/callback:: OpenDocument Spreadsheet tmp29audaaa/callback:: OpenDocument Spreadsheet tmp29avdaaa/callback:: OpenDocument Spreadsheet tmp29awdaaa/callback:: OpenDocument Spreadsheet tmp29axdaaa/callback:: OpenDocument Spreadsheet tmp29aydaaa/callback:: OpenDocument Presentation tmp29azdaaa/callback:: OpenDocument Spreadsheet tmp29baeaaa/callback:: OpenDocument Spreadsheet tmp29bbeaaa/callback:: OpenDocument Spreadsheet tmp29bceaaa/callback:: OpenDocument Spreadsheet tmp29bdeaaa/callback:: OpenDocument Spreadsheet tmp29beeaaa/callback:: OpenDocument Presentation tmp29btdaaa/callback:: OpenDocument Spreadsheet tmp29budaaa/callback:: OpenDocument Spreadsheet tmp29bvdaaa/callback:: OpenDocument Spreadsheet tmp29bwdaaa/callback:: OpenDocument Spreadsheet tmp29bxdaaa/callback:: OpenDocument Spreadsheet tmp29bydaaa/callback:: OpenDocument Spreadsheet tmp29bzdaaa/callback:: OpenDocument Spreadsheet tmp29caeaaa/callback:: Microsoft OOXML tmp29cbeaaa/callback:: OpenDocument Spreadsheet tmp29cceaaa/callback:: OpenDocument Spreadsheet tmp29cdeaaa/callback:: OpenDocument Spreadsheet tmp29ceeaaa/callback:: OpenDocument Spreadsheet tmp29ctdaaa/callback:: OpenDocument Spreadsheet tmp29cudaaa/callback:: OpenDocument Spreadsheet tmp29cvdaaa/callback:: OpenDocument Spreadsheet tmp29cwdaaa/callback:: OpenDocument Spreadsheet tmp29cxdaaa/callback:: OpenDocument Presentation tmp29cydaaa/callback:: OpenDocument Presentation tmp29czdaaa/callback:: OpenDocument Spreadsheet tmp29daeaaa/callback:: OpenDocument Spreadsheet tmp29dbeaaa/callback:: OpenDocument Spreadsheet tmp29dceaaa/callback:: OpenDocument Spreadsheet tmp29ddeaaa/callback:: OpenDocument Spreadsheet tmp29deeaaa/callback:: OpenDocument Spreadsheet tmp29dtdaaa/callback:: OpenDocument Presentation tmp29dudaaa/callback:: OpenDocument Spreadsheet tmp29dvdaaa/callback:: OpenDocument Spreadsheet tmp29dwdaaa/callback:: OpenDocument Presentation tmp29dxdaaa/callback:: OpenDocument Spreadsheet tmp29dydaaa/callback:: Microsoft Word 2007+ tmp29dzdaaa/callback:: OpenDocument Spreadsheet tmp29eaeaaa/callback:: OpenDocument Spreadsheet tmp29ebeaaa/callback:: OpenDocument Spreadsheet tmp29eceaaa/callback:: OpenDocument Spreadsheet tmp29edeaaa/callback:: OpenDocument Presentation tmp29eeeaaa/callback:: OpenDocument Spreadsheet tmp29etdaaa/callback:: OpenDocument Spreadsheet tmp29eudaaa/callback:: OpenDocument Spreadsheet tmp29evdaaa/callback:: OpenDocument Presentation tmp29ewdaaa/callback:: OpenDocument Spreadsheet tmp29exdaaa/callback:: OpenDocument Spreadsheet tmp29eydaaa/callback:: Microsoft Excel 2007+ tmp29ezdaaa/callback:: OpenDocument Presentation tmp29faeaaa/callback:: OpenDocument Spreadsheet tmp29fbeaaa/callback:: OpenDocument Spreadsheet tmp29fceaaa/callback:: OpenDocument Spreadsheet tmp29fdeaaa/callback:: OpenDocument Spreadsheet tmp29ftdaaa/callback:: OpenDocument Spreadsheet tmp29fudaaa/callback:: OpenDocument Spreadsheet tmp29fvdaaa/callback:: OpenDocument Spreadsheet tmp29fwdaaa/callback:: OpenDocument Spreadsheet tmp29fxdaaa/callback:: OpenDocument Presentation tmp29fydaaa/callback:: OpenDocument Presentation tmp29fzdaaa/callback:: OpenDocument Spreadsheet tmp29gaeaaa/callback:: OpenDocument Spreadsheet tmp29gbeaaa/callback:: OpenDocument Presentation tmp29gceaaa/callback:: OpenDocument Presentation tmp29gdeaaa/callback:: OpenDocument Spreadsheet tmp29gtdaaa/callback:: OpenDocument Spreadsheet tmp29gudaaa/callback:: OpenDocument Spreadsheet tmp29gvdaaa/callback:: OpenDocument Spreadsheet tmp29gwdaaa/callback:: OpenDocument Spreadsheet tmp29gxdaaa/callback:: OpenDocument Spreadsheet tmp29gydaaa/callback:: OpenDocument Spreadsheet tmp29gzdaaa/callback:: OpenDocument Spreadsheet tmp29haeaaa/callback:: OpenDocument Spreadsheet tmp29hbeaaa/callback:: OpenDocument Spreadsheet tmp29hceaaa/callback:: OpenDocument Spreadsheet tmp29hdeaaa/callback:: OpenDocument Spreadsheet tmp29htdaaa/callback:: OpenDocument Spreadsheet tmp29hudaaa/callback:: OpenDocument Spreadsheet tmp29hvdaaa/callback:: OpenDocument Spreadsheet tmp29hwdaaa/callback:: OpenDocument Spreadsheet tmp29hxdaaa/callback:: OpenDocument Spreadsheet tmp29hydaaa/callback:: OpenDocument Presentation tmp29hzdaaa/callback:: OpenDocument Spreadsheet tmp29iaeaaa/callback:: OpenDocument Spreadsheet tmp29ibeaaa/callback:: OpenDocument Spreadsheet tmp29iceaaa/callback:: OpenDocument Spreadsheet tmp29ideaaa/callback:: OpenDocument Spreadsheet tmp29itdaaa/callback:: OpenDocument Presentation tmp29iudaaa/callback:: OpenDocument Spreadsheet tmp29ivdaaa/callback:: OpenDocument Spreadsheet tmp29iwdaaa/callback:: OpenDocument Spreadsheet tmp29ixdaaa/callback:: OpenDocument Spreadsheet tmp29iydaaa/callback:: OpenDocument Presentation tmp29izdaaa/callback:: OpenDocument Spreadsheet tmp29jaeaaa/callback:: OpenDocument Spreadsheet tmp29jbeaaa/callback:: OpenDocument Spreadsheet tmp29jceaaa/callback:: OpenDocument Spreadsheet tmp29jdeaaa/callback:: OpenDocument Spreadsheet tmp29jtdaaa/callback:: OpenDocument Spreadsheet tmp29judaaa/callback:: OpenDocument Presentation tmp29jvdaaa/callback:: OpenDocument Spreadsheet tmp29jwdaaa/callback:: OpenDocument Spreadsheet tmp29jxdaaa/callback:: OpenDocument Spreadsheet tmp29jydaaa/callback:: OpenDocument Spreadsheet tmp29jzdaaa/callback:: OpenDocument Spreadsheet tmp29kaeaaa/callback:: OpenDocument Spreadsheet tmp29kbeaaa/callback:: OpenDocument Spreadsheet tmp29kceaaa/callback:: OpenDocument Spreadsheet tmp29kdeaaa/callback:: OpenDocument Spreadsheet tmp29ktdaaa/callback:: OpenDocument Spreadsheet tmp29kudaaa/callback:: OpenDocument Spreadsheet tmp29kvdaaa/callback:: OpenDocument Spreadsheet tmp29kwdaaa/callback:: OpenDocument Spreadsheet tmp29kxdaaa/callback:: OpenDocument Spreadsheet tmp29kydaaa/callback:: OpenDocument Spreadsheet tmp29kzdaaa/callback:: OpenDocument Spreadsheet tmp29laeaaa/callback:: OpenDocument Spreadsheet tmp29lbeaaa/callback:: OpenDocument Spreadsheet tmp29lceaaa/callback:: OpenDocument Spreadsheet tmp29ldeaaa/callback:: OpenDocument Spreadsheet tmp29ltdaaa/callback:: OpenDocument Spreadsheet tmp29ludaaa/callback:: OpenDocument Spreadsheet tmp29lvdaaa/callback:: OpenDocument Spreadsheet tmp29lwdaaa/callback:: OpenDocument Spreadsheet tmp29lxdaaa/callback:: OpenDocument Spreadsheet tmp29lydaaa/callback:: OpenDocument Presentation tmp29lzdaaa/callback:: OpenDocument Spreadsheet tmp29maeaaa/callback:: OpenDocument Spreadsheet tmp29mbeaaa/callback:: OpenDocument Spreadsheet tmp29mceaaa/callback:: OpenDocument Spreadsheet tmp29mdeaaa/callback:: OpenDocument Spreadsheet tmp29mtdaaa/callback:: OpenDocument Spreadsheet tmp29mudaaa/callback:: OpenDocument Spreadsheet tmp29mvdaaa/callback:: OpenDocument Spreadsheet tmp29mwdaaa/callback:: OpenDocument Spreadsheet tmp29mxdaaa/callback:: OpenDocument Spreadsheet tmp29mydaaa/callback:: OpenDocument Spreadsheet tmp29mzdaaa/callback:: OpenDocument Spreadsheet tmp29naeaaa/callback:: OpenDocument Spreadsheet tmp29nbeaaa/callback:: OpenDocument Spreadsheet tmp29nceaaa/callback:: OpenDocument Spreadsheet tmp29ndeaaa/callback:: OpenDocument Spreadsheet tmp29ntdaaa/callback:: OpenDocument Spreadsheet tmp29nudaaa/callback:: OpenDocument Spreadsheet tmp29nvdaaa/callback:: OpenDocument Spreadsheet tmp29nwdaaa/callback:: OpenDocument Presentation tmp29nxdaaa/callback:: OpenDocument Presentation tmp29nydaaa/callback:: OpenDocument Spreadsheet tmp29nzdaaa/callback:: OpenDocument Spreadsheet tmp29oaeaaa/callback:: OpenDocument Spreadsheet tmp29obeaaa/callback:: OpenDocument Presentation tmp29oceaaa/callback:: OpenDocument Spreadsheet tmp29odeaaa/callback:: OpenDocument Spreadsheet tmp29otdaaa/callback:: OpenDocument Spreadsheet tmp29oudaaa/callback:: OpenDocument Presentation tmp29ovdaaa/callback:: OpenDocument Spreadsheet tmp29owdaaa/callback:: OpenDocument Spreadsheet tmp29oxdaaa/callback:: OpenDocument Spreadsheet tmp29oydaaa/callback:: OpenDocument Presentation tmp29ozdaaa/callback:: OpenDocument Presentation tmp29paeaaa/callback:: OpenDocument Spreadsheet tmp29pbeaaa/callback:: OpenDocument Spreadsheet tmp29pceaaa/callback:: OpenDocument Spreadsheet tmp29pdeaaa/callback:: OpenDocument Presentation tmp29ptdaaa/callback:: OpenDocument Spreadsheet tmp29pudaaa/callback:: OpenDocument Spreadsheet tmp29pvdaaa/callback:: OpenDocument Spreadsheet tmp29pwdaaa/callback:: OpenDocument Spreadsheet tmp29pxdaaa/callback:: OpenDocument Spreadsheet tmp29pydaaa/callback:: OpenDocument Spreadsheet tmp29pzdaaa/callback:: OpenDocument Spreadsheet tmp29qaeaaa/callback:: OpenDocument Spreadsheet tmp29qbeaaa/callback:: OpenDocument Spreadsheet tmp29qceaaa/callback:: OpenDocument Presentation tmp29qdeaaa/callback:: OpenDocument Spreadsheet tmp29qtdaaa/callback:: OpenDocument Spreadsheet tmp29qudaaa/callback:: OpenDocument Spreadsheet tmp29qvdaaa/callback:: OpenDocument Spreadsheet tmp29qwdaaa/callback:: OpenDocument Spreadsheet tmp29qxdaaa/callback:: OpenDocument Spreadsheet tmp29qydaaa/callback:: OpenDocument Spreadsheet tmp29qzdaaa/callback:: OpenDocument Spreadsheet tmp29raeaaa/callback:: OpenDocument Spreadsheet tmp29rbeaaa/callback:: OpenDocument Spreadsheet tmp29rceaaa/callback:: OpenDocument Spreadsheet tmp29rdeaaa/callback:: OpenDocument Spreadsheet tmp29rtdaaa/callback:: OpenDocument Presentation tmp29rudaaa/callback:: OpenDocument Spreadsheet tmp29rvdaaa/callback:: OpenDocument Spreadsheet tmp29rwdaaa/callback:: OpenDocument Text tmp29rxdaaa/callback:: OpenDocument Spreadsheet tmp29rydaaa/callback:: OpenDocument Presentation tmp29rzdaaa/callback:: OpenDocument Spreadsheet tmp29saeaaa/callback:: OpenDocument Spreadsheet tmp29sbeaaa/callback:: OpenDocument Spreadsheet tmp29sceaaa/callback:: OpenDocument Spreadsheet tmp29sdeaaa/callback:: OpenDocument Spreadsheet tmp29stdaaa/callback:: OpenDocument Spreadsheet tmp29sudaaa/callback:: OpenDocument Spreadsheet tmp29svdaaa/callback:: OpenDocument Spreadsheet tmp29swdaaa/callback:: OpenDocument Spreadsheet tmp29sxdaaa/callback:: OpenDocument Presentation tmp29sydaaa/callback:: OpenDocument Spreadsheet tmp29szdaaa/callback:: OpenDocument Spreadsheet tmp29taeaaa/callback:: OpenDocument Spreadsheet tmp29tbeaaa/callback:: OpenDocument Spreadsheet tmp29tceaaa/callback:: OpenDocument Spreadsheet tmp29tdeaaa/callback:: OpenDocument Spreadsheet tmp29ttdaaa/callback:: OpenDocument Spreadsheet tmp29tudaaa/callback:: OpenDocument Spreadsheet tmp29tvdaaa/callback:: OpenDocument Presentation tmp29twdaaa/callback:: OpenDocument Spreadsheet tmp29txdaaa/callback:: OpenDocument Spreadsheet tmp29tydaaa/callback:: OpenDocument Spreadsheet tmp29tzdaaa/callback:: OpenDocument Spreadsheet tmp29uaeaaa/callback:: OpenDocument Spreadsheet tmp29ubeaaa/callback:: OpenDocument Spreadsheet tmp29uceaaa/callback:: OpenDocument Spreadsheet tmp29udeaaa/callback:: OpenDocument Spreadsheet tmp29utdaaa/callback:: OpenDocument Spreadsheet tmp29uudaaa/callback:: OpenDocument Spreadsheet tmp29uvdaaa/callback:: OpenDocument Spreadsheet tmp29uwdaaa/callback:: OpenDocument Spreadsheet tmp29uxdaaa/callback:: OpenDocument Spreadsheet tmp29uydaaa/callback:: OpenDocument Presentation tmp29uzdaaa/callback:: Microsoft OOXML tmp29vaeaaa/callback:: OpenDocument Spreadsheet tmp29vbeaaa/callback:: OpenDocument Spreadsheet tmp29vceaaa/callback:: OpenDocument Spreadsheet tmp29vdeaaa/callback:: OpenDocument Spreadsheet tmp29vtdaaa/callback:: OpenDocument Spreadsheet tmp29vudaaa/callback:: OpenDocument Spreadsheet tmp29vvdaaa/callback:: OpenDocument Spreadsheet tmp29vwdaaa/callback:: OpenDocument Spreadsheet tmp29vxdaaa/callback:: OpenDocument Spreadsheet tmp29vydaaa/callback:: OpenDocument Spreadsheet tmp29vzdaaa/callback:: OpenDocument Spreadsheet tmp29waeaaa/callback:: OpenDocument Spreadsheet tmp29wbeaaa/callback:: OpenDocument Spreadsheet tmp29wceaaa/callback:: OpenDocument Spreadsheet tmp29wdeaaa/callback:: OpenDocument Spreadsheet tmp29wsdaaa/callback:: OpenDocument Spreadsheet tmp29wtdaaa/callback:: OpenDocument Spreadsheet tmp29wudaaa/callback:: OpenDocument Spreadsheet tmp29wvdaaa/callback:: OpenDocument Spreadsheet tmp29wwdaaa/callback:: OpenDocument Presentation tmp29wxdaaa/callback:: OpenDocument Presentation tmp29wydaaa/callback:: OpenDocument Spreadsheet tmp29wzdaaa/callback:: OpenDocument Spreadsheet tmp29xaeaaa/callback:: OpenDocument Spreadsheet tmp29xbeaaa/callback:: OpenDocument Spreadsheet tmp29xceaaa/callback:: OpenDocument Spreadsheet tmp29xdeaaa/callback:: OpenDocument Spreadsheet tmp29xsdaaa/callback:: OpenDocument Spreadsheet tmp29xtdaaa/callback:: OpenDocument Spreadsheet tmp29xudaaa/callback:: Microsoft OOXML tmp29xvdaaa/callback:: OpenDocument Spreadsheet tmp29xwdaaa/callback:: OpenDocument Spreadsheet tmp29xxdaaa/callback:: OpenDocument Spreadsheet tmp29xydaaa/callback:: OpenDocument Spreadsheet tmp29xzdaaa/callback:: OpenDocument Spreadsheet tmp29yaeaaa/callback:: OpenDocument Spreadsheet tmp29ybeaaa/callback:: OpenDocument Presentation tmp29yceaaa/callback:: OpenDocument Presentation tmp29ydeaaa/callback:: OpenDocument Spreadsheet tmp29ysdaaa/callback:: OpenDocument Spreadsheet tmp29ytdaaa/callback:: OpenDocument Spreadsheet tmp29yudaaa/callback:: OpenDocument Spreadsheet tmp29yvdaaa/callback:: OpenDocument Spreadsheet tmp29ywdaaa/callback:: OpenDocument Spreadsheet tmp29yxdaaa/callback:: OpenDocument Spreadsheet tmp29yydaaa/callback:: OpenDocument Spreadsheet tmp29yzdaaa/callback:: OpenDocument Spreadsheet tmp29zaeaaa/callback:: OpenDocument Spreadsheet tmp29zbeaaa/callback:: OpenDocument Spreadsheet tmp29zceaaa/callback:: OpenDocument Spreadsheet tmp29zdeaaa/callback:: OpenDocument Spreadsheet tmp29zsdaaa/callback:: OpenDocument Spreadsheet tmp29ztdaaa/callback:: OpenDocument Presentation tmp29zudaaa/callback:: OpenDocument Spreadsheet tmp29zvdaaa/callback:: OpenDocument Spreadsheet tmp29zwdaaa/callback:: OpenDocument Spreadsheet tmp29zxdaaa/callback:: OpenDocument Text tmp29zydaaa/callback:: OpenDocument Spreadsheet tmp29zzdaaa/callback:: OpenDocument Spreadsheet root@7f5fcf32a9ca:/tmp/convert-to#
Thanks - any chance you can try loading one or two in LibreOffice to see if that fails or gives a warning prompt in some interesting way ? would love to know =) or attach one or two here perhaps.
Created attachment 149685 [details] Contents of /tmp/convert-to shortly after completely cleaning the directory. I created a simple test spreadsheet, removed all contents from /tmp/convert-to and opened the file once from within nextcloud. The tar contains the resulting /tmp/convert-to directory tree containing four copies of my test spreadsheet and three copies of the default Nextcloud welcome document file which was in the same directory, but which I didn't even open explicitly. $ file */* tmp30bbaaaa/callback:: OpenDocument Text tmp30dbaaaa/callback:: OpenDocument Spreadsheet tmp30ebaaaa/callback:: OpenDocument Spreadsheet tmp30fbaaaa/callback:: OpenDocument Text tmp30hbaaaa/callback:: OpenDocument Spreadsheet tmp30ibaaaa/callback:: OpenDocument Text tmp30kbaaaa/callback:: OpenDocument Spreadsheet $ I'm not wondering any longer why CODE fills up all available disk space within a few days... ;)
Hmm - those files look completely normal - and should thumbnail perfectly - very odd; anyhow hopefully this fix will help. Aron - any chance of some testing ? the files are just the stock 'Nextcloud' sample documents being thumbnailed by new users.
I just noticed that the following log messages in the CODE docker container seem to be related to failing preview images: wsd-00030-00040 2019-03-04 17:28:52.610993 [ websrv_poll ] ERR Requesting address is denied: ::ffff:172.17.0.1| wsd/LOOLWSD.cpp:1971 I'm using the CODE docker container which only listens on the localhost IPv4 address, and any Apache reverse proxy forwarding to this address. So IPv6 should not be involved...
Michael Meeks committed a patch related to this issue. It has been pushed to "libreoffice-6-1": https://git.libreoffice.org/online/+/a788dd238a27b36ca6f3cf98a20c52c56654b3f4%5E%21 tdf#123482 - cleanup convert-to folder more reliably.
It appears this issue is still present even though the patch is there in the latest Collabora Online 4.0.4 and master - I suspect it is down to an exception while processing a multi-part message; and will dig into it in a bit. In the meantime, just deleting any files that are more than a few minutes old is a workaround.
An even better fix waiting for review here: https://gerrit.libreoffice.org/#/c/72695/ and an even-nicer one (to fix the root cause with multi-part messages from Nextcloud) is in the works - although a more recent richdocuments may help there that streams from a local temporary file rather than direct from object storage; still looking ...
Michael Meeks committed a patch related to this issue. It has been pushed to "master": https://git.libreoffice.org/online/+/1845ed42af661dbb7b10a2fb4686ce2c8ef7e521%5E%21 tdf#123482 - cleanup convert-to folder even more reliably.