Bug 42684 - FILEOPEN very slow, network/dns related, it takes about 15 minutes
Summary: FILEOPEN very slow, network/dns related, it takes about 15 minutes
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Calc (show other bugs)
Version:
(earliest affected)
3.3.1 release
Hardware: All Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-11-07 14:15 UTC by mbohets
Modified: 2013-11-25 14:58 UTC (History)
6 users (show)

See Also:
Crash report or crash signature:


Attachments
strace while opening the file (2.64 MB, application/x-bzip)
2011-11-07 14:15 UTC, mbohets
Details
strace of sheet open in LO 3.3 (446.35 KB, application/x-bzip)
2011-11-12 01:17 UTC, mbohets
Details
strace after throwing out personal stuff and after a LO upgrade in Ubuntu (1.95 MB, application/x-bzip)
2011-11-17 14:17 UTC, mbohets
Details
sheet showing the problem described in the bug report (129.87 KB, application/vnd.oasis.opendocument.spreadsheet)
2011-11-17 14:18 UTC, mbohets
Details
strace taken with Chackra distro instead of Kubuntu 11.10 (1.80 MB, application/x-bzip2)
2011-11-27 04:10 UTC, mbohets
Details

Note You need to log in before you can comment on or make changes to this bug.
Description mbohets 2011-11-07 14:15:29 UTC
Created attachment 53273 [details]
strace while opening the file

When I open the file homesheet2011.ods, it takes about 15 minutes before it is open.
When I open the same document with openoffice 3.1, the document opens normally.
OS: Kubuntu 11.10, upgrade on one PC, fresh install on laptop (same problem)
Installed via the Kubuntu repositories.
SW release: LibreOffice 3.4.3   OOO340m1 (Build:302)

> The top command shows my CPU is 100% occupied
> asks: 170 total,   2 running, 167 sleeping,   0 stopped,   1 zombie
> Cpu(s): 53.6%us,  3.2%sy,  0.0%ni, 40.3%id,  2.8%wa,  0.0%hi,  0.2%si,
> 0.0%st
> Mem:   2054052k total,  1637320k used,   416732k free,    36568k buffers
> Swap:  2087932k total,    19180k used,  2068752k free,   576984k cached
> 
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 15890 marc      20   0  371m 209m  60m R  100 10.4   3:13.20 soffice.bin
>  1055 root      20   0  146m  46m  33m S    4  2.3   3:57.71 Xorg
>  1601 marc      20   0  299m  50m  20m S    3  2.5   5:29.93 kwin
>  3562 marc      20   0  497m 241m  25m S    3 12.0   3:54.00 rekonq
>  1611 marc      20   0  260m  21m  13m S    2  1.1   2:57.86 knotify4
>  2288 marc      20   0  138m  40m  23m S    2  2.0   3:08.79 dolphin
>  8076 marc      20   0  109m  24m  17m S    2  1.2   0:01.84 konsole

The problem is network/dns related (see network trace), 
Calc wants to resolve some urls and their thawte and verisign certificates.  

marc@marc-6320:~/Documents/OOO330_m20_native_packed-1_en-US.9567/DEBS$ sudo 
tcpdump -nnl -i wlan0 -s 1536 dst port 53
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wlan0, link-type EN10MB (Ethernet), capture size 1536 bytes
00:08:34.412749 IP 192.168.0.184.50638 > 195.130.130.1.53: 11395+ A? 
marc-6320.(none). (34)
00:08:34.427846 IP 192.168.0.184.56845 > 195.130.130.1.53: 60767+ A? 
marc-6320.(none).lan_dn. (41)
00:09:08.680435 IP 192.168.0.184.37999 > 195.130.130.1.53: 53290+ A? 
secure.keytradebank.com. (41)
00:09:08.817358 IP 192.168.0.184.57854 > 195.130.130.1.53: 54384+ A? 
SVRIntl-aia.verisign.com. (42)
00:09:09.117648 IP 192.168.0.184.52596 > 195.130.130.1.53: 5621+ A? 
crl.verisign.com. (34)
00:09:09.542308 IP 192.168.0.184.49393 > 195.130.130.1.53: 31157+ A? 
SVRIntl-crl.verisign.com. (42)
00:09:10.457164 IP 192.168.0.184.43200 > 195.130.130.1.53: 14130+ A? 
www.keytradebank.com. (38)
00:09:10.549438 IP 192.168.0.184.53301 > 195.130.130.1.53: 64786+ A? 
www.thawte.com. (32)
00:09:10.930726 IP 192.168.0.184.36438 > 195.130.130.1.53: 57057+ A? 
crl.thawte.com. (32)
00:09:11.323096 IP 192.168.0.184.49990 > 195.130.130.1.53: 43965+ A? 
crl.thawte.com. (32)
00:09:11.881908 IP 192.168.0.184.55908 > 195.130.130.1.53: 50958+ A? 
secure.keytradebank.com. (41)
00:09:11.953006 IP 192.168.0.184.60660 > 195.130.130.1.53: 44754+ A? 
SVRIntl-aia.verisign.com. (42)
00:09:12.340463 IP 192.168.0.184.37211 > 195.130.130.1.53: 9710+ A? 
www.keytradebank.com. (38)
00:09:12.431319 IP 192.168.0.184.38837 > 195.130.130.1.53: 47319+ A? 
www.thawte.com. (32)
00:09:14.133780 IP 192.168.0.184.52429 > 195.130.130.1.53: 6557+ A? 
citibankonline.citibank.be. (44)
00:09:14.304999 IP 192.168.0.184.60316 > 195.130.130.1.53: 29831+ A? EVIntl-
aia.verisign.com. (41)
00:09:14.889239 IP 192.168.0.184.44654 > 195.130.130.1.53: 43734+ A? 
EVSecure-crl.verisign.com. (43)
00:09:15.304504 IP 192.168.0.184.56166 > 195.130.130.1.53: 58082+ A? EVIntl-
crl.verisign.com. (41)
00:09:16.678805 IP 192.168.0.184.46363 > 195.130.130.1.53: 40532+ A? 
citibankonline.citibank.be. (44)
00:09:16.818202 IP 192.168.0.184.46140 > 195.130.130.1.53: 3943+ A? EVIntl-
aia.verisign.com. (41)
00:19:18.689824 IP 192.168.0.184.34335 > 195.130.130.1.53: 41972+ A? 
citibankonline.citibank.be. (44)
00:19:18.840882 IP 192.168.0.184.38792 > 195.130.130.1.53: 7530+ A? EVIntl-
aia.verisign.com. (41)

While googling for this problem, I saw some messages where calc wabts to resolve the hostname of its own system, which can easily be solved with an entry in /etc/host, but here calc also wants to resolve urls inside the spreadsheet.

In strace.log, you see excactly how long it takes to open the file and you can also see what url resolving is going on (the file contains www.citibank.be and keytrade.be) and what additional ssl and certificates that are not mentioned in the file calc also wants to resolve.

The problem is perfectly reproducable, please let me know if you need additional information.
I did not attach the file itself as it contains my bank account data. 

How can this be turned off (I did not find an option in settings)
Why is calc doing this ? I don't see the purpose of this, it only seems to be a wast of bandwidth and dns resources.
Comment 1 NoOp 2011-11-10 10:57:00 UTC
I wonder if this could be related to:
https://bugs.freedesktop.org/show_bug.cgi?id=32640
[LibreOffice localized as BrOffice (auto) takes a long time to start]
which was supposedly fixed in 3.3. Can you install 3.3.4 directly and test?
Comment 2 mbohets 2011-11-12 01:17:40 UTC
Created attachment 53430 [details]
strace of sheet open in LO 3.3
Comment 3 mbohets 2011-11-12 01:21:13 UTC
When opening the file in LO 3.3, LO stops.
I think some dependencies are missing.
I add an strace and below is the command line output 

arc@VASTEPC:~$ strace -o /tmp/strace.log -f -tt -s 512 libreoffice

(soffice:2265): Gtk-WARNING **: /opt/libreoffice/program/../basis-link/ure-link/lib/libstdc++.so.6: version `GLIBCXX_3.4.11' not found (required by /usr/lib/i386-linux-gnu/gtk-2.0/2.10.0/engines/liboxygen-gtk.so)
marc@VASTEPC:~$ strace -o /tmp/strace.log -f -tt -s 512 libreoffice

(soffice:2351): Gtk-WARNING **: /opt/libreoffice/program/../basis-link/ure-link/lib/libstdc++.so.6: version `GLIBCXX_3.4.11' not found (required by /usr/lib/i386-linux-gnu/gtk-2.0/2.10.0/engines/liboxygen-gtk.so)
marc@VASTEPC:~$ strace -o /tmp/strace.log -f -tt -s 512 libreoffice

(soffice:2392): Gtk-WARNING **: /opt/libreoffice/program/../basis-link/ure-link/lib/libstdc++.so.6: version `GLIBCXX_3.4.11' not found (required by /usr/lib/i386-linux-gnu/gtk-2.0/2.10.0/engines/liboxygen-gtk.so)
terminate called after throwing an instance of 'eIDMW::CMWException'
  what():  std::exception


When opening a simple file with just a few numbers in it, it works.
Comment 4 mbohets 2011-11-17 14:15:34 UTC
Hi,
Sorry for the delay, it was kind of hectic at work.
I also did not want to attach the sheet causing the problem, as it contains all my personal data.
I threw out my personal stuff, and the sheet still has the problem, so I'll attach it to the bug report,
along with a new strace.log

As you see on the tcpdump output, the problem is dns related.
 here I start tcpdump and opened the sheet test4homesheet2011.ods from file/recent documents
marc@VASTEPC:~$ sudo tcpdump -nnl -i eth0 -s 1536 dst port 53
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 1536 bytes
22:08:36.516401 IP 192.168.0.205.57638 > 195.130.131.129.53: 32882+ A? citibankonline.citibank.be. (44)
22:08:36.690849 IP 192.168.0.205.47362 > 195.130.131.129.53: 56456+ A? EVIntl-aia.verisign.com. (41)
22:08:37.105135 IP 192.168.0.205.50653 > 195.130.131.129.53: 32734+ A? crl.verisign.com. (34)
22:08:37.375983 IP 192.168.0.205.55451 > 195.130.131.129.53: 64940+ A? EVSecure-crl.verisign.com. (43)
22:08:37.758145 IP 192.168.0.205.53332 > 195.130.131.129.53: 48144+ A? EVIntl-crl.verisign.com. (41)
22:08:39.148293 IP 192.168.0.205.42231 > 195.130.131.129.53: 1009+ A? citibankonline.citibank.be. (44)
22:08:39.289922 IP 192.168.0.205.45455 > 195.130.131.129.53: 27165+ A? EVIntl-aia.verisign.com. (41)
22:17:12.191380 IP 192.168.0.205.41869 > 195.130.131.129.53: 34958+ A? citibankonline.citibank.be. (44)
22:17:12.360221 IP 192.168.0.205.55684 > 195.130.131.129.53: 37503+ A? EVIntl-aia.verisign.com. (41)
^C
9 packets captured
9 packets received by filter
0 packets dropped by kernel

marc@VASTEPC:~$ date
Thu Nov 17 22:17:45 CET 2011 (at this moment, the sheet opened)

- this week there was a libreoffice update from Kubuntu, the problem is still 
  there.

Some info from the ubuntu-bug command:
ApportVersion: 1.23-0ubuntu4
Architecture: i386
DistroRelease: Ubuntu 11.10
InstallationMedia: Kubuntu 11.10 "Oneiric Ocelot" - Release i386 (20111012)
NonfreeKernelModules: nvidia
Package: libreoffice 1:3.4.4-0ubuntu1
PackageArchitecture: i386
ProcEnviron:
 LANGUAGE=
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 3.0.0-13.22-generic 3.0.6
Tags: oneiric
Uname: Linux 3.0.0-13-generic i686
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm admin cdrom dialout lpadmin plugdev sambashare
Comment 5 mbohets 2011-11-17 14:17:14 UTC
Created attachment 53635 [details]
strace after throwing out personal stuff and after a LO upgrade in Ubuntu
Comment 6 mbohets 2011-11-17 14:18:23 UTC
Created attachment 53636 [details]
sheet showing the problem described in the bug report
Comment 7 NoOp 2011-11-17 14:52:38 UTC
Confirmed in LO 3.4.4 final (linux):
LibreOffice 3.4.4 
OOO340m1 (Build:402)

I finally had to kill the process as it was taking over 95% of my cpu and heating up my system. 

Opens fine in LO 3.3.4 final (linux):
LibreOffice 3.3.4 
OOO330m19 (Build:401)
tag libreoffice-3.3.4.

And go-oo Ubuntu distribution 3.2.1 (linux)
OpenOffice.org 3.2.1 
OOO320m19 (Build:9505)
ooo-build 3.2.1.4, Ubuntu package 1:3.2.1-7ubuntu1.1

And fine in:
OOo-dev 3.4.0 
DEV300m106 (Build: 9582)

So this appears to be a regression between LO 3.3.4 and LO 3.4.4. Can you please try with 3.3.4?
Comment 8 Jean-Baptiste Faure 2011-11-19 06:42:59 UTC
It seems that there is 2 problems in this spreadsheet:
1/ named range with unknown references
2/ the sheet Zichtrekening
If you remove this sheet and the named range with unknown references, the file opens in less than few seconds.
If you only remove the sheet Zichtrekening, the file opens in fews seconds.

Best regards. JBF
Comment 9 mbohets 2011-11-27 04:04:18 UTC
Hi, 
I decided to try out some distros.
When I use the same file in Chackra, it only takes 3 minutes instead of 15
to load it.
Any ideas on why this is ?
LibreOffice 3.4.4 OOO340m1 (Build:401)

I'll upload a new strace

Marc
Comment 10 mbohets 2011-11-27 04:10:02 UTC
Created attachment 53882 [details]
strace taken with Chackra distro instead of Kubuntu 11.10

strace taken with Chackra distro (latest version and packages up to date) instead of Kubuntu 11.10.
opening the file is much quicker.
Comment 11 Chris Peñalver 2011-11-27 08:44:01 UTC
As per downstream bug:
https://bugs.launchpad.net/df-libreoffice/+bug/734424

this occurs in both Ubuntu 32 and 64-bit, with earliest version found 3.3.1.
Platform -> All
Version -> 3.3.1
Comment 12 NoOp 2011-11-28 08:45:06 UTC
@ Christopher: Please see my Comment #7: The test file opens in less than 10 seconds in:
And go-oo Ubuntu distribution 3.2.1 (linux)
OpenOffice.org 3.2.1 
OOO320m19 (Build:9505)
ooo-build 3.2.1.4, Ubuntu package 1:3.2.1-7ubuntu1.1

and

Opens fine in LO 3.3.4 final (linux):
LibreOffice 3.3.4 
OOO330m19 (Build:401)
tag libreoffice-3.3.4.
(Standard LO on Ubuntu 10.10 - 32 and 64bit)

It is only in LO 3.4.4 that I encounter the problem. This is regardless of the /etc/hosts workaround suggested in https://bugs.launchpad.net/df-libreoffice/+bug/734424.

@ mbohets: As previously requested, can you (or anyone else) please try with 3.3.4 linux?
Comment 13 mbohets 2011-12-03 04:06:48 UTC
I already did that, see comment 3.
Comment 14 NoOp 2011-12-03 10:35:45 UTC
I saw comment #3. Did you resolve the glibc dependency issues & then try 3.3.4?
Comment 15 mbohets 2011-12-03 12:46:32 UTC
I have no idea how to solve the dependency problem.
I searched for GLIB in synaptic but it is not there.
When I google around a bit, I see it has something to do with libstcc++.
In Kubuntu, libstdc++6 and libstdc++6-dev are installed.

I guess my linux skills are not enough to solve this.

I googled a bit further and found a lot of people with this GLIBCXX_3.4.11' not found problem,but no solution.
some people suggested to delete the link to libstdc++.so.6 but when I did that,
LO threw another error

libreoffice -calc
/opt/libreoffice/program/soffice.bin: symbol lookup error: /usr/lib/i386-linux-gnu/gtk-2.0/2.10.0/engines/liboxygen-gtk.so: undefined symbol: cairo_surface_create_for_rectangle
marc@VASTEPC:/opt/libreoffice/ure/lib$ sudo mv libstdc++.so.6.orig libstdc++.so.6

So I put it back
Comment 16 NoOp 2011-12-03 12:50:29 UTC
Have a look through this bug report:
<https://bugs.freedesktop.org/show_bug.cgi?id=32852>
[[Linux] video / `GLIBCXX_3.4.11' not found]
This is how I ended up resolving the issue:
<https://bugs.freedesktop.org/show_bug.cgi?id=32852#c19>
Comment 17 mbohets 2011-12-03 13:46:25 UTC
No luck, I removed libavcodec-extra-52, and libavcodec was pulled in.
When starting LO, I got the same error, also after a reboot.

After that, I also tried
  export LD_LIBRARY_PATH=/usr/lib:$LD_LIBRARY_PATH

in /opt/libreoffice3.4/program/soffice 
so that the /usr/lib/libstdc++.so.6 library 
is loaded instead of the one which comes with LO

But no luck either
Comment 18 mbohets 2011-12-17 06:18:15 UTC
Hi, I was not able to solve the dependancies.

The LO version I got when installing Aptosid distro on a laptop from work
3.3.3 build 301

My file opens there in less than 3 seconds (full contents, without the deletions
I did in the file I uploaded here.

If someone can point me to a distro with LO 3.3.4 to try this out as NooP requested, I'll be glad to install it and see what happens.
Comment 19 Kohei Yoshida 2012-01-11 12:57:46 UTC
Taking this off the 3.4 most annoying.

The delay is caused by a slow connection to the file that is externally linked.  Removing that externally linked image makes the file open instantly, using the latest 3.5 build.
Comment 20 Cor Nouws 2012-01-13 14:34:10 UTC
(In reply to comment #19)

> The delay is caused by a slow connection to the file that is externally linked.

Is it clear what causes that delay? If it is just the connection, then this one should be closed as "notabug" IMO
Comment 21 mbohets 2012-01-13 16:12:11 UTC
If it is not a bug, why do previous versions of LO open the exact same file without any problem ?
Comment 22 mbohets 2012-01-29 07:53:52 UTC
I just installed Kubuntu 12.04 alpha1.
It contains LibreOffice 3.5.0beta2, and there the file opens fine again, without any delays, and without having to change any contents

Thanks a lot for fixing it, now I won't have to use an old Kubuntu version anymore to be able to use my homesheet again.

It is a sheet in which I keep all my home stats like
- bank records
- electricity, water use
- solar panel production
....

So I needed it fairly often.

I guess the bug report can be closed with status solved.
Comment 23 Rainer Bielefeld Retired 2012-08-26 09:25:57 UTC
<https://bugs.freedesktop.org/page.cgi?id=fields.html#status>
We should use FIXED only if a known fix has solved the problem; if the problem only disappeared without known reason, please use WORKSFORME.