Download it now!
Bug 49657 - FILESAVE: Base freezes when exporting a very large table to Calc
Summary: FILESAVE: Base freezes when exporting a very large table to Calc
Status: RESOLVED INSUFFICIENTDATA
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Base (show other bugs)
Version:
(earliest affected)
3.5.2 release
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-05-08 13:38 UTC by bullgard4
Modified: 2017-01-31 00:25 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description bullgard4 2012-05-08 13:38:03 UTC
LibreOffice Base and Calc freeze after ~14 mins when exporting data of a very large (~10 MB) table from an embedded HSQLDB to a LibreOffice Calc spreadsheet. For this, I am using the procedure outlined in LibreOffice Help.
I can move the cursor after the LibreOffice freeze though.
Top shows a processor load of 102% on average. Memory usage increases slowly from ~9% to 14%.
Exporting a small (~10 kB) table works all right though.
For further analyzing this bug please let me know if I need to install the DEB program package libreoffice-dbg and repeat the export.
Or what should I do to support debugging?
What should I look for in ~/.xsession-errors?

I did report this bug in Ubuntu's Launchpad as Bug #996732 too with some more details.
Comment 1 bullgard4 2012-05-08 14:16:06 UTC
I made a typo:
For (~10 MB) please read (~10O MB).
Comment 2 Michael Meeks 2012-05-09 03:02:21 UTC
Do you have a script-generated CSV file that can be used to reproduce this ?

Having said that HSQLDB is a rather noddy database written in Java that is most likely not suitable for your use-case ;-) I'd encourage using a remote postgresql database, or somesuch.

Possibly as/when/if we get the SQLite backend implemented, life will be better for you - but that is waiting for a volunteer to help out cf. bug#38811
Comment 3 bullgard4 2012-05-09 04:51:06 UTC
I made a typo:
For (~10 MB) please read (~10O MB).
Comment 4 bullgard4 2012-05-09 05:33:23 UTC
> Do you have a script-generated CSV file that can be used to reproduce this ?

No, I don't. In addition, I do not know how a script-generated CSV file could help to reproduce the reported bug. As I said my aim was to create a spreadsheet file. But this target failed. So I only have my embedded HSQLDB. 

My efforts to extract a CSV file from that same HSQLDB table have not been successful yet.

> Having said that HSQLDB is a rather noddy database written in Java that is most
> likely not suitable for your use-case ;-) 

My use case is one large table in a HSQLDB including one memo [LONGVARCHAR] field. Where do you know from that this is not a use-case for HSQLDB?

> I'd encourage using a remote postgresql database, or somesuch.

I'm going to use MySQL. But before I can do that I need to export my data. So please let me know how I can export my data from that one large embedded HSQLDB table. As I said before, the export to a CSV file was not successful either. 

> Possibly as/when/if we get the SQLite backend implemented, life will be better
> for you - but that is waiting for a volunteer to help out cf. bug#38811

I cannot wait that long. I am using my HSQLDB almost every single day.
Comment 5 Jochen 2012-08-27 19:00:27 UTC
Hi Robert,

please: have a look at this bugreport. What is your opinion?
Comment 6 Robert Großkopf 2012-08-28 16:02:11 UTC
I haven't such a big database. I have tried it with
https://bugs.freedesktop.org/attachment.cgi?id=63257
When I open the table in LO 3.3.4-Base it is shown at once. Scrolling to the end for over 30000 rows lasts 3 s.
When I register the database and then want to import the table into calc by drag and drop the prozessor is working at his maximum for a longer time - but with this table it works.
Could be that the freezing is a part of the problem which appears under https://bugs.freedesktop.org/show_bug.cgi?id=51239 .
When I try the same as described above with LO 3.6.1.2 rc LO freeezes - I have breaked the prozess after 10 minutes.
I have only tested with the internal HSQLDB. I don't think that it differs to other external DBs, because it is a problem while importing the data to Calc, not reading the data from the table, as you see in the first example with LO 3.3.4. Whith LO 3.6.1.2 rc the reading of the data is slow and the import into Calc is slow - so it won't work any more.
I have changed the status to New.
Comment 7 tommy27 2013-09-08 06:59:43 UTC
@robert
could you try replicating your previous test with current 4.1.1 release and tell us if the performance has now got any better? 1 year passed by and many LibO fixes and improvements have been added meanwhile.
Comment 8 Robert Großkopf 2013-09-08 08:27:58 UTC
(In reply to comment #7)
> @robert
> could you try replicating your previous test with current 4.1.1 release and
> tell us if the performance has now got any better? 1 year passed by and many
> LibO fixes and improvements have been added meanwhile.

I have tried with LO 4.1.2.1. For the example-database of comment6 the time to paste it to calc is about 60 seconds. With LO 3.3.4 on the same machine it's 50 seconds. No big difference.
Comment 9 Alex Thurgood 2015-01-03 17:38:44 UTC Comment hidden (no-value)
Comment 10 QA Administrators 2016-01-17 20:03:21 UTC Comment hidden (obsolete)
Comment 11 Robert Großkopf 2016-01-27 18:32:41 UTC
Bug couldn't be confirmed by me with LO 5.1.0.2, OpenSUSE 42.1 Leap, 64bit rpm Linux. It's the same as written in comment8.

I can't create such a big database as written in the description (100MB!) (except using tables with images ...) without getting 0-pointer-exceptions of the internal HSQLDB. So I can't test the original bug.

I will set this bug to unconfirmed.
Comment 12 Alex Thurgood 2016-06-10 07:44:02 UTC
@bullgard4 : if you want us to be able to triage this bug report appropriately, we need a sample hsqldb database that is as big as the one you describe. As such a file would be too big for bugzilla to handle, you would need to provide a link to where we could download it.

Setting to needinfo. If nothing is posted, the report will eventually receive the status RESOLVED INSUFFICIENTDATA
Comment 13 QA Administrators 2016-12-07 12:53:19 UTC Comment hidden (obsolete)
Comment 14 QA Administrators 2017-01-31 00:25:05 UTC
Dear Bug Submitter,

Please read this message in its entirety before proceeding.

Your bug report is being closed as INSUFFICIENTDATA due to inactivity and
a lack of information which is needed in order to accurately
reproduce and confirm the problem. We encourage you to retest
your bug against the latest release. If the issue is still
present in the latest stable release, we need the following
information (please ignore any that you've already provided):

a) Provide details of your system including your operating
   system and the latest version of LibreOffice that you have
   confirmed the bug to be present

b) Provide easy to reproduce steps – the simpler the better

c) Provide any test case(s) which will help us confirm the problem

d) Provide screenshots of the problem if you think it might help

e) Read all comments and provide any requested information

Once all of this is done, please set the bug back to UNCONFIRMED
and we will attempt to reproduce the issue. Please do not:

a) respond via email 

b) update the version field in the bug or any of the other details
   on the top section of our bug tracker

Warm Regards,
QA Team

MassPing-NeedInfo-20170131