Bug 83999 - MySQL direct driver: Hang / poor performance when reading table containing binary encoded data string
Summary: MySQL direct driver: Hang / poor performance when reading table containing bi...
Status: RESOLVED WORKSFORME
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Base (show other bugs)
Version:
(earliest affected)
3.6.7.2 release
Hardware: x86-64 (AMD64) All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: perf
: 119386 (view as bug list)
Depends on:
Blocks: Database-Tables
  Show dependency treegraph
 
Reported: 2014-09-17 14:24 UTC by Alex Thurgood
Modified: 2023-06-19 08:40 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
test data (93.65 KB, application/text)
2014-09-17 14:30 UTC, Alex Thurgood
Details
test odb emb firebird (883.75 KB, application/vnd.oasis.opendocument.database)
2018-08-22 02:36 UTC, Drew Jensen
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alex Thurgood 2014-09-17 14:24:32 UTC
Reproducible poor performance / hang os OSX when attempting to open, move cursor within, or edit a table containing binary encoded data.

CREATE TABLE `obintm` (
  `id` int(11) DEFAULT NULL,
  `goods` mediumtext
) ENGINE=MyISAM DEFAULT CHARSET=latin1

test data will be attached
Comment 1 Alex Thurgood 2014-09-17 14:30:32 UTC
Created attachment 106435 [details]
test data
Comment 2 Alex Thurgood 2014-09-17 14:35:46 UTC
The cli, or mysql workbench handles this data without any noticeable perfomance hit.
Comment 3 Alex Thurgood 2014-09-19 06:59:59 UTC
The problem is also apparent on Linux Mint 17 64bit, so confirming, and with LibreOffice 

Version: 4.2.6.3
Build ID: 420m0(Build:3)

With this version, there is clearly a display refresh issue as well as the performance of moving the cursor from one position to another position. 

Alex
Comment 4 Alex Thurgood 2014-09-19 07:07:49 UTC
If I try and click at more than one cursor position over a 20s interval, I can get the system to bring up the "This application has stopped responding message".
Comment 5 Alex Thurgood 2014-09-19 08:19:08 UTC
Also tested on OSX 10.9.4 with 32bit LibreOffice 3.6.7.2 with mysql connector 1.0.1

The first load of the table is slow, after that, cursor movements, e.g start to end are noticeably quicker, although the spinning beachball still takes pride of place on screen while waiting for the new display to be updated
Comment 6 Alex Thurgood 2014-09-19 08:25:39 UTC
In fact 3.6.7.2 is disticntly quicker, the previous lags with this version came as a result of an erroneous tuple copy operation.

The performance hit becomes noticeable in 3.6.7.2 when a copy/paste of a tuple is attempted.
Comment 7 QA Administrators 2015-10-14 19:58:21 UTC Comment hidden (obsolete)
Comment 8 ribotb 2015-10-15 09:11:23 UTC
Hi,

Configuration :
- Windows 7 x86
- LibreOffice 5.0.2.2
- MariaDB  5.5.46
- MySQL connector ODBC 5.1.7

I created table obintm with SQL and test data of Alex.
 I can edit table but LO Base hangs when opening table.

Bernard
Comment 9 QA Administrators 2016-11-08 11:42:02 UTC Comment hidden (obsolete)
Comment 10 ribotb 2016-11-08 13:03:55 UTC
Hi,

Configuration :
- Windows 7 x86
- LibreOffice 5.2.3.2
- MariaDB  5.5.53
- MySQL connector ODBC 5.2

I created table obintm with SQL and test data of Alex.
 I can edit table but LO Base hangs when opening table.

Bernard
Comment 11 ribotb 2016-11-08 13:24:42 UTC
I forgot: no problem to open the table with MySQL Administrator or HeidiSQL.
Comment 12 Alex Thurgood 2017-09-15 12:59:40 UTC
Still present in :

Version: 5.4.1.2
Build ID: ea7cb86e6eeb2bf3a5af73a8f7777ac570321527
CPU threads: 4; OS: Mac OS X 10.12.6; UI render: default; 
Locale: fr-FR (fr_FR.UTF-8); Calc: group
Comment 13 Alex Thurgood 2017-09-15 13:04:01 UTC
I have a work dataset that is 20 col (fields) by 39 tuples.
2 of the columns have binary data stored in them, but not all are filled with binary data. The data lengths for columns for which binary data is present are on average 1500 bytes.

This table refuse to open for me with 

Version: 5.4.1.2
Build ID: ea7cb86e6eeb2bf3a5af73a8f7777ac570321527
CPU threads: 4; OS: Mac OS X 10.12.6; UI render: default; 
Locale: fr-FR (fr_FR.UTF-8); Calc: group

irrespective of the connector being JBDC or native mysql


even after waiting for more than 5 minutes.

If I open the same table in MySQLWorkbench, the table data is displayed almost immediately (fraction of a second).
Comment 14 Alex Thurgood 2017-09-15 13:08:45 UTC
 The problem is also visible with MySQL JDBC connector and JB Faure has indicated, also via ODBC.

Changing title to remove the specific mention of the native connector and Component to Database.
Comment 15 Alex Thurgood 2017-09-15 13:09:35 UTC
(In reply to Alex Thurgood from comment #14)
>  The problem is also visible with MySQL JDBC connector and JB Faure has
> indicated, also via ODBC.
> 
> Changing title to remove the specific mention of the native connector and
> Component to Database.

Sorry not JB Faure, Bernard Ribot.
Comment 16 Alex Thurgood 2017-09-15 13:40:50 UTC
For comparison, the using the test data provided here, and on a recent OSX10.12.6 MBPro(2015) core i7 with 16Mb RAM, the same problem exists in OpenOffice.org3.2.1 (OOo320m18) with regard to slow opening and navigation, but in comparison to LO541 the performance penalty is negligible. In other words, things have got worse over time with successive versions of LO.
Comment 17 Drew Jensen 2018-08-21 10:45:11 UTC
*** Bug 119386 has been marked as a duplicate of this bug. ***
Comment 18 Drew Jensen 2018-08-21 11:04:57 UTC
So - there are three places where this is really bad at the moment.

Table_View (open a Table object)
Query)View (run a Query Definition or execute a query in the designer)
SQL_Window with 'Show results' enabled.

The text control setting 'Multi-Line Input' is a factor. 

For columns in a tablecontrol using a TextControl this setting is default TRUE for CLOB fiels in the Table_View and Query_View. The setting is FALSE by default when a TableControl is added to a form. Changing the setting in the embedded form instance to TRUE significantly effects the performance there also.
Comment 19 Drew Jensen 2018-08-22 02:36:09 UTC
Created attachment 144360 [details]
test odb emb firebird

Using the attached file.

Another factor is GTK apparently. using Ubuntu 18.04 (Gnome) 

Opening the 'Databases' Table in the attached file.

If the application is started with SAL_USE_VCLPLUGIN=gen there is change for the better particularly in TABLE_VIEW window and QUERY_VIEW window

If I run an install from downloaded Daily Build binaries for with SAL_USE_VCLPLUGIN=gtk Libreoffice uses GTK2 and performance becomes much worse, and has an added problem of multiple screen paints. 

If I run the latest refreshed build from the Daily_build bibisect repo with SAL_USE_VCLPLUGIN=gtk LibreOffice uses GTK3 and things get worse again this time with severe results in the TABLE_VIEW window.


There are also two forms based on a query in the attached file that shows the effect 'multi-line input' has on the tablecontrol. They are all named Query2. 

If you open form Query2 the tablecontrol scrolls quickly.
Try Multi-Line/Query2 and the tablecontrol scrolls more erratically. 
The querydefinition Query2 is the worst performing of the three.
And again the difference bewteen VCLPLUGIN GEN/GTK2/GTK3 is significant.
Comment 20 QA Administrators 2021-04-12 03:32:07 UTC Comment hidden (obsolete)
Comment 21 Alex Thurgood 2021-04-15 06:46:06 UTC
Bug still present in 


Version: 7.1.2.2 / LibreOffice Community
Build ID: 8a45595d069ef5570103caea1b71cc9d82b2aae4
CPU threads: 8; OS: Mac OS X 10.16; UI render: default; VCL: osx
Locale: fr-FR (fr_FR.UTF-8); UI: fr-FR
Calc: threaded


On Apple M1 silicon, macOS BigSur
Comment 22 Julien Nabet 2021-04-15 17:22:53 UTC
On pc Debian x86-64 here are the tests I did:
- install MariaDb server 10.5.9-MariaDB-1
- create obintm table and load data from first attachment with phpmyadmin

Then with master sources updated today:
- create a brand new odb file with Mysql direct connection
- open the table
=> no pb

- Open emb Firebird file
- open Query2
- scroll down then up
=> no pb
- close Query2
- open Multi-Line/Query2
- scroll down then up
=> no pb



On LO Debian package 7.0.4
- open the same file
- open the table
=> it hangs, I must kill LO

- Open emb Firebird file
- open Query2
- scroll down then up
=> no pb
- close Query2
- open Multi-Line/Query2
- scroll down then up
=> it slows down a lot

Perhaps I missed something, has someone so time to give a try with a master build?

(Alex: how do you use LO with Mac and its new processor M1? Do you use some emulator or do you try to build LO sources with some Arm64 options? there's nothing about it on https://wiki.documentfoundation.org/Development/BuildingOnMac)
Comment 23 Robert Großkopf 2021-04-15 17:47:46 UTC
Tested this with MyariaDB and testdata from Alex.
LO 7.1.2.2 on OpenSUSE 15.2 64bit rpm Linux
Content of table isn't shown directly, scrolling through 'obintm' stops with direct connection. 
Content is shown directly, scrolling don't stop wit Mariadb JDBC driver.

The buggy behavior is the same with daily build of 2021-04-14.
Comment 24 Julien Nabet 2021-04-15 18:38:25 UTC
(In reply to Robert Großkopf from comment #23)
> ...
> The buggy behavior is the same with daily build of 2021-04-14.
Ok so I must have missed something.
Can't help here=>uncc myself.
Comment 25 QA Administrators 2023-06-19 03:13:15 UTC Comment hidden (obsolete)
Comment 26 Robert Großkopf 2023-06-19 06:42:02 UTC
Have tested again with LO 7.5.4.2. Couldn't reproduce the bug. 
Then I tested older versions. With LO 7.2.5.1 and older versions I could reproduce. Table shows, when opening, grey background while loading.
Since LO 7.3.6.2 (might be earlier versions of 7.3) I couldn't reproduce this behavior any more with OpenSUSE 15.4 64bit rpm Linux.

I will close this bug as WORKSFORME. Feel free to reopen, if you could reproduce the behavior with a current version.
Comment 27 Alex Thurgood 2023-06-19 08:40:58 UTC
Confirming WFM with

Version: 7.5.4.2 (AARCH64) / LibreOffice Community
Build ID: 36ccfdc35048b057fd9854c757a8b67ec53977b6
CPU threads: 8; OS: Mac OS X 13.4; UI render: Skia/Raster; VCL: osx
Locale: fr-FR (fr_FR.UTF-8); UI: fr-FR
Calc: threaded