Download it now!
Bug 61142 - ODBC: (LONG)(VAR)CHAR fields over 1023 characters, but below 2048 characters, cause garbage text
Summary: ODBC: (LONG)(VAR)CHAR fields over 1023 characters, but below 2048 characters,...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Base (show other bugs)
Version:
(earliest affected)
4.0.0.3 release
Hardware: x86-64 (AMD64) Windows (All)
: medium normal
Assignee: Lionel Elie Mamane
URL:
Whiteboard: target:4.1.0 target:4.0.3
Keywords: regression
Depends on:
Blocks:
 
Reported: 2013-02-19 23:36 UTC by Kurt
Modified: 2013-04-15 07:14 UTC (History)
4 users (show)

See Also:
Crash report or crash signature:


Attachments
Sample ODBC database (SQLite) and BASE document (2.23 KB, application/zip)
2013-02-19 23:36 UTC, Kurt
Details
typescript: under valgrind on linux, ending with segfault (45.46 KB, text/plain)
2013-02-20 16:21 UTC, Terrence Enger
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kurt 2013-02-19 23:36:20 UTC
Created attachment 75141 [details]
Sample ODBC database (SQLite) and BASE document

When connecting to an ODBC database, a memo field with greater than 1023 bytes of text causes extremely slow performance and displays garbage at the end of the field.  It is strongly suspected there is an overflow of a 1k buffer.

Attached is a sample SQLite database illustrating the problem, and a "Base" database that connects to it.

To reproduce the problem with my sample file...
REQUIRED:
- SQLite ODBC driver.  You need the 32-bit driver even on a x64 system in Windows since there is no 64 bit LibreOffice build for Windows.  You can download this at http://www.ch-werner.de/sqliteodbc/

Steps:
1) Set up the ODBC DSN for the sample database.  Use the normal ODBC administrator through your control panel for 32-bit Windows.  For 64-bit Windows, you must create the ODBC datasource using "c:\windows\sysWOW64\odbcad32.exe" to access the 32-bit ODBC system.  In either case, create a datasource named "ODBCTEST1" and point it to the "ODBCTest.db" sample in the zip file attached to this report.
2) Open the sample "Base" database ODBCTest.odb.  Select tables and open the "Test1" table.
3) The Memorandum field in row 1 has 1023 bytes.  In row 2 it has 1024 bytes.  If you click on that field in row 1 and hit the "END" button on your keyboard, you will notice it displays properly.  If you click on that column in row 2 and hit "END", you will notice a bunch of junk oriental characters.  Scrolling through that field is extremely slow.  Any handling of rows with such fields is very slow.
4) If you copy (CRTL-C) an affected field, then paste it into another program, the text pastes correctly.  If you try and display it in any way within "Base" (table, queries, forms), all the junk characters are added and performance degrades.

This same ODBC database works in 32 and 64 bit versions of Windows with the same ODBC driver using 3.5 versions of LibreOffice.  It is unknown what interim version introduced this bug.
Comment 1 Terrence Enger 2013-02-20 16:21:11 UTC
Created attachment 75181 [details]
typescript: under valgrind on linux, ending with segfault

For what little it is worth, I see no problem with master commit
2e367c0, pulled 2013-02-17, built and running on ubuntu-natty (11.04)
32-bit, with sqlite3 version 3.7.4-2ubuntu5.

Execution under valgrind, however, caused a segfault.
Comment 2 Kurt 2013-02-21 22:43:03 UTC
(In reply to comment #1)

> For what little it is worth, I see no problem with master commit
> 2e367c0, pulled 2013-02-17, built and running on ubuntu-natty (11.04)
> 32-bit, with sqlite3 version 3.7.4-2ubuntu5.

I'm not certain the problem manifests in Linux.  It's possible, if not likely, that a buffer overflow would have different symptoms in Linux anyway.

I've tried release 3.6.5.2 and the problem does not manifest.  So far the 4.0.0.3 release, and daily builds for today of 4.0.2 and master both do manifest the problem.  I do not know if any early 4.0 pre-releases exhibit the problem.
Comment 3 Terrence Enger 2013-03-04 13:29:54 UTC
Having tried to dig into my segfault, I now think that it is nothing
to do with this bug.  Sorry for the noise.
Comment 4 Joel Madero 2013-04-14 18:38:12 UTC
Lionel - this one seems vaguely familiar, do you have a few minutes to take a look at it and confirm it?
Comment 5 Lionel Elie Mamane 2013-04-14 20:34:24 UTC
The buffer is 2048 bytes, but that corresponds to 1024 UTF-16 code units (including terminating NULL -> 1023 "real" code units), so this is plausible.

With my testing under GNU/Linux, the SQLite ODBC driver reports SQL_CHAR type for the column (8-bit text encoding). Probably the Windows version reports SQL_WCHAR (16-bit text encoding)...
Comment 6 Lionel Elie Mamane 2013-04-14 23:23:20 UTC
Sorry, I was mistaken. The buffer is actually 2048 *characters*, so in the WCHAR case, it is 4096 bytes. So the "1024 characters" limit makes no sense to me, unless there is some mixup between SQL_CHAR and SQL_WCHAR...

Ah, found it! I mixed up length (number of UTF16 code units) and size (number of bytes) again...
Comment 7 Commit Notification 2013-04-14 23:32:24 UTC
Lionel Elie Mamane committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=a63c9bac6a6223ac716b7bcb590dac4eb01d5def

fdo#61142 mixup between size and length



The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.
Comment 8 Commit Notification 2013-04-15 07:12:26 UTC
Lionel Elie Mamane committed a patch related to this issue.
It has been pushed to "libreoffice-4-0":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=b475a63edea91daccfc179506b53a5f2ae7fc12b&h=libreoffice-4-0

fdo#61142 mixup between size and length


It will be available in LibreOffice 4.0.3.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds
Affected users are encouraged to test the fix and report feedback.