Bug 90520 - Base crashes with Oracle ODBC driver when sorting/filtering
Summary: Base crashes with Oracle ODBC driver when sorting/filtering
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Base (show other bugs)
Version:
(earliest affected)
4.4.2.2 release
Hardware: x86-64 (AMD64) Windows (All)
: medium major
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: haveBacktrace
Depends on:
Blocks: Database-Connectivity Crash
  Show dependency treegraph
 
Reported: 2015-04-08 14:59 UTC by grongo2
Modified: 2023-12-27 02:40 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
callstack of crash with 5.1.1.1 testing (4.33 KB, text/plain)
2016-03-03 08:46 UTC, grongo2
Details
bt Windows (windbg) (43.35 KB, text/plain)
2020-05-20 12:40 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description grongo2 2015-04-08 14:59:53 UTC
Base on windows crashes consistently when accessing table or queries using the Oracle ODBC driver and attempting to sort or filter. Works with oracle jdbc driver.
Steps to reproduce:
- Create/open a database connected with oracle odbc driver to an oracle database (mine is 11.2)
- select a table or query and open it
- when the table opens click on toolbar button with funnel or hand + z/a (autofilter, standard filter, sort). 
- crash. 
- Not crashing if you sort with arrows buttons (sort ascending/descending)

I'm working with Win 7 pro 64bit and LO 4.4.2.2 32bit release

callstack (no symbols, I'm sorry, symbol server didn't find any)
sal3!rtl_uString_new_WithLength+0xba
sal3!rtl_uStringbuffer_ensureCapacity+0x39
sal3!rtl_uStringbuffer_insert+0x2a
odbclo!connectivity::odbc::OTools::getStringValue+0x276
odbclo!connectivity::odbc::ODatabaseMetaDataResultSet::getString+0xde
dbtoolslo!connectivity::ORowSetValue::getString+0x59a
dbtoolslo!connectivity::ORowSetValue::impl_fill+0x66
dbtoolslo!connectivity::ORowSetValue::fill+0x68
dbtoolslo!connectivity::ODatabaseMetaDataBase::getTypeInfo+0x400
dbtoolslo!dbtools::getSearchColumnFlag+0x49
dbulo!makeDbaIndexFieldsControl+0x6a39
dbulo!cppu::WeakImplHelper1<com::sun::star::sdb::XInteractionSupplyParameters>::operator=+0x36da
dbulo!cppu::WeakImplHelper1<com::sun::star::sdb::XInteractionSupplyParameters>::operator=+0x2ccf
dbulo!Point::Y+0x13767
dbulo!dbaui::OGenericUnoController::executeChecked+0xcb
dbulo!dbaui::OGenericUnoController::dispatch+0x45
fwklo!com_sun_star_comp_framework_ControlMenuController_get_implementation+0x2ea0
tllo!Link::Call+0x11
vcllo!vcl::Window::ImplAsyncFocusHdl+0x3452
vcllo!vcl::Window::ImplAsyncFocusHdl+0x3cc4
vcllo!vcl::OpenTTFontBuffer+0x47ab9
vcllo!vcl::OpenTTFontBuffer+0x4a39b
vcllo!vcl::OpenTTFontBuffer+0x4a990
USER32!gapfnScSendMessage+0x332
USER32!GetThreadDesktop+0xd7
USER32!CharPrevW+0x138
USER32!DispatchMessageW+0xf
vcllo!vcl::OpenTTFontBuffer+0x171d4
vcllo!vcl::OpenTTFontBuffer+0x17a63
vcllo!Application::Execute+0x69
vcllo!DeInitVCL+0x634
sofficeapp!soffice_main+0x80
soffice+0x101e
kernel32!BaseThreadInitThunk+0x12
ntdll!RtlInitializeExceptionChain+0x63
ntdll!RtlInitializeExceptionChain+0x36
Comment 1 grongo2 2015-04-08 15:10:11 UTC
works also with MS odbc driver for oracle
Comment 2 raal 2015-04-12 13:32:01 UTC
Hello,
where can I download Oracle ODBC driver?
Comment 3 grongo2 2015-04-12 18:48:38 UTC
Is downloadable from oracle site (you need to register).  It's in the "oracle client" package.

I've done a bit more debugging, however, and the problem seems due to a combination of driver bug and incorrect LO handling of SQLGetData.

Oracle ODBC driver < 11.2.0.1.0 has a bug (Bug 6801797), returning wrong length in SQLGetData last parameter (read as pcbValue by LO).
This bug is amplificated by LO incorrect behavior.

OTools::getStringValue tries to read a column with more than 2048 bytes by iterating over SQLGetData, but uses only pcbValue to detect when stopping, instead of looking also at the function return value.
Further, it iterates even with fixed-length column, while ODBC allows iterating only over variable lenght columns (eg VARCHAR).
Calling multiple times SQLGetData on a fixed lenght columns results in the driver doing nothing but returning SQL_NO_DATA for calls after the first.

When opening the filter/sort dialog for the first time, LO apparently asks the driver about supported data types, thus getting CHAR data and triggering the bug.
AFAIK the sequence is

1. first call to SQLGetData. Buggy oracle driver returns 2048 in pcbValue
2. LO thinks the column has more than 2048 bytes and tries to read another chunk
- this time the call to SQLGetData returns SQL_NO_DATA and does nothing, but pcbValue still contains 2048 from previous result
3. LO ignores the return value and looking only at pcbValue copies another 2048 bytes (of garbage) in the result buffer, then tries to read another chunk

repeat steps 2 and 3 until crash

So a workaround would be resetting pcbValue to zero just before calling SQLGetData.
Something like
        while ((pcbValue == SQL_NO_TOTAL ) || (pcbValue >= nMaxLen) )
        {
            pcbValue = 0;  <--- insert 0 here
            OTools::ThrowException(_pConnection,
                           (*(T3SQLGetData)_pConnection->getOdbcFunction(ODBC3SQLGetData))(
                                       _aStatementHandle,
                                       (SQLUSMALLINT)columnIndex,
                                       SQL_C_CHAR,
                                       &aCharArray,
                                       nMaxLen,
                                       &pcbValue),
         

A better fix should be check SQLGetData return value and act accordingly, but perhaps LO does things this way as a workaround to other driver bugs ?
Comment 4 Alex Thurgood 2015-04-30 14:44:37 UTC
Is this a duplicate of bug 87370 ?
Comment 5 grongo2 2015-05-01 14:37:13 UTC
(In reply to Alex Thurgood from comment #4)
> Is this a duplicate of bug 87370 ?

No, I think not.
Bug 87370 involves the jdbc driver instead of odbc and the callstack seems different, involving 
  connectivity::ORowSetValue::operator=
instead of 
  connectivity::odbc::OTools::getStringValue
Comment 6 Buovjaga 2016-03-02 20:10:03 UTC
Could you try with these instructions: https://wiki.documentfoundation.org/How_to_get_a_backtrace_with_WinDbg

Maybe try with the latest version 5.1.1 will be released this week. You can install the release candidate right now separately with this tool: https://wiki.documentfoundation.org/SI-GUI

Set to NEEDINFO.
Change back to UNCONFIRMED after further testing.
Comment 7 grongo2 2016-03-03 08:46:22 UTC
Created attachment 123181 [details]
callstack of crash with 5.1.1.1 testing
Comment 8 grongo2 2016-03-03 08:57:15 UTC
The bug is still present in 5.1.1.1, I've attached a callstack as requested.
Error callstack and probable cause is unchanged from my first bug report.
As you can see, in comment #3 is outlined a possibile fix, but I don't know enough of the LO codebase to know if the code knowingly ignores SQLGetData return values as a workaround for another buggy driver or not.
Perhaps the best solution would be checking SQLGetData return values, but at least pcbValue should be cleared between calls.
Comment 9 Julien Nabet 2016-06-19 21:05:43 UTC
Lionel: thought you might be interested in the Grongo2's analysis about this crash (above all at comment 3)
Comment 10 tommy27 2017-05-11 12:51:22 UTC
has anyone tried latest LibO 5.2.7 or 5.3.2?
is issue still present?
Comment 11 Alex Thurgood 2017-05-11 13:29:39 UTC
A git log grep of the source code tree doesn't appear to show any changes in SQLGetData later than 2012, so it is unlikely that this behaviour has changed in the meantime.
Comment 12 Alex Thurgood 2017-05-11 13:38:50 UTC
Hmm, from

commit 2f54f2a4ac508de3984d2865da984b9ecf30f602 

it would appear that at least some of the problem was addressed by Lionel already back in 2012
Comment 13 Alex Thurgood 2017-05-11 13:39:56 UTC
Give the already quite detailed analysis, setting to NEW
Comment 14 Alex Thurgood 2017-05-11 13:43:07 UTC
Also seems that

https://cgit.freedesktop.org/libreoffice/core/commit/?id=ee5e1bca4b1d0f972eb3f9ab0ec79cf6423fd642

might be relevant
Comment 15 QA Administrators 2018-05-12 02:28:39 UTC Comment hidden (obsolete)
Comment 16 QA Administrators 2020-05-12 03:51:34 UTC Comment hidden (obsolete)
Comment 17 Julien Nabet 2020-05-16 18:37:57 UTC
grongo2: following Alex's comment about 
https://cgit.freedesktop.org/libreoffice/core/commit/?id=ee5e1bca4b1d0f972eb3f9ab0ec79cf6423fd642
author	Lionel Elie Mamane <lionel@mamane.lu>	2012-12-05 06:07:11 +0100
committer	Lionel Elie Mamane <lionel@mamane.lu>	2012-12-06 19:20:16 +0100
commit ee5e1bca4b1d0f972eb3f9ab0ec79cf6423fd642 (patch)
tree 76f91df8a8bdf29ffb249d44c0a85df6f742a985
parent a62e53830c95eb3db5976232d3375add94749896 (diff)
ODBC: clean up our use of SQLGetData

do you still reproduce this with LO 6.4.3?
Comment 18 Julien Nabet 2020-05-20 12:40:13 UTC
Created attachment 161037 [details]
bt Windows (windbg)

On Win10 with master sources updated today, I got an assertion.

Indeed, I did:
diff --git a/dbaccess/source/ui/browser/brwctrlr.cxx b/dbaccess/source/ui/browser/brwctrlr.cxx
index 4fb84775099a..9873e334aefe 100644
--- a/dbaccess/source/ui/browser/brwctrlr.cxx
+++ b/dbaccess/source/ui/browser/brwctrlr.cxx
@@ -1590,7 +1590,9 @@ FeatureState SbaXDataBrowserController::GetState(sal_uInt16 nId) const
     }
     catch(const Exception& )
     {
-        DBG_UNHANDLED_EXCEPTION("dbaccess");
+
+        //DBG_UNHANDLED_EXCEPTION("dbaccess");
+        assert(false);
     }

to retrieve a bt more easily
Comment 19 Aimee Britt 2020-11-17 06:35:11 UTC Comment hidden (spam)
Comment 20 Nishu Shah 2021-12-25 09:26:52 UTC Comment hidden (spam)
Comment 21 QA Administrators 2023-12-26 03:09:18 UTC Comment hidden (obsolete)
Comment 22 grongo2 2023-12-26 09:22:12 UTC
To tell the truth, the handling of this bug is disheartening: eight year ago I reported it, carrying out a detailed analysis and suggesting a fix.
After that, no action, only a "please try with the latest release", even if the source code remains unchanged.

Anyway, the bug is still present.
Reproducing is simple: open an oracle database via odbc, do a query with a varchar2 field, click on autofilter/filter and see the memory usage grow until crash.
I've tried with oo x64 on windows 10, connecting with oracle 19c odbc driver to a 19c database with codepage WE8MSWIN1252 hosted on a linux x64 machine (I don´t think the bug depends on the database details, however)


Version: 7.6.3.2 (X86_64) / LibreOffice Community
Build ID: 29d686fea9f6705b262d369fede658f824154cc0
CPU threads: 16; OS: Windows 10.0 Build 19045; UI render: Skia/Vulkan; VCL: win
Locale: it-IT (it_IT); UI: en-US
Calc: threaded
Comment 23 Noel Grandin 2023-12-26 17:27:15 UTC
(In reply to grongo2 from comment #22)
> To tell the truth, the handling of this bug is disheartening: eight year ago

Welcome to how open-source works. 

This is a 10 million line codebase with only a handful of maintainers. Identifying the problem is great, but often only the first step in a much larger process.

Feel free to get involved and submit a code fix.
Comment 24 Lionel Elie Mamane 2023-12-27 02:40:05 UTC
I'd be happy to review/approve a well-tested patch along the lines of the discussion below. Put me as reviewer on gerrit, and feel free to ping me every week by email if I don't respond.

(In reply to grongo2 from comment #3)
> OTools::getStringValue tries to read a column with more than 2048 bytes by
> iterating over SQLGetData, but uses only pcbValue to detect when stopping,
> instead of looking also at the function return value.

Good point. Looking also at the function return value would make the code more robust and would be a welcome improvement.

> Further, it iterates even with fixed-length column, while ODBC allows
> iterating only over variable lenght columns (eg VARCHAR).

Not 100% sure what you mean by "fixed-length column", but I note that
https://learn.microsoft.com/en-us/sql/odbc/reference/syntax/sqlgetdata-function#retrieving-variable-length-data-in-parts
explicitly allows it for SQL_CHAR, SQL_WCHAR and SQL_BINARY.

In the context of this bug, you say the data being read is SQL_CHAR so the iteration seems legit.

However, there does seem to be a codepath where indeed LibreOffice will try to do that iteration on SQL_DECIMAL and SQL_NUMERIC: in OResultSet.cxx, method OResultSet::fillColumn

Plus anytime any code (even user-written LibreOffice Basic, Python or other API binding) calls getString on a non-string column but only on a DatabaseMetadataResultSet, not on a regular ResultSet.

> So a workaround would be resetting pcbValue to zero just before calling
> SQLGetData.

While I'm generally wary of driver-specific work-arounds, something like that could indeed be seen as a general robustification of the code, so I could be tempted to approve it. One could discuss whether one should set it to some other value than zero, such as e.g. a constant not allowed to be written there by the driver (such as a constant chosen to be different than all valid "Length and Indicator Values", or one valid only for SQLPutData and not SQLGetData).

> A better fix should be check SQLGetData return value and act accordingly,
> but perhaps LO does things this way as a workaround to other driver bugs ?

In the absence of a comment in the code that specifically says so, any such putative driver bug is lost to time immemorial, and could have been fixed since then. I wouldn't let that keep us from moving forward with that better fix.

(In reply to grongo2 from comment #22)
> To tell the truth, the handling of this bug is disheartening: eight year ago
> I reported it, carrying out a detailed analysis and suggesting a fix.
> After that, no action, only a "please try with the latest release", even if
> the source code remains unchanged.

> Anyway, the bug is still present.

You mean in the same eight years, Oracle has (also) not fixed their bug?

> Reproducing is simple:

Well, no. It requires an Oracle database, which is not quite an "apt-get install" away, and I expect requires agreeing to a bizarre funky anti-social agreement forbidding use for any purpose on any machine one wants and/or sharing it with one's neighbour and/or improving it and/or sharing one's improvements? The first download page I could find https://www.oracle.com/database/technologies/xe-downloads.html also doesn't have any download for any source code (so I cannot compile/install it), nor for binaries for my main work machine (GNU/Linux on ppc64le).