Bug 72987 - [firebird db integration] big-endian LibreOffice unable to open little-endian embedded firebird db (found 2944.512, support 11.2)
Summary: [firebird db integration] big-endian LibreOffice unable to open little-endian...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Base (show other bugs)
Version:
(earliest affected)
4.3.0.0.alpha0+ Master
Hardware: PowerPC All
: medium normal
Assignee: Tamas Bunth
URL:
Whiteboard: target:5.3.0
Keywords:
Depends on:
Blocks: Database-Firebird
  Show dependency treegraph
 
Reported: 2013-12-23 14:06 UTC by Douglas Mencken
Modified: 2016-09-07 17:58 UTC (History)
7 users (show)

See Also:
Crash report or crash signature:


Attachments
doesn't work yet (12.50 KB, patch)
2014-01-08 12:28 UTC, Caolán McNamara
Details
no need for rebuild indexes (5.47 KB, patch)
2015-03-23 13:51 UTC, Popa Adrian Marius
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Douglas Mencken 2013-12-23 14:06:17 UTC
I got

##Failure Location unknown## : Error
Test name: FirebirdTest::testIntegerDatabase
An uncaught exception of type com.sun.star.sdbc.SQLException
- firebird_sdbc error:
*unsupported on-disk structure for file /private/var/folders/+o/+oY8OrlkHRyUosC
PilWtLE+++TI/-Tmp-/lu35xfyn.tmp/firebird.fdb; found 2944.512, support 11.2
caused by
'isc_attach_database'

Failures !!!
Run: 2   Failure total: 1   Failures: 0   Errors: 1

Error: a unit test failed, please do one of:

export DEBUGCPPUNIT=TRUE            # for exception catching
export CPPUNITTRACE="gdb --args" # for interactive debugging on linux
...

when I tried to build LibreOffice 9e118293188607ac883ed65754aaccc9ef2c6abb 4.3.0.0alpha+ master.

It looks like that current firebird is unable to correctly handle I/O on Big Endian architectures.

I'm forced to make --disable-firebird-sdbc builds until it is fixed.
Comment 1 Caolán McNamara 2014-01-02 20:18:28 UTC
hmm, firebird saves using the native platform endianness and alignment as far as I can see so its a platform-dependent file format. Which rather sucks for something to haul around in the embedded database case where the db can be opened on any arbitrary system. The exact failure in this case is big-endian ppc is unable to open the little-endian sample database.

One solution might be to not store the .fbd in the odb but instead always "backup" to the odb and "restore" from the odb which would use the platform-neutral .fbk archival format IIRC
Comment 2 Andrzej Hunt 2014-01-02 21:26:50 UTC
(In reply to comment #1)
> One solution might be to not store the .fbd in the odb but instead always
> "backup" to the odb and "restore" from the odb which would use the
> platform-neutral .fbk archival format IIRC

Yes, that's the plan (discussed on IRC but otherwise undocumented).

The backup API is unfortunately somewhat unintuitive, but there shouldn't
be any huge issues replacing the storage format -- however I have no
idea what sort of impact this will have on saving/opening times.
Comment 3 Caolán McNamara 2014-01-08 12:28:17 UTC
Created attachment 91663 [details]
doesn't work yet

A prototype of backing up the db instead of saving it in native format.

Catch is it appears that one has to actually login to the services api with a username/password which means firebird wants to look up those in /usr/local/firebird/security2.fdb, sigh... So presumably we would have to bundle a security2.fdb and somehow tell firebird to use that to auth against unless there is some other way to workaround that problem of isc_service_attach. 

Hopefully get to debug firebird a bit more to see what options are available here.
Comment 4 Andrzej Hunt 2014-01-13 07:53:21 UTC
(In reply to comment #3)
> Created attachment 91663 [details] [review]
> doesn't work yet
> 
> A prototype of backing up the db instead of saving it in native format.
> 
> Catch is it appears that one has to actually login to the services api with
> a username/password which means firebird wants to look up those in
> /usr/local/firebird/security2.fdb, sigh... So presumably we would have to
> bundle a security2.fdb and somehow tell firebird to use that to auth against
> unless there is some other way to workaround that problem of
> isc_service_attach. 
> 
> Hopefully get to debug firebird a bit more to see what options are available
> here.

Ahh sorry, looks like we're duplicating work here, I've also got a
(non-working) alternative patch in gerrit ;)
( https://gerrit.libreoffice.org/#/c/7299/ )

However it doesn't actually work, as I can't convince the backup service to
work.

Based on the docs, as we're using embedded mode, we specifically *shouldn't* pass a username/password in the spb (i.e. the same as when connecting to the
db -- no username/pass for embedded mode), and I'm not getting any authentication errors with my version suggesting that's not where the
problem lies (but I'll try and experiment anyway just in case).


But I have no idea where the problem could be, as the only error output
given by the backup service is:

 gbak:Exiting before completion due to errors

As far as I can tell we're not missing any libraries or other firebird files that might be needed, but I suspect that it might be necessary to debug firebird itself to figure out what's happening.
Comment 5 Popa Adrian Marius 2014-04-28 08:23:38 UTC
I have asked on Firebird devel and no password / access to security2.fdb is required for embedded mode 
Please add a backtrace with the error if possible
Comment 6 Alex Thurgood 2015-01-03 17:39:20 UTC Comment hidden (no-value)
Comment 7 Popa Adrian Marius 2015-03-23 13:34:30 UTC
I will retest and add patches in gerrit 


1. First i will remove rebuildIndexes() it will not be necesarry when we keep fbk format in odb file 

2. I will retest isc_action_svc_backup , it is not necessary to have a password database with firebird embedded and i will add the secondary patch
Comment 8 Popa Adrian Marius 2015-03-23 13:51:43 UTC
Created attachment 114274 [details]
no need for rebuild indexes
Comment 9 Popa Adrian Marius 2015-03-23 15:53:35 UTC
added also to gerrit 

https://gerrit.libreoffice.org/14968
Comment 10 Popa Adrian Marius 2015-03-24 16:10:42 UTC

First atttempt for secondary patch 
https://gist.github.com/mariuz/60d4d1edfe83c0670648

, i will investigate the error bellow 


unknown:0:FirebirdTest::testIntegerDatabase
An uncaught exception of type com.sun.star.uno.RuntimeException
- unexpected UNO exception caught: firebird_sdbc error:
*invalid database handle (no active connection)
caused by
'isc_detach_database'


##Failure Location unknown## : Error
Test name: FirebirdTest::testIntegerDatabase
An uncaught exception of type com.sun.star.uno.RuntimeException
- unexpected UNO exception caught: firebird_sdbc error:
*invalid database handle (no active connection)
caused by
'isc_detach_database'


Failures !!!
Run: 2   Failure total: 1   Failures: 0   Errors: 1

Error: a unit test failed, please do one of:

export DEBUGCPPUNIT=TRUE            # for exception catching
export CPPUNITTRACE="gdb --args"    # for interactive debugging on Linux
export VALGRIND=memcheck            # for memory checking

and retry using: make CppunitTest_dbaccess_firebird_test
Comment 11 Björn Michaelsen 2015-07-06 16:46:23 UTC
Comment on attachment 91663 [details]
doesn't work yet

Mark patches obsolete, as work is ongoing on gerrit now.
Comment 12 Commit Notification 2016-01-05 11:44:02 UTC
Popa Adrian Marius committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=f961fef03906fc059a4a0c008799f68fc22727c1

WIP tdf#72987 Use firebird backup format for .odb no need for rebuild indexes

It will be available in 5.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 13 Commit Notification 2016-01-05 12:01:57 UTC
Lionel Elie Mamane committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=c88fd50c4e8815546e9830a41bf08edf9f165923

Revert "WIP tdf#72987 Use firebird backup format for .odb no need for rebuild indexes"

It will be available in 5.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 14 Lionel Elie Mamane 2016-09-07 17:58:36 UTC
fixed by the switch to Firebird "fbk" (backup) format