Bug 162115 - Charset/collation data is missing in embedded Firebird
Summary: Charset/collation data is missing in embedded Firebird
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Base (show other bugs)
Version:
(earliest affected)
unspecified
Hardware: All Windows (All)
: medium normal
Assignee: Mike Kaganski
URL:
Whiteboard: target:25.2.0 target:24.8.0.2
Keywords:
Depends on:
Blocks: Database-Firebird-Default
  Show dependency treegraph
 
Reported: 2024-07-20 11:22 UTC by Mike Kaganski
Modified: 2024-07-31 10:33 UTC (History)
2 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mike Kaganski 2024-07-20 11:22:53 UTC
In a Firebird embedded database, execute this SQL (Tools->SQL):

CREATE TABLE mytable (c char(2) character set WIN1252)

An error appears in the Status box:

> 1: firebird_sdbc error:
> *unsuccessful metadata update
> *EXT_LOG
> *CHARACTER SET WIN1252 is not installed
> caused by
> 'CREATE TABLE mytable (c char(2) character set WIN1252)'
>  at .../connectivity/source/drivers/firebird/Util.cxx:69

This is because we simply don't add intl data required by Firebird. Using a filesystem monitoring tools like Sysinternals' ProcMon, initialization of Firebird in LibreOffice attempts to access <instdir>/program/intl. Copying this 'intl' directory from workdir/UnpackedTarball/firebird/gen/<Debug|Release>/firebird directory to the wanted place in instdir makes the command succeed.

We need to bundle:

program/intl/fbintl.dll
program/intl/fbintl.conf
Comment 1 Mike Kaganski 2024-07-20 12:07:54 UTC
https://gerrit.libreoffice.org/c/core/+/170791
Comment 2 Commit Notification 2024-07-20 14:43:18 UTC
Mike Kaganski committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/07cc2fdc3d447479ca3afd2f91c9bc80be745dff

tdf#162115 Add intl data for Firebird

It will be available in 25.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 3 Commit Notification 2024-07-23 07:00:52 UTC
Mike Kaganski committed a patch related to this issue.
It has been pushed to "libreoffice-24-8":

https://git.libreoffice.org/core/commit/3a9bcbd04e65d3312808f1014af854b406bc6d3e

tdf#162115 Add intl data for Firebird

It will be available in 24.8.0.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 4 Xisco Faulí 2024-07-30 16:14:10 UTC
Hi Mike,
With

Version: 25.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: dc9486f2443fa52588b625c0a2a288bff56a7a45
CPU threads: 8; OS: Linux 6.1; UI render: default; VCL: gtk3
Locale: es-ES (es_ES.UTF-8); UI: en-US
Calc: threaded

I'm still getting

1: firebird_sdbc error:
*unsuccessful metadata update
*MYTABLE
*CHARACTER SET WIN1252 is not installed
caused by
'CREATE TABLE mytable (c char(2) character set WIN1252)'
 at /home/tdf/lode/bibisect/core252/connectivity/source/drivers/firebird/Util.cxx:69

Is there anything else I have to do before running

CREATE TABLE mytable (c char(2) character set WIN1252)
Comment 5 Mike Kaganski 2024-07-30 18:20:33 UTC
(In reply to Xisco Faulí from comment #4)

No, nothing else shouldn't be necessary. I have just checked that this command now works in my Windows build's instdir; but maybe it were remnants of my direct copy experiments - I will make clean and make again.

On the other hand, that's instdir; I don't know what is in lode/bibisect. Can you test id your Linux-based build's instdir also has that problem? It could be either my failure to make it correctly for Linux, or something with bibisect.
Comment 6 Xisco Faulí 2024-07-30 21:23:57 UTC
(In reply to Mike Kaganski from comment #5)
> (In reply to Xisco Faulí from comment #4)
> 
> No, nothing else shouldn't be necessary. I have just checked that this
> command now works in my Windows build's instdir; but maybe it were remnants
> of my direct copy experiments - I will make clean and make again.
> 
> On the other hand, that's instdir; I don't know what is in lode/bibisect.
> Can you test id your Linux-based build's instdir also has that problem? It
> could be either my failure to make it correctly for Linux, or something with
> bibisect.

it also fails with my local build

Version: 25.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 142ce290a2a29f9794ba9a689f35d6f7e3ff588c
CPU threads: 8; OS: Linux 6.1; UI render: default; VCL: gtk3
Locale: es-ES (es_ES.UTF-8); UI: en-US
Calc: threaded

I see this log:

warn:connectivity.parse:506884:506884:connectivity/source/parse/sqliterator.cxx:2021: Adding error com.sun.star.sdbc.SQLException message: "The database does not contain a table named "mytable". at /home/xisco/libreoffice/connectivity/source/parse/sqliterator.cxx:2015" SQLState: HY000 ErrorCode: 1000
    wrapped: 
warn:connectivity.firebird:506884:506884:connectivity/source/drivers/firebird/Statement.cxx:114: isc_dsql_execute failed
warn:connectivity.firebird:506884:506884:connectivity/source/drivers/firebird/Util.cxx:58: firebird_sdbc error:
*unsuccessful metadata update
*MYTABLE
*CHARACTER SET WIN1252 is not installed
caused by
'CREATE TABLE mytable (c char(2) character set WIN1252)'
Comment 7 Mike Kaganski 2024-07-31 04:32:34 UTC
I can confirm, that the fix works on Windows after a clean build.

I also confirm that it doesn't work on Linux. The files are copied to program/intl, but the SQL fails.

Needs debugging to understand what's the issue. Also, it will likely have a similar problem on macOS / other.
Comment 8 Mike Kaganski 2024-07-31 04:54:32 UTC
Aha; it seems that it looks for the 'intl' under /usr/local/firebird on Linux (Ubuntu that I'm testing with). We should make sure that when we use our built-in FB, we look under our own directories.
Comment 9 Mike Kaganski 2024-07-31 05:57:54 UTC
Relevant code for debugging:

gds__prefix function, InitPrefix class, and fb_prefix static variable in workdir/UnpackedTarball/firebird/src/yvalve/gds.cpp

Possibly workdir/UnpackedTarball/firebird/src/yvalve/config/os/posix/config_root.cpp may be important; maybe the build should use --enable-binreloc - these all are variables that currently surround the intl path lookup. However, I don't know enough of Linux library path lookup specifics to attempt to fix this. So let me call this Windows-specific, and allow Linux / macOS cases be resolved separately (now that the files are copied, and the task remaining is to make FB find them).
Comment 10 Xisco Faulí 2024-07-31 10:33:22 UTC
(In reply to Mike Kaganski from comment #9)
> So let me call this Windows-specific, and allow Linux / macOS cases be
> resolved separately (now that the files are copied, and the task remaining
> is to make FB find them).

Reported in bug 162281