Bug 153431 - PostgreSQL native connector: Field Memo [Text] is filled with 0 instead of character content where postgres server version is greater than or equal to 13
Summary: PostgreSQL native connector: Field Memo [Text] is filled with 0 instead of ch...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Base (show other bugs)
Version:
(earliest affected)
7.3.7.2 release
Hardware: All All
: high major
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: dataLoss
Depends on:
Blocks: Database-Connectivity
  Show dependency treegraph
 
Reported: 2023-02-07 08:32 UTC by Robert Großkopf
Modified: 2023-03-30 13:25 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Großkopf 2023-02-07 08:32:20 UTC
Open a connection to PostgreSQL with native connector.
Create a table. Should contain one field with field type Memo[Text].
Save the table and open the table for input data.
Fill content to the field for Memo[text] (and other needed content for primary key).
Change to next row.
'0' will be saved instead of any content in this field.
You couldn't save anything in this field instead of '0'.

Now close the connection to the database. Connect by JDBC (or ODBC). 
Open the table - you could type text in the field for Memo[text] and it will be saved.

Close the connection to the database. Connect with native connector. Now you could type text into the field Memo[text].
Comment 1 Michael 2023-02-07 08:52:56 UTC
Same here, using 
Version: 7.3.6.2 / LibreOffice Community
Build ID: c28ca90fd6e1a19e189fc16c05f8f8924961e12e
CPU threads: 16; OS: Linux 5.15; UI render: default; VCL: kf5 (cairo+xcb)
Locale: de-DE (de_DE.UTF-8); UI: de-DE
Calc: threaded
on
Operating System: Manjaro Linux
KDE Plasma Version: 5.26.5
KDE Frameworks Version: 5.102.0
Qt Version: 5.15.8
Kernel Version: 5.15.91-1-MANJARO (64-bit)

I have an existing table, not created with BASE but directly in PostgresSQL, having fields type TEXT. Browsing through a form, based on a quite complex query from that table, with native Connector produces data-losses, as the original content is replaced by '0'. With JDBC-Connector everything is fine.
Comment 2 Alex Thurgood 2023-02-14 12:20:48 UTC
I don't seem to be able to reproduce this on

Version: 7.4.4.2 / LibreOffice Community
Build ID: 85569322deea74ec9134968a29af2df5663baa21
CPU threads: 8; OS: Mac OS X 13.0.1; UI render: Skia/Raster; VCL: osx
Locale: fr-FR (fr_FR.UTF-8); UI: fr-FR
Calc: threaded
Comment 3 Alex Thurgood 2023-02-14 14:52:02 UTC
@Robert :

can you explain what would be a typical sequence of characters for your varchar field so that I can try and reproduce ?

Is the varchar field also the primary key field, or do you have a separate INT field for that ?

I set up my table as follows :

                       Table "public.SportClub"
   Column   |          Type          | Collation | Nullable | Default 
------------+------------------------+-----------+----------+---------
 ID         | integer                |           | not null | 
 ClubName   | character varying(100) |           |          | 
 Country_ID | integer                |           |          | 
Indexes:
    "SportClub_pkey" PRIMARY KEY, btree ("ID")

The primary key directive was added to the table definition via the LO UI (Edit Table) after I had initially defined the table without a PK.
Comment 4 Alex Thurgood 2023-02-14 14:53:33 UTC
Ah, I've just noticed you mentioned MEMO fields and not VARCHAR.
Will test again, sorry.
Comment 5 Alex Thurgood 2023-02-14 15:02:29 UTC
(In reply to Alex Thurgood from comment #4)
> Ah, I've just noticed you mentioned MEMO fields and not VARCHAR.
> Will test again, sorry.

Hmm, still no repro with just :

ID INT(4) NOT NULL
Name MEMO (text)

even if I enter a number into the sequence of characters (at end or beginning) into the MEMO field.


Note : when I choose MEMO (text) from the LO UI dropdown in table design, it gets converted to 

character varying (100) in the postgres database (postgres 10).
Comment 6 Robert Großkopf 2023-02-14 15:38:28 UTC
(In reply to Alex Thurgood from comment #5)
> 
> 
> Note : when I choose MEMO (text) from the LO UI dropdown in table design, it
> gets converted to 
> 
> character varying (100) in the postgres database (postgres 10).

It is created here as field type TEXT.
ProstgreSQL 13.9 is the database server here.

Datatype TEXT should also be there in PostgreSQL 10.

Where did you get the information it has been changed to character varying (100)? I get the information it is "TEXT" when opening the table for editing in dBeaver.
Comment 7 Alex Thurgood 2023-02-15 11:28:46 UTC
(In reply to Robert Großkopf from comment #6)

 
> Where did you get the information it has been changed to character varying
> (100)? I get the information it is "TEXT" when opening the table for editing
> in dBeaver.

From a postgres console session, using 

\d "TABLENAME"


I tried to create a new table via the LO GUI, and chose MEMO (text) when defining the character field, with an ID INT (4) field as the NON NULL PK.

When I query the table definition from the console, I get :

\d "Pets"
                Table "public.Pets"
 Column |  Type   | Collation | Nullable | Default 
--------+---------+-----------+----------+---------
 ID     | integer |           | not null | 
 Pet    | text    |           |          | 
Indexes:
    "Pets_pkey" PRIMARY KEY, btree ("ID")




I still can't reproduce the problem.

Could this be a difference between PG10 and PG13 ?
Comment 8 Robert Großkopf 2023-02-15 11:35:49 UTC
(In reply to Alex Thurgood from comment #7)
> (In reply to Robert Großkopf from comment #6)

Hi Alex,
>                 Table "public.Pets"
>  Column |  Type   | Collation | Nullable | Default 
> --------+---------+-----------+----------+---------
>  ID     | integer |           | not null | 
>  Pet    | text    |           |          | 
> Indexes:
>     "Pets_pkey" PRIMARY KEY, btree ("ID")

This should be the right definition. If I define this in LO GUI with direct connection and try to input text I always get '0'. I only could input numbers.
If there is text in field "Pet" (while inserting by a JDBC-connection) I could also insert text.
> 
> Could this be a difference between PG10 and PG13 ?

Might be it is a difference between 10 and 13, but the type "text" in PostgreSQL is very old …
Comment 9 Alex Thurgood 2023-02-15 11:57:10 UTC
> Could this be a difference between PG10 and PG13 ?


Well, I fired up a PG15 instance, and then could reproduce the problem as described.

It sounds like the SDBC driver would need to be built against a more current release of postgres than PG10.

Adjusting title.

@lionel, @julien : would you happen to know which version of postgres our SDBC driver is built against ?
Comment 10 Alex Thurgood 2023-02-15 12:01:36 UTC
(In reply to Alex Thurgood from comment #9)
> > Could this be a difference between PG10 and PG13 ?
> 


Note that the title only reflects current testing.

Robert reported against PG13
I reproduced against PG15
I did not reproduce against PG10.
Comment 11 Alex Thurgood 2023-02-15 12:11:20 UTC
It turns out that postgres 10 was EOL in November 2022:
https://endoflife.date/postgresql

if we haven't done so, I guess the driver should be updated to something current and not nearing EOL, as even v11 is EOL in 8 months time.
Comment 12 Julien Nabet 2023-02-15 15:47:06 UTC
(In reply to Alex Thurgood from comment #9)
> > Could this be a difference between PG10 and PG13 ?
> ...
> @lionel, @julien : would you happen to know which version of postgres our
> SDBC driver is built against ?

taking a look at download.lst, I see:
    464 POSTGRESQL_TARBALL := postgresql-13.8.tar.bz2
(see https://opengrok.libreoffice.org/xref/core/download.lst?r=3c988ef4#464)

so I'd say Postgresql 13.8

It migrated from 9.2 to 13.1 with 234833f7823a1424b62c93e145f0cfe2c6b6efd5
(see https://cgit.freedesktop.org/libreoffice/core/commit/?id=234833f7823a1424b62c93e145f0cfe2c6b6efd5)

you can see the migrations to 13.5 and 13.8 with https://cgit.freedesktop.org/libreoffice/core/log/?qt=grep&q=postgresql
Comment 13 Robert Großkopf 2023-02-15 18:20:12 UTC
Have tested this a little bit more.

Bug will only appear if primary key is a numeric field (int 4). If primary key is varchar(100) text content is saved in field for Meme[text] as text.

Bug will only appear directly after creating the table. If you close the database file, close and reopen LO and connect again to PostgreSQL with direct connection text content will be saved.
Comment 14 Stéphane Guillou (stragu) 2023-03-28 14:56:36 UTC
Alex, you marked as a regression, but Michael in Comment 1 reproduced in 7.3.
Any evidence that this is actually a regression?
Comment 15 Alex Thurgood 2023-03-28 16:23:59 UTC
@Stephane - probably because I couldn't reproduce the buggy behaviour against a pg10 instance, cf. comment 5.

It was only when I tried against a pg13 instance and later that I saw the problem.

I thought that it might possibly be down to a change introduced in the TEXT datatype in pg13, but couldn't see anything in the release notes that might be indicative of such a change.

I can't test any 6.x on macOS Arm, they refuse to run.

Ideally, someone should test with a 6.x against at least a pg13 instance on Linux.
Comment 16 Stéphane Guillou (stragu) 2023-03-29 13:44:03 UTC
I get "permission denied" errors when creating the table in LO 6.0 or 6.4.

Making sure the primary key was Integer[int4] before the Memo[text] var, I could reproduce with PG13.10 and:

Version: 7.3.7.2 / LibreOffice Community
Build ID: e114eadc50a9ff8d8c8a0567d6da8f454beeb84f
CPU threads: 8; OS: Linux 5.15; UI render: default; VCL: gtk3
Locale: en-AU (en_AU.UTF-8); UI: en-US
Calc: threaded

For people not reproducing with PG10, maybe double-check that you had it setup as Robert described in Comment 13.

Works as expected in:

Version: 7.2.7.2 / LibreOffice Community
Build ID: 8d71d29d553c0f7dcbfa38fbfda25ee34cce99a2
CPU threads: 8; OS: Linux 5.15; UI render: default; VCL: gtk3
Locale: en-AU (en_AU.UTF-8); UI: en-US
Calc: threaded

So regression.
Comment 17 Stéphane Guillou (stragu) 2023-03-29 14:14:38 UTC
tried to bibisect but couldn't reproduce in the bibisect repo, then tried again to reproduce in 7.3.7.2 where I reproduced before, and couldn't reproduce either.

I used a new table every time, and made sure a primary int4 ID was before the memo[text]var, no luck. I think I need more precise steps...
Comment 18 Robert Großkopf 2023-03-29 16:12:21 UTC
1. Table has to be created with 2 fields, 
"ID" INTEGER[int 4]
"Memotext" MEMO[Text]
"ID" should be primary key

2. Try to input in GUI: "ID" → 1, "MemoText" → 'New Text'

3. Go to next row. 'New Text' will be set as '0'.

All this only happens after creating a new table here. Doesn't happen after closing the connection and reopening the connection.

All this happens only if there are other tables in schema "public". If there is no table in schema "public" the first table won't show this bug here.

Tested with PostgreSQL 14.7 on OpenSUSE 15.4 64bit rpm Linux.
Bug appears in LO 7.0.5.2, also in LO 7.5.1.2 and all other versions installed here.
Think this isn't a regression, has never worked and was never detected …
Comment 19 Robert Großkopf 2023-03-29 17:29:34 UTC
Another hint: Seems to be a problem with more than on the field type "text". 
'0' appears also in field type array if table has been created new.

Created a array in this way through Tools → SQL
CREATE TABLE "public"."Table_Ar" ("ID" int4 NOT NULL,"Surname" varchar(100),"Forename" varchar(200)[], PRIMARY KEY  ("ID"));

I'm not able to enter data in the array in the fresh table. It will produce '0' and this won't be accepted by an array, because it needs {…}.

When inserting through Tools → SQL

INSERT INTO "public"."Table_Ar" ("ID", "Surname", "Forename") VALUES (1, 'Müller', '{Liese, Gerd}');

The row will be inserted and the next row could be created in GUI without any problem.