Bug 138394 - FIREBIRD: truncated CLOB data
Summary: FIREBIRD: truncated CLOB data
Status: RESOLVED INSUFFICIENTDATA
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Base (show other bugs)
Version:
(earliest affected)
6.4.7.2 release
Hardware: x86-64 (AMD64) Windows (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: Database-Firebird-Default
  Show dependency treegraph
 
Reported: 2020-11-21 16:52 UTC by trowelandmattock
Modified: 2022-04-12 04:38 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description trowelandmattock 2020-11-21 16:52:12 UTC
Description:

CLOB fields which are larger than standard VARCHAR max do not display correctly in LO - they appear to be truncated to VARCHAR max, eg in form text-fields, and (unsurprisingly) in calc cells....
...since there is also an issue exporting .csv directly from Base+Firebird (oops-lost link) this becomes a potentially big issue>>

ie: CLOB fields larger than VARCHAR max may become truncated and irrecoverable by 'normal' means !!! :O

(perhaps a LO project could  be fully reconstructed in external DB...but not exactly for the standard user)

Steps to Reproduce:
1.make CLOB larger than VARCHAR max
2.try to extract or view....:/
3...sadly macro based.csv export also dosn't appear to work in LO-Firebird, so stuck with truncation

Actual Results:
large CLOB truncated

Expected Results:
large CLOB not truncated :)


Reproducible: Always


User Profile Reset: No


OpenGL enabled: Yes

Additional Info:
a .csv/text export would be great, But relevant SQL not working for firebird in LO---
of course large CLOBS are not necessarily 'everyday' issue,but seems potential serious data loss issue here...
...if LO-firebird CLOBS can not be fixed, maybe they need 'removing' or have a BIG 'use-with-care' warning slapped on them !
Comment 1 Julien Nabet 2020-11-22 10:40:03 UTC
You spoke about Firebird, do you mean it's on embedded Firebird only?
Could you provide an example file?
Comment 2 trowelandmattock 2020-11-22 17:19:03 UTC
Yes - i am using firebird - i don't know the situation with LO-HSQLDB, but i understand that .csv export is not a problem via macros ("TEXT"), while I have seen comments that a similar approach is not working with LO-firebird ( LO with an external firebird i believe does do the export-to-TEXT-function correctly)

Has LO internally treated CLOB as max varchar at some point? IDK
However there would would appear to be no easy way to extract the e.g. full text of large CLOB without truncation (e.g. through calc, or in a form-text-field)

I will check this out more when i get chance > maybe there is something the data or queries (in this particular case multiple CAST( aaa AS BLOB SUB_TYPE 1) || CAST (bbb AS BLOB SUB_TYPE 1)|| CAST (etc... with aaa/bbb/etc all under varchar max , and the whole then all CAST as a single CLOB.

I have not tried direct data entry of single CLOB over varchar max > to big for past-into-table-cell >>> i should try pate into text-field of form

>>>+ the export to .csv (for LO-firebird) could also do with resolving (can this work or not? >> if not, then surely potential issues for CLOB usage?)
Comment 3 QA Administrators 2020-11-23 04:58:18 UTC Comment hidden (obsolete)
Comment 4 Robert Großkopf 2020-11-23 21:15:00 UTC
Have tested this with a query, which concates the content of a CLOB-field with 12700 characters 10 times. The whole content (127000 Characters) of this concated content is shown in the field of a form. The content isn't truncated here.

Have tested this with LO 6.4.6.2 on OpenSUSE 15.1 64bit rpm Linux.
Comment 5 trowelandmattock 2020-12-08 10:52:29 UTC
(In reply to Robert Großkopf from comment #4)
> Have tested this with a query, which concates the content of a CLOB-field
> with 12700 characters 10 times. The whole content (127000 Characters) of
> this concated content is shown in the field of a form. The content isn't
> truncated here.
> 
> Have tested this with LO 6.4.6.2 on OpenSUSE 15.1 64bit rpm Linux.





That is good to know Robert. The initial problem I had was when trying to export to a .csv format, for use external to LO. I have been rather lax and not delved into this much beyond dealing with the initial task at hand...or followed up the apparent lack working of macro-based method to export to csv under Firebird. Will report back if I do :)
Comment 6 Buovjaga 2021-08-31 15:26:21 UTC
(In reply to trowelandmattock from comment #5)
> (In reply to Robert Großkopf from comment #4)
> > Have tested this with a query, which concates the content of a CLOB-field
> > with 12700 characters 10 times. The whole content (127000 Characters) of
> > this concated content is shown in the field of a form. The content isn't
> > truncated here.
> > 
> > Have tested this with LO 6.4.6.2 on OpenSUSE 15.1 64bit rpm Linux.
>
> That is good to know Robert. The initial problem I had was when trying to
> export to a .csv format, for use external to LO. I have been rather lax and
> not delved into this much beyond dealing with the initial task at hand...or
> followed up the apparent lack working of macro-based method to export to csv
> under Firebird. Will report back if I do :)

Do you have an update on this? How does 7.2 behave?

Set to NEEDINFO.
Change back to UNCONFIRMED, if the problem persists. Change to RESOLVED WORKSFORME, if the problem went away.
Comment 7 Mike Kaganski 2021-09-12 10:18:42 UTC
I tested with a query using a LIST statement; and running it to concatenate 20000 values 6 characters with 2-char separator each gives a CLOB larger than 150000 characters (VARCHAR limit is 32K in FB). Copying the query, and pasting into Writer produces the full non-truncated data in 7.2.1.2.

It is difficult to do anything here without repro steps. A database, maybe with macros that generate the export; or clear method that produces the problem is required.

The problem could even be in an export filter (CSV), not in CLOB handling.
Comment 8 QA Administrators 2022-03-12 03:38:12 UTC Comment hidden (obsolete)
Comment 9 QA Administrators 2022-04-12 04:38:22 UTC
Dear trowelandmattock,

Please read this message in its entirety before proceeding.

Your bug report is being closed as INSUFFICIENTDATA due to inactivity and
a lack of information which is needed in order to accurately
reproduce and confirm the problem. We encourage you to retest
your bug against the latest release. If the issue is still
present in the latest stable release, we need the following
information (please ignore any that you've already provided):

a) Provide details of your system including your operating
   system and the latest version of LibreOffice that you have
   confirmed the bug to be present

b) Provide easy to reproduce steps – the simpler the better

c) Provide any test case(s) which will help us confirm the problem

d) Provide screenshots of the problem if you think it might help

e) Read all comments and provide any requested information

Once all of this is done, please set the bug back to UNCONFIRMED
and we will attempt to reproduce the issue. Please do not:

a) respond via email 

b) update the version field in the bug or any of the other details
   on the top section of our bug tracker

Warm Regards,
QA Team

MassPing-NeedInfo-FollowUp