Bug 118043 - Firebird: Migration Data loss (change) when moving decimal datatype data defined in HSQL schema
Summary: Firebird: Migration Data loss (change) when moving decimal datatype data defi...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Base (show other bugs)
Version:
(earliest affected)
6.1.0.0.beta1+
Hardware: All All
: high major
Assignee: Not Assigned
URL:
Whiteboard: target:6.2.0
Keywords:
Depends on:
Blocks: Database-Firebird-Migration
  Show dependency treegraph
 
Reported: 2018-06-06 21:07 UTC by Drew Jensen
Modified: 2019-07-10 17:27 UTC (History)
1 user (show)

See Also:
Crash report or crash signature:


Attachments
file to test migration (5.90 KB, application/vnd.oasis.opendocument.database)
2018-06-06 21:08 UTC, Drew Jensen
Details
screen shot comparing before and after migration (337.54 KB, image/png)
2018-06-06 21:09 UTC, Drew Jensen
Details
new test file (6.17 KB, application/vnd.oasis.opendocument.database)
2018-06-06 22:39 UTC, Drew Jensen
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Drew Jensen 2018-06-06 21:07:17 UTC
Description:
Both HSQL and Firebird define DECIMAL data types with user supplied precision setting.

HSQL supports a larger precision setting than does Firebird. 
Firebird limit is 18 and HSQL supports values up to 100.

The attached example with HSQL decimal(20, 2) [precision 20, 2 decimal places]) and a range of values displayed as Euros. 

 

Steps to Reproduce:
1. download the attached file and make a copy of it
2. open the copy and click on tables, say yes to start migration assistant
3. when it finishes open the data table
4. open the original file you download, open the same data table


Actual Results:  
the values do not match in every row
check out the attached screen shot for highlighted rows.

Expected Results:
the values stored properly in a decimal(18,2) column in the new schema. 


Reproducible: Always


User Profile Reset: No



Additional Info:


User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/66.0.3359.181 Chrome/66.0.3359.181 Safari/537.36
Comment 1 Drew Jensen 2018-06-06 21:08:10 UTC
Created attachment 142574 [details]
file to test migration

hsql file with three columns defined as decimal(20,0) and data
Comment 2 Drew Jensen 2018-06-06 21:09:48 UTC
Created attachment 142575 [details]
screen shot comparing before and after migration

screen shot,

Table on the left is in HSQL file

Table on the right is in Firebird after migration.

Rows with data mismatch are highlighted.
Comment 3 Xisco Faulí 2018-06-06 21:56:21 UTC
Reproduced in

Version: 6.1.0.0.beta1+
Build ID: 8b96445766efe237eb47608ade6c147673466e2e
CPU threads: 4; OS: Linux 4.13; UI render: default; VCL: gtk3; 
Locale: ca-ES (ca_ES.UTF-8); Calc: group threaded

Thank you very much for investigating this problem and creating a testcase!!
Comment 4 Drew Jensen 2018-06-06 22:39:51 UTC
Created attachment 142576 [details]
new test file

This isn't just where the precision in the column definition is greater than 18.

I updated the test file with a second table, decimal(10,2) and get some of the same data changes.
Comment 5 Commit Notification 2018-06-14 11:16:16 UTC
Tamas Bunth committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=1f91ebb586356aa77ecced15cd5e049f1ebca70c

tdf#118043 dbahsql: Fix putDot funciton

It will be available in 6.2.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 6 Drew Jensen 2018-06-18 13:23:55 UTC
Checked this with Version: 
6.2.0.0.alpha0+
Build ID: cbd0d1da85a7ae6b35023274c7267872b3d920e7
CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk2; 
TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time: 2018-06-18_00:24:33
Locale: en-US (en_US.UTF-8); Calc: group threaded

That does include the patch.

The problem is unchanged when trying to migrate the test file.
Comment 7 Drew Jensen 2018-06-18 15:13:30 UTC
set the component type to Base vs Basic so this shows up in QA team canned searchs
Comment 8 Xisco Faulí 2019-07-10 14:16:27 UTC
(In reply to Drew Jensen from comment #6)
> Checked this with Version: 
> 6.2.0.0.alpha0+
> Build ID: cbd0d1da85a7ae6b35023274c7267872b3d920e7
> CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk2; 
> TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time:
> 2018-06-18_00:24:33
> Locale: en-US (en_US.UTF-8); Calc: group threaded
> 
> That does include the patch.
> 
> The problem is unchanged when trying to migrate the test file.

I believe, it's the same root cause as bug 126268
Comment 9 Xisco Faulí 2019-07-10 17:27:44 UTC
(In reply to Xisco Faulí from comment #8)
> (In reply to Drew Jensen from comment #6)
> > Checked this with Version: 
> > 6.2.0.0.alpha0+
> > Build ID: cbd0d1da85a7ae6b35023274c7267872b3d920e7
> > CPU threads: 4; OS: Linux 4.15; UI render: default; VCL: gtk2; 
> > TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time:
> > 2018-06-18_00:24:33
> > Locale: en-US (en_US.UTF-8); Calc: group threaded
> > 
> > That does include the patch.
> > 
> > The problem is unchanged when trying to migrate the test file.
> 
> I believe, it's the same root cause as bug 126268

Yes, it's, I have a patch in gerrit already.
Since this issue has a commit, let's close it as RESOLVED FIXED