Bug 116944 - Firebird-migration must happen only after user consent and advise to backup data prior to migration at 6.1
Summary: Firebird-migration must happen only after user consent and advise to backup d...
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Base (show other bugs)
Version:
(earliest affected)
6.1.0.0.alpha0+
Hardware: x86-64 (AMD64) All
: high major
Assignee: Not Assigned
URL:
Whiteboard: target:6.1.0
Keywords:
Depends on:
Blocks: Database-Firebird-Migration
  Show dependency treegraph
 
Reported: 2018-04-11 15:17 UTC by Robert Großkopf
Modified: 2018-04-19 18:35 UTC (History)
13 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Großkopf 2018-04-11 15:17:50 UTC
Have downloaded 
Version: 6.1.0.0.alpha0+
Build-ID: dc823f5fa4a5d2eca56297b9045e5962536c00f9
CPU-Threads: 4; BS: Linux 4.4; UI-Render: Standard; VCL: kde4; 
TinderBox: Linux-rpm_deb-x86_64@70-TDF, Branch:master, Time: 2018-04-10_23:32:35
Gebietsschema: de-DE (de_DE.UTF-8); Calc: group

Seen in mailinglists there should exist an automatically migration from internal HSQLDB-files to internal Firebird. The I wrote the following to the global users mailnglist:

Opened the database, which is part of the Base Handbook:
"Media_without_Macros".
No comment like "Internal HSQLB" appears at the bottom of the window.
Tried to open the tablecontainer.
No table appears, but after some time the following message:

Connection to datasource "Media_without_Macros" couldn't be established.

firebird_sdbc error:
*Dynamic SQL Error
*SQL error code = -104
*Token unknown - line 1, column 163
*BLOB
caused by
'isc_dsql_prepare'

I haven't set the experimental features to "yes". I wasn't asked if I
want to change the database from HSQLDB to Firebird.

Base should never be a playground for such a function. People will loos
their data and will never again use LO after starting one database with
this automatic.

1. An automatic migration should never be installed from a working
internal database to a database, which is an "experimental feature"
(Firebird) - and never means: also never with a daily build.
2. A migration should only be offered, never automatically, when
internal Firebird database has been activated.
3. The migration should create a new database with the migrated content,
not overwrite the old database.
Comment 1 Florian Reisinger 2018-04-11 16:26:54 UTC
Hi,

It seems that existing files are irreversibly broken.

Automatic conversion is always dangerous if not only the driver itself is changed, but the internal data structure is.

Setting status to NEW and doing some research in the IRC chat
Comment 2 Adolfo Jayme Barrientos 2018-04-12 12:23:28 UTC
> I haven't set the experimental features to "yes".

Of course you haven’t; it’s that Firebird is no longer experimental. Just thought I should point that out.
Comment 3 V Stuart Foote 2018-04-12 14:31:03 UTC
The summary "Firebird-migration should never work automatically" is invalid. 

Conversion from embedded 1.8.0 HSQLDB to Firebird 3.0. is explicit and happens automatically as otherwise a legacy database would not open in LibreOffice master/6.1.0

If the concern that the user needs more hand holding, and better notice that the database will be restructured (with potential for data loss, and therefor a suggestion to manually archive a backup)--that is a reasonable UI/UX requirement. 

Summary adjusted...
Comment 4 Heiko Tietze 2018-04-12 15:19:28 UTC
How about an infobar telling the user that migration was successful, or not?
Comment 5 Lionel Elie Mamane 2018-04-12 15:29:11 UTC
IMO we should either:
1) eschew the need for backup altogether by _not_ overwriting the *same*
   .odb file, but saving the converted odb under a new name (e.g. one
   that we ask the user via a filepicker?)
2) do the backup unconditionally by saving the original .odb file under
   a new name

Solution "1" is probably somewhat easier.
Comment 6 Florian Reisinger 2018-04-12 17:05:00 UTC
@4: The file would be irreversible damaged I the conversation was not successful?

@5: We can only get problems when automatically saving to a new location:

 - space on disc can be full
 - user could not have permissions

Therefore I would prefer prompting the user on fileopen to convert the database and save it in a new location.

However the transition from experimental to default is too quick. We should wait a few releases (at least one) to stabilize the new database. 

Therefore I believe HSQLDB should not be removed yet, but in the short future. Conversation in the next release should be triggered only manually (and should result in a new file.

Such a drastic transition would scare users away not only from Base, but from LibreOffice. This is even more probable when no backup of a file is created and the file is destroyed....
Comment 7 Gerhard Weydt 2018-04-12 18:01:03 UTC
I totally agree with comment #6. I just want to point out some more problems.
Showing a warning if the migration didn't work or even before migrating is too late, because the user has already updated LibreOffice, so he has no chance to get at his data with that release. Surely, it is possible to do a parallel installation, but many people will not know that or might not be able to do it or simply give up, frustrated, and are lost for LibreOffice.
There has to be at least one release, say 6.0, where HSQLDB and Firebird run in parallel, and informing the user that the database should be migrated to Firebird for the future, but that he should first try it using a copy of his database document.
Some might say that we have already had that period of parallel database systems, but I am sure that most people will not have known that because you could know about Firebird only if you had activated the experimental features, or had read it somewhere, for example in Roberts Base handbook. I am using Bae myself, with MariaDB when it's possible, but also with HSQLDB, but I haven't yet tried Firebird, because I always heard it still was lacking functionality. So I underline Florian's remark that the change is too quick.
But even an infobar in 6.0 when opening a database document with HSQLDB that the database should be migrated to Firebird, because in 6.1 HSQLDB will no longer be supported (the version nummbers only as an example) will not be sufficient, there may be people updating from 5.4 to 6.1, they will never have got that message.
I have no convincing proposal how to cope with this problem, yet, the only idea I have for the moment is warning at the start of the installation describing the problem and, if applicable, proposing to update first to the intermediate version and test the migration before doing the second update. That's not very comfortable.
All would be easier if someone could guarantee that the migration to Firebird will always work, but who would be able to do that? Robert's experience shows that we are far away from that.
Comment 8 Florian Reisinger 2018-04-12 18:23:15 UTC
If (as currently built in) the content of the file is changed there is no way going back!

If (in some way) the conversion is happening in a separate file, the user at least has the change of moving back to an older (probable insecure) release of LibreOffice.

I guess we all agree that the currently implemented behavior is not in any way user friendly. It - in fact - will scare users away.

However, especially if we have a release with both database drivers we can automatically compare the result of reports. Most read-only reports will change if executed not in the same moment.

The hardest requirement would be to try converting to a new file and checking all reports and fail (or give a warning with the name of the reports) when the result are not identical.

Only with such a dynamic test suite we would have a chance to make the transition as smooth as possible,

I hope you agree to my statement that the current situation must not be released and that the conversation must happen to a new file.

Some might complain that this is similar to introducing a new algorithm for encrypting documents, which also break compatibility with AOO and older versions of LibO. However the complexity of implementing / using a different encryption function cannot be compared to switching to a different database backend. Therefore a staged approach (marking Firebird non-experimental, creating new ODB files with Firebird, converting old databases OPTIONALLY, removing HSQLDB) should be spitted up to more than one maybe 3 or more releases. Bugs will appear and we need to fix them with the fewest impact for the user. A staged roll-out therefore is from the end user's point of view the only real option!
Comment 9 Thomas Lendo 2018-04-12 18:45:33 UTC
From a UX point of view, we should not suppose that the user is creating a backup file manually even if we warn him by an info bar.

We should force the user 1) either to save a backup file within an automatically opened dialog where the user can choose a file name and a location or 2) to save the new firebird file with an identical dialog.

This user prompted solution prevents failed auto-backups and informs the user that there is something happen.
Comment 10 Florian Reisinger 2018-04-12 19:03:25 UTC
I would let the user save a new file. I guess the awareness might be higher that he is able to use the old file with the old LibreOffice version. Let him store a new file for the new behavior. The old file for the new behavior would be strange for me.

Bur I am not an UX expert, just my 2 cents!
Comment 11 V Stuart Foote 2018-04-12 19:06:09 UTC
Hate to be the lone voice of dissent here, but Tamás remains under tender from the foundation to get the HSQLDB migration tool done correctly.

There is ESC support for the implementation (Lionel is a member) and rather than stretching out the migration of internal HSQLDB to Firebird (that should have happened long ago starting at bug 38811) if the migration code is ready and gets well tested--as Tamás requested [1]--there really is no practical UX reason to stretch out the pain point of the DB transition.

In fact there is much to be said for an ESC led effort to see this conversion completed, diverting dev cycles where needed to get the DB and the Base UIs finished. The project--and users--gain little by pushing it down the road with a "transitional" period. 

Otherwise, deprecation notice for internal HSQLDB should be made now in 6.0 release notes.

=-ref-=
[1] http://nabble.documentfoundation.org/Database-migration-in-Base-tc4237588.html
Comment 12 Lionel Elie Mamane 2018-04-12 19:35:34 UTC
(In reply to V Stuart Foote from comment #11)

> There is ESC support for the implementation (Lionel is a member)

There is ESC support for the migration, and the change of default,
but not for the timing of it. HSQLDB and Firebird are two different
systems with two different SQL syntaxes, different feature sets, etc.
People that use Base as a development platform to make their own application
need a transition period to adapt their code.

IMO, the timing of the change of default for *new* databases is still not
decided for certain. It might happen for the next release, or later.
Depends on when bugs like 104918, 116935 and 116936 are fixed. See bug
51780 for the tracking of that. We may also want to wait for bug 116936
to be fixed.

Removing HSQLDB needs _time_ from the moment Firebird-in-LibreOffice is
stable, good quality, and has - broadly speaking - feature-parity with
HSQLDB. Bugs like 116936 are blocker for that. See bug 116970 for the
tracking of that.

> rather than stretching out the migration of internal HSQLDB to Firebird
> if the migration code is ready and gets well tested, there really is no
> practical UX reason to stretch out the pain point of the DB transition.

I feel that this POV comes from a view that all SQL engines are interchangeable, they only differ in on-disk data storage format. For users that only use Base as a graphical GUI to do stuff, that may even be, to a moderate to large degree, true. However, for people using Base as an application development platform, that is very much not true.
Comment 13 Gerhard Weydt 2018-04-12 19:47:32 UTC Comment hidden (no-value)
Comment 14 Robert Großkopf 2018-04-12 19:49:40 UTC Comment hidden (no-value)
Comment 15 Florian Reisinger 2018-04-12 19:57:57 UTC Comment hidden (no-value)
Comment 16 V Stuart Foote 2018-04-12 20:04:39 UTC
Who said ANYTHING about the migration engine being backported to a 6.0 release?

Deprecation notice needs to have gone in to the 6.0 release notes (and probably should have gone in to the 5.4 release notes--given the TDF Tender approval).

This change only affects master toward a 6.1.0 (RC1 ~ July 2018).
Comment 17 Robert Großkopf 2018-04-12 20:23:56 UTC
(In reply to V Stuart Foote from comment #16)
> Who said ANYTHING about the migration engine being backported to a 6.0
> release?
> 
Okay, have understand it the wrong way while reading about bugs migration from HSQLDB to Firebird. Only 6.1.0.0.alpha0 will automatically migrate the databases.
Comment 18 jonathon 2018-04-12 21:17:38 UTC
If it is going to automatically convert the database, could the first step in the conversion process be copying the file from filename.odb to filename.HSQLDB.date.time.odb.
The second step would be converting filename.odb to filename.firebird.date.time.odb.
The third step would be to copy filename.firebird.date.time.odb to filename.odb.
All done with no user interaction.
Comment 19 Florian Reisinger 2018-04-13 04:43:20 UTC
(In reply to jonathon from comment #18)
> If it is going to automatically convert the database, could the first step
> in the conversion process be copying the file from filename.odb to
> filename.HSQLDB.date.time.odb.
> The second step would be converting filename.odb to
> filename.firebird.date.time.odb.
> The third step would be to copy filename.firebird.date.time.odb to
> filename.odb.
> All done with no user interaction.

Doing to automatically won't work as the user might not have permission to create a new file.

I honestly do not believe that a backup without user involvement is possible

Still looking forward to answers to #15
Comment 20 V Stuart Foote 2018-04-13 05:16:17 UTC
(In reply to Florian Reisinger from comment #19)
> 
> Doing to automatically won't work as the user might not have permission to
> create a new file.
> 
> I honestly do not believe that a backup without user involvement is possible
> 
> Still looking forward to answers to #15

Removal of HSQLDB and the Firebird migration was reviewed at ESC today [1] and backed out [2], if it can be redone near term it will instead be optional at 6.1 with just NEW Base ODF using Firebird. This evenings builds of master have HSQLDB restored.

=-ref-=
[1] http://nabble.documentfoundation.org/Libreoffice-qa-minutes-of-ESC-call-td4237769.html
[2] https://gerrit.libreoffice.org/#/c/52731/
Comment 21 Lionel Elie Mamane 2018-04-13 05:59:47 UTC
(In reply to Florian Reisinger from comment #15)
> If you care to explain why
> 
> - removing a DB backend without deprication notice
> - keeping the HSQLDB data in the ODB file WITHOUT possibility to open it
> with an old version of LibreOffice
> - hushing the transition to the new database backend
> 
> is a good idea.

It is not, and will not happen. Please, there was some over-enthusiasm, things got committed to master, they were discussed and the plan was established. Do not take every commit to the master branch as a firm, well thought out, plan.

Let's stop excoriating this transitory situation and focus on the reasonable plan.

> - Putting it in the 6.0 release notes is impossible - already released.
>   --> Mark it for removal in 6.1 with removal in 6.2 / 6.3

My current thought is: deprecation when Firebird is in good shape (cf the tracker bug for that), and removal at least one major distro LTS cycle after that.

>   --> New databases will be created with Firebird (by default?) in 6.1

_If_ the Firebird driver is in a good enough shape, which we have good hope it will be.

> - Why keep the old data in the file?
>   --> Creating a new file with just Firebird data makes more sense as the
> end user is not able to recover the data!

Yes.

> - Why do we need to get rid of HSQLDB so quickly?

We don't.
Comment 22 Lionel Elie Mamane 2018-04-13 06:01:07 UTC
(In reply to robert from comment #17)

> Okay, have understand it the wrong way while reading about bugs migration
> from HSQLDB to Firebird. Only 6.1.0.0.alpha0 will automatically migrate the
> databases.

No, it will not. It will _propose_ migration, and allow continued use of HSQLDB, for at least a few releases.
Comment 23 Robert Großkopf 2018-04-13 06:25:56 UTC
(In reply to Lionel Elie Mamane from comment #21)
> 
> Deprecation when Firebird is in good shape (cf the
> tracker bug for that), and removal at least one major distro LTS cycle after
> that.
> 
> >   --> New databases will be created with Firebird (by default?) in 6.1
> 
> _If_ the Firebird driver is in a good enough shape, which we have good hope
> it will be.
> 
> > - Why keep the old data in the file?
> >   --> Creating a new file with just Firebird data makes more sense as the
> > end user is not able to recover the data!
> 
> Yes.
> 
> > - Why do we need to get rid of HSQLDB so quickly?
> 
> We don't.

This would be the best solution. 

Changing step by step. Let us first try to get Firebird better working than HSQLDB for a normal user. Do not deprecate HSQLDB before. Test the migration. Migration has to save all: Tables, relations and views. Could be views will be saved as queries, if code is to complicated. After this is working we should set HSQLDB as deprecated.

I have seen the long time for getting Firebird to work in Base - experimental since LO 4.2! We will need also time for a well working Firebird as non experimental database.
Comment 24 Gerhard Weydt 2018-04-13 20:24:18 UTC
I think we all agree that the presently communicated concept for the change-over from HSQLDB to Firebird - which I hope all will accept as necessary in the long run - is much more convincing than what we understood from the bits of information we had got (the information policy could have been better).

Now those of us that are concerned with HSQL databases and have more knowledge than "simple" users should intensively test Firebird and the migration program and report our findings, in order that the migration, when it will be in effect, should be easy, secure and complete, as far as it is possible, for all users, and hence be accepted as a positive step even by those users who might not care for the reasons behind the screen.

- The discusion I followed was in the german discuss and in this bug. How will the international community be involved?
- Where can we report? Please define some address to collect these reports, so that Tamás can use the information to improve his migration program (and maybe there are also possible enhancements for Firebird within LibO).

I will test with my databases, and I am also willing to test again and again with newer versions of LibO, if asked. I am a fan of databases, and I will do much to maintain Base as a useful tool.
Comment 25 Lionel Elie Mamane 2018-04-13 20:45:15 UTC
(In reply to Gerhard Weydt from comment #24)

> (...) should intensively test Firebird and the migration program

> - Where can we report?

File bugs, put Tamás in CC, and make it block bug 116968, 116970 or 51780 as appropriate.
Comment 26 V Stuart Foote 2018-04-13 21:35:19 UTC
(In reply to Lionel Elie Mamane from comment #25)
> File bugs, put Tamás in CC, and make it block bug 116968, 116970 or 51780 as
> appropriate.

To make it just a bit easier for folks here are active links to the META issues...

bug 116968 -- "Migrating existing embedded HSQLDB databases to Firebird"
bug 116970 -- "HSQLDB removal blockers"
bug 51780 -- "Default to Firebird not HSQLDB in Base (for _new_ files)"
Comment 27 Gerhard Weydt 2018-04-13 22:21:04 UTC
@25: That's just the problem I wanted to address: you people in the heart of LibreOffice have many connections, but we simple users (as which I consider myself: I have 30 years of experience in software development, but not in LibreOffice; I am active in using LibO and writing handbooks, but not involved in development; I certainly can help in testing, perhaps in suggesting, but probably not in sketching solutions) do not have the information. I simply do not know how to "put Tamás in cc" because I do not have his mail address. I once registered with nabble (don't know why), and I think this will reveal the address for me, but this was not what I asked for. I would have had the address plainly: his mail address or some other for the same purpose. That's the same issue as Stuart talking of ESC (comment #11): I simply don't know what this means, I assumed that SC meant Steering Committee, but I still don't know.
So please give some more detailed information for us interested and new and willing contributors.
Comment 28 Lionel Elie Mamane 2018-04-14 09:58:42 UTC
(In reply to Gerhard Weydt from comment #27)
> I simply do not know how to "put Tamás in cc"

Near top right of bug, to the right of "CC list", click "edit". In the "add" textbox, enter "tamas bunth", wait about a second, click on the search result. Below the "additioanl comments" textbox, click "save changes".


> That's the same issue as Stuart talking of ESC
> (comment #11): I simply don't know what this means,

https://wiki.documentfoundation.org/Development/ESC
Comment 29 Commit Notification 2018-04-14 19:15:56 UTC
Tamas Bunth committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=5b0ceb86c342754d8f4e83408c7ae0da0e3931d3

tdf#116944 Warn user before database migration

It will be available in 6.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 30 Florian Reisinger 2018-04-14 19:29:02 UTC
As no kind of relationships (foreign key relations) or views can be migrated to Firebird users should not be prompted just yet!

With the current migration quality, this either should not be included in 6.1 (we have quite some time until there to improve). In the current state it should be AT MOST an experimental feature.

Note: I am not talking about making the database experimental again, I am explicitly talking about the (semi) automatic conversion!

Just showing a dialog is no solution, the user will click anything to just be able to work.

This (in my opinion) is the opposite direction to what we seemed to agree on. This most certainly is not a stated approach but a big bang again.

Please assure me, that I am interpreting the commit wrong...
Comment 31 Tamas Bunth 2018-04-14 19:32:30 UTC
from bug title:
"advise to backup data prior to migration at 6.1"

You can revert the migration by opening "content.xml" in the zipped structure and replace the URL "sdbc:embedded:firebird" with "sdbc:embedded:hsqldb". This will work, because the old HSQLDB data is still in the odb structure. You may also want to remove "database/firebird.fbk" so it won't make any surprises when you try to migrate next time.

Of course, the above method is not user friendly. I would like to suggest that - instead of advising the user to backup it's data - we should revert the changes automatically (with the above mentioned method) if the migration wasn't successful.

By "wasn't successful" I mean that the sdbc driver throws an exception - which indicates a lot of errors, but not all of them (e.g. what if something is just missing in the new format). In these cases the user could still revert the migration manually. Because of that, the above mentioned way to revert the migration should be put into a wiki or something.
Comment 32 Tamas Bunth 2018-04-14 19:53:45 UTC
Hi Florian,

(In reply to Florian Reisinger from comment #30)
> As no kind of relationships (foreign key relations) or views can be migrated
> to Firebird users should not be prompted just yet!

These are already on gerrit.
https://gerrit.libreoffice.org/52892/
https://gerrit.libreoffice.org/52893/
https://gerrit.libreoffice.org/52879/
Comment 33 Florian Reisinger 2018-04-14 20:09:44 UTC Comment hidden (no-value)
Comment 34 V Stuart Foote 2018-04-14 20:52:58 UTC
(In reply to Florian Reisinger from comment #33)

"Users" are not going to be working with master/6.1.0 at this point!

Rather, it remains in the preview of project QA effort and expert users who of necessity must test and advise the developers if the code is suitable for release. What role do you intend to play here?

Leaving the migration code and Firebird stuck in experimental mode as it has been does nothing to move the effort along.

Please rest assured that if we get to the end of the 6.1 dev cycle--and the code is not ready for release--hopefully not because it has not adequately been tested and engaged feedback provided--it would be grounds for blocker status and the release then does not roll until fixed.

Production use belongs currently on 5.4.6, or an earlier stable EOL release. Production on current fresh 6.0.3 is probably safe--and can safely remain on 6.0 through the 6.0.7 and beyond its EOL in November 2018.

Point is folks are not and will not be forced to migrate to Firebird 3 for internal data before they are comfortable that they can protect their processes and data--they just will not be on the current "Fresh" build. 

Consider for example that currently we do not even recommend a 6.0 update for 5.4 and earlier users.

Not saying go away, but please roll up your sleeves and either build from source and test, or tune in to the nightlies as they roll from our TinderBox suite and test on that cycle as Tamás notes.

Constructive feedback on explicit issues with samples to Tamás (and the other devs) is a lot more effective here than dropping an anchor and pestering the devs.

Just saying...
Comment 35 V Stuart Foote 2018-04-14 21:15:40 UTC
@Tamás, Miklos, Lionel, *

With internal HSQLDB support now restored through 6.1 (and likely through later releases) why shouldn't we make the offer of odb migration to internal Firebird 3 an opt-in action?

Then could also provide a note in the GUI that reversion HSQLDB is possible and point to a good source how-to (Wiki or Help).
Comment 36 V Stuart Foote 2018-04-14 21:38:51 UTC
(In reply to V Stuart Foote from comment #35)
> @Tamás, Miklos, Lionel, *
> 
> With internal HSQLDB support now restored through 6.1 (and likely through
> later releases) why shouldn't we make the offer of odb migration to internal
> Firebird 3 an opt-in action?

Oops, nevermind. Just read the migrwarndlg.ui in https://gerrit.libreoffice.org/#/c/52876/

But still, give some note of possibility of and how-to revert?
Comment 37 Robert Großkopf 2018-04-15 07:18:59 UTC
(In reply to V Stuart Foote from comment #36)
> 
> Oops, nevermind. Just read the migrwarndlg.ui in
> https://gerrit.libreoffice.org/#/c/52876/
> 
Seems there should appear a popup. Better would be: You choose the migration explicit. A popup will appear every time I open a HSQLDB-database with LO 6.1. One unconcentrated moment and the user presses "YES" - all data will have been gone for him.

The migration isn't working as intended. I don't get any database here working with that - all data have gone after migration, views also, relations also. See https://bugs.documentfoundation.org/show_bug.cgi?id=116968

Putting the backup in the *.odb-file couldn't be the solution for normal users. I wouldn't trust a office-suite which tells me: You are looking for your data? Please use a packing-program, open the *.odb-file ...
Comment 38 Heiko Tietze 2018-04-15 08:02:07 UTC
(In reply to V Stuart Foote from comment #36)
> https://gerrit.libreoffice.org/#/c/52876/

Would be nice to get a screenshot.

(In reply to robert from comment #37)
> One unconcentrated moment and the user presses "YES" - all data will
> have been gone for him.

Safety-focused dialogs revert the logic and Yes, remaining the default, cancels the operation. We use the other, more annoying, approach with a couple of confirmation and backup dialogs when LibreOffice has crashed. 

But we are talking about a temporary solution for master, right? Lost the thread with the many comments ;-).
Comment 39 V Stuart Foote 2018-04-15 13:36:36 UTC
(In reply to robert from comment #37)

> Seems there should appear a popup. Better would be: You choose the migration
> explicit. A popup will appear every time I open a HSQLDB-database with LO
> 6.1.

A opening dialog is appropriate here because the migration can occur. I'd expect additional dev work to control offering the migration, including an expert configuration stanza or Options entry to not convert while we continue to carry HSQLDB 

> One unconcentrated moment and the user presses "YES" - all data will
> have been gone for him.
> 

Maybe, but lets get the GUI pieces working. 

> The migration isn't working as intended. I don't get any database here
> working with that - all data have gone after migration, views also,
> relations also. See
> https://bugs.documentfoundation.org/show_bug.cgi?id=116968
> 

Nor would they be expected to until the code referenced in comment 32 is actually committed and present in a build for use.
Comment 40 V Stuart Foote 2018-04-18 20:02:47 UTC
Closing this resolved fixed. The automatic migration has been quashed, internal HSQLDB support restored and a suitable user dialog to perform the migration (Yes) or not (Later).

Obviously the fidelity of the migration needs continued work--but this facet is fixed.

=-ref-=
https://gerrit.libreoffice.org/#/c/52876/

https://gerrit.libreoffice.org/gitweb?p=core.git;a=commitdiff;h=5b0ceb86c342754d8f4e83408c7ae0da0e3931d3

and ongoing tweaks.
Comment 41 Florian Reisinger 2018-04-19 05:59:20 UTC
I do not think this is fixed as of yet, as per comment #38 this has only been accepted as "a temporary workaround for master".

Personally I would not offer the migration in any 6.1 release builds (side question: Are experimental features automatically enabled in master?).

I do not see the issue resolved until the comments in #39 are treated.

Furthermore this dialog should be "safety focused" and revert the logic by definition. 

As already said using more words: This does not protect the user and as data can be lost and corrupted we need a longer testing period.

I furthermore see the "backup" solution (keeping the HSQLDB database in the file) as not ideal by far. If a file dialog opens user would question the action / have a working backup copy.
Comment 42 Heiko Tietze 2018-04-19 12:37:22 UTC
(In reply to Florian Reisinger from comment #41)
> I do not think this is fixed as of yet...

Suggest to close this ticket as no dev will ever read all the messages, in particular when a patch has been submitted. So ideally you come to a final decision what should be done (not what's wrong) and file another ticket for it.
Comment 43 V Stuart Foote 2018-04-19 14:17:20 UTC
Closing again as per comment 40; and Florian R. opened bug 117106
Comment 44 Drew Jensen 2018-04-19 18:35:26 UTC
(In reply to Tamas Bunth from comment #31)
> from bug title:
> "advise to backup data prior to migration at 6.1"
> 
> You can revert the migration by opening "content.xml" in the zipped
> structure and replace the URL "sdbc:embedded:firebird" with
> "sdbc:embedded:hsqldb". This will work, because the old HSQLDB data is still
> in the odb structure. You may also want to remove "database/firebird.fbk" so
> it won't make any surprises when you try to migrate next time.
> 
> Of course, the above method is not user friendly. I would like to suggest
> that - instead of advising the user to backup it's data - we should revert
> the changes automatically (with the above mentioned method) if the migration
> wasn't successful.
> 
> By "wasn't successful" I mean that the sdbc driver throws an exception -
> which indicates a lot of errors, but not all of them (e.g. what if something
> is just missing in the new format). In these cases the user could still
> revert the migration manually. Because of that, the above mentioned way to
> revert the migration should be put into a wiki or something.

Right you are and there is a place holder for that on the wiki now. https://wiki.documentfoundation.org/Documentation/FirebirdMigration

I just made sure that I have edit rights there and I do, so I'm willing to help put that page together with you folks, if you'll have me.