Bug 46259 - rewrite mysql connector to use mariadb client library
Summary: rewrite mysql connector to use mariadb client library
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: LibreOffice (show other bugs)
Version:
(earliest affected)
Master old -3.6
Hardware: Other All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords: difficultyInteresting, easyHack, skillCpp, skillSql, topicCleanup
: 88965 (view as bug list)
Depends on:
Blocks: Database-Connectivity
  Show dependency treegraph
 
Reported: 2012-02-18 11:52 UTC by Michael Meeks
Modified: 2019-07-29 12:19 UTC (History)
16 users (show)

See Also:
Crash report or crash signature:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Meeks 2012-02-18 11:52:58 UTC
The MariaDB chaps have a functional equivalent, API idential (but not GPL) connector library for MySQL.

Unfortunately, it is 'C' only - so we need to re-write our mysql connector, to target that C library instead of the C++ one.

That means a fair bit of hackery in mysqlc/ and some re-writing action there, to use 'libmysql' and not 'libmysqlcppcon'.

Having done that, we can then use (and presuambly distribute) MariaDB's connector.

Thanks.
Comment 1 Julien Nabet 2012-03-17 15:37:02 UTC
Just some questions about Mysql/MariaDB :

1)
- do we want to replace Mysql connector by MariaDB connector or do we want both ? 2 things in mind :
a) should the bugs concerning Mysql connector should be still investigated ?
b) If we only keep MariaDB, with the time there'll api/compatibility pb to connect Mysql DB.

2) Could MariaDB connector licence be a pb for LO ? (http://kb.askmonty.org/en/what-is-mariadb-53 indicates CC BY-SA / Gnu FDL).
Comment 2 Lionel Elie Mamane 2012-03-18 23:50:34 UTC
(In reply to comment #1)
> 1)
> - do we want to replace Mysql connector by MariaDB connector or do we want both
> ? 2 things in mind :
> a) should the bugs concerning Mysql connector should be still investigated ?
> b) If we only keep MariaDB, with the time there'll api/compatibility pb to
> connect Mysql DB.

No, a single connector should suffice. The MariaDB guys commit to maintaining on-the-wire binary compatibility. That is, linking to the MariaDB client library, one can connect to MySQL.

> 2) Could MariaDB connector licence be a pb for LO ?
> (http://kb.askmonty.org/en/what-is-mariadb-53 indicates CC BY-SA / Gnu FDL).

That's the licence of the webpage itself.
Comment 3 Florian Reisinger 2012-05-18 09:21:10 UTC
Das Dreifach-Date
Comment 4 Anthony Youngman 2013-03-04 19:56:44 UTC
Looking for database easyhacks, I found this (not as easy as it seems for me :-)

BUT. A cursory glance implies it has been rebased to the ASL and as such is now MPL-licenced. And I think the original problem is it was (L)GPL of the wrong sort? The MariaDB connector says it is LGPL 2.1+.

Anyways, does that mean this bug is now moot and can be closed thanks to the licence change of the LO code? If so, do I just change status to resolved?
Comment 6 Anthony Youngman 2013-03-04 20:38:04 UTC
Julien: Ah, I think I've sussed what you're saying.

At present the existing C++ connector talks to the MySQL GPL'd connector. We can't incorporate and distribute the MySQL code.

But if we change it to a C api, we can incorporate and distribute the MariaDB LGPL connector. Okay.

As a good post-relational-database and (if rusty) C programmer, I'm looking at db easyhacks to try and get into working on the lo codebase. But I'm in a bit over my depth with the C++. Is there anywhere I can get a LOT of handholding to start with without annoying the regular devs? Or am I best just struggling on chatzilla in the dev forum anyway?
Comment 7 Julien Nabet 2013-03-04 20:50:33 UTC
Anthony:
I'm not a core dev but if you want to hack, you should start by this page:
https://wiki.documentfoundation.org/Development

You must know that most of LO is in C++ but there is also C, Python, Java, assembly and Perl.
About getting help, a good link here (for users and devs): http://www.libreoffice.org/get-help/
Comment 8 Lionel Elie Mamane 2013-03-05 05:06:14 UTC
(In reply to comment #6)

> As a good post-relational-database and (if rusty) C programmer, I'm looking
> at db easyhacks to try and get into working on the lo codebase. But I'm in a
> bit over my depth with the C++. Is there anywhere I can get a LOT of
> handholding to start with without annoying the regular devs? Or am I best
> just struggling on chatzilla in the dev forum anyway?

"Without annoying the regular devs" I'm not sure, but we as a bunch are usually willing to invest time into getting a new contributor :)

In the past, Fridrich (which I just added to CC) orally expressed some interest in getting into this bug. He has the C++ knowledge, but lacks the relational/SQL database knowledge, which you have, so maybe the two of you would make a good team to collaborate on this bug.

I'll be available for any issue/question you two may have while working on this bug.

As to MariaDB/MySQL-specific knowledge, such as "how to list all tables", "how to list all columns of a table and their type", "how to get the current user's permissions on this table", etc, I see two roads to getting it:

1) Take it from the JDBC connector
   https://kb.askmonty.org/en/mariadb-faq-is-there-any-java-connector-for-mariadb-than-the-one-from-mysql/
   As the LibreOffice (SDBC(X)) connector is supposed to implement a JDBC-like interface for the LibreOffice dbms-independent code, this could be a good match.

2) Try to get some MariaDB guy as an oracle for that kind of questions (sometimes the answer will be "the C api has a list_tables call", sometimes the answer will be "execute this SQL: SELECT table_name FROM information_schema.tables, or SELECT table_name FROM mysql.tables WHERE tables.database='THE_SCHEMA_YOU_WANT'" or some such.
Comment 9 Fridrich Strba 2013-03-05 07:40:56 UTC
Lionel, Anthony: Yes, I am thrilled by the possibility to see that done. Nevertheless, since FOSDEM where we discussed it with Lionel, I did not have time to put myself on it. I had many little issues in Visio and CorelDraw importers that were eating my time.
I propose the following: 1) first to replace the mysqlc library with mariadb c library, leaving in the cpp layer; 2) rewrite the cpp layer using the lgpl-ed jdbc connector to mariadb and the knowledge embedded in it. I am ready to find some time to get the (1) done soon-ish.
Comment 10 Anthony Youngman 2013-03-05 21:22:45 UTC
Okay. I'm digging into this (and getting myself thoroughly confused :-). I've downloaded the MariaDB source and am digging into it. I've also read the readme in LO in the mysqlc directory.

The readme says you need to pull in libmysql and libmysqlcppcon - I guess the idea behind this bug is to use MariaDB's libmysql rather than MySQL's libmysqlcppcon? OUCH!

Because if, as I think, we want to try and remove all GPL-only code, then this is a waste of time. Because I've been looking at the source of libmysql in MariaDB and the headers say GPL. Okay, the EXCEPTIONS-CLIENT file applies, which allows FLOSS code under GPL-incompatible licences to be linked, but this will then blow up with a vengeance if somebody wants to take advantage of the MPL and mix it with closed code!

What next?
Comment 11 Michael Meeks 2013-03-05 21:48:45 UTC
Hi Anthony,

> The readme says you need to pull in libmysql and libmysqlcppcon -
> I guess the idea behind this bug is to use MariaDB's libmysql
> rather than MySQL's libmysqlcppcon? OUCH!

That's right. So the hope is to use this guy:

https://kb.askmonty.org/en/mariadb-client-library-for-c/

instead (I think) - which is (last I looked) LGPL licensed. That should replace the libmysql I believe without the compound oddness issue, and let us distribute it [ as/when we've nailed libmysqlcppcon ].

Thanks so much for digging into this though ! :-)
Comment 12 Lionel Elie Mamane 2013-03-05 22:43:39 UTC
(In reply to comment #10)
> I've also read the readme in LO in the mysqlc directory.

I've recently significantly updated it; make sure to read the one in the master branch, not the stable (libreoffice-4-0) branch.

See e.g. http://cgit.freedesktop.org/libreoffice/core/plain/mysqlc/README

> The readme says you need to pull in libmysql and libmysqlcppcon - I guess
> the idea behind this bug is to use MariaDB's libmysql rather than MySQL's
> libmysqlcppcon? (...) I've been looking at the source of libmysql
> in MariaDB and the headers say GPL.

The idea is to use the MariaDB Client Library for C, which is LGPL-licensed, not GPL. It has the same API as libmysql (which is GPL-licensed), though. And yes, the idea is to remove libmysqlcppconn from the picture.

MariaDB Client Library for C:
 https://kb.askmonty.org/en/mariadb-client-library-for-c/
It is a separate download, not part of the MariaDB server sources (and it is distinct from the copy/fork of libmysql that is in the MariaDB server sources).
Comment 13 Fridrich Strba 2013-03-06 16:31:58 UTC
OK, I just switched LO to use libmariadb and I added a libmariadb module where a static library (lib)mariadblib is built in case internal mariadb is requiered. There is still the mysqlcppconn in the play in this scenario, so whoever wants can try to ditch it out by rewriting using the way Lionel outlined.
Comment 14 Lionel Elie Mamane 2013-03-18 09:10:17 UTC
There is now a C++ connector for MariaDB. Dunno if it is in an usable state yet, but we could consider being the "launch customer" for it.

See https://kb.askmonty.org/en/c-client-library-mariadb/
Comment 15 Lionel Elie Mamane 2013-04-15 08:27:28 UTC
There is also this older C++ MySQL API wrapper: http://tangentsoft.net/mysql++/

Does not seem extremely active, but there is a commit to their SVN "7 weeks ago", so not dead :)
Comment 16 Björn Michaelsen 2013-10-04 18:46:59 UTC
adding LibreOffice developer list as CC to unresolved EasyHacks for better visibility.

see e.g. http://nabble.documentfoundation.org/minutes-of-ESC-call-td4076214.html for details
Comment 17 Matthew Pottage 2014-07-31 19:44:26 UTC
I've started working on this. At the moment I am investigating the codebase and how much it depends on the MySQL C++ Library.
Comment 18 Michael Stahl (allotropia) 2014-07-31 21:45:31 UTC
good to see somebody interested in this!

by the way i notice a ridiculously large patch in external/libmariadb;
probably there's a new release by now that would include that, if
anybody has time to upgrade the bundled libmariadb  :)
Comment 19 Matthew Pottage 2014-08-23 17:53:08 UTC
Sorry, I can no longer work on this (school summer holidays are ending). Hopefully, at some time in the future, I will be able to work on it.
Comment 20 Alex Thurgood 2015-01-03 17:39:19 UTC Comment hidden (no-value)
Comment 21 Lionel Elie Mamane 2015-10-25 15:55:46 UTC
*** Bug 88965 has been marked as a duplicate of this bug. ***
Comment 22 Robinson Tryon (qubit) 2015-12-14 04:53:17 UTC Comment hidden (obsolete)
Comment 23 Robinson Tryon (qubit) 2016-02-18 14:52:19 UTC Comment hidden (obsolete)
Comment 24 zaxebo zaxebo 2016-05-03 11:33:46 UTC
 "MariaDB Connector/C" library is used to connect applications developed in C/C++ to MariaDB and MySQL databases BOTH. It is written in C language.

VERY IMPORTANTLY, It is LGPL licensed.

It is well maintained library. Its version 2.2.3 came on 26 April 2016

Please see:
https://mariadb.com/kb/en/mariadb/mariadb-connector-c/

So this library "MariaDB Connector/C" can be used in libreoffice, for the above mentioned purpose as stated in the filed bug report
Comment 25 Julien Nabet 2016-05-03 11:43:27 UTC
(In reply to zaxebo zaxebo from comment #24)
>  "MariaDB Connector/C" library is used to connect applications developed in
> C/C++ to MariaDB and MySQL databases BOTH. It is written in C language.
...
> 
> So this library "MariaDB Connector/C" can be used in libreoffice, for the
> above mentioned purpose as stated in the filed bug report

If I don't misunderstand the previous comment, it seems we need a C++ lib, not a C lib.
So for the moment, we're stuck.
Comment 26 Lionel Elie Mamane 2016-05-03 11:52:34 UTC
(In reply to Julien Nabet from comment #25)
> If I don't misunderstand the previous comment, it seems we need a C++ lib,
> not a C lib.
> So for the moment, we're stuck.

We can use the C library directly. It is just more work on our side.
Comment 27 zaxebo zaxebo 2016-05-03 13:39:58 UTC
native database driver API for libreoffice is called "SDBC"

for example: SDBC driver for postgresql is at
https://wiki.documentfoundation.org/PostgreSQL-SDBC

HENCE Now the task is that:
 In the same way just WRITE THE "SDBC driver wrapper" over  "MariaDB Connector/C" library mentioned in my comment 24.
 https://bugs.documentfoundation.org/show_bug.cgi?id=46259#c24
Comment 28 Julien Nabet 2017-09-24 16:04:40 UTC
Just to be sure.
Now here's the chain:
SDBC part: <LO core>/mysqlc
  -> MySql C++ connector : <LO core>/external/mysql-connector-cpp
     -> MariaDB C connector : <LO core>/external/mariadb-connector-c

1) Is it right?
2) Have we still got license pb?
3) Do we want to rewrite SDBC part to directly call MariaDB C connector as zaxebo indicated in previous comment?
Unless https://launchpad.net/mariadb++ may interest us? (license is:
Boost Software License (BSL) http://www.boost.org/users/license.html
(I don't know if it's ok for LO))

The goal is to be sure the bugtracker title corresponds to what we want.
Comment 29 Michael Meeks 2017-09-25 08:54:14 UTC
Ooh - so, the MariaDB++ thing looks rather interesting indeed ... my hope is that it is reasonably compatible; if so - then porting from the libmysql <-> mariadb++ thing -may- be reasonably easy. Well worth checking out the launchpad code and poking at it I guess ...
Comment 30 Julien Nabet 2017-09-25 10:06:10 UTC
(In reply to Michael Meeks from comment #29)
> Ooh - so, the MariaDB++ thing looks rather interesting indeed ... my hope is
> that it is reasonably compatible; if so - then porting from the libmysql <->
> mariadb++ thing -may- be reasonably easy. Well worth checking out the
> launchpad code and poking at it I guess ...

I just noticed dates and it seems quite old and unmaintained :-(
I thought it could be due to the fact that the code has been migrated toward other website but found nothing.
I only noticed this https://github.com/Leandros/MariaDB-Connector-Cxx but it's unrelated to the original connector and no commit since 2 years.

So it seems we must stick to keep both connectors c++ -> C and C -> DB
Comment 31 Lionel Elie Mamane 2017-09-25 10:36:51 UTC
(In reply to Julien Nabet from comment #28)
> Just to be sure.
> Now here's the chain:
> SDBC part: <LO core>/mysqlc
>   -> MySql C++ connector : <LO core>/external/mysql-connector-cpp
>      -> MariaDB C connector : <LO core>/external/mariadb-connector-c
> 
> 1) Is it right?

yes

> 2) Have we still got license pb?

yes, at the MySQL C++ connector level

> 3) Do we want to rewrite SDBC part to directly call MariaDB C connector as
> zaxebo indicated in previous comment?

<shrug>

If we have a *qualified* volunteer willing to essentially replicate MySQL C++ connector (modulo a few corner cases, that is what the work would be, because MySQL C++ connector and SDBC are nearly the same API, both being a C++-isation of JDBC), do it without introducing too many bugs, etc... sure, we take patches.
Comment 32 Hanno Meyer-Thurow 2017-12-03 12:35:15 UTC
JFYI, there is another thin C++ wrapper[0] for "MariaDB Connector/C" library licensed LGPL. Though, I am with zaxebo to have a native SDBC driver and wonder if this C++ wrapper helps there.

See comment 24, comment 27.

[0] https://github.com/Roslaniec/MariaCpp
Comment 33 Komal 2019-01-09 03:16:04 UTC
Hey, I am new to LO. I want to contribute. Is this issue still open? If at all, can I work? How do I start about this? Thanks!
Comment 34 Xisco Faulí 2019-01-28 10:55:18 UTC
Hello Michael Meeks,
do you think we can close this issue as FIXED after Tamas' work in https://cgit.freedesktop.org/libreoffice/core/commit/?id=3478d7453a3d65b3d8d164e8f898a0b79f005c58 and the consecutive mysqlc commits -> https://cgit.freedesktop.org/libreoffice/core/log/?qt=grep&q=mysqlc ?
BTW, should it be mentioned in the 6.2 release notes ?
Comment 35 Michael Meeks 2019-01-28 10:56:38 UTC
Yes - good point; thanks Xisco - and yes it should be mentioned in the release notes - of course ! =) would be appreciated if you can add it.
Comment 36 Xisco Faulí 2019-01-28 11:25:08 UTC
(In reply to Michael Meeks from comment #35)
> Yes - good point; thanks Xisco - and yes it should be mentioned in the
> release notes - of course ! =) would be appreciated if you can add it.

Done in https://wiki.documentfoundation.org/ReleaseNotes/6.2#Base.