Bug 103736 - libreoffice-5.2.2.2 AVG function produces SQL syntax error in query design view
Summary: libreoffice-5.2.2.2 AVG function produces SQL syntax error in query design view
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Base (show other bugs)
Version:
(earliest affected)
5.2.2.2 release
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: Not Assigned
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-11-06 17:42 UTC by jimishol
Modified: 2020-04-28 20:54 UTC (History)
6 users (show)

See Also:
Crash report or crash signature:


Attachments
just for avg testing (3.94 KB, application/vnd.oasis.opendocument.database)
2016-11-07 22:09 UTC, jimishol
Details
103736bug-video for average function_on greek translation (8.35 MB, video/mp4)
2018-03-23 17:50 UTC, jimishol
Details
bt + gdb traces (7.15 KB, text/plain)
2018-04-06 22:25 UTC, Julien Nabet
Details
bts (9.47 KB, application/zip)
2019-02-09 19:23 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description jimishol 2016-11-06 17:42:59 UTC
libreoffice version 5.2.2.2 on gentoo-testing

When designing query with librebase then

average AVG function in query design view, produces a syntax error and one can not run query, save or go to SQL view.

A work around is to use MAX function instead, go to SQL view, change by hand the MAX to AVG and save without going to design view again.

Using wizard for creating query the AVG function does not produce any errors and query is saved and running successfully. you can see it by opening it directly in SQL.
But again if you open the query in design view and try in there the SQL view the problem appears again.

Same behavior with libreoffice-4.3.3.2 on debian-jessie
the error says something (i translate from greek)

Status SQL: HY000
error: 1000
syntax error, unexpected $end, expecting
BETWEEN or IN or SQL_TOKEN_LIKE
Comment 1 Julien Nabet 2016-11-06 18:36:20 UTC
Which DB do you use?
Would you have some example file so we can give it a try?
Comment 2 Robert Großkopf 2016-11-07 15:42:12 UTC
Have tested with 
Version: 5.2.3.3
Build-ID: d54a8868f08a7b39642414cf2c8ef2f228f780cf
CPU-Threads: 4; BS-Version: Linux 4.1; UI-Render: Standard; 
Gebietsschema: de-DE (de_DE.UTF-8); Calc: group

In a internal HSQLDB I created a query, set AVG by GUI to a numeric field. No problem to run a query.

The error, which is reported, is the standard if Base doesn't understand the code. 

@jimishol: Which *.deb-packages do you use? From LO directly or packages of your distribution?
Comment 3 jimishol 2016-11-07 22:09:27 UTC
Created attachment 128557 [details]
just for avg testing

right click query to edit on sql view so as to see sql code
run (f5) to see 2 averages
click view on design view
try run again or save or leave design view
syntax error
Comment 4 jimishol 2016-11-07 23:32:15 UTC
i use packages as it comes from my distributions
and i used HSQLDB

i tried to see if it happens with postgres as well. So on a virtualbox with Arch i installed libreoffice and postgres. Upon query design LO said it needed JRE and i installed jre8-openjdk and problem disappeared!! at HSQLDB too!

Now on gentoo i have virtual/jre-1.8.0-r1 and 
on debian-jessie openjdk-7-jre and default-jre

I had no time to test with different jre s and specially on gentoo it isnt easy because of compilation time it will take

It seems it is a jre matter
Comment 5 jimishol 2016-11-08 14:12:25 UTC
It was not jre matter after all. Sorry.
Since I'm Greek I had user interface in greek language. As soon as I choosed from Tools/Options/Language Settings/User interface: English (USA)
problem disappeared both on debian and gentoo
Comment 6 QA Administrators 2017-05-31 10:51:04 UTC Comment hidden (obsolete)
Comment 7 jimishol 2017-06-25 14:31:08 UTC
I can not update right now, may be later.
i use 
Version: 5.2.5.1
Build ID: Gentoo official package
CPU Threads: 4; OS Version: Linux 4.6; UI Render: default; VCL: gtk2; 
Locale: el-GR (el_GR.utf8); Calc: group

I installed from gentoo portage
I installed app-office/libreoffice-l10n too

When i use Greek interface for libreoffice i get this error
So i use the default English interface where there is no problem
Comment 8 Xisco Faulí 2017-06-25 15:08:01 UTC
Put it back to UNCONFIRMED once you test it, otherwise, better to keep it in NEEDINFO
Comment 9 QA Administrators 2018-01-02 10:14:36 UTC Comment hidden (obsolete)
Comment 10 jimishol 2018-01-15 14:23:39 UTC
Problem fixed by itself at

Version: 5.4.4.2
Build ID: Gentoo official package
CPU threads: 4; OS: Linux 4.13; UI render: default; VCL: gtk2; 
Locale: el-GR (el_GR.utf8); Calc: group

Now i can switch to greek interface with no problem

I mark as resolved-fixed. Please, put the right tag if it is not the right ones.
Comment 11 jimishol 2018-01-15 14:34:24 UTC
I'm deeply sorry. I failed to check properly. On greek interface it continue the same problem when i go to design mode. So  i switch back to english interface. 
I switch to uncorfimed tag, because i have nothing else to add.
Comment 12 Alex Thurgood 2018-01-17 07:47:48 UTC
No repro with:

Version: 5.4.4.2
Build ID: 2524958677847fb3bb44820e40380acbe820f960
Threads CPU : 4; OS : Mac OS X 10.13.2; UI Render : par défaut; 
Locale : fr-FR (fr_FR.UTF-8); Calc: group
Comment 13 Alex Thurgood 2018-01-17 07:54:20 UTC
No repro either with GR langpack :

Έκδοση: 5.4.4.2
Αναγνωριστικό δόμησης: 2524958677847fb3bb44820e40380acbe820f960
Νήματα CPU:4; Λειτουργικό σύστημα: Mac OS X 10.13.2; Απόδοση διεπαφής χρήστη: προεπιλογή; 
Τοπικό: el-GR (fr_FR.UTF-8); Calc: group
Comment 14 Alex Thurgood 2018-02-06 08:19:40 UTC
@jimishol : 5.2 is no longer current.

Given other people's testing without a problem, it appears that your problem is a Gentoo package issue, rather than a problem with a current release of LibreOffice, I am setting this to NOTOURBUG.
Comment 15 Markos Chandras 2018-03-20 11:14:48 UTC
I am reopening this bug since I was able to reproduce it on openSUSE Leap 42.3 using 5.4.5.1
Comment 16 Xisco Faulí 2018-03-20 11:20:21 UTC
(In reply to Markos Chandras from comment #15)
> I am reopening this bug since I was able to reproduce it on openSUSE Leap
> 42.3 using 5.4.5.1

Could you please try to reproduce it with a master build from http://dev-builds.libreoffice.org/daily/master/ ?
You can install it alongside the standard version.
I have set the bug's status to 'NEEDINFO'. Please change it back to 'UNCONFIRMED' if the bug is still present in the master build
Comment 17 jimishol 2018-03-23 17:50:27 UTC
Created attachment 140832 [details]
103736bug-video for average function_on greek translation

On updated Arch i installed libreoffice-still-el package
Comment 18 Buovjaga 2018-04-02 14:59:52 UTC
jimishol, Markos: can you try with an appimage of a daily version: https://www.libreoffice.org/download/appimage
Comment 19 jimishol 2018-04-03 18:41:10 UTC
I tried appimage. It was full libreoffice-still version 5.4.6.2
Same bug, same behavior.
And since i had full version i tried german, italian, french and russian interfaces. None had the greek interface problem.
Comment 20 Buovjaga 2018-04-04 08:32:02 UTC
(In reply to jimishol from comment #19)
> I tried appimage. It was full libreoffice-still version 5.4.6.2
> Same bug, same behavior.
> And since i had full version i tried german, italian, french and russian
> interfaces. None had the greek interface problem.

No, please try with the DAILY VERSION.
Comment 21 jimishol 2018-04-04 12:54:11 UTC
I tried daily and alpha. Both have only English default interface, so I cann't test the Greek interface. Nevertheless, both crashed when i clicked "edit query" even at english interface.
Comment 22 Buovjaga 2018-04-04 13:14:51 UTC
Right, I forgot about the Greekness along the way.
Markos: how were you able to reproduce this? With Greek interface or something else?
Comment 23 Markos Chandras 2018-04-04 13:16:22 UTC
(In reply to Buovjaga from comment #22)
> Right, I forgot about the Greekness along the way.
> Markos: how were you able to reproduce this? With Greek interface or
> something else?

Yep, I did the exact same thing with jimishol
Comment 24 Julien Nabet 2018-04-06 22:17:42 UTC
On pc Debian x86-64 with master sources updated today, I could reproduce this with Greek language.
Comment 25 Julien Nabet 2018-04-06 22:25:47 UTC
Created attachment 141180 [details]
bt + gdb traces

On gdb, I did a ctrl-C when error message box appeared.
Then I used:
thread apply all bt
and finally got this bt

Pb is in OQueryController::translateStatement (see https://opengrok.libreoffice.org/xref/core/dbaccess/source/ui/querydesign/querycontroller.cxx#1683)
Comment 26 Julien Nabet 2018-04-07 06:05:02 UTC
Lionel: I'm not sure but wonder if the pb may be due to the fact that AVG corresponds to a 2 words "Μέσος όρος" in Greek.
Perhaps putting an unbreakable space could make it.
Comment 27 Julien Nabet 2018-04-07 07:13:44 UTC
I confirm putting an unbreakable space worked.

Lionel: do you think it's ok for the unbreakable space or is it just a workaround and tokenization process should be changed? (I couldn't help for this since I know too little about lex and yacc)
Comment 28 jimishol 2018-04-07 15:37:38 UTC
Excuse my  interference, I, as simple user for this kind of problems, use visible Shift+"-" as a pointer that space could cause problems.
May be it is totally irrelevant but the "Μέσος όρος" appears 4 times at https://opengrok.libreoffice.org/xref/translations/source/el/sc/messages.po
Comment 29 Lionel Elie Mamane 2018-04-07 17:03:23 UTC
Try putting adding double quotes around "Μέσος όρος"; it that doesn't work, try simple quotes. I don't know if it works, but worth trying.
If that doesn't work, yes, put an unbreakable space, a dash "-", an underscore "_" or just remove the space and use CamelCase, whatever the Greek translation community thinks is the least horrible.

Thanks
Comment 30 Lionel Elie Mamane 2018-04-07 17:21:18 UTC
I mean, if you solve the bug at a deeper level and truly allow the process to handle spaces, then all the better. To do that, the right place is probably not the yacc/lex grammar per se. We need to find the code that gets the localised string chosen in the menu and maps it back to SQL keywords (which happen to resemble English). That's the point that needs to be fixed. Hopefully it just needs to treat the whole string gotten from the menu as an indivisible string instead of tokenizing it. To get started on that, look where the error is raised and go from there. If there is already Greek at that point, go back and find where that Greek is coming from.

But failing that, at least the non-breaking space will solve this particular bug which is problematic for Greek users.
Comment 31 Julien Nabet 2018-04-07 22:17:53 UTC
I tried to play with m_bGraphicalDesign + impl_setViewMode to call getContainer()->switchView just when needed* + revert back the state, I could retrieve "AVG" instead of the localized name.
However, when trying trying sql, execute, design, execute, ... I don't remember the order, it still failed at a point.
In brief, I can put an unbreakable space in Pootle but can't do more.

*
    688         case ID_BROWSER_QUERY_EXECUTE:
    689             grabFocusFromLimitBox(*this);
    690             if ( getContainer()->checkStatement() )
    691             {
from dbaccess/source/ui/querydesign/querycontroller.cxx (see https://opengrok.libreoffice.org/xref/core/dbaccess/source/ui/querydesign/querycontroller.cxx#692)

Here why I tried this:
68	OUString OQueryViewSwitch::getStatement()
69	{
70	    if(m_pTextView->IsVisible())
71	        return m_pTextView->getStatement();
72	    return m_pDesignView->getStatement();
73	}
(gdb) p m_pTextView->IsVisible()
$11 = false
(gdb) p m_pTextView->getStatement()
$12 = "SELECT AVG( \"table-number\".\"number-field\" ) AS \"number-field\" FROM \"table-number\" GROUP BY \"group\""
(gdb) p m_pDesignView->getStatement()
$13 = "SELECT Moyenne(\"number-field\") AS \"number-field\" FROM \"table-number\" GROUP BY \"group\" "

from
(gdb) bt
#0  0x00007fffc8193a14 in dbaui::OQueryViewSwitch::getStatement() (this=0x55555895b500) at /home/julien/lo/libreoffice/dbaccess/source/ui/querydesign/QueryViewSwitch.cxx:69
#1  0x00007fffc815c6d9 in dbaui::OQueryContainerWindow::getStatement() (this=0x5555580bd0d0) at /home/julien/lo/libreoffice/dbaccess/source/ui/inc/querycontainerwindow.hxx:82
#2  0x00007fffc8158b69 in dbaui::OQueryController::translateStatement(bool) (this=0x555558e7d650, _bFireStatementChange=false)
    at /home/julien/lo/libreoffice/dbaccess/source/ui/querydesign/querycontroller.cxx:1686
#3  0x00007fffc81559ac in dbaui::OQueryController::executeQuery() (this=0x555558e7d650) at /home/julien/lo/libreoffice/dbaccess/source/ui/querydesign/querycontroller.cxx:1246
#4  0x00007fffc8152713 in dbaui::OQueryController::Execute(unsigned short, com::sun::star::uno::Sequence<com::sun::star::beans::PropertyValue> const&) (this=0x555558e7d650, _nId=10721, aArgs=uno::Sequence of length 1 = {...}) at /home/julien/lo/libreoffice/dbaccess/source/ui/querydesign/querycontroller.cxx:690
#5  0x00007fffc7ef54d2 in dbaui::OGenericUnoController::executeChecked(com::sun::star::util::URL const&, com::sun::star::uno::Sequence<com::sun::star::beans::PropertyValue> const&) (this=0x555558e7d650, _rCommand=..., aArgs=uno::Sequence of length 1 = {...}) at /home/julien/lo/libreoffice/dbaccess/source/ui/browser/genericcontroller.cxx:1049

I searched about IsVisible and found Show function in OQueryViewSwitch::impl_postViewSwitch itself called from OQueryViewSwitch::switchView
Comment 32 Julien Nabet 2019-02-09 19:23:18 UTC
Created attachment 149056 [details]
bts

I found where the translation was done.

I retrieved 7 bts:
- 3 when loading query editor (so before typing F5)
- 4 after I typed F5

Interestingly, I noticed that in bt4 we got:
#0  0x00007ffff3bac2a9 in svxform::OSystemParseContext::getIntlKeywordAscii(connectivity::IParseContext::InternationalKeyCode) const
    (this=0x555558dbc620, _eKey=connectivity::IParseContext::InternationalKeyCode::Avg) at /home/julien/lo/libreoffice/svx/source/form/ParseContext.cxx:119
#1  0x00007ffff3bac386 in svxform::OSystemParseContext::getIntlKeyCode(rtl::OString const&) const (this=0x555558dbc620, rToken="Μέσος")
    at /home/julien/lo/libreoffice/svx/source/form/ParseContext.cxx:140
#2  0x00007fffec04b9d0 in connectivity::OSQLScanner::getInternationalTokenID(char const*) const (this=0x5555588aa850, sToken=0x555558766220 "Μέσος")
    at /home/julien/lo/libreoffice/workdir/LexTarget/connectivity/source/parse/sqlflex.cxx:7309
#3  0x00007fffec04b046 in gatherName(sal_Char const*) (text=0x555558766220 "Μέσος")
    at /home/julien/lo/libreoffice/workdir/LexTarget/connectivity/source/parse/sqlflex.cxx:7130
#4  0x00007fffec046bd4 in SQLyylex() () at /home/julien/lo/libreoffice/workdir/LexTarget/connectivity/source/parse/sqlflex.cxx:5884
#5  0x00007fffec04ba9b in connectivity::OSQLScanner::SQLlex() () at /home/julien/lo/libreoffice/workdir/LexTarget/connectivity/source/parse/sqlflex.cxx:7322
#6  0x00007fffec040629 in connectivity::OSQLParser::SQLlex() () at /home/julien/lo/libreoffice/workdir/YaccTarget/connectivity/source/parse/sqlbison.cxx:11158
#7  0x00007fffec03dcd1 in SQLyyparse() () at /home/julien/lo/libreoffice/workdir/YaccTarget/connectivity/source/parse/sqlbison.cxx:10339
#8  0x00007fffec03f237 in connectivity::OSQLParser::parseTree(rtl::OUString&, rtl::OUString const&, bool)
    (this=0x555557b38b80, rErrorMessage="", rStatement="SELECT Μέσος όρος(\"number-field\") AS \"number-field\" FROM \"table-number\" GROUP BY \"group\" ", bInternational=true) at /home/julien/lo/libreoffice/workdir/YaccTarget/connectivity/source/parse/sqlbison.cxx:10921

bt5:
#0  0x00007ffff3bac2a9 in svxform::OSystemParseContext::getIntlKeywordAscii(connectivity::IParseContext::InternationalKeyCode) const
    (this=0x555558dbc620, _eKey=connectivity::IParseContext::InternationalKeyCode::Avg) at /home/julien/lo/libreoffice/svx/source/form/ParseContext.cxx:119
#1  0x00007ffff3bac386 in svxform::OSystemParseContext::getIntlKeyCode(rtl::OString const&) const (this=0x555558dbc620, rToken="όρος")
    at /home/julien/lo/libreoffice/svx/source/form/ParseContext.cxx:140
#2  0x00007fffec04b9d0 in connectivity::OSQLScanner::getInternationalTokenID(char const*) const (this=0x5555588aa850, sToken=0x555558766220 "όρος")
    at /home/julien/lo/libreoffice/workdir/LexTarget/connectivity/source/parse/sqlflex.cxx:7309
#3  0x00007fffec04b046 in gatherName(sal_Char const*) (text=0x555558766220 "όρος")
    at /home/julien/lo/libreoffice/workdir/LexTarget/connectivity/source/parse/sqlflex.cxx:7130
#4  0x00007fffec046bd4 in SQLyylex() () at /home/julien/lo/libreoffice/workdir/LexTarget/connectivity/source/parse/sqlflex.cxx:5884
#5  0x00007fffec04ba9b in connectivity::OSQLScanner::SQLlex() () at /home/julien/lo/libreoffice/workdir/LexTarget/connectivity/source/parse/sqlflex.cxx:7322
#6  0x00007fffec040629 in connectivity::OSQLParser::SQLlex() () at /home/julien/lo/libreoffice/workdir/YaccTarget/connectivity/source/parse/sqlbison.cxx:11158
#7  0x00007fffec03dcd1 in SQLyyparse() () at /home/julien/lo/libreoffice/workdir/YaccTarget/connectivity/source/parse/sqlbison.cxx:10339
#8  0x00007fffec03f237 in connectivity::OSQLParser::parseTree(rtl::OUString&, rtl::OUString const&, bool)
    (this=0x555557b38b80, rErrorMessage="", rStatement="SELECT Μέσος όρος(\"number-field\") AS \"number-field\" FROM \"table-number\" GROUP BY \"group\" ", bInternational=true) at /home/julien/lo/libreoffice/workdir/YaccTarget/connectivity/source/parse/sqlbison.cxx:10921
Comment 33 Julien Nabet 2019-02-10 09:06:03 UTC
In dbaccess/source/ui/querydesign/QueryDesignView.cxx, line 666 from GenerateSelectList method I tried to change this line:
OUStringBuffer aTmpStr2( field->GetFunction());
to:
OUStringBuffer aTmpStr2(field->GetFunction().replaceAll(OUStringLiteral1(" "), OUStringLiteral1(a0)));
to replace plain space by an unbreakable space, still the same result.
I'm stuck :-(
Comment 35 Julien Nabet 2020-04-27 20:45:39 UTC
Eike's response indicates that even in Calc there shouldn't be a space.

Is everyone ok for putting an unbreakable space here?
Comment 36 Julien Nabet 2020-04-28 20:54:23 UTC
I changed it for an unbreakable space but it's been changed again by Dimitris Spingos (dmtrs32) to put "AVERAGE".
Since I'm not Greek, I don't know what to think.
At least, it shouldn't fail so let's put this one to FIXED then.