Bug 143081 - Basic Replace() function crashes LO
Summary: Basic Replace() function crashes LO
Status: RESOLVED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: BASIC (show other bugs)
Version:
(earliest affected)
7.0.2.2 release
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: Julien Nabet
URL:
Whiteboard: target:7.3.0 target:7.2.0.0.beta2 tar...
Keywords: haveBacktrace
Depends on:
Blocks:
 
Reported: 2021-06-26 09:42 UTC by Robert Großkopf
Modified: 2021-07-13 09:06 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:
Regression By:


Attachments
Try to document this database with BaseDocumenter (213.97 KB, application/vnd.oasis.opendocument.database)
2021-06-28 14:57 UTC, Robert Großkopf
Details
Screenshot of BaseDocumenter extension in action (224.81 KB, image/png)
2021-06-29 08:04 UTC, Alex Thurgood
Details
bt with debug symbols (11.96 KB, text/plain)
2021-06-30 20:29 UTC, Julien Nabet
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Großkopf 2021-06-26 09:42:55 UTC
For Base there is a extension Base Documenter available.

The extension works well with LO 6.4.7.2 and fails with all LO 7.x-versions I installed here (LO 7.0.5.2 and LO 7.1.4.2 on OpenSUSE 15.2 64bit rpm Linux).

It will crash LO immediately when trying to open any form to read the content of the form.
Comment 1 Julien Nabet 2021-06-26 21:05:58 UTC
On pc Debian x86-64 with master sources updated today, it fails to create a BaseDocumenter repository because of the error
"Object variable not set" there:

Dim iTables As Integer, bReinit As Boolean, iMaster As Integer

	InitRepository = False
	If IsNull(Repository) Then GoTo Exit_Function

	With Repository
		'Identify if repo should be (re)initialized
		bReinit = False
		iTables = .TableDefs().Count
		If iTables = 0 Then
			bReinit = True
		ElseIf iTables = 2 Then
			'Do nothing if the database is not a repository !!
			If .TableDefs(0).Name <> "DATABASES" Or .TableDefs(1).Name <> "OBJECTS" Then Set Repository = Nothing
		Else
			Set Repository = Nothing
		End If

		'Check if a master database record exists
		If Not bReinit Then
   iMaster = .DCount("ID", "DATABASES", "ID=0") <<<<<<< THERE


So can't test the crash.
Comment 2 Robert Großkopf 2021-06-27 06:34:00 UTC
(In reply to Julien Nabet from comment #1)
Have had the same error sometimes ago. Didn't know what has solved the problem. Did you register the database BaseDocumenter.odb?
Comment 3 Julien Nabet 2021-06-27 14:55:50 UTC
I gave a new try.
This time I installed Access2Base since it seems used by Base Documenter.

Then I tried to initialize repository following http://www.access2base.com/basedocumenter/basedocumenter.html.

1) with use of creating automatically new repository, I must recognize I don't understand why it would need to disable macros but I tested both, still no response with security at high or low.

2) I created manually a database (HSQLDB) and registered it.
Then I tried "Open the repository" in BaseDocumenter menu
=> The repository could not be opened or initialized.


Some error/warn/info message during repo generation + info about why disabling macro during repo generation would be useful.
Comment 4 Julien Nabet 2021-06-27 14:56:50 UTC
Jean-Pierre: since the extension Base Documenter requires Access2Base, thought you might be interested here.
Comment 5 Robert Großkopf 2021-06-27 15:13:38 UTC
(In reply to Julien Nabet from comment #3)
> I gave a new try.
> This time I installed Access2Base since it seems used by Base Documenter.

Yes, without Access2Base it won't work.

Then you have to create a folder. You have to unzip http://www.access2base.com/basedocumenter/_download/Templates.zip in this folder.

This should be all. BaseDocumenter could be seen in the menue of Base. You have to create a new "Sammel-DB" (don't know it in English). This should be created better 
from a database, which doesn't contain macros.

The database you will document could be documented with enabled macros. Works here with LO 6, but not with LO 7.*
Comment 6 Julien Nabet 2021-06-27 15:43:49 UTC
I removed the registered BaseDocumenter + extension BaseDocumenter and started over.
I installed again BaseDocumenter (this time "only for me" instead of "For all users").
I created a new dir: /home/julien/Documents/testBaseDocumenter
I downloaded Templates.zip and unzip it there
=> so I got jquery, BD_Template.html, etc. in this dir.

Then I launched LO and tried again the menu "new Repository", it still failed with the same error.

Just for the record, at another moment (I don't remember exactly when), I had another popup message:
Public Function _BD_ScriptPython(ByVal psFunction As String) As Object
'	Return the script to invoke to call the given Python function
'	https://wiki.openoffice.org/wiki/Documentation/DevGuide/Scripting/Scripting_Framework_URI_Specification

	'Manual storage of python script
	'Set _BD_ScriptPython = _BD_.ScriptProvider.getScript("vnd.sun.star.script:BaseDocumenter/BD_json.py$" & psFunction & "?language=Python&location=user")

	On Local Error GoTo Install_ForAllUsers
	'Storage from extension => Install "only for me"
	Set _BD_ScriptPython = _BD_.ScriptProvider.getScript( _
	"vnd.sun.star.script:BaseDocumenter.oxt|python|BD_json.py$" & psFunction & "?language=Python&location=user:uno_packages") ================ Error here

Exit_Function:
	Exit Function
Install_ForAllUsers:
	'Storage from extension => Install "for all users"
	On Local Error GoTo Error_NoPython
	Set _BD_ScriptPython = _BD_.ScriptProvider.getScript( _
	"vnd.sun.star.script:BaseDocumenter.oxt|python|BD_json.py$" & psFunction & "?language=Python&location=share:uno_packages")
	GoTo Exit_Function
Error_NoPython:
	_BD_ForceExecutionStop(_BD_GetLabel("ERR" & ERRINTERNAL))
End Function	'	_BD_ScriptPython

(I copied it by hand since I couldn't copy paste the error message).
Comment 7 Robert Großkopf 2021-06-27 16:07:15 UTC
I haven't tested to create the database with LO 7.*. I had installed it with LO 6.2 or something else and only used it with LO 7.*

The database will only be an empty database, which is registered as "BaseDocumenter" in LO.

I had described the steps I made (only in German) here: https://www.familiegrosskopf.de/robert/index.php?&Inhalt=base_documenter
Comment 8 Julien Nabet 2021-06-27 16:49:30 UTC
Ok certainly someone may reproduce this.
Comment 9 Alex Thurgood 2021-06-28 07:50:30 UTC
With Version: 7.1.2.2 / LibreOffice Community
Build ID: 8a45595d069ef5570103caea1b71cc9d82b2aae4
CPU threads: 8; OS: Mac OS X 10.16; UI render: default; VCL: osx
Locale: fr-FR (fr_FR.UTF-8); UI: fr-FR
Calc: threaded

As per the documentation, I opened an existing database without any embedded macros, then I go to the BaseDocumenter menu, and choose Create New Repository (Nouveau Référentiel in French).
The dialog asks me to select a location for the new empty database. I chose a folder named BaseDocumenter that is writable.  

However, the macro fails to complete correctly with a Basic execution error :

Erreur d'exécution BASIC.
Une exception s'est produite : 
Type: com.sun.star.script.provider.ScriptFrameworkErrorException
Message: <class 'KeyError'>: 'BaseDocumenter.oxt'
  File "/Applications/LibreOffice.app/Contents/Resources/pythonscript.py", line 1046, in getScript
    storageUri = self.provCtx.getStorageUrlFromPersistentUrl(
  File "/Applications/LibreOffice.app/Contents/Resources/pythonscript.py", line 417, in getStorageUrlFromPersistentUrl
    package = self.mapPackageName2Path[ packageName ]

I see that the very same error is already reported on the Extensions web page for the BaseDocumenter extension (5 months ago).

I don't have a 6.4.x version anymore to test with to see if the extension setup works there.
Comment 10 Jean-Pierre Ledure 2021-06-28 10:57:37 UTC
Hi all,

I could not reproduce the crash described by Robert but could reproduce the behaviour described by Alex.

The cause of the latter incident is that the file downloaded from the extensions web site is called
    basedocumenter.oxt
while it should be 
    BaseDocumenter.oxt

I had never uploaded a file with a name in lower case, but the extensions site has been revamped/redesigned in the mean time.

The wrong name has as unexpected side-effect that the installation creates somewhere a subdirectory
    .../uno_packages/lu...tmp/basedocumenter.oxt
i.o. .../BaseDocumenter.oxt. This prevents the software to find a Python piece of code and causes the error.

Most extensions do not care about the name of that directory and, additionally, the error probably does not happen in Windows that ignores lower/upper case in file names. Anyway I discovered the file name change only today ...

May I suggest to Robert to re-test after having:
- uninstalled the extension
- downloaded the extension
- renamed the received file with correctly cased characters
- reinstalled the extension
?

Thanks to all for the tests done.
Comment 11 Robert Großkopf 2021-06-28 14:57:49 UTC
Created attachment 173249 [details]
Try to document this database with BaseDocumenter

I have tested it now with a simple database without any macros and a database, which contains macros. The attached database couldn't be documented here with LO 7.0.5 and newer database. No difference if I allow to execute macros or don't allow. The simple database could documented with the newer versions of LO.

If I start BaseDocumenter in LO 7.0.0 or 7.0.1 I get "Error #13 (Datentypen unverträglich) aufgetreten in Zeile 486 in Tabledef._PropertyGet". BaseDocumenter won't start in this versions. With LO 7.0.2 it started but crashes during documentation.
Comment 12 QA Administrators 2021-06-29 03:49:16 UTC Comment hidden (obsolete)
Comment 13 Alex Thurgood 2021-06-29 07:57:01 UTC
(In reply to Jean-Pierre Ledure from comment #10)

 
> May I suggest to Robert to re-test after having:
> - uninstalled the extension
> - downloaded the extension
> - renamed the received file with correctly cased characters
> - reinstalled the extension
> ?

1) I removed the package using the GUI Extension Manager, then restarted LO.

2) Although the UI indicated that the extension wasn't present, a verification check with unopkg from the command line showed that it was still installed.

3) To force removal, I used unopkg remove BaseDocumenter from the command line. Restarted LO.


4) Then I renamed the OXT file to BaseDocumenter.oxt instead of the original all lower case characters name.

5) Next, I tried re-installing the extension via the GUI Extensions Manager. The installation hung, forcing me to force quit LO.

6) Restarted LO to find that the extension showed up in the Extension Manager, but didn't contain any menu entries when a Base file was loaded.

7) Removed the extension again via unopkg, and reinstalled via unopkg. Restarted LO. Now, the extension was finally reinstalled and I could see the corresponding menu entry.

8) Opened an ODB file. Created a new BaseDocumenter.odb file.

9) With the previously opened ODB file open on the desktop, chose Document this database from the BaseDocumenter menu - no crash for me.

From my testing on macOS with 7.1.2.2, this is WFM.
Comment 14 Alex Thurgood 2021-06-29 07:58:58 UTC
I should add that the initial test of the extension was only on the available Tables object, I still need to try a database containing Forms.
Comment 15 Alex Thurgood 2021-06-29 08:04:33 UTC
Testing with an ODB file containing forms seems to work too, cf. attached screenshot.
Comment 16 Alex Thurgood 2021-06-29 08:04:59 UTC
Created attachment 173263 [details]
Screenshot of BaseDocumenter extension in action
Comment 17 Robert Großkopf 2021-06-29 13:19:31 UTC
(In reply to Alex Thurgood from comment #15)
> Testing with an ODB file containing forms seems to work too, cf. attached
> screenshot.

Could you please test it with the attached database?

With the simple example-database of the Base handbook ("Example_Sports.odb") ist will work here. But with the attached database it will fail every time with any LO 7.*-version.
Comment 18 Jean-Pierre Ledure 2021-06-29 14:13:52 UTC
Crash is confirmed in version

Version: 7.1.3.2 / LibreOffice Community
Build ID: 47f78053abe362b9384784d31a6e56f8511eb1c1
CPU threads: 6; OS: Linux 5.4; UI render: default; VCL: kf5
Locale: en-US (en_US.UTF-8); UI: en-US
Calc: threaded

with the provided database.

It limit the scope of the crash more precisely: it happens when removing all forms and reports, and all queries except "Rechnung_Bericht".

The run Basic routine is parsing and beautifying the SQL statement stored in that single query (when deselecting "Beautify SQL ?" i the settings, the crash does not happen).

I have no idea so far what is specific to that query to cause the crash.
I will try to identify the statement that causes it. "Try" because not obvious.

In the mean time I change the status to "NEW".
Comment 19 Jean-Pierre Ledure 2021-06-29 14:37:02 UTC
When I replace in above-mentioned query "Rechnung_Bericht" the identifier
    "Straße"
with
    "Strasse",
the crash does not happen.

The root cause however (Unicode in LO, Unicode in Basic, Unicode in HSQLDB ?) remains unknown.
Comment 20 Robert Großkopf 2021-06-29 14:56:16 UTC
(In reply to Jean-Pierre Ledure from comment #19)
> When I replace in above-mentioned query "Rechnung_Bericht" the identifier
>     "Straße"
> with
>     "Strasse",
> the crash does not happen.

Field "Straße" is part of table "Kunde". I tried to create a view from the query and deleted the query but the crash will appear also.

Don't know what is running better here with LO 6.* instead of LO 7.*
Comment 21 Jean-Pierre Ledure 2021-06-30 10:57:17 UTC
I could reproduce the crash with a minimalist scenario.

It has nothing to do with Base or BaseDocumenter.

Run simply next Basic statements in any empty document:

Sub Main
Dim a As String, b As String
	a = """Straße"""
	b = Replace(a, """", "&quot;")
End Sub

LO crashes instantly.

I presume it has to do with the fact that
- the replacement is by default case-insensitive
- the upper case of character "ß" is 2 characters long, and not only 1

I change the title of the bug to "Basic Replace() function crashes LO"
Comment 22 Robert Großkopf 2021-06-30 15:38:17 UTC
(In reply to Jean-Pierre Ledure from comment #21)
> I could reproduce the crash with a minimalist scenario.
> 
> It has nothing to do with Base or BaseDocumenter.
> 
> Run simply next Basic statements in any empty document:
> 
> Sub Main
> Dim a As String, b As String
> 	a = """Straße"""
> 	b = Replace(a, """", "&quot;")
> End Sub
> 
> LO crashes instantly.
> 
> I presume it has to do with the fact that
> - the replacement is by default case-insensitive
> - the upper case of character "ß" is 2 characters long, and not only 1
> 
> I change the title of the bug to "Basic Replace() function crashes LO"

Following this comment I change the component to BASIC.
Comment 23 Julien Nabet 2021-06-30 20:29:29 UTC
Created attachment 173298 [details]
bt with debug symbols

On pc Debian x86-64 with master sources updated today, I got an assertion by following Jean-Pierre's example in comment 21.
Comment 24 Julien Nabet 2021-06-30 21:16:05 UTC
I gave a try with https://bugs.documentfoundation.org/show_bug.cgi?id=143081
Comment 25 Commit Notification 2021-07-01 06:39:06 UTC
Julien Nabet committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/4a0b40f1be9f6773c8ebc5331c257911a76a5cee

tdf#143081: fix Basic Replace() function crashes LO

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 26 Commit Notification 2021-07-01 14:06:58 UTC
Julien Nabet committed a patch related to this issue.
It has been pushed to "libreoffice-7-2":

https://git.libreoffice.org/core/commit/89e19634a775d53ea855db8767113f3ab08a3479

tdf#143081: fix Basic Replace() function crashes LO

It will be available in 7.2.0.0.beta2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 27 Commit Notification 2021-07-01 15:35:41 UTC
Julien Nabet committed a patch related to this issue.
It has been pushed to "libreoffice-7-1":

https://git.libreoffice.org/core/commit/bf3c4e49a48178849f75e6449ec47ab3ff456878

tdf#143081: fix Basic Replace() function crashes LO

It will be available in 7.1.5.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 28 Commit Notification 2021-07-02 17:37:47 UTC
Xisco Fauli committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/79657c9f5392a9c243d3f3c14fa190b04b6806e9

tdf#143081: basic_macros: Add unittest

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 29 Commit Notification 2021-07-13 09:06:28 UTC
Xisco Fauli committed a patch related to this issue.
It has been pushed to "libreoffice-7-2":

https://git.libreoffice.org/core/commit/fadccf7e6e9a46c54de9578dd708623495ba8e80

tdf#143081: basic_macros: Add unittest

It will be available in 7.2.0.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.