Bug 107501

Summary:	Convert manual tests to automated UI tests
Product:	LibreOffice	Reporter:	Markus Mohrhard <markus.mohrhard>
Component:	LibreOffice	Assignee:	Not Assigned <libreoffice-bugs>
Status:	RESOLVED FIXED
Severity:	normal	CC:	ilmari.lauhakangas, mentoring, xiscofauli
Priority:	medium	Keywords:	difficultyBeginner, easyHack, skillPython
Version:	unspecified
Hardware:	All
OS:	All
Whiteboard:	target:6.1.0
Crash report or crash signature:		Regression By:

Description Markus Mohrhard 2017-04-28 10:48:54 UTC

The new structured manual testing instance at https://manual-test.libreoffice.org suffers again from the problem that many of the manual tests could easily be automated tests.

Since the 5.3 and even more since the 5.4 release we have an automated UI testing framework that allows us to handle even more as automated tests.

This task is about converting existing manual tests to automated UI tests.

The UI testing framework can be found at https://cgit.freedesktop.org/libreoffice/core/tree/uitest with some additional tests being at https://cgit.freedesktop.org/libreoffice/core/tree/sc/qa/uitest

https://cgit.freedesktop.org/libreoffice/contrib/dev-tools/tree/uitest/execute.sh contains a script that allows executing UI tests outside of the test framework with the UI visible. It helps to add time.sleep into the test to see the state after specific actions.

https://mmohrhard.wordpress.com/2016/10/03/libreoffice-ui-test-tutorial-part-1-adding-a-simple-test/ contains a short introduction into writing UI tests.


Depending on the manual test you'll find a number of items that are currently not possible to do with the UI testing framework. Please list them here and push the incomplete test to gerrit and I will try to add the support for missing pieces on the LibreOffice side.

Comment 1 Xisco Faulí 2017-04-28 10:59:29 UTC

List of .uno commands: https://wiki.documentfoundation.org/Development/DispatchCommands

Comment 2 Markus Mohrhard 2017-04-28 11:01:04 UTC

Even better: by setting LO_COLLECT_UIINFO="log.txt" in the terminal you get a log with many UI actions including the uno commands in instdir/uitest.

Comment 3 Markus Mohrhard 2017-05-14 20:20:18 UTC

At least the following tests should not be converted to UI tests:

EN-5:Array formula that references a cell
EN-7:Named range robustness
EN-9:Recalculation of grouped formulas
EN-10:Array formula

These need to be converted to automated tests in sc's ucalc.

Comment 4 Buovjaga 2018-01-29 07:38:38 UTC

There has been some confusion on how to get an account at TestLink. Currently the way is to register at our transitory single sign-on site: https://user.documentfoundation.org/
As the site is transitory, expect this to change in the future.

Comment 5 Commit Notification 2018-02-05 08:47:34 UTC

Saurav Chirania committed a patch related to this issue.
It has been pushed to "master":

http://cgit.freedesktop.org/libreoffice/core/commit/?id=b57ad983e6ed5d976903b1d6588b472bb3051e7d

tdf#107501 automatize EN-8 manual test, related tdf#84810

It will be available in 6.1.0.

The patch should be included in the daily builds available at
http://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
http://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.

Comment 6 Saurav Chirania 2018-02-25 17:36:37 UTC

I was not able to use the given execute.sh script.

Since there is no file named python in my instdir/program directory, I got the following error:
https://pastebin.com/raw/5mVWxGTf

I also tried modifying the command to use the python compiler I got pre-installed in my Ubuntu 16.04, but I got an ImportError:
https://pastebin.com/raw/6Lh9Bqys

Have I done something wrong? Please help me.

Comment 7 Markus Mohrhard 2018-02-25 18:22:03 UTC

(In reply to Saurav Chirania from comment #6)
> I was not able to use the given execute.sh script.
> 
> Since there is no file named python in my instdir/program directory, I got
> the following error:
> https://pastebin.com/raw/5mVWxGTf
> 
> I also tried modifying the command to use the python compiler I got
> pre-installed in my Ubuntu 16.04, but I got an ImportError:
> https://pastebin.com/raw/6Lh9Bqys
> 
> Have I done something wrong? Please help me.

You can use --enable-python=fully-internal as a autogen flag (e.g. in autogen.input or autogen.lastrun) or set PYTHONPATH and URE_BOOTSTRAP in the execute.sh script.

Comment 8 Buovjaga 2018-10-09 12:57:18 UTC

We do not have a manual test system anymore and I believe all the test cases were converted. Closing.