Skip to content

Commit

Permalink
doc: robot: Add docs about robotframework usage
Browse files Browse the repository at this point in the history
This change adds documentation about the usage of the newly implemented
robotframework harness. Hereby, general setup and usage, together with
added ZephyrLibrary keywords, are described.

Signed-off-by: Stefan Kraus <stefan.kraus@ul.com>
  • Loading branch information
MP-StefanKraus committed Mar 19, 2024
1 parent 7d893be commit af4c13e
Show file tree
Hide file tree
Showing 2 changed files with 124 additions and 0 deletions.
1 change: 1 addition & 0 deletions doc/develop/test/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Testing
ztest
twister
pytest
robot
coverage
BabbleSim<bsim>
ztest_deprecated
123 changes: 123 additions & 0 deletions doc/develop/test/robot.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
.. integration-with-robot:
Integration with robot framework
################################

*Please mind that integration of twister with robot is still work in progress. Not every platform
type is supported in robot (yet). If you find any issue with the integration or have an idea for
an improvement, please, let us know about it and open a GitHub issue/enhancement.*

.. note::

This section of the documentation describes the integration of generic Python Robot Framework
library -- currently called `robotframework` -- in twister; in contrast to the specific renode
harness which is called just `robot`.

Introduction
************

*“Robot Framework is a generic open source automation framework. It can be used for test automation
and robotic process automation (RPA).“* (`<https://robotframework.org/>`_).
The focus of Robot Framework is especially on acceptance tests.

The basic behaviour and implementation is similar to the pytest implementation.

Integration with twister
************************

By default, there is nothing to be done to enable robot support in twister. The plugin is
developed as a part of Zephyr’s tree. To enable install-less operation, twister first extends
``PYTHONPATH`` with path to this plugin. If one prefers to use the installed version of the plugin,
they must add ``--allow-installed-plugin`` flag to twister’s call.

Robot Framework test are discovered the same way as other twister tests, i.e., by a presence
of testcase.yaml or sample.yaml. Inside, a keyword ``harness`` tells twister how to handle a given test.
In the case of ``harness: robotframework``, most of twister’s workflow (test suites discovery,
parallelization, building and reporting) remains the same as for other harnesses. The change
happens during the execution step.

If ``harness: robotframework`` is used, twister calls a robot_wrapper entrypoint in `pytest-twister-harness` that
handles proper argument parsing from twister (e.g., baudrate, serial device) and calls the
Python3 Robot Framework library afterwards.
This is needed to not replicate all argument parsing from pytest and robot, but having all configuration
available in the robot execution later.
After test execution, twister searches for a XUnit report file (robot_report.xml) containing test results, and
set its own results accordingly.


How to create a robot test
**************************

An example of a robot test is given at :zephyr_file:`samples/subsys/testsuite/robotframework/shell/robot/shell_module.robot`.
Twister calls robot for each configuration from the .yaml file which uses ``harness: robotframework``.
By default, it points to ``robot`` directory, located next to a directory with binary sources.
A keyword ``robot_test_path`` placed under ``harness_config`` section can be used to point to other
files, directories or subtests.

One can also pass some extra arguments to robot from yaml file using ``robot_args`` keyword
under ``harness_config``, e.g.: ``robot_args: [‘-t=test_method’, ‘-t=test_method’]``.
There is also an option to pass ``--robot-args`` through Twister command line parameters.
This can be particularly useful when one wants to select a specific testcase from a test suite.
For instance, one can use a command:

.. code-block:: console
$ ./scripts/twister -p qemu_cortex_m3 --device-testing \
-T samples/subsys/testsuite/robotframework/shell/ \
--robot-args="--SuiteStatLevel 2 --Metadata Version:3"
Note that ``--robot-args`` can be passed multiple times to pass several arguments to the robot framework.

Robot keywords
==============

Besides some common keywords provided by Robot Framework themselves, there is a
'ZephyrLibrary' that can be loaded and used in the Robot test files.

.. code-block:: robotframework
*** Settings ***
Library twister_harness.robot_framework.ZephyrLibrary
This library provides additional, Zephyr specific keywords and phrases that
can be used to handle interaction with the device.


Get a Device
------------

Give access to a ``DeviceAdapter`` type object, that represents Device Under Test.
This is the core of the following interaction.
It is required to launch and interact with other keywords.
It yields a device prepared according to the requested type
(``native``, ``qemu``, ``hardware``, etc.). All types of devices share the same API.
This allows for writing tests which are device-type-agnostic.


Run Device
----------

This keyword flashes and launches a device that was previously acquired by 'Get a Device'.
Otherwise, the Device will keep its previous state.


Run Command <command>
------------------------

This connects to the running device or emulator via a serial console and executes
the given command ``<command>`` via shell. Note that the target must be
compiled with ``CONFIG_SHELL=y``.
It waits for a proper response of the ``wait_for_prompt`` method, so it does not
start the scenario until the DUT is ready.


Close Device
------------

This closes the serial connection to the device.


Limitations
***********

* Not every platform type is supported in the plugin (yet).

0 comments on commit af4c13e

Please sign in to comment.