Robot Framework's actual testing capabilities are provided by test libraries. There are many existing libraries, some of which are even bundled with the core framework, but there is still often a need to create new ones. This task is not too complicated because, as this chapter illustrates, Robot Framework's library API is simple and straightforward.
4.1.1 Introduction Supported programming languagesRobot Framework itself is written with Python and naturally test libraries extending it can be implemented using the same language. It is also possible to implement libraries with C using Python C API, although it is often easier to interact with C code from Python libraries using ctypes module.
Libraries implemented using Python can also act as wrappers to functionality implemented using other programming languages. A good example of this approach is the Remote library, and another widely used approaches is running external scripts or tools as separate processes.
Different test library APIsRobot Framework has three different test library APIs.
Static API
The simplest approach is having a module or a class with functions/methods which map directly to keyword names. Keywords also take the same arguments as the methods implementing them. Keywords report failures with exceptions, log by writing to standard output and can return values using the return statement.
Dynamic API
Dynamic libraries are classes that implement a method to get the names of the keywords they implement, and another method to execute a named keyword with given arguments. The names of the keywords to implement, as well as how they are executed, can be determined dynamically at runtime, but reporting the status, logging and returning values is done similarly as in the static API.
Hybrid API
This is a hybrid between the static and the dynamic API. Libraries are classes with a method telling what keywords they implement, but those keywords must be available directly. Everything else except discovering what keywords are implemented is similar as in the static API.
All these APIs are described in this chapter. Everything is based on how the static API works, so its functions are discussed first. How the dynamic library API and the hybrid library API differ from it is then discussed in sections of their own.
4.1.2 Creating test library class or moduleTest libraries can be implemented as Python modules or classes.
Library nameAs discussed under the Using test libraries section, libraries can be imported by name or path:
*** Settings *** Library MyLibrary Library module.LibraryClass Library path/AnotherLibrary.py
When a library is imported by a name, the library module must be in the module search path and the name can either refer to a library module or to a library class. When a name refers directly to a library class, the name must be in format like modulename.ClassName. Paths to libraries always refer to modules.
Even when a library import refers to a module, either by a name or by a path, a class in the module, not the module itself, is used as a library in these cases:
If the module contains a class that has the same name as the module. The class can be either implemented in the module or imported into it.
This makes it possible to import libraries using simple names like MyLibrary instead of specifying both the module and the class like module.MyLibrary or MyLibrary.MyLibrary. When importing a library by a path, it is not even possible to directly refer to a library class and automatically using a class from the imported module is the only option.
If the module contains exactly one class decorated with the @library decorator. In this case the class needs to be implemented in the module, not imported to it.
This approach has all the same benefits as the earlier one, but it also allows the class name to differ from the module name.
Using the @library decorator for this purpose is new in Robot Framework 7.2.
Tip
If the library name is really long, it is often a good idea to give it a simpler alias at the import time.
Providing arguments to librariesAll test libraries implemented as classes can take arguments. These arguments are specified after the library name when the library is imported, and when Robot Framework creates an instance of the imported library, it passes them to its constructor. Libraries implemented as a module cannot take any arguments.
The number of arguments needed by the library is the same as the number of arguments accepted by the library's __init__ method. The default values, argument conversion, and other such features work the same way as with keyword arguments. Arguments passed to the library, as well as the library name itself, can be specified using variables, so it is possible to alter them, for example, from the command line.
*** Settings ***
Library MyLibrary 10.0.0.1 8080
Library AnotherLib ${ENVIRONMENT}
Example implementations for the libraries used in the above example:
from example import Connection
class MyLibrary:
def __init__(self, host, port=80):
self.connection = Connection(host, port)
def send_message(self, message):
self.connection.send(message)
class AnotherLib:
def __init__(self, environment):
self.environment = environment
def do_something(self):
if self.environment == 'test':
do_something_in_test_environment()
else:
do_something_in_other_environments()
Library scope
Libraries implemented as classes can have an internal state, which can be altered by keywords and with arguments to the constructor of the library. Because the state can affect how keywords actually behave, it is important to make sure that changes in one test case do not accidentally affect other test cases. These kind of dependencies may create hard-to-debug problems, for example, when new test cases are added and they use the library inconsistently.
Robot Framework attempts to keep test cases independent from each other: by default, it creates new instances of test libraries for every test case. However, this behavior is not always desirable, because sometimes test cases should be able to share a common state. Additionally, all libraries do not have a state and creating new instances of them is simply not needed.
Test libraries can control when new libraries are created with a class attribute ROBOT_LIBRARY_SCOPE . This attribute must be a string and it can have the following three values:
TEST
A new instance is created for every test case. A possible suite setup and suite teardown share yet another instance.
Prior to Robot Framework 3.2 this value was TEST CASE, but nowadays TEST is recommended. Because all unrecognized values are considered same as TEST, both values work with all versions. For the same reason it is possible to also use value TASK if the library is targeted for RPA usage more than testing. TEST is also the default value if the ROBOT_LIBRARY_SCOPE attribute is not set.
SUITE
A new instance is created for every test suite. The lowest-level test suites, created from test case files and containing test cases, have instances of their own, and higher-level suites all get their own instances for their possible setups and teardowns.
Prior to Robot Framework 3.2 this value was TEST SUITE. That value still works, but SUITE is recommended with libraries targeting Robot Framework 3.2 and newer.
GLOBAL
Note
If a library is imported multiple times with different arguments, a new instance is created every time regardless the scope.
When the SUITE or GLOBAL scopes are used with libraries that have a state, it is recommended that libraries have some special keyword for cleaning up the state. This keyword can then be used, for example, in a suite setup or teardown to ensure that test cases in the next test suites can start from a known state. For example, SeleniumLibrary uses the GLOBAL scope to enable using the same browser in different test cases without having to reopen it, and it also has the Close All Browsers keyword for easily closing all opened browsers.
Example library using the SUITE scope:
class ExampleLibrary:
ROBOT_LIBRARY_SCOPE = 'SUITE'
def __init__(self):
self._counter = 0
def count(self):
self._counter += 1
print(self._counter)
def clear_count(self):
self._counter = 0
Library version
When a test library is taken into use, Robot Framework tries to determine its version. This information is then written into the syslog to provide debugging information. Library documentation tool Libdoc also writes this information into the keyword documentations it generates.
Version information is read from attribute ROBOT_LIBRARY_VERSION, similarly as library scope is read from ROBOT_LIBRARY_SCOPE. If ROBOT_LIBRARY_VERSION does not exist, information is tried to be read from __version__ attribute. These attributes must be class or module attributes, depending whether the library is implemented as a class or a module.
An example module using __version__:
__version__ = '0.1'
def keyword():
pass
Documentation format
Library documentation tool Libdoc supports documentation in multiple formats. If you want to use something else than Robot Framework's own documentation formatting, you can specify the format in the source code using ROBOT_LIBRARY_DOC_FORMAT attribute similarly as scope and version are set with their own ROBOT_LIBRARY_* attributes.
The possible case-insensitive values for documentation format are ROBOT (default), HTML, TEXT (plain text), and reST (reStructuredText). Using the reST format requires the docutils module to be installed when documentation is generated.
Setting the documentation format is illustrated by the following example that uses reStructuredText format. See Documenting libraries section and Libdoc chapter for more information about documenting test libraries in general.
"""A library for *documentation format* demonstration purposes.
This documentation is created using reStructuredText__. Here is a link
to the only \`Keyword\`.
__ http://docutils.sourceforge.net
"""
ROBOT_LIBRARY_DOC_FORMAT = 'reST'
def keyword():
"""**Nothing** to see here. Not even in the table below.
======= ===== =====
Table here has
nothing to see.
======= ===== =====
"""
pass
Library acting as listener
Listener interface allows external listeners to get notifications about test execution. They are called, for example, when suites, tests, and keywords start and end. Sometimes getting such notifications is also useful for test libraries, and they can register a custom listener by using ROBOT_LIBRARY_LISTENER attribute. The value of this attribute should be an instance of the listener to use, possibly the library itself.
For more information and examples see Libraries as listeners section.
@library decorator
An easy way to configure libraries implemented as classes is using the robot.api.deco.library class decorator. It allows configuring library's scope, version, custom argument converters, documentation format and listener with optional arguments scope, version, converter, doc_format and listener, respectively. When these arguments are used, they set the matching ROBOT_LIBRARY_SCOPE, ROBOT_LIBRARY_VERSION, ROBOT_LIBRARY_CONVERTERS, ROBOT_LIBRARY_DOC_FORMAT and ROBOT_LIBRARY_LISTENER attributes automatically:
from robot.api.deco import library
from example import Listener
@library(scope='GLOBAL', version='3.2b1', doc_format='reST', listener=Listener())
class Example:
...
The @library decorator also disables the automatic keyword discovery by setting the ROBOT_AUTO_KEYWORDS argument to False by default. This means that it is mandatory to decorate methods with the @keyword decorator to expose them as keywords. If only that behavior is desired and no further configuration is needed, the decorator can also be used without parenthesis like:
from robot.api.deco import library
@library
class Example:
...
If needed, the automatic keyword discovery can be enabled by using the auto_keywords argument:
from robot.api.deco import library
@library(scope='GLOBAL', auto_keywords=True)
class Example:
...
The @library decorator only sets class attributes ROBOT_LIBRARY_SCOPE, ROBOT_LIBRARY_VERSION, ROBOT_LIBRARY_CONVERTERS, ROBOT_LIBRARY_DOC_FORMAT and ROBOT_LIBRARY_LISTENER if the respective arguments scope, version, converters, doc_format and listener are used. The ROBOT_AUTO_KEYWORDS attribute is set always and its presence can be used as an indication that the @library decorator has been used. When attributes are set, they override possible existing class attributes.
When a class is decorated with the @library decorator, it is used as a library even when a library import refers only to a module containing it. This is done regardless does the class name match the module name or not.
Note
The @library decorator is new in Robot Framework 3.2, the converters argument is new in Robot Framework 5.0, and specifying that a class in an imported module should be used as a library is new in Robot Framework 7.2.
When the static library API is used, Robot Framework uses introspection to find out what keywords the library class or module implements. By default it excludes methods and functions starting with an underscore. All the methods and functions that are not ignored are considered keywords. For example, the library below implements a single keyword My Keyword.
class MyLibrary:
def my_keyword(self, arg):
return self._helper_method(arg)
def _helper_method(self, arg):
return arg.upper()
Limiting public methods becoming keywords
Automatically considering all public methods and functions keywords typically works well, but there are cases where it is not desired. There are also situations where keywords are created when not expected. For example, when implementing a library as class, it can be a surprise that also methods in possible base classes are considered keywords. When implementing a library as a module, functions imported into the module namespace becoming keywords is probably even a bigger surprise.
This section explains how to prevent methods and functions becoming keywords.
Class based librariesWhen a library is implemented as a class, it is possible to tell Robot Framework not to automatically expose methods as keywords by setting the ROBOT_AUTO_KEYWORDS attribute to the class with a false value:
class Example:
ROBOT_AUTO_KEYWORDS = False
When the ROBOT_AUTO_KEYWORDS attribute is set like this, only methods that have explicitly been decorated with the @keyword decorator or otherwise have the robot_name attribute become keywords. The @keyword decorator can also be used for setting a custom name, tags and argument types to the keyword.
Although the ROBOT_AUTO_KEYWORDS attribute can be set to the class explicitly, it is more convenient to use the @library decorator that sets it to False by default:
from robot.api.deco import keyword, library
@library
class Example:
@keyword
def this_is_keyword(self):
pass
@keyword('This is keyword with custom name')
def xxx(self):
pass
def this_is_not_keyword(self):
pass
Note
Both limiting what methods become keywords using the ROBOT_AUTO_KEYWORDS attribute and the @library decorator are new in Robot Framework 3.2.
Another way to explicitly specify what keywords a library implements is using the dynamic or the hybrid library API.
Module based librariesWhen implementing a library as a module, all functions in the module namespace become keywords. This is true also with imported functions, and that can cause nasty surprises. For example, if the module below would be used as a library, it would contain a keyword Example Keyword, as expected, but also a keyword Current Thread.
from threading import current_thread
def example_keyword():
thread_name = current_thread().name
print(f"Running in thread '{thread_name}'.")
A simple way to avoid imported functions becoming keywords is to only import modules (e.g. import threading) and to use functions via the module (e.g threading.current_thread()). Alternatively functions could be given an alias starting with an underscore at the import time (e.g. from threading import current_thread as _current_thread).
A more explicit way to limit what functions become keywords is using the module level __all__ attribute that Python itself uses for similar purposes. If it is used, only the listed functions can be keywords. For example, the library below implements only one keyword Example Keyword:
from threading import current_thread
__all__ = ['example_keyword']
def example_keyword():
thread_name = current_thread().name
print(f"Running in thread '{thread_name}'.")
def this_is_not_keyword():
pass
If the library is big, maintaining the __all__ attribute when keywords are added, removed or renamed can be a somewhat big task. Another way to explicitly mark what functions are keywords is using the ROBOT_AUTO_KEYWORDS attribute similarly as it can be used with class based libraries. When this attribute is set to a false value, only functions explicitly decorated with the @keyword decorator become keywords. For example, also this library implements only one keyword Example Keyword:
from threading import current_thread
from robot.api.deco import keyword
ROBOT_AUTO_KEYWORDS = False
@keyword
def example_keyword():
thread_name = current_thread().name
print(f"Running in thread '{thread_name}'.")
def this_is_not_keyword():
pass
Note
Limiting what functions become keywords using ROBOT_AUTO_KEYWORDS is a new feature in Robot Framework 3.2.
@not_keyword decorator
Functions in modules and methods in classes can be explicitly marked as "not keywords" by using the @not_keyword decorator. When a library is implemented as a module, this decorator can also be used to avoid imported functions becoming keywords.
from threading import current_thread
from robot.api.deco import not_keyword
not_keyword(current_thread) # Don't expose `current_thread` as a keyword.
def example_keyword():
thread_name = current_thread().name
print(f"Running in thread '{thread_name}'.")
@not_keyword
def this_is_not_keyword():
pass
Using the @not_keyword decorator is pretty much the opposite way to avoid functions or methods becoming keywords compared to disabling the automatic keyword discovery with the @library decorator or by setting the ROBOT_AUTO_KEYWORDS to a false value. Which one to use depends on the context.
Note
The @not_keyword decorator is new in Robot Framework 3.2.
Keyword names used in the test data are compared with method names to find the method implementing these keywords. Name comparison is case-insensitive, and also spaces and underscores are ignored. For example, the method hello maps to the keyword name Hello, hello or even h e l l o. Similarly both the do_nothing and doNothing methods can be used as the Do Nothing keyword in the test data.
Example library implemented as a module in the MyLibrary.py file:
def hello(name):
print(f"Hello, {name}!")
def do_nothing():
pass
The example below illustrates how the example library above can be used. If you want to try this yourself, make sure that the library is in the module search path.
*** Settings ***
Library MyLibrary
*** Test Cases ***
My Test
Do Nothing
Hello world
Setting custom name
It is possible to expose a different name for a keyword instead of the default keyword name which maps to the method name. This can be accomplished by setting the robot_name attribute on the method to the desired custom name:
def login(username, password):
...
login.robot_name = 'Login via user panel'
*** Test Cases ***
My Test
Login Via User Panel ${username} ${password}
Instead of explicitly setting the robot_name attribute like in the above example, it is typically easiest to use the @keyword decorator:
from robot.api.deco import keyword
@keyword('Login via user panel')
def login(username, password):
...
Using this decorator without an argument will have no effect on the exposed keyword name, but will still set the robot_name attribute. This allows marking methods to expose as keywords without actually changing keyword names. Methods that have the robot_name attribute also create keywords even if the method name itself would start with an underscore.
Setting a custom keyword name can also enable library keywords to accept arguments using the embedded arguments syntax.
Keyword argumentsWith a static and hybrid API, the information on how many arguments a keyword needs is got directly from the method that implements it. Libraries using the dynamic library API have other means for sharing this information, so this section is not relevant to them.
The most common and also the simplest situation is when a keyword needs an exact number of arguments. In this case, the method simply take exactly those arguments. For example, a method implementing a keyword with no arguments takes no arguments either, a method implementing a keyword with one argument also takes one argument, and so on.
Example keywords taking different numbers of arguments:
def no_arguments():
print("Keyword got no arguments.")
def one_argument(arg):
print(f"Keyword got one argument '{arg}'.")
def three_arguments(a1, a2, a3):
print(f"Keyword got three arguments '{a1}', '{a2}' and '{a3}'.")
Default values to keywords
It is often useful that some of the arguments that a keyword uses have default values.
In Python a method has always exactly one implementation and possible default values are specified in the method signature. The syntax, which is familiar to all Python programmers, is illustrated below:
def one_default(arg='default'):
print(f"Got argument '{arg}'.")
def multiple_defaults(arg1, arg2='default 1', arg3='default 2'):
print(f"Got arguments '{arg1}', '{arg2}' and '{arg3}'.")
The first example keyword above can be used either with zero or one arguments. If no arguments are given, arg gets the value default. If there is one argument, arg gets that value, and calling the keyword with more than one argument fails. In the second example, one argument is always required, but the second and the third one have default values, so it is possible to use the keyword with one to three arguments.
*** Test Cases ***
Defaults
One Default
One Default argument
Multiple Defaults required arg
Multiple Defaults required arg optional
Multiple Defaults required arg optional 1 optional 2
Variable number of arguments (*varargs)
Robot Framework supports also keywords that take any number of arguments.
Python supports methods accepting any number of arguments. The same syntax works in libraries and, as the examples below show, it can also be combined with other ways of specifying arguments:
def any_arguments(*args):
print("Got arguments:")
for arg in args:
print(arg)
def one_required(required, *others):
print(f"Required: {required}\nOthers:")
for arg in others:
print(arg)
def also_defaults(req, def1="default 1", def2="default 2", *rest):
print(req, def1, def2, rest)
*** Test Cases ***
Varargs
Any Arguments
Any Arguments argument
Any Arguments arg 1 arg 2 arg 3 arg 4 arg 5
One Required required arg
One Required required arg another arg yet another
Also Defaults required
Also Defaults required these two have defaults
Also Defaults 1 2 3 4 5 6
Free keyword arguments (**kwargs)
Robot Framework supports Python's **kwargs syntax. How to use use keywords that accept free keyword arguments, also known as free named arguments, is discussed under the Creating test cases section. In this section we take a look at how to create such keywords.
If you are already familiar how kwargs work with Python, understanding how they work with Robot Framework test libraries is rather simple. The example below shows the basic functionality:
def example_keyword(**stuff):
for name, value in stuff.items():
print(name, value)
*** Test Cases ***
Keyword Arguments
Example Keyword hello=world # Logs 'hello world'.
Example Keyword foo=1 bar=42 # Logs 'foo 1' and 'bar 42'.
Basically, all arguments at the end of the keyword call that use the named argument syntax name=value, and that do not match any other arguments, are passed to the keyword as kwargs. To avoid using a literal value like foo=quux as a free keyword argument, it must be escaped like foo\=quux.
The following example illustrates how normal arguments, varargs, and kwargs work together:
def various_args(arg=None, *varargs, **kwargs):
if arg is not None:
print('arg:', arg)
for value in varargs:
print('vararg:', value)
for name, value in sorted(kwargs.items()):
print('kwarg:', name, value)
*** Test Cases ***
Positional
Various Args hello world # Logs 'arg: hello' and 'vararg: world'.
Named
Various Args arg=value # Logs 'arg: value'.
Kwargs
Various Args a=1 b=2 c=3 # Logs 'kwarg: a 1', 'kwarg: b 2' and 'kwarg: c 3'.
Various Args c=3 a=1 b=2 # Same as above. Order does not matter.
Positional and kwargs
Various Args 1 2 kw=3 # Logs 'arg: 1', 'vararg: 2' and 'kwarg: kw 3'.
Named and kwargs
Various Args arg=value hello=world # Logs 'arg: value' and 'kwarg: hello world'.
Various Args hello=world arg=value # Same as above. Order does not matter.
For a real world example of using a signature exactly like in the above example, see Run Process and Start Keyword keywords in the Process library.
Keyword-only argumentsStarting from Robot Framework 3.1, it is possible to use named-only arguments with different keywords. This support is provided by Python's keyword-only arguments. Keyword-only arguments are specified after possible *varargs or after a dedicated * marker when *varargs are not needed. Possible **kwargs are specified after keyword-only arguments.
Example:
def sort_words(*words, case_sensitive=False):
key = str.lower if case_sensitive else None
return sorted(words, key=key)
def strip_spaces(word, *, left=True, right=True):
if left:
word = word.lstrip()
if right:
word = word.rstrip()
return word
*** Test Cases ***
Example
Sort Words Foo bar baZ
Sort Words Foo bar baZ case_sensitive=True
Strip Spaces ${word} left=False
Positional-only arguments
Python supports so called positional-only arguments that make it possible to specify that an argument can only be given as a positional argument, not as a named argument like name=value. Positional-only arguments are specified before normal arguments and a special / marker must be used after them:
def keyword(posonly, /, normal):
print(f"Got positional-only argument {posonly} and normal argument {normal}.")
The above keyword could be used like this:
*** Test Cases ***
Example
# Positional-only and normal argument used as positional arguments.
Keyword foo bar
# Normal argument can also be named.
Keyword foo normal=bar
If a positional-only argument is used with a value that contains an equal sign like example=usage, it is not considered to mean named argument syntax even if the part before the = would match the argument name. This rule only applies if the positional-only argument is used in its correct position without other arguments using the name argument syntax before it, though.
*** Test Cases ***
Example
# Positional-only argument gets literal value `posonly=foo` in this case.
Keyword posonly=foo normal=bar
# This fails.
Keyword normal=bar posonly=foo
Positional-only arguments are fully supported starting from Robot Framework 4.0. Using them as positional arguments works also with earlier versions, but using them as named arguments causes an error on Python side.
Argument conversionArguments defined in Robot Framework test data are, by default, passed to keywords as Unicode strings. There are, however, several ways to use non-string values as well:
Automatic argument conversion based on function annotations, types specified using the @keyword decorator, and argument default values are all new features in Robot Framework 3.1. The Supported conversions section specifies which argument conversion are supported in these cases.
Prior to Robot Framework 4.0, automatic conversion was done only if the given argument was a string. Nowadays it is done regardless the argument type.
Manual argument conversionIf no type information is specified to Robot Framework, all arguments not passed as variables are given to keywords as Unicode strings. This includes cases like this:
*** Test Cases ***
Example
Example Keyword 42 False
It is always possible to convert arguments passed as strings insider keywords. In simple cases this means using int() or float() to convert arguments to numbers, but other kind of conversion is possible as well. When working with Boolean values, care must be taken because all non-empty strings, including string False, are considered true by Python. Robot Framework's own robot.utils.is_truthy() utility handles this nicely as it considers strings like FALSE, NO and NONE (case-insensitively) to be false:
from robot.utils import is_truthy
def example_keyword(count, case_insensitive):
count = int(count)
if is_truthy(case_insensitive):
...
Keywords can also use Robot Framework's argument conversion functionality via the robot.api.TypeInfo class and its convert method. This can be useful if the needed conversion logic is more complicated or the are needs for better error reporting than what simply using, for example, int() provides.
from robot.api import TypeInfo
def example_keyword(count, case_insensitive):
count = TypeInfo.from_type(int).convert(count)
if TypeInfo.from_type(bool).convert(case_insensitive):
...
Tip
It is generally recommended to specify types using type hints or otherwise and let Robot Framework handle argument conversion automatically. Manual argument conversion should only be needed in special cases.
Note
robot.api.TypeInfo is new in Robot Framework 7.0.
Starting from Robot Framework 3.1, arguments passed to keywords are automatically converted if argument type information is available and the type is recognized. The most natural way to specify types is using Python function annotations. For example, the keyword in the previous example could be implemented as follows and arguments would be converted automatically:
def example_keyword(count: int, case_insensitive: bool = True):
if case_insensitive:
...
See the Supported conversions section below for a list of types that are automatically converted and what values these types accept. It is an error if an argument having one of the supported types is given a value that cannot be converted. Annotating only some of the arguments is fine.
Annotating arguments with other than the supported types is not an error, and it is also possible to use annotations for other than typing purposes. In those cases no conversion is done, but annotations are nevertheless shown in the documentation generated by Libdoc.
Keywords can also have a return type annotation specified using the -> notation at the end of the signature like def example() -> int:. This information is not used for anything during execution, but starting from Robot Framework 7.0 it is shown by Libdoc for documentation purposes.
@keyword decorator
An alternative way to specify explicit argument types is using the @keyword decorator. Starting from Robot Framework 3.1, it accepts an optional types argument that can be used to specify argument types either as a dictionary mapping argument names to types or as a list mapping arguments to types based on position. These approaches are shown below implementing the same keyword as in earlier examples:
from robot.api.deco import keyword
@keyword(types={'count': int, 'case_insensitive': bool})
def example_keyword(count, case_insensitive=True):
if case_insensitive:
...
@keyword(types=[int, bool])
def example_keyword(count, case_insensitive=True):
if case_insensitive:
...
Regardless of the approach that is used, it is not necessarily to specify types for all arguments. When specifying types as a list, it is possible to use None to mark that a certain argument does not have type information and arguments at the end can be omitted altogether. For example, both of these keywords specify the type only for the second argument:
@keyword(types={'second': float})
def example1(first, second, third):
...
@keyword(types=[None, float])
def example2(first, second, third):
...
Starting from Robot Framework 7.0, it is possible to specify the keyword return type by using key 'return' with an appropriate type in the type dictionary. This information is not used for anything during execution, but it is shown by Libdoc for documentation purposes.
If any types are specified using the @keyword decorator, type information got from annotations is ignored with that keyword. Setting types to None like @keyword(types=None) disables type conversion altogether so that also type information got from default values is ignored.
If type information is not got explicitly using annotations or the @keyword decorator, Robot Framework 3.1 and newer tries to get it based on possible argument default value. In this example count and case_insensitive get types int and bool, respectively:
def example_keyword(count=-1, case_insensitive=True):
if case_insensitive:
...
When type information is got implicitly based on the default values, argument conversion itself is not as strict as when the information is got explicitly:
If an argument has an explicit type and a default value, conversion is first attempted based on the explicit type. If that fails, then conversion is attempted based on the default value. In this special case conversion based on the default value is strict and a conversion failure causes an error.
If argument conversion based on default values is not desired, the whole argument conversion can be disabled with the @keyword decorator like @keyword(types=None).
Note
Prior to Robot Framework 4.0 conversion was done based on the default value only if the argument did not have an explict type.
Supported conversionsThe table below lists the types that Robot Framework 3.1 and newer convert arguments to. These characteristics apply to all conversions:
Note
If an argument has both a type hint and a default value, conversion is first attempted based on the type hint and then, if that fails, based on the default value type. This behavior is likely to change in the future so that conversion based on the default value is done only if the argument does not have a type hint. That will change conversion behavior in cases like arg: list = None where None conversion will not be attempted anymore. Library creators are strongly recommended to specify the default value type explicitly like arg: list | None = None already now.
The type to use can be specified either using concrete types (e.g. list), by using abstract base classes (ABC) (e.g. Sequence), or by using sub classes of these types (e.g. MutableSequence). Also types in in the typing module that map to the supported concrete types or ABCs (e.g. List) are supported. In all these cases the argument is converted to the concrete type.
In addition to using the actual types (e.g. int), it is possible to specify the type using type names as a string (e.g. 'int') and some types also have aliases (e.g. 'integer'). Matching types to names and aliases is case-insensitive.
The Accepts column specifies which given argument types are converted. If the given argument already has the expected type, no conversion is done. Other types cause conversion failures.
Supported argument conversions Type ABC Aliases Accepts Explanation Examples bool boolean str, int, float, NoneStrings TRUE, YES, ON and 1 are converted to True, the empty string as well as FALSE, NO, OFF and 0 are converted to False, and the string NONE is converted to None. Other strings and other accepted values are passed as-is, allowing keywords to handle them specially if needed. All string comparisons are case-insensitive.
True and false strings can be localized. See the Translations appendix for supported translations.
TRUE (converted to True)
off (converted to False)
example (used as-is)
Conversion is done using the int built-in function. Floats are accepted only if they can be represented as integers exactly. For example, 1.0 is accepted and 1.1 is not. If converting a string to an integer fails and the type is got implicitly based on a default value, conversion to float is attempted as well.
Starting from Robot Framework 4.1, it is possible to use hexadecimal, octal and binary numbers by prefixing values with 0x, 0o and 0b, respectively.
Starting from Robot Framework 4.1, spaces and underscores can be used as visual separators for digit grouping purposes.
Starting from Robot Framework 7.0, strings representing floats are accepted as long as their decimal part is zero. This includes using the scientific notation like 1e100.
42
-1
10 000 000
1e100
0xFF
0o777
0b1010
0xBAD_C0FFEE
${1}
${1.0}
Conversion is done using the float built-in.
Starting from Robot Framework 4.1, spaces and underscores can be used as visual separators for digit grouping purposes.
3.14
2.9979e8
10 000.000 01
10_000.000_01
Conversion is done using the Decimal class. Decimal is recommended over float when decimal numbers need to be represented exactly.
Starting from Robot Framework 4.1, spaces and underscores can be used as visual separators for digit grouping purposes.
3.14
10 000.000 01
10_000.000_01
All arguments are converted to Unicode strings.
New in Robot Framework 4.0.
bytes str, bytearray Strings are converted to bytes so that each Unicode code point below 256 is directly mapped to a matching byte. Higher code points are not allowed.good
hyvä (converted to hyv\xe4)
\x00 (the null byte)
String timestamps are expected to be in ISO 8601 like format YYYY-MM-DD hh:mm:ss.mmmmmm, where any non-digit character can be used as a separator or separators can be omitted altogether. Additionally, only the date part is mandatory, all possibly missing time components are considered to be zeros.
Special values NOW and TODAY (case-insensitive) can be used to get the current local datetime. This is new in Robot Framework 7.3.
Integers and floats are considered to represent seconds since the Unix epoch.
2022-02-09T16:39:43.632269
20220209 16:39
2022-02-09
now (current local date and time)
TODAY (same as above)
${1644417583.632269} (Epoch time)
Same timestamp conversion as with datetime, but all time components are expected to be omitted or to be zeros.
Special values NOW and TODAY (case-insensitive) can be used to get the current local date. This is new in Robot Framework 7.3.
2018-09-12
20180912
today (current local date)
NOW (same as above)
42 (42 seconds)
1 minute 2 seconds
01:02 (same as above)
Strings are converted to pathlib.Path objects. On Windows / is converted to \ automatically.
New in Robot Framework 6.0.
/tmp/absolute/path
relative/path/to/file.ext
name.txt
The specified type must be an enumeration (a subclass of Enum or Flag) and given arguments must match its member names.
Matching member names is case, space, underscore and hyphen insensitive, but exact matches have precedence over normalized matches. Ignoring hyphens is new in Robot Framework 7.0.
Enumeration documentation and members are shown in documentation generated by Libdoc automatically.
class Direction(Enum):
"""Move direction."""
NORTH = auto()
NORTH_WEST = auto()
def kw(arg: Direction):
...
NORTH (Direction.NORTH)
north west (Direction.NORTH_WEST)
The specified type must be an integer based enumeration (a subclass of IntEnum or IntFlag) and given arguments must match its member names or values.
Matching member names works the same way as with Enum. Values can be given as integers and as strings that can be converted to integers.
Enumeration documentation and members are shown in documentation generated by Libdoc automatically.
New in Robot Framework 4.1.
class PowerState(IntEnum):
"""Turn system ON or OFF."""
OFF = 0
ON = 1
def kw(arg: PowerState):
...
OFF (PowerState.OFF)
1 (PowerState.ON)
Only specified values are accepted. Values can be strings, integers, bytes, Booleans, enums and None, and used arguments are converted using the value type specific conversion logic.
Strings are case, space, underscore and hyphen insensitive, but exact matches have precedence over normalized matches.
Literal provides similar functionality as Enum, but does not support custom documentation.
New in Robot Framework 7.0.
def kw(arg: Literal['ON', 'OFF']):
...
OFF
on
NONE (case-insensitive) is converted to the Python None object. Other values cause an error.
None
Any value is accepted. No conversion is done.
New in Robot Framework 6.1.
list Sequence sequence str, SequenceStrings must be Python list literals. They are converted to actual lists using the ast.literal_eval function. They can contain any values ast.literal_eval supports, including lists and other containers.
If the used type hint is list (e.g. arg: list), sequences that are not lists are converted to lists. If the type hint is generic Sequence, sequences are used without conversion.
Alias sequence is new in Robot Framework 7.0.
['one', 'two']
[('one', 1), ('two', 2)]
list, but string arguments must be tuple literals.
('one', 'two')
list, but string arguments must be set literals or set() to create an empty set.
{1, 2, 3, 42}
set()
set, but the result is a frozenset.
{1, 2, 3, 42}
frozenset()
Same as list, but string arguments must be dictionary literals.
Alias mapping is new in Robot Framework 7.0.
{'a': 1, 'b': 2}
{'key': 1, 'nested': {'key': 2}}
Same as dict, but dictionary items are also converted to the specified types and items not included in the type spec are not allowed.
New in Robot Framework 6.0. Normal dict conversion was used earlier.
class Config(TypedDict):
width: int
enabled: bool
{'width': 1600, 'enabled': True}
Note
Starting from Robot Framework 5.0, types that have a converted are automatically shown in Libdoc outputs.
Note
Prior to Robot Framework 4.0, most types supported converting string NONE (case-insensitively) to Python None. That support has been removed and None conversion is only done if an argument has None as an explicit type or as a default value.
Starting from Robot Framework 4.0, it is possible to specify that an argument has multiple possible types. In this situation argument conversion is attempted based on each type and the whole conversion fails if none of these conversions succeed.
When using function annotations, the natural syntax to specify that an argument has multiple possible types is using Union:
from typing import Union
def example(length: Union[int, float], padding: Union[int, str, None] = None):
...
When using Python 3.10 or newer, it is possible to use the native type1 | type2 syntax instead:
def example(length: int | float, padding: int | str | None = None):
...
Robot Framework 7.0 enhanced the support for the union syntax so that also "stringly typed" unions like 'type1 | type2' work. This syntax works also with older Python versions:
def example(length: 'int | float', padding: 'int | str | None' = None):
...
An alternative is specifying types as a tuple. It is not recommended with annotations, because that syntax is not supported by other tools, but it works well with the @keyword decorator:
from robot.api.deco import keyword
@keyword(types={'length': (int, float), 'padding': (int, str, None)})
def example(length, padding=None):
...
With the above examples the length argument would first be converted to an integer and if that fails then to a float. The padding would be first converted to an integer, then to a string, and finally to None.
If the given argument has one of the accepted types, then no conversion is done and the argument is used as-is. For example, if the length argument gets value 1.5 as a float, it would not be converted to an integer. Notice that using non-string values like floats as an argument requires using variables as these examples giving different values to the length argument demonstrate:
*** Test Cases ***
Conversion
Example 10 # Argument is a string. Converted to an integer.
Example 1.5 # Argument is a string. Converted to a float.
Example ${10} # Argument is an integer. Accepted as-is.
Example ${1.5} # Argument is a float. Accepted as-is.
If one of the accepted types is string, then no conversion is done if the given argument is a string. As the following examples giving different values to the padding argument demonstrate, also in these cases passing other types is possible using variables:
*** Test Cases ***
Conversion
Example 1 big # Argument is a string. Accepted as-is.
Example 1 10 # Argument is a string. Accepted as-is.
Example 1 ${10} # Argument is an integer. Accepted as-is.
Example 1 ${None} # Argument is `None`. Accepted as-is.
Example 1 ${1.5} # Argument is a float. Converted to an integer.
If the given argument does not have any of the accepted types, conversion is attempted in the order types are specified. If any conversion succeeds, the resulting value is used without attempting remaining conversions. If no individual conversion succeeds, the whole conversion fails.
If a specified type is not recognized by Robot Framework, then the original argument value is used as-is. For example, with this keyword conversion would first be attempted to an integer, but if that fails the keyword would get the original argument:
def example(argument: Union[int, Unrecognized]):
...
Starting from Robot Framework 6.1, the above logic works also if an unrecognized type is listed before a recognized type like Union[Unrecognized, int]. Also in this case int conversion is attempted, and the argument id passed as-is if it fails. With earlier Robot Framework versions, int conversion would not be attempted at all.
With generics also the parameterized syntax like list[int] or dict[str, int] works. When this syntax is used, the given value is first converted to the base type and then individual items are converted to the nested types. Conversion with different generic types works according to these rules:
list[float]. All list items are converted to that type.tuple[int, int] and tuple[str, int, bool]. Tuples used as arguments are expected to have exactly that amount of items and they are converted to matching types.tuple[int, ...]. In this case tuple can have any number of items and they are all converted to the specified type.dict[str, int]. Dictionary keys are converted using the former type and values using the latter.set[float]. Conversion logic is the same as with lists.Using the native list[int] syntax requires Python 3.9 or newer. If there is a need to support also earlier Python versions, it is possible to either use matching types from the typing module like List[int] or use the "stringly typed" syntax like 'list[int]'.
Note
Support for converting nested types with generics is new in Robot Framework 6.0. Same syntax works also with earlier versions, but arguments are only converted to the base type and nested types are not used for anything.
Note
Support for "stringly typed" parameterized generics is new in Robot Framework 7.0.
Custom argument convertersIn addition to doing argument conversion automatically as explained in the previous sections, Robot Framework supports custom argument conversion. This functionality has two main use cases:
Argument converters are functions or other callables that get arguments used in data and convert them to desired format before arguments are passed to keywords. Converters are registered for libraries by setting ROBOT_LIBRARY_CONVERTERS attribute (case-sensitive) to a dictionary mapping desired types to converts. When implementing a library as a module, this attribute must be set on the module level, and with class based libraries it must be a class attribute. With libraries implemented as classes, it is also possible to use the converters argument with the @library decorator. Both of these approaches are illustrated by examples in the following sections.
Note
Custom argument converters are new in Robot Framework 5.0.
Overriding default convertersLet's assume we wanted to create a keyword that accepts date objects for users in Finland where the commonly used date format is dd.mm.yyyy. The usage could look something like this:
*** Test Cases ***
Example
Keyword 25.1.2022
Automatic argument conversion supports dates, but it expects them to be in yyyy-mm-dd format so it will not work. A solution is creating a custom converter and registering it to handle date conversion:
from datetime import date
# Converter function.
def parse_fi_date(value):
day, month, year = value.split('.')
return date(int(year), int(month), int(day))
# Register converter function for the specified type.
ROBOT_LIBRARY_CONVERTERS = {date: parse_fi_date}
# Keyword using custom converter. Converter is resolved based on argument type.
def keyword(arg: date):
print(f'year: {arg.year}, month: {arg.month}, day: {arg.day}')
Conversion errors
If we try using the above keyword with invalid argument like invalid, it fails with this error:
ValueError: Argument 'arg' got value 'invalid' that cannot be converted to date: not enough values to unpack (expected 3, got 1)
This error is not too informative and does not tell anything about the expected format. Robot Framework cannot provide more information automatically, but the converter itself can be enhanced to validate the input. If the input is invalid, the converter should raise a ValueError with an appropriate message. In this particular case there would be several ways to validate the input, but using regular expressions makes it possible to validate both that the input has dots (.) in correct places and that date parts contain correct amount of digits:
from datetime import date
import re
def parse_fi_date(value):
# Validate input using regular expression and raise ValueError if not valid.
match = re.match(r'(\d{1,2})\.(\d{1,2})\.(\d{4})$', value)
if not match:
raise ValueError(f"Expected date in format 'dd.mm.yyyy', got '{value}'.")
day, month, year = match.groups()
return date(int(year), int(month), int(day))
ROBOT_LIBRARY_CONVERTERS = {date: parse_fi_date}
def keyword(arg: date):
print(f'year: {arg.year}, month: {arg.month}, day: {arg.day}')
With the above converter code, using the keyword with argument invalid fails with a lot more helpful error message:
ValueError: Argument 'arg' got value 'invalid' that cannot be converted to date: Expected date in format 'dd.mm.yyyy', got 'invalid'.Restricting value types
By default Robot Framework tries to use converters with all given arguments regardless their type. This means that if the earlier example keyword would be used with a variable containing something else than a string, conversion code would fail in the re.match call. For example, trying to use it with argument ${42} would fail like this:
ValueError: Argument 'arg' got value '42' (integer) that cannot be converted to date: TypeError: expected string or bytes-like object
This error situation could naturally handled in the converter code by checking the value type, but if the converter only accepts certain types, it is typically easier to just restrict the value to that type. Doing it requires only adding appropriate type hint to the converter:
def parse_fi_date(value: str):
...
Notice that this type hint is not used for converting the value before calling the converter, it is used for strictly restricting which types can be used. With the above addition calling the keyword with ${42} would fail like this:
ValueError: Argument 'arg' got value '42' (integer) that cannot be converted to date.
If the converter can accept multiple types, it is possible to specify types as a Union. For example, if we wanted to enhance our keyword to accept also integers so that they would be considered seconds since the Unix epoch, we could change the converter like this:
from datetime import date
import re
from typing import Union
# Accept both strings and integers.
def parse_fi_date(value: Union[str, int]):
# Integers are converted separately.
if isinstance(value, int):
return date.fromtimestamp(value)
match = re.match(r'(\d{1,2})\.(\d{1,2})\.(\d{4})$', value)
if not match:
raise ValueError(f"Expected date in format 'dd.mm.yyyy', got '{value}'.")
day, month, year = match.groups()
return date(int(year), int(month), int(day))
ROBOT_LIBRARY_CONVERTERS = {date: parse_fi_date}
def keyword(arg: date):
print(f'year: {arg.year}, month: {arg.month}, day: {arg.day}')
Converting custom types
A problem with the earlier example is that date objects could only be given in dd.mm.yyyy format. It would not work if there was a need to support dates in different formats like in this example:
*** Test Cases ***
Example
Finnish 25.1.2022
US 1/25/2022
ISO 8601 2022-01-22
A solution to this problem is creating custom types instead of overriding the default date conversion:
from datetime import date
import re
from typing import Union
from robot.api.deco import keyword, library
# Custom type. Extends an existing type but that is not required.
class FiDate(date):
# Converter function implemented as a classmethod. It could be a normal
# function as well, but this way all code is in the same class.
@classmethod
def from_string(cls, value: str):
match = re.match(r'(\d{1,2})\.(\d{1,2})\.(\d{4})$', value)
if not match:
raise ValueError(f"Expected date in format 'dd.mm.yyyy', got '{value}'.")
day, month, year = match.groups()
return cls(int(year), int(month), int(day))
# Another custom type.
class UsDate(date):
@classmethod
def from_string(cls, value: str):
match = re.match(r'(\d{1,2})/(\d{1,2})/(\d{4})$', value)
if not match:
raise ValueError(f"Expected date in format 'mm/dd/yyyy', got '{value}'.")
month, day, year = match.groups()
return cls(int(year), int(month), int(day))
# Register converters using '@library' decorator.
@library(converters={FiDate: FiDate.from_string, UsDate: UsDate.from_string})
class Library:
# Uses custom converter supporting 'dd.mm.yyyy' format.
@keyword
def finnish(self, arg: FiDate):
print(f'year: {arg.year}, month: {arg.month}, day: {arg.day}')
# Uses custom converter supporting 'mm/dd/yyyy' format.
@keyword
def us(self, arg: UsDate):
print(f'year: {arg.year}, month: {arg.month}, day: {arg.day}')
# Uses IS0-8601 compatible default conversion.
@keyword
def iso_8601(self, arg: date):
print(f'year: {arg.year}, month: {arg.month}, day: {arg.day}')
# Accepts date in different formats.
@keyword
def any(self, arg: Union[FiDate, UsDate, date]):
print(f'year: {arg.year}, month: {arg.month}, day: {arg.day}')
Strict type validation
Converters are not used at all if the argument is of the specified type to begin with. It is thus easy to enable strict type validation with a custom converter that does not accept any value. For example, the Example keyword accepts only StrictType instances:
class StrictType:
pass
def strict_converter(arg):
raise TypeError(f'Only StrictType instances accepted, got {type(arg).__name__}.')
ROBOT_LIBRARY_CONVERTERS = {StrictType: strict_converter}
def example(argument: StrictType):
assert isinstance(argument, StrictType)
As a convenience, Robot Framework allows setting converter to None to get the same effect. For example, this code behaves exactly the same way as the code above:
class StrictType:
pass
ROBOT_LIBRARY_CONVERTERS = {StrictType: None}
def example(argument: StrictType):
assert isinstance(argument, StrictType)
Note
Using None as a strict converter is new in Robot Framework 6.0. An explicit converter function needs to be used with earlier versions.
Starting from Robot Framework 6.1, it is possible to access the library instance from a converter function. This allows defining dynamic type conversions that depend on the library state. For example, if the library can be configured to test particular locale, you might use the library state to determine how a date should be parsed like this:
from datetime import date
import re
def parse_date(value, library):
# Validate input using regular expression and raise ValueError if not valid.
# Use locale based from library state to determine parsing format.
if library.locale == 'en_US':
match = re.match(r'(?P<month>\d{1,2})/(?P<day>\d{1,2})/(?P<year>\d{4})$', value)
format = 'mm/dd/yyyy'
else:
match = re.match(r'(?P<day>\d{1,2})\.(?P<month>\d{1,2})\.(?P<year>\d{4})$', value)
format = 'dd.mm.yyyy'
if not match:
raise ValueError(f"Expected date in format '{format}', got '{value}'.")
return date(int(match.group('year')), int(match.group('month')), int(match.group('day')))
ROBOT_LIBRARY_CONVERTERS = {date: parse_date}
def keyword(arg: date):
print(f'year: {arg.year}, month: {arg.month}, day: {arg.day}')
The library argument to converter function is optional, i.e. if the converter function only accepts one argument, the library argument is omitted. Similar result can be achieved by making the converter function accept only variadic arguments, e.g. def parse_date(*varargs).
Information about converters is added to outputs produced by Libdoc automatically. This information includes the name of the type, accepted values (if specified using type hints) and documentation. Type information is automatically linked to all keywords using these types.
Used documentation is got from the converter function by default. If it does not have any documentation, documentation is got from the type. Both of these approaches to add documentation to converters in the previous example thus produce the same result:
class FiDate(date):
@classmethod
def from_string(cls, value: str):
"""Date in ``dd.mm.yyyy`` format."""
...
class UsDate(date):
"""Date in ``mm/dd/yyyy`` format."""
@classmethod
def from_string(cls, value: str):
...
Adding documentation is in general recommended to provide users more information about conversion. It is especially important to document converter functions registered for existing types, because their own documentation is likely not very useful in this context.
Using custom decoratorsWhen implementing keywords, it is sometimes useful to modify them with Python decorators. However, decorators often modify function signatures and can thus confuse Robot Framework's introspection when determining which arguments keywords accept. This is especially problematic when creating library documentation with Libdoc and when using external tools like RIDE. The easiest way to avoid this problem is decorating the decorator itself using functools.wraps. Other solutions include using external modules like decorator and wrapt that allow creating fully signature-preserving decorators.
Note
Support for "unwrapping" decorators decorated with functools.wraps is a new feature in Robot Framework 3.2.
Library keywords can also accept embedded arguments the same way as user keywords. This section mainly covers the Python syntax to use to create such keywords, the embedded arguments syntax itself is covered in detail as part of user keyword documentation.
Library keywords with embedded arguments need to have a custom name that is typically set using the @keyword decorator. Values matching embedded arguments are passed to the function or method implementing the keyword as positional arguments. If the function or method accepts more arguments, they can be passed to the keyword as normal positional or named arguments. Argument names do not need to match the embedded argument names, but that is generally a good convention.
Keywords accepting embedded arguments:
from robot.api.deco import keyword
@keyword('Select ${animal} from list')
def select_animal_from_list(animal):
...
@keyword('Number of ${animals} should be')
def number_of_animals_should_be(animals, count):
...
Tests using the above keywords:
*** Test Cases ***
Embedded arguments
Select cat from list
Select dog from list
Embedded and normal arguments
Number of cats should be 2
Number of dogs should be count=3
If type information is specified, automatic argument conversion works also with embedded arguments:
@keyword('Add ${quantity} copies of ${item} to cart')
def add_copies_to_cart(quantity: int, item: str):
...
Note
Embedding type information to keyword names like Add ${quantity: int} copies of ${item: str} to cart similarly as with user keywords is not supported with library keywords.
Note
Support for mixing embedded arguments and normal arguments is new in Robot Framework 7.0.
Asynchronous keywordsStarting from Robot Framework 6.1, it is possible to run native asynchronous functions (created by async def) just like normal functions:
import asyncio
from robot.api.deco import keyword
@keyword
async def this_keyword_waits():
await asyncio.sleep(5)
You can get the reference of the loop using asyncio.get_running_loop() or asyncio.get_event_loop(). Be careful when modifying how the loop runs, it is a global resource. For example, never call loop.close() because it will make it impossible to run any further coroutines. If you have any function or resource that requires the event loop, even though await is not used explicitly, you have to define your function as async to have the event loop available.
More examples of functionality:
import asyncio
from robot.api.deco import keyword
async def task_async():
await asyncio.sleep(5)
@keyword
async def examples():
tasks = [task_async() for _ in range(10)]
results = await asyncio.gather(*tasks)
background_task = asyncio.create_task(task_async())
await background_task
# If running with Python 3.10 or higher
async with asyncio.TaskGroup() as tg:
task1 = tg.create_task(task_async())
task2 = tg.create_task(task_async())
Note
Robot Framework waits for the function to complete. If you want to have a task that runs for a long time, use, for example, asyncio.create_task(). It is your responsibility to manage the task and save a reference to avoid it being garbage collected. If the event loop closes and a task is still pending, a message will be printed to the console.
Note
If execution of keyword cannot continue for some reason, for example a signal stop, Robot Framework will cancel the async task and any of its children. Other async tasks will continue running normally.
4.1.4 Communicating with Robot FrameworkAfter a method implementing a keyword is called, it can use any mechanism to communicate with the system under test. It can then also send messages to Robot Framework's log file, return information that can be saved to variables and, most importantly, report if the keyword passed or not.
Reporting keyword statusReporting keyword status is done simply using exceptions. If an executed method raises an exception, the keyword status is FAIL, and if it returns normally, the status is PASS.
Normal execution failures and errors can be reported using the standard exceptions such as AssertionError, ValueError and RuntimeError. There are, however, some special cases explained in the subsequent sections where special exceptions are needed.
The error message shown in logs, reports and the console is created from the exception type and its message. With generic exceptions (for example, AssertionError, Exception, and RuntimeError), only the exception message is used, and with others, the message is created in the format ExceptionType: Actual message.
It is possible to avoid adding the exception type as a prefix to failure message also with non generic exceptions. This is done by adding a special ROBOT_SUPPRESS_NAME attribute with value True to your exception.
Python:
class MyError(RuntimeError):
ROBOT_SUPPRESS_NAME = True
In all cases, it is important for the users that the exception message is as informative as possible.
HTML in error messagesIt is also possible to have HTML formatted error messages by starting the message with text *HTML*:
raise AssertionError("*HTML* <a href='robotframework.org'>Robot Framework</a> rulez!!")
This method can be used both when raising an exception in a library, like in the example above, and when users provide an error message in the test data.
Cutting long messages automaticallyIf the error message is longer than 40 lines, it will be automatically cut from the middle to prevent reports from getting too long and difficult to read. The full error message is always shown in the log message of the failed keyword.
TracebacksThe traceback of the exception is also logged using DEBUG log level. These messages are not visible in log files by default because they are very rarely interesting for normal users. When developing libraries, it is often a good idea to run tests using --loglevel DEBUG.
Robot Framework provides some exceptions that libraries can use for reporting failures and other events. These exceptions are exposed via the robot.api package and contain the following:
Failure
AssertionError. The main benefit of using this exception is that its name is consistent with other provided exceptions.
Error
Failure exception or the standard AssertionError. This exception can be used, for example, if the keyword is used incorrectly. There is no practical difference, other than consistent naming with other provided exceptions, compared to using this exception and the standard RuntimeError.
ContinuableFailure
SkipExecution
FatalError
Note
All these exceptions are new in Robot Framework 4.0. Other features than skipping tests, which is also new in Robot Framework 4.0, are available by other means in earlier versions.
Continuable failuresIt is possible to continue test execution even when there are failures. The easiest way to do that is using the provided robot.api.ContinuableFailure exception:
from robot.api import ContinuableFailure
def example_keyword():
if something_is_wrong():
raise ContinuableFailure('Something is wrong but execution can continue.')
...
An alternative is creating a custom exception that has a special ROBOT_CONTINUE_ON_FAILURE attribute set to a True value. This is demonstrated by the example below.
class MyContinuableError(RuntimeError):
ROBOT_CONTINUE_ON_FAILURE = True
Skipping tests
It is possible to skip tests with a library keyword. The easiest way to do that is using the provided robot.api.SkipExecution exception:
from robot.api import SkipExecution
def example_keyword():
if test_should_be_skipped():
raise SkipExecution('Cannot proceed, skipping test.')
...
An alternative is creating a custom exception that has a special ROBOT_SKIP_EXECUTION attribute set to a True value. This is demonstrated by the example below.
class MySkippingError(RuntimeError):
ROBOT_SKIP_EXECUTION = True
Stopping test execution
It is possible to fail a test case so that the whole test execution is stopped. The easiest way to accomplish this is using the provided robot.api.FatalError exception:
from robot.api import FatalError
def example_keyword():
if system_is_not_running():
raise FatalError('System is not running!')
...
In addition to using the robot.api.FatalError exception, it is possible create a custom exception that has a special ROBOT_EXIT_ON_FAILURE attribute set to a True value. This is illustrated by the example below.
class MyFatalError(RuntimeError):
ROBOT_EXIT_ON_FAILURE = True
Logging information
Exception messages are not the only way to give information to the users. In addition to them, methods can also send messages to log files simply by writing to the standard output stream (stdout) or to the standard error stream (stderr), and they can even use different log levels. Another, and often better, logging possibility is using the programmatic logging APIs.
By default, everything written by a method into the standard output is written to the log file as a single entry with the log level INFO. Messages written into the standard error are handled similarly otherwise, but they are echoed back to the original stderr after the keyword execution has finished. It is thus possible to use the stderr if you need some messages to be visible on the console where tests are executed.
To use other log levels than INFO, or to create several messages, specify the log level explicitly by embedding the level into the message in the format *LEVEL* Actual log message. In this formant *LEVEL* must be in the beginning of a line and LEVEL must be one of the available concrete log levels TRACE, DEBUG, INFO, WARN or ERROR, or a pseudo log level HTML or CONSOLE. The pseudo levels can be used for logging HTML and logging to console, respectively.
Messages with ERROR or WARN level are automatically written to the console and a separate Test Execution Errors section in the log files. This makes these messages more visible than others and allows using them for reporting important but non-critical problems to users.
Everything normally logged by the library will be converted into a format that can be safely represented as HTML. For example, <b>foo</b> will be displayed in the log exactly like that and not as foo. If libraries want to use formatting, links, display images and so on, they can use a special pseudo log level HTML. Robot Framework will write these messages directly into the log with the INFO level, so they can use any HTML syntax they want. Notice that this feature needs to be used with care, because, for example, one badly placed </table> tag can ruin the log file quite badly.
When using the public logging API, various logging methods have optional html attribute that can be set to True to enable logging in HTML format.
By default messages logged via the standard output or error streams get their timestamps when the executed keyword ends. This means that the timestamps are not accurate and debugging problems especially with longer running keywords can be problematic.
Keywords have a possibility to add an accurate timestamp to the messages they log if there is a need. The timestamp must be given as milliseconds since the Unix epoch and it must be placed after the log level separated from it with a colon:
*INFO:1308435758660* Message with timestamp *HTML:1308435758661* <b>HTML</b> message with timestamp
As illustrated by the examples below, adding the timestamp is easy. It is, however, even easier to get accurate timestamps using the programmatic logging APIs. A big benefit of adding timestamps explicitly is that this approach works also with the remote library interface.
import time
def example_keyword():
timestamp = int(time.time() * 1000)
print(f'*INFO:{timestamp}* Message with timestamp')
Logging to console
Libraries have several options for writing messages to the console. As already discussed, warnings and all messages written to the standard error stream are written both to the log file and to the console. Both of these options have a limitation that the messages end up to the console only after the currently executing keyword finishes.
Starting from Robot Framework 6.1, libraries can use a pseudo log level CONSOLE for logging messages both to the log file and to the console:
def my_keyword(arg):
print('*CONSOLE* Message both to log and to console.')
These messages will be logged to the log file using the INFO level similarly as with the HTML pseudo log level. When using this approach, messages are logged to the console only after the keyword execution ends.
Another option is writing messages to sys.__stdout__ or sys.__stderr__. When using this approach, messages are written to the console immediately and are not written to the log file at all:
import sys
def my_keyword(arg):
print('Message only to console.', file=sys.__stdout__)
The final option is using the public logging API. Also in with this approach messages are written to the console immediately:
from robot.api import logger
def log_to_console(arg):
logger.console('Message only to console.')
def log_to_console_and_log_file(arg):
logger.info('Message both to log and to console.', also_console=True)
Logging example
In most cases, the INFO level is adequate. The levels below it, DEBUG and TRACE, are useful for writing debug information. These messages are normally not shown, but they can facilitate debugging possible problems in the library itself. The WARN or ERROR level can be used to make messages more visible and HTML is useful if any kind of formatting is needed. Level CONSOLE can be used when the message needs to shown both in console and in the log file.
The following examples clarify how logging with different levels works.
print('Hello from a library.')
print('*WARN* Warning from a library.')
print('*ERROR* Something unexpected happen that may indicate a problem in the test.')
print('*INFO* Hello again!')
print('This will be part of the previous message.')
print('*INFO* This is a new message.')
print('*INFO* This is <b>normal text</b>.')
print('*CONSOLE* This logs into console and log file.')
print('*HTML* This is <b>bold</b>.')
print('*HTML* <a href="http://robotframework.org">Robot Framework</a>')
16:18:42.123 INFO Hello from a library. 16:18:42.123 WARN Warning from a library. 16:18:42.123 ERROR Something unexpected happen that may indicate a problem in the test. 16:18:42.123 INFO Hello again!
Programmatic APIs provide somewhat cleaner way to log information than using the standard output and error streams.
Public logging APIRobot Framework has a Python based logging API for writing messages to the log file and to the console. Test libraries can use this API like logger.info('My message') instead of logging through the standard output like print('*INFO* My message'). In addition to a programmatic interface being a lot cleaner to use, this API has a benefit that the log messages have accurate timestamps.
The public logging API is thoroughly documented as part of the API documentation at https://robot-framework.readthedocs.org. Below is a simple usage example:
from robot.api import logger
def my_keyword(arg):
logger.debug(f"Got argument '{arg}'.")
do_something()
logger.info('<i>This</i> is a boring example', html=True)
logger.console('Hello, console!')
An obvious limitation is that test libraries using this logging API have a dependency to Robot Framework. If Robot Framework is not running, the messages are redirected automatically to Python's standard logging module.
Using Python's standardlogging module
In addition to the new public logging API, Robot Framework offers a built-in support to Python's standard logging module. This works so that all messages that are received by the root logger of the module are automatically propagated to Robot Framework's log file. Also this API produces log messages with accurate timestamps, but logging HTML messages or writing messages to the console are not supported. A big benefit, illustrated also by the simple example below, is that using this logging API creates no dependency to Robot Framework.
import logging
def my_keyword(arg):
logging.debug(f"Got argument '{arg}'.")
do_something()
logging.info('This is a boring example')
The logging module has slightly different log levels than Robot Framework. Its levels DEBUG, INFO, WARNING and ERROR are mapped directly to the matching Robot Framework log levels, and CRITICAL is mapped to ERROR. Custom log levels are mapped to the closest standard level smaller than the custom level. For example, a level between INFO and WARNING is mapped to Robot Framework's INFO level.
Libraries can also log during the test library import and initialization. These messages do not appear in the log file like the normal log messages, but are instead written to the syslog. This allows logging any kind of useful debug information about the library initialization. Messages logged using the WARN or ERROR levels are also visible in the test execution errors section in the log file.
Logging during the import and initialization is possible both using the standard output and error streams and the programmatic logging APIs. Both of these are demonstrated below.
Library logging using the logging API during import:
from robot.api import logger
logger.debug("Importing library")
def keyword():
...
Note
If you log something during initialization, i.e. in Python __init__, the messages may be logged multiple times depending on the library scope.
The final way for keywords to communicate back to the core framework is returning information retrieved from the system under test or generated by some other means. The returned values can be assigned to variables in the test data and then used as inputs for other keywords, even from different test libraries.
Values are returned using the return statement in methods. Normally, one value is assigned into one scalar variable, as illustrated in the example below. This example also illustrates that it is possible to return any objects and to use extended variable syntax to access object attributes.
from mymodule import MyObject
def return_string():
return "Hello, world!"
def return_object(name):
return MyObject(name)
*** Test Cases ***
Returning one value
${string} = Return String
Should Be Equal ${string} Hello, world!
${object} = Return Object Robot
Should Be Equal ${object.name} Robot
Keywords can also return values so that they can be assigned into several scalar variables at once, into a list variable, or into scalar variables and a list variable. All these usages require that returned values are lists or list-like objects.
def return_two_values():
return 'first value', 'second value'
def return_multiple_values():
return ['a', 'list', 'of', 'strings']
*** Test Cases ***
Returning multiple values
${var1} ${var2} = Return Two Values
Should Be Equal ${var1} first value
Should Be Equal ${var2} second value
@{list} = Return Two Values
Should Be Equal @{list}[0] first value
Should Be Equal @{list}[1] second value
${s1} ${s2} @{li} = Return Multiple Values
Should Be Equal ${s1} ${s2} a list
Should Be Equal @{li}[0] @{li}[1] of strings
Detecting is Robot Framework running
Starting from Robot Framework 6.1, it is easy to detect is Robot Framework running at all and is the dry-run mode active by using the robot_running and dry_run_active properties of the BuiltIn library. A relatively common use case is that library initializers may want to avoid doing some work if the library is not used during execution but is initialized, for example, by Libdoc:
from robot.libraries.BuiltIn import BuiltIn
class MyLibrary:
def __init__(self):
builtin = BuiltIn()
if builtin.robot_running and not builtin.dry_run_active:
# Do some initialization that only makes sense during real execution.
For more information about using the BuiltIn library as a programmatic API, including another example using robot_running, see the Using BuiltIn library section.
If a library uses threads, it should generally communicate with the framework only from the main thread. If a worker thread has, for example, a failure to report or something to log, it should pass the information first to the main thread, which can then use exceptions or other mechanisms explained in this section for communication with the framework.
This is especially important when threads are run on background while other keywords are running. Results of communicating with the framework in that case are undefined and can in the worst case cause a crash or a corrupted output file. If a keyword starts something on background, there should be another keyword that checks the status of the worker thread and reports gathered information accordingly.
Messages logged by non-main threads using the normal logging methods from programmatic logging APIs are silently ignored.
There is also a BackgroundLogger in separate robotbackgroundlogger project, with a similar API as the standard robot.api.logger. Normal logging methods will ignore messages from other than main thread, but the BackgroundLogger will save the background messages so that they can be later logged to Robot's log.
A test library without documentation about what keywords it contains and what those keywords do is rather useless. To ease maintenance, it is highly recommended that library documentation is included in the source code and generated from it. Basically, that means using docstrings as in the example below.
class MyLibrary:
"""This is an example library with some documentation."""
def keyword_with_short_documentation(self, argument):
"""This keyword has only a short documentation"""
pass
def keyword_with_longer_documentation(self):
"""First line of the documentation is here.
Longer documentation continues here and it can contain
multiple lines or paragraphs.
"""
pass
Python has tools for creating an API documentation of a library documented as above. However, outputs from these tools can be slightly technical for some users. Another alternative is using Robot Framework's own documentation tool Libdoc. This tool can create a library documentation from libraries using the static library API, such as the ones above, but it also handles libraries using the dynamic library API and hybrid library API.
The first logical line of a keyword documentation, until the first empty line, is used for a special purpose and should contain a short overall description of the keyword. It is used as a short documentation by Libdoc (for example, as a tool tip) and also shown in the test logs.
By default documentation is considered to follow Robot Framework's documentation formatting rules. This simple format allows often used styles like *bold* and _italic_, tables, lists, links, etc. It is possible to use also HTML, plain text and reStructuredText formats. See the Documentation format section for information how to set the format in the library source code and Libdoc chapter for more information about the formats in general.
Note
Prior to Robot Framework 3.1, the short documentation contained only the first physical line of the keyword documentation.
Testing librariesAny non-trivial test library needs to be thoroughly tested to prevent bugs in them. Of course, this testing should be automated to make it easy to rerun tests when libraries are changed.
Python has excellent unit testing tools, and they suite very well for testing libraries. There are no major differences in using them for this purpose compared to using them for some other testing. The developers familiar with these tools do not need to learn anything new, and the developers not familiar with them should learn them anyway.
It is also easy to use Robot Framework itself for testing libraries and that way have actual end-to-end acceptance tests for them. There are plenty of useful keywords in the BuiltIn library for this purpose. One worth mentioning specifically is Run Keyword And Expect Error, which is useful for testing that keywords report errors correctly.
Whether to use a unit- or acceptance-level testing approach depends on the context. If there is a need to simulate the actual system under test, it is often easier on the unit level. On the other hand, acceptance tests ensure that keywords do work through Robot Framework. If you cannot decide, of course it is possible to use both the approaches.
Packaging librariesAfter a library is implemented, documented, and tested, it still needs to be distributed to the users. With simple libraries consisting of a single file, it is often enough to ask the users to copy that file somewhere and set the module search path accordingly. More complicated libraries should be packaged to make the installation easier.
Since libraries are normal programming code, they can be packaged using normal packaging tools. For information about packaging and distributing Python code see https://packaging.python.org/. When such a package is installed using pip or other tools, it is automatically in the module search path.
Deprecating keywordsSometimes there is a need to replace existing keywords with new ones or remove them altogether. Just informing the users about the change may not always be enough, and it is more efficient to get warnings at runtime. To support that, Robot Framework has a capability to mark keywords deprecated. This makes it easier to find old keywords from the test data and remove or replace them.
Keywords can be deprecated by starting their documentation with text *DEPRECATED, case-sensitive, and having a closing * also on the first line of the documentation. For example, *DEPRECATED*, *DEPRECATED.*, and *DEPRECATED in version 1.5.* are all valid markers.
When a deprecated keyword is executed, a deprecation warning is logged and the warning is shown also in the console and the Test Execution Errors section in log files. The deprecation warning starts with text Keyword '<name>' is deprecated. and has rest of the short documentation after the deprecation marker, if any, afterwards. For example, if the following keyword is executed, there will be a warning like shown below in the log file.
def example_keyword(argument):
"""*DEPRECATED!!* Use keyword `Other Keyword` instead.
This keyword does something to given ``argument`` and returns results.
"""
return do_something(argument)
20080911 16:00:22.650 WARN Keyword 'SomeLibrary.Example Keyword' is deprecated. Use keyword `Other Keyword` instead.
This deprecation system works with most test libraries and also with user keywords.
4.1.6 Dynamic library APIThe dynamic API is in most ways similar to the static API. For example, reporting the keyword status, logging, and returning values works exactly the same way. Most importantly, there are no differences in importing dynamic libraries and using their keywords compared to other libraries. In other words, users do not need to know what APIs their libraries use.
Only differences between static and dynamic libraries are how Robot Framework discovers what keywords a library implements, what arguments and documentation these keywords have, and how the keywords are actually executed. With the static API, all this is done using reflection, but dynamic libraries have special methods that are used for these purposes.
One of the benefits of the dynamic API is that you have more flexibility in organizing your library. With the static API, you must have all keywords in one class or module, whereas with the dynamic API, you can, for example, implement each keyword as a separate class. This use case is not so important with Python, though, because its dynamic capabilities and multi-inheritance already give plenty of flexibility, and there is also possibility to use the hybrid library API.
Another major use case for the dynamic API is implementing a library so that it works as proxy for an actual library possibly running on some other process or even on another machine. This kind of a proxy library can be very thin, and because keyword names and all other information is got dynamically, there is no need to update the proxy when new keywords are added to the actual library.
This section explains how the dynamic API works between Robot Framework and dynamic libraries. It does not matter for Robot Framework how these libraries are actually implemented (for example, how calls to the run_keyword method are mapped to a correct keyword implementation), and many different approaches are possible. Python users may also find the PythonLibCore project useful.
Dynamic libraries tell what keywords they implement with the get_keyword_names method. This method cannot take any arguments, and it must return a list or array of strings containing the names of the keywords that the library implements.
If the returned keyword names contain several words, they can be returned separated with spaces or underscores, or in the camelCase format. For example, ['first keyword', 'second keyword'], ['first_keyword', 'second_keyword'], and ['firstKeyword', 'secondKeyword'] would all be mapped to keywords First Keyword and Second Keyword.
Dynamic libraries must always have this method. If it is missing, or if calling it fails for some reason, the library is considered a static library.
Marking methods to expose as keywordsIf a dynamic library should contain both methods which are meant to be keywords and methods which are meant to be private helper methods, it may be wise to mark the keyword methods as such so it is easier to implement get_keyword_names. The robot.api.deco.keyword decorator allows an easy way to do this since it creates a custom 'robot_name' attribute on the decorated method. This allows generating the list of keywords just by checking for the robot_name attribute on every method in the library during get_keyword_names.
from robot.api.deco import keyword
class DynamicExample:
def get_keyword_names(self):
# Get all attributes and their values from the library.
attributes = [(name, getattr(self, name)) for name in dir(self)]
# Filter out attributes that do not have 'robot_name' set.
keywords = [(name, value) for name, value in attributes
if hasattr(value, 'robot_name')]
# Return value of 'robot_name', if given, or the original 'name'.
return [value.robot_name or name for name, value in keywords]
def helper_method(self):
...
@keyword
def keyword_method(self):
...
Running keywords
Dynamic libraries have a special run_keyword (alias runKeyword) method for executing their keywords. When a keyword from a dynamic library is used in the test data, Robot Framework uses the run_keyword method to get it executed. This method takes two or three arguments. The first argument is a string containing the name of the keyword to be executed in the same format as returned by get_keyword_names. The second argument is a list of positional arguments given to the keyword in the test data, and the optional third argument is a dictionary containing named arguments. If the third argument is missing, free named arguments and named-only arguments are not supported, and other named arguments are mapped to positional arguments.
Note
Prior to Robot Framework 3.1, normal named arguments were mapped to positional arguments regardless did run_keyword accept two or three arguments. The third argument only got possible free named arguments.
After getting keyword name and arguments, the library can execute the keyword freely, but it must use the same mechanism to communicate with the framework as static libraries. This means using exceptions for reporting keyword status, logging by writing to the standard output or by using the provided logging APIs, and using the return statement in run_keyword for returning something.
Every dynamic library must have both the get_keyword_names and run_keyword methods but rest of the methods in the dynamic API are optional. The example below shows a working, albeit trivial, dynamic library.
class DynamicExample:
def get_keyword_names(self):
return ['first keyword', 'second keyword']
def run_keyword(self, name, args, named_args):
print(f"Running keyword '{name}' with positional arguments {args} "
f"and named arguments {named_args}.")
Getting keyword arguments
If a dynamic library only implements the get_keyword_names and run_keyword methods, Robot Framework does not have any information about the arguments that the implemented keywords accept. For example, both First Keyword and Second Keyword in the example above could be used with any arguments. This is problematic, because most real keywords expect a certain number of keywords, and under these circumstances they would need to check the argument counts themselves.
Dynamic libraries can communicate what arguments their keywords expect by using the get_keyword_arguments (alias getKeywordArguments) method. This method gets the name of a keyword as an argument, and it must return a list of strings containing the arguments accepted by that keyword.
Similarly as other keywords, dynamic keywords can require any number of positional arguments, have default values, accept variable number of arguments, accept free named arguments and have named-only arguments. The syntax how to represent all these different variables is derived from how they are specified in Python and explained in the following table.
Representing different arguments withget_keyword_arguments Argument type How to represent Examples No arguments Empty list.
[]
['argument']
['arg1', 'arg2', 'arg3']
Two ways how to represent the argument name and the default value:
=.String with = separator:
['name=default']
['a', 'b=1', 'c=2']
Tuple:
[('name', 'default')]
['a', ('b', 1), ('c', 2)]
/ marker. New in Robot Framework 6.1.
['posonly', '/']
['p', 'q', '/', 'normal']
* prefix
['*varargs']
['argument', '*rest']
['a', 'b=42', '*c']
* if there are no varargs. With or without defaults. Requires run_keyword to support named-only arguments. New in Robot Framework 3.1.
['*varargs', 'named']
['*', 'named']
['*', 'x', 'y=default']
['a', '*b', ('c', 42)]
** prefix. Requires run_keyword to support free named arguments.
['**named']
['a', ('b', 42), '**c']
['*varargs', '**kwargs']
['*', 'kwo', '**kws']
When the get_keyword_arguments is used, Robot Framework automatically calculates how many positional arguments the keyword requires and does it support free named arguments or not. If a keyword is used with invalid arguments, an error occurs and run_keyword is not even called.
The actual argument names and default values that are returned are also important. They are needed for named argument support and the Libdoc tool needs them to be able to create a meaningful library documentation.
As explained in the above table, default values can be specified with argument names either as a string like 'name=default' or as a tuple like ('name', 'default'). The main problem with the former syntax is that all default values are considered strings whereas the latter syntax allows using all objects like ('inteter', 1) or ('boolean', True). When using other objects than strings, Robot Framework can do automatic argument conversion based on them.
For consistency reasons, also arguments that do not accept default values can be specified as one item tuples. For example, ['a', 'b=c', '*d'] and [('a',), ('b', 'c'), ('*d',)] are equivalent.
If get_keyword_arguments is missing or returns Python None for a certain keyword, that keyword gets an argument specification accepting all arguments. This automatic argument spec is either [*varargs, **kwargs] or [*varargs], depending does run_keyword support free named arguments or not.
Note
Support to specify arguments as tuples like ('name', 'default') is new in Robot Framework 3.2. Support for positional-only arguments in dynamic library API is new in Robot Framework 6.1.
Robot Framework 3.1 introduced support for automatic argument conversion and the dynamic library API supports that as well. The conversion logic works exactly like with static libraries, but how the type information is specified is naturally different.
With dynamic libraries types can be returned using the optional get_keyword_types method (alias getKeywordTypes). It can return types using a list or a dictionary exactly like types can be specified when using the @keyword decorator. Type information can be specified using actual types like int, but especially if a dynamic library gets this information from external systems, using strings like 'int' or 'integer' may be easier. See the Supported conversions section for more information about supported types and how to specify them.
Robot Framework does automatic argument conversion also based on the argument default values. Earlier this did not work with the dynamic API because it was possible to specify arguments only as strings. As discussed in the previous section, this was changed in Robot Framework 3.2 and nowadays default values returned like ('example', True) are automatically used for this purpose.
Starting from Robot Framework 7.0, dynamic libraries can also specify the keyword return type by using key 'return' with an appropriate type in the returned type dictionary. This information is not used for anything during execution, but it is shown by Libdoc for documentation purposes.
If dynamic libraries want to provide keyword documentation, they can implement the get_keyword_documentation method (alias getKeywordDocumentation). It takes a keyword name as an argument and, as the method name implies, returns its documentation as a string.
The returned documentation is used similarly as the keyword documentation string with static libraries. The main use case is getting keywords' documentations into a library documentation generated by Libdoc. Additionally, the first line of the documentation (until the first \n) is shown in test logs.
The get_keyword_documentation method can also be used for specifying overall library documentation. This documentation is not used when tests are executed, but it can make the documentation generated by Libdoc much better.
Dynamic libraries can provide both general library documentation and documentation related to taking the library into use. The former is got by calling get_keyword_documentation with special value __intro__, and the latter is got using value __init__. How the documentation is presented is best tested with Libdoc in practice.
Dynamic libraries can also specify the general library documentation directly in the code as the docstring of the library class and its __init__ method. If a non-empty documentation is got both directly from the code and from the get_keyword_documentation method, the latter has precedence.
The dynamic API masks the real implementation of keywords from Robot Framework and thus makes it impossible to see where keywords are implemented. This means that editors and other tools utilizing Robot Framework APIs cannot implement features such as go-to-definition. This problem can be solved by implementing yet another optional dynamic method named get_keyword_source (alias getKeywordSource) that returns the source information.
The return value from the get_keyword_source method must be a string or None if no source information is available. In the simple case it is enough to simply return an absolute path to the file implementing the keyword. If the line number where the keyword implementation starts is known, it can be embedded to the return value like path:lineno. Returning only the line number is possible like :lineno.
The source information of the library itself is got automatically from the imported library class the same way as with other library APIs. The library source path is used with all keywords that do not have their own source path defined.
Note
Returning source information for keywords is a new feature in Robot Framework 3.2.
Named argument syntax with dynamic librariesAlso the dynamic library API supports the named argument syntax. Using the syntax works based on the argument names and default values got from the library using the get_keyword_arguments method.
If the run_keyword method accepts three arguments, the second argument gets all positional arguments as a list and the last arguments gets all named arguments as a mapping. If it accepts only two arguments, named arguments are mapped to positional arguments. In the latter case, if a keyword has multiple arguments with default values and only some of the latter ones are given, the framework fills the skipped optional arguments based on the default values returned by the get_keyword_arguments method.
Using the named argument syntax with dynamic libraries is illustrated by the following examples. All the examples use a keyword Dynamic that has an argument specification [a, b=d1, c=d2]. The comment on each row shows how run_keyword would be called in these cases if it has two arguments (i.e. signature is name, args) and if it has three arguments (i.e. name, args, kwargs).
*** Test Cases *** # args # args, kwargs
Positional only
Dynamic x # [x] # [x], {}
Dynamic x y # [x, y] # [x, y], {}
Dynamic x y z # [x, y, z] # [x, y, z], {}
Named only
Dynamic a=x # [x] # [], {a: x}
Dynamic c=z a=x b=y # [x, y, z] # [], {a: x, b: y, c: z}
Positional and named
Dynamic x b=y # [x, y] # [x], {b: y}
Dynamic x y c=z # [x, y, z] # [x, y], {c: z}
Dynamic x b=y c=z # [x, y, z] # [x], {y: b, c: z}
Intermediate missing
Dynamic x c=z # [x, d1, z] # [x], {c: z}
Note
Prior to Robot Framework 3.1, all normal named arguments were mapped to positional arguments and the optional kwargs was only used with free named arguments. With the above examples run_keyword was always called like it is nowadays called if it does not support kwargs.
Dynamic libraries can also support free named arguments (**named). A mandatory precondition for this support is that the run_keyword method takes three arguments: the third one will get the free named arguments along with possible other named arguments. These arguments are passed to the keyword as a mapping.
What arguments a keyword accepts depends on what get_keyword_arguments returns for it. If the last argument starts with **, that keyword is recognized to accept free named arguments.
Using the free named argument syntax with dynamic libraries is illustrated by the following examples. All the examples use a keyword Dynamic that has an argument specification [a=d1, b=d2, **named]. The comment shows the arguments that the run_keyword method is actually called with.
*** Test Cases *** # args, kwargs
No arguments
Dynamic # [], {}
Only positional
Dynamic x # [x], {}
Dynamic x y # [x, y], {}
Only free named
Dynamic x=1 # [], {x: 1}
Dynamic x=1 y=2 z=3 # [], {x: 1, y: 2, z: 3}
Positional and free named
Dynamic x y=2 # [x], {y: 2}
Dynamic x y=2 z=3 # [x], {y: 2, z: 3}
Positional as named and free named
Dynamic a=1 x=1 # [], {a: 1, x: 1}
Dynamic b=2 x=1 a=1 # [], {a: 1, b: 2, x: 1}
Note
Prior to Robot Framework 3.1, normal named arguments were mapped to positional arguments but nowadays they are part of the kwargs along with the free named arguments.
Starting from Robot Framework 3.1, dynamic libraries can have named-only arguments. This requires that the run_keyword method takes three arguments: the third getting the named-only arguments along with the other named arguments.
In the argument specification returned by the get_keyword_arguments method named-only arguments are specified after possible variable number of arguments (*varargs) or a lone asterisk (*) if the keyword does not accept varargs. Named-only arguments can have default values, and the order of arguments with and without default values does not matter.
Using the named-only argument syntax with dynamic libraries is illustrated by the following examples. All the examples use a keyword Dynamic that has been specified to have argument specification [positional=default, *varargs, named, named2=default, **free]. The comment shows the arguments that the run_keyword method is actually called with.
*** Test Cases *** # args, kwargs
Only named-only
Dynamic named=value # [], {named: value}
Dynamic named=value named2=2 # [], {named: value, named2: 2}
Named-only with positional and varargs
Dynamic argument named=xxx # [argument], {named: xxx}
Dynamic a1 a2 named=3 # [a1, a2], {named: 3}
Named-only with positional as named
Dynamic named=foo positional=bar # [], {positional: bar, named: foo}
Named-only with free named
Dynamic named=value foo=bar # [], {named: value, foo=bar}
Dynamic named2=2 third=3 named=1 # [], {named: 1, named2: 2, third: 3}
Summary
All special methods in the dynamic API are listed in the table below. Method names are listed in the underscore format, but their camelCase aliases work exactly the same way.
All special methods in the dynamic API Name Arguments Purposeget_keyword_names Return names of the implemented keywords. run_keyword name, arguments, kwargs Execute the specified keyword with given arguments. kwargs is optional. get_keyword_arguments name Return keywords' argument specification. Optional method. get_keyword_types name Return keywords' argument type information. Optional method. New in RF 3.1. get_keyword_tags name Return keywords' tags. Optional method. get_keyword_documentation name Return keywords' and library's documentation. Optional method. get_keyword_source name Return keywords' source. Optional method. New in RF 3.2.
A good example of using the dynamic API is Robot Framework's own Remote library.
Note
Starting from Robot Framework 7.0, dynamic libraries can have asynchronous implementations of their special methods.
4.1.7 Hybrid library APIThe hybrid library API is, as its name implies, a hybrid between the static API and the dynamic API. Just as with the dynamic API, it is possible to implement a library using the hybrid API only as a class.
Getting keyword namesKeyword names are got in the exactly same way as with the dynamic API. In practice, the library needs to have the get_keyword_names or getKeywordNames method returning a list of keyword names that the library implements.
In the hybrid API, there is no run_keyword method for executing keywords. Instead, Robot Framework uses reflection to find methods implementing keywords, similarly as with the static API. A library using the hybrid API can either have those methods implemented directly or, more importantly, it can handle them dynamically.
In Python, it is easy to handle missing methods dynamically with the __getattr__ method. This special method is probably familiar to most Python programmers and they can immediately understand the following example. Others may find it easier to consult Python Reference Manual first.
from somewhere import external_keyword
class HybridExample:
def get_keyword_names(self):
return ['my_keyword', 'external_keyword']
def my_keyword(self, arg):
print(f"My Keyword called with '{args}'.")
def __getattr__(self, name):
if name == 'external_keyword':
return external_keyword
raise AttributeError(f"Non-existing attribute '{name}'.")
Note that __getattr__ does not execute the actual keyword like run_keyword does with the dynamic API. Instead, it only returns a callable object that is then executed by Robot Framework.
Another point to be noted is that Robot Framework uses the same names that are returned from get_keyword_names for finding the methods implementing them. Thus the names of the methods that are implemented in the class itself must be returned in the same format as they are defined. For example, the library above would not work correctly, if get_keyword_names returned My Keyword instead of my_keyword.
When this API is used, Robot Framework uses reflection to find the methods implementing keywords, similarly as with the static API. After getting a reference to the method, it searches for arguments and documentation from it, in the same way as when using the static API. Thus there is no need for special methods for getting arguments and documentation like there is with the dynamic API.
SummaryWhen implementing a test library, the hybrid API has the same dynamic capabilities as the actual dynamic API. A great benefit with it is that there is no need to have special methods for getting keyword arguments and documentation. It is also often practical that the only real dynamic keywords need to be handled in __getattr__ and others can be implemented directly in the main library class.
Because of the clear benefits and equal capabilities, the hybrid API is in most cases a better alternative than the dynamic API. One notable exception is implementing a library as a proxy for an actual library implementation elsewhere, because then the actual keyword must be executed elsewhere and the proxy can only pass forward the keyword name and arguments.
A good example of using the hybrid API is Robot Framework's own Telnet library.
4.1.8 Handling Robot Framework's timeoutsRobot Framework has its own timeouts that can be used for stopping keyword execution if a test or a keyword takes too much time. There are two things to take into account related to them.
Doing cleanup if timeout occursTimeouts are technically implemented using robot.errors.TimeoutExceeded exception that can occur any time during a keyword execution. If a keyword wants to make sure possible cleanup activities are always done, it needs to handle these exceptions. Probably the simplest way to handle exceptions is using Python's try/finally structure:
def example():
try:
do_something()
finally:
do_cleanup()
A benefit of the above is that cleanup is done regardless of the exception. If there is a need to handle timeouts specially, it is possible to catch TimeoutExceeded explicitly. In that case it is important to re-raise the original exception afterwards:
from robot.errors import TimeoutExceeded
def example():
try:
do_something()
except TimeoutExceeded:
do_cleanup()
raise
Note
The TimeoutExceeded exception was named TimeoutError prior to Robot Framework 7.3. It was renamed to avoid a conflict with Python's standard exception with the same name. The old name still exists as a backwards compatible alias in the robot.errors module and can be used if older Robot Framework versions need to be supported.
Robot Framework's timeouts can stop normal Python code, but if the code calls functionality implemented using C or some other language, timeouts may not work. Well behaving keywords should thus avoid long blocking calls that cannot be interrupted.
As an example, subprocess.run cannot be interrupted on Windows, so the following simple keyword cannot be stopped by timeouts there:
import subprocess
def run_command(command, *args):
result = subprocess.run([command, *args], encoding='UTF-8')
print(f'stdout: {result.stdout}\nstderr: {result.stderr}')
This problem can be avoided by using the lower level subprocess.Popen and handling waiting in a loop with short timeouts. This adds quite a lot of complexity, though, so it may not be worth the effort in all cases.
import subprocess
def run_command(command, *args):
process = subprocess.Popen([command, *args], encoding='UTF-8',
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
while True:
try:
stdout, stderr = process.communicate(timeout=0.1)
except subprocess.TimeoutExpired:
continue
else:
break
print(f'stdout: {stdout}\nstderr: {stderr}')
4.1.9 Using Robot Framework's internal modules
Test libraries can use Robot Framework's internal modules, for example, to get information about the executed tests and the settings that are used. This powerful mechanism to communicate with the framework should be used with care, though, because all Robot Framework's APIs are not meant to be used by externally and they might change radically between different framework versions.
Using BuiltIn libraryThe safest API to use are methods implementing keywords in the BuiltIn library. Changes to keywords are rare and they are always done so that old usage is first deprecated. One of the most useful methods is replace_variables which allows accessing currently available variables. The following example demonstrates how to get ${OUTPUT_DIR} which is one of the many handy automatic variables. It is also possible to set new variables from libraries using set_test_variable, set_suite_variable and set_global_variable.
import os.path
from robot.libraries.BuiltIn import BuiltIn
def do_something(argument):
builtin = BuiltIn()
output = do_something_that_creates_a_lot_of_output(argument)
if builtin.robot_running:
output_dir = builtin.replace_variables('${OUTPUT_DIR}')
else:
output_dir = '.'
with open(os.path.join(output_dir, 'output.txt'), 'w') as file:
file.write(output)
print('*HTML* Output written to <a href="output.txt">output.txt</a>')
As the above examples illustrates, BuiltIn also has a convenient robot_running property for detecting is Robot Framework running.
The only catch with using methods from BuiltIn is that all run_keyword method variants must be handled specially. Methods that use run_keyword methods have to be registered as run keywords themselves using register_run_keyword method in BuiltIn module. This method's documentation explains why this needs to be done and obviously also how to do it.
This section explains different approaches how to add new functionality to existing test libraries and how to use them in your own libraries otherwise.
Modifying original source codeIf you have access to the source code of the library you want to extend, you can naturally modify the source code directly. The biggest problem of this approach is that it can be hard for you to update the original library without affecting your changes. For users it may also be confusing to use a library that has different functionality than the original one. Repackaging the library may also be a big extra task.
This approach works extremely well if the enhancements are generic and you plan to submit them back to the original developers. If your changes are applied to the original library, they are included in the future releases and all the problems discussed above are mitigated. If changes are non-generic, or you for some other reason cannot submit them back, the approaches explained in the subsequent sections probably work better.
Using inheritanceAnother straightforward way to extend an existing library is using inheritance. This is illustrated by the example below that adds new Title Should Start With keyword to the SeleniumLibrary.
from robot.api.deco import keyword
from SeleniumLibrary import SeleniumLibrary
class ExtendedSeleniumLibrary(SeleniumLibrary):
@keyword
def title_should_start_with(self, expected):
title = self.get_title()
if not title.startswith(expected):
raise AssertionError(f"Title '{title}' did not start with '{expected}'.")
A big difference with this approach compared to modifying the original library is that the new library has a different name than the original. A benefit is that you can easily tell that you are using a custom library, but a big problem is that you cannot easily use the new library with the original. First of all your new library will have same keywords as the original meaning that there is always conflict. Another problem is that the libraries do not share their state.
This approach works well when you start to use a new library and want to add custom enhancements to it from the beginning. Otherwise other mechanisms explained in this section are probably better.
Using other libraries directlyBecause test libraries are technically just classes or modules, a simple way to use another library is importing it and using its methods. This approach works great when the methods are static and do not depend on the library state. This is illustrated by the earlier example that uses Robot Framework's BuiltIn library.
If the library has state, however, things may not work as you would hope. The library instance you use in your library will not be the same as the framework uses, and thus changes done by executed keywords are not visible to your library. The next section explains how to get an access to the same library instance that the framework uses.
Getting active library instance from Robot FrameworkBuiltIn keyword Get Library Instance can be used to get the currently active library instance from the framework itself. The library instance returned by this keyword is the same as the framework itself uses, and thus there is no problem seeing the correct library state. Although this functionality is available as a keyword, it is typically used in test libraries directly by importing the BuiltIn library class as discussed earlier. The following example illustrates how to implement the same Title Should Start With keyword as in the earlier example about using inheritance.
from robot.libraries.BuiltIn import BuiltIn
def title_should_start_with(expected):
seleniumlib = BuiltIn().get_library_instance('SeleniumLibrary')
title = seleniumlib.get_title()
if not title.startswith(expected):
raise AssertionError(f"Title '{title}' did not start with '{expected}'.")
This approach is clearly better than importing the library directly and using it when the library has a state. The biggest benefit over inheritance is that you can use the original library normally and use the new library in addition to it when needed. That is demonstrated in the example below where the code from the previous examples is expected to be available in a new library SeLibExtensions.
*** Settings ***
Library SeleniumLibrary
Library SeLibExtensions
*** Test Cases ***
Example
Open Browser http://example # SeleniumLibrary
Title Should Start With Example # SeLibExtensions
4.2 Remote library interface
The remote library interface provides means for having test libraries on different machines than where Robot Framework itself is running, and also for implementing libraries using other languages than the natively supported Python. For a test library, user remote libraries look pretty much the same as any other test library, and developing test libraries using the remote library interface is also very close to creating normal test libraries.
4.2.1 IntroductionThere are two main reasons for using the remote library API:
The remote library interface is provided by the Remote library that is one of the standard libraries. This library does not have any keywords of its own, but it works as a proxy between the core framework and keywords implemented elsewhere. The Remote library interacts with actual library implementations through remote servers, and the Remote library and servers communicate using a simple remote protocol on top of an XML-RPC channel. The high level architecture of all this is illustrated in the picture below:
Robot Framework architecture with Remote library
Note
The remote client uses Python's standard XML-RPC module. It does not support custom XML-RPC extensions implemented by some XML-RPC servers.
4.2.2 Putting Remote library to use Importing Remote libraryThe Remote library needs to know the address of the remote server but otherwise importing it and using keywords that it provides is no different to how other libraries are used. If you need to use the Remote library multiple times in a suite, or just want to give it a more descriptive name, you can give it an alias when importing it.
*** Settings *** Library Remote http://127.0.0.1:8270 AS Example1 Library Remote http://example.com:8080/ AS Example2 Library Remote http://10.0.0.2/example 1 minute AS Example3
The URL used by the first example above is also the default address that the Remote library uses if no address is given.
The last example above shows how to give a custom timeout to the Remote library as an optional second argument. The timeout is used when initially connecting to the server and if a connection accidentally closes. Timeout can be given in Robot Framework time format like 60s or 2 minutes 10 seconds. The default timeout is typically several minutes, but it depends on the operating system and its configuration. Notice that setting a timeout that is shorter than keyword execution time will interrupt the keyword.
Note
Port 8270 is the default port that remote servers are expected to use and it has been registered by IANA for this purpose. This port number was selected because 82 and 70 are the ASCII codes of letters R and F, respectively.
Note
When connecting to the local machine, it is recommended to use IP address 127.0.0.1 instead of machine name localhost. This avoids address resolution that can be extremely slow at least on Windows.
Note
If the URI contains no path after the server address, the XML-RPC module used by the Remote library will use /RPC2 path by default. In practice using http://127.0.0.1:8270 is thus identical to using http://127.0.0.1:8270/RPC2. Depending on the remote server this may or may not be a problem. No extra path is appended if the address has a path even if the path is just /. For example, neither http://127.0.0.1:8270/ nor http://127.0.0.1:8270/my/path will be modified.
Before the Remote library can be imported, the remote server providing the actual keywords must be started. If the server is started before launching the test execution, it is possible to use the normal Library setting like in the above example. Alternatively other keywords, for example from Process or SSH libraries, can start the server up, but then you may need to use Import Library keyword because the library is not available when the test execution starts.
How a remote server can be stopped depends on how it is implemented. Typically servers support the following methods:
stop_remote_server method in their XML-RPC interface.Ctrl-C on the console where the server is running should stop the server.Note
Servers may be configured so that users cannot stop it with Stop Remote Server keyword or stop_remote_server method.
Because the XML-RPC protocol does not support all possible object types, the values transferred between the Remote library and remote servers must be converted to compatible types. This applies to the keyword arguments the Remote library passes to remote servers and to the return values servers give back to the Remote library.
Both the Remote library and the Python remote server handle Python values according to the following rules. Other remote servers should behave similarly.
None is converted to an empty string.${result.key}. This works also with nested dictionaries like ${root.child.leaf}.This section explains the protocol that is used between the Remote library and remote servers. This information is mainly targeted for people who want to create new remote servers.
The remote protocol is implemented on top of XML-RPC, which is a simple remote procedure call protocol using XML over HTTP. Most mainstream languages (Python, Java, C, Ruby, Perl, Javascript, PHP, ...) have a support for XML-RPC either built-in or as an extension.
The Python remote server can be used as a reference implementation.
Required methodsThere are two possibilities how remote servers can provide information about the keywords they contain. They are briefly explained below and documented more thoroughly in the subsequent sections.
get_keyword_names method and optional get_keyword_arguments, get_keyword_types, get_keyword_tags and get_keyword_documentation methods. Notice that using "camel-case names" like getKeywordNames is not possible similarly as in the normal dynamic API.get_library_information method that returns all library and keyword information as a single dictionary. If a remote server has this method, the other getter methods like get_keyword_names are not used at all. This approach has the benefit that there is only one XML-RPC call to get information while the approach explained above requires several calls per keyword. With bigger libraries the difference can be significant.Regardless how remote servers provide information about their keywords, they must have run_keyword method that is used when keywords are executed. How the actual keywords are implemented is not relevant for the Remote library. Remote servers can either act as wrappers for the real test libraries, like the available generic remote servers do, or they can implement keywords themselves.
Remote servers should additionally have stop_remote_server method in their public interface to ease stopping them. They should also automatically expose this method as Stop Remote Server keyword to allow using it in the test data regardless of the test library. Allowing users to stop the server is not always desirable, and servers may support disabling this functionality somehow. The method, and also the exposed keyword, should return True or False depending on whether stopping is allowed or not. That makes it possible for external tools to know if stopping the server succeeded.
get_keyword_names and keyword specific getters
This section explains how the Remote library gets keyword names and other information when the server implements get_keyword_names. The next sections covers using the newer get_library_info method.
The get_keyword_names method must return names of the keyword the server contains as a list of strings. Remote servers can, and should, also implement get_keyword_arguments, get_keyword_types, get_keyword_tags and get_keyword_documentation methods to provide more information about the keywords. All these methods take the name of the keyword as an argument and what they must return is explained in the table below.
get_keyword_arguments Arguments as a list of strings in the same format as with dynamic libraries. get_keyword_types Type information as a list or dictionary of strings. See below for details. get_keyword_documentation Documentation as a string. get_keyword_tags Tags as a list of strings.
Type information used for argument conversion can be returned either as a list mapping type names to arguments based on position or as a dictionary mapping argument names to type names directly. In practice this works the same way as when specifying types using the @keyword decorator with normal libraries. The difference is that because the XML-RPC protocol does not support arbitrary values, type information needs to be specified using type names or aliases like 'int' or 'integer', not using actual types like int. Additionally None or null values may not be allowed by the XML-RPC server, but an empty string can be used to indicate that certain argument does not have type information instead.
Argument conversion is supported also based on default values using the same logic as with normal libraries. For this to work, arguments with default values must be returned as tuples, not as strings, the same way as with dynamic libraries. For example, argument conversion works if argument information is returned like [('count', 1), ('caseless', True)] but not if it is ['count=1', 'caseless=True'].
Remote servers can also provide general library documentation to be used when generating documentation with the Libdoc tool. This information is got by calling get_keyword_documentation with special values __intro__ and __init__.
Note
get_keyword_types is new in Robot Framework 3.1 and support for argument conversion based on defaults is new in Robot Framework 4.0.
get_library_information
The get_library_information method allows returning information about the whole library in one XML-RPC call. The information must be returned as a dictionary where keys are keyword names and values are nested dictionaries containing keyword information. The dictionary can also contain separate entries for generic library information.
The keyword information dictionary can contain keyword arguments, documentation, tags and types, and the respective keys are args, doc, tags and types. Information must be provided using same semantics as when get_keyword_arguments, get_keyword_documentation, get_keyword_tags and get_keyword_types discussed in the previous section. If some information is not available, it can be omitted from the info dictionary altogether.
get_library_information supports also returning general library documentation to be used with Libdoc. It is done by including special __intro__ and __init__ entries into the returned library information dictionary.
For example, a Python library like
"""Library documentation."""
from robot.api.deco import keyword
@keyword(tags=['x', 'y'])
def example(a: int, b=True):
"""Keyword documentation."""
pass
def another():
pass
could be mapped into this kind of library information dictionary:
{
'__intro__': {'doc': 'Library documentation'}
'example': {'args': ['a', 'b=True'],
'types': ['int'],
'doc': 'Keyword documentation.',
'tags': ['x', 'y']}
'another: {'args': []}
}
Note
get_library_information is new in Robot Framework 4.0.
When the Remote library wants the server to execute some keyword, it calls the remote server's run_keyword method and passes it the keyword name, a list of arguments, and possibly a dictionary of free named arguments. Base types can be used as arguments directly, but more complex types are converted to supported types.
The server must return results of the execution in a result dictionary (or map, depending on terminology) containing items explained in the following table. Notice that only the status entry is mandatory, others can be omitted if they are not applicable.
*INFO* First message\n*HTML* <b>2nd</b>\n*WARN* Another message. It is also possible to embed timestamps to the log messages like *INFO:1308435758660* Message with timestamp. return Possible return value. Must be one of the supported types. error Possible error message. Used only when the execution fails. traceback Possible stack trace to write into the log file using DEBUG level when the execution fails. continuable When set to True, or any value considered True in Python, the occurred failure is considered continuable. fatal Like continuable, but denotes that the occurred failure is fatal. Different argument syntaxes
The Remote library is a dynamic library, and in general it handles different argument syntaxes according to the same rules as any other dynamic library. This includes mandatory arguments, default values, varargs, as well as named argument syntax.
Also free named arguments (**kwargs) works mostly the same way as with other dynamic libraries. First of all, the get_keyword_arguments must return an argument specification that contains **kwargs exactly like with any other dynamic library. The main difference is that remote servers' run_keyword method must have an optional third argument that gets the kwargs specified by the user. The third argument must be optional because, for backwards-compatibility reasons, the Remote library passes kwargs to the run_keyword method only when they have been used in the test data.
In practice run_keyword should look something like the following Python and Java examples, depending on how the language handles optional arguments.
def run_keyword(name, args, kwargs=None):
# ...
public Map run_keyword(String name, List args) {
// ...
}
public Map run_keyword(String name, List args, Map kwargs) {
// ...
}
4.3 Listener interface
Robot Framework's listener interface provides a powerful mechanism for getting notifications and for inspecting and modifying data and results during execution. Listeners are called, for example, when suites, tests and keywords start and end, when output files are ready, and finally when the whole execution ends. Example usages include communicating with external test management systems, sending a message when a test fails, and modifying tests during execution.
Listeners are implemented as classes or modules with certain special methods. They can be taken into use from the command line and be registered by libraries. The former listeners are active during the whole execution while the latter are active only when executing suites where libraries registering them are imported.
There are two supported listener interface versions, listener version 2 and listener version 3. They have mostly the same methods, but these methods are called with different arguments. The newer listener version 3 is more powerful and generally recommended.
4.3.1 Listener structureListeners are implement as modules or classes similarly as libraries. They can implement certain named hook methods depending on what events they are interested in. For example, if a listener wants to get a notification when a test starts, it can implement the start_test method. As discussed in the subsequent sections, different listener versions have slightly different set of available methods and they also are called with different arguments.
# Listener implemented as a module using the listener API version 3.
def start_suite(data, result):
print(f"Suite '{data.name}' starting.")
def end_test(data, result):
print(f"Test '{result.name}' ended with status {result.status}.")
Listeners do not need to implement any explicit interface, it is enough to simply implement needed methods and they will be recognized automatically. There are, however, base classes robot.api.interfaces.ListenerV2 and robot.api.interfaces.ListenerV3 that can be used to get method name completion in editors, type hints, and so on.
# Same as the above example, but uses an optional base class and type hints.
from robot import result, running
from robot.api.interfaces import ListenerV3
class Example(ListenerV3):
def start_suite(self, data: running.TestSuite, result: result.TestSuite):
print(f"Suite '{data.name}' starting.")
def end_test(self, data: running.TestCase, result: result.TestCase):
print(f"Test '{result.name}' ended with status {result.status}.")
Note
Optional listener base classes are new in Robot Framework 6.1.
In addition to using "snake case" like start_test with listener method names, it is possible to use "camel case" like startTest. This support was added when it was possible to run Robot Framework on Jython and implement listeners using Java. It is preserved for backwards compatibility reasons, but not recommended with new listeners.
There are two supported listener interface versions with version numbers 2 and 3. A listener can specify which version to use by having a ROBOT_LISTENER_API_VERSION attribute with value 2 or 3, respectively. Starting from Robot Framework 7.0, the listener version 3 is used by default if the version is not specified.
Listener version 2 and listener version 3 have mostly the same methods, but arguments passed to these methods are different. Arguments given to listener 2 methods are strings and dictionaries containing information about execution. This information can be inspected and sent further, but it is not possible to modify it directly. Listener 3 methods get the same model objects that Robot Framework itself uses, and these model objects can be both inspected and modified.
Listener version 3 is more powerful than the older listener version 2 and generally recommended.
Listener version 2Listeners using the listener API version 2 get notifications about various events during execution, but they do not have access to actually executed tests and thus cannot directly affect the execution or created results.
Listener methods in the API version 2 are listed in the following table and in the API docs of the optional ListenerV2 base class. All methods related to test execution progress have the same signature method(name, attributes), where attributes is a dictionary containing details of the event. Listener methods are free to do whatever they want to do with the information they receive, but they cannot directly change it. If that is needed, listener version 3 can be used instead.
Called when a test suite starts.
Contents of the attribute dictionary:
id: Suite id. s1 for the top level suite, s1-s1 for its first child suite, s1-s2 for the second child, and so on.longname: Suite name including parent suites.doc: Suite documentation.metadata: Free suite metadata as a dictionary.source: An absolute path of the file/directory the suite was created from.suites: Names of the direct child suites this suite has as a list.tests: Names of the tests this suite has as a list. Does not include tests of the possible child suites.totaltests: The total number of tests in this suite. and all its sub-suites as an integer.starttime: Suite execution start time.Called when a test suite ends.
Contents of the attribute dictionary:
id: Same as in start_suite.longname: Same as in start_suite.doc: Same as in start_suite.metadata: Same as in start_suite.source: Same as in start_suite.starttime: Same as in start_suite.endtime: Suite execution end time.elapsedtime: Total execution time in milliseconds as an integerstatus: Suite status as string PASS, FAIL or SKIP.statistics: Suite statistics (number of passed and failed tests in the suite) as a string.message: Error message if suite setup or teardown has failed, empty otherwise.Called when a test case starts.
Contents of the attribute dictionary:
id: Test id in format like s1-s2-t2, where the beginning is the parent suite id and the last part shows test index in that suite.longname: Test name including parent suites.originalname: Test name with possible variables unresolved. New in RF 3.2.doc: Test documentation.tags: Test tags as a list of strings.template: The name of the template used for the test. An empty string if the test not templated.source: An absolute path of the test case source file. New in RF 4.0.lineno: Line number where the test starts in the source file. New in RF 3.2.starttime: Test execution execution start time.Called when a test case ends.
Contents of the attribute dictionary:
id: Same as in start_test.longname: Same as in start_test.originalname: Same as in start_test.doc: Same as in start_test.tags: Same as in start_test.template: Same as in start_test.source: Same as in start_test.lineno: Same as in start_test.starttime: Same as in start_test.endtime: Test execution execution end time.elapsedtime: Total execution time in milliseconds as an integerstatus: Test status as string PASS, FAIL or SKIP.message: Status message. Normally an error message or an empty string.Called when a keyword or a control structure such as IF/ELSE or TRY/EXCEPT starts.
With keywords name is the full keyword name containing possible library or resource name as a prefix like MyLibrary.Example Keyword. With control structures name contains string representation of parameters.
Keywords and control structures share most of attributes, but control structures can have additional attributes depending on their type.
Shared attributes:
type: String specifying type of the started item. Possible values are: KEYWORD, SETUP, TEARDOWN, FOR, WHILE, ITERATION, IF, ELSE IF, ELSE, TRY, EXCEPT, FINALLY, VAR, RETURN, BREAK, CONTINUE and ERROR. All type values were changed in RF 4.0 and in RF 5.0 FOR ITERATION was changed to ITERATION.kwname: Name of the keyword without library or resource prefix. String representation of parameters with control structures.libname: Name of the library or resource file the keyword belongs to. An empty string with user keywords in a test case file and with control structures.doc: Keyword documentation.args: Keyword's arguments as a list of strings.assign: A list of variable names that keyword's return value is assigned to.tags: Keyword tags as a list of strings.source: An absolute path of the file where the keyword was used. New in RF 4.0.lineno: Line where the keyword was used. Typically an integer, but can be None if a keyword has been executed by a listener. New in RF 4.0.status: Initial keyword status. NOT RUN if keyword is not executed (e.g. due to an earlier failure), NOT SET otherwise. New in RF 4.0.starttime: Keyword execution start time.Additional attributes for FOR types:
variables: Assigned variables for each loop iteration as a list or strings.flavor: Type of loop (e.g. IN RANGE).values: List of values being looped over as a list or strings.start: Start configuration. Only used with IN ENUMERATE loops. New in RF 6.1.mode: Mode configuration. Only used with IN ZIP loops. New in RF 6.1.fill: Fill value configuration. Only used with IN ZIP loops. New in RF 6.1.Additional attributes for ITERATION types with FOR loops:
variables: Variables and string representations of their contents for one FOR loop iteration as a dictionary.Additional attributes for WHILE types:
condition: The looping condition.limit: The maximum iteration limit.on_limit: What to do if the limit is exceeded. Valid values are pass and fail. New in RF 7.0.on_limit_message: The custom error raised when the limit of the WHILE loop is reached. New in RF 6.1.Additional attributes for IF and ELSE IF types:
condition: The conditional expression being evaluated. With ELSE IF new in RF 6.1.Additional attributes for EXCEPT types:
patterns: The exception patterns being matched as a list or strings.pattern_type: The type of pattern match (e.g. GLOB).variable: The variable containing the captured exception.Additional attributes for RETURN types:
values: Return values from a keyword as a list or strings.Additional attributes for VAR types:
name: Variable name.value: Variable value. A string with scalar variables and a list otherwise.scope: Variable scope (e.g. GLOBAL) as a string.Additional attributes for control structures are in general new in RF 6.0. VAR is new in RF 7.0.
Called when a keyword or a control structure ends.
name is the full keyword name containing possible library or resource name as a prefix. For example, MyLibrary.Example Keyword.
Control structures have additional attributes, which change based on the type attribute. For descriptions of all possible attributes, see the start_keyword section.
Contents of the attribute dictionary:
type: Same as with start_keyword.kwname: Same as with start_keyword.libname: Same as with start_keyword.doc: Same as with start_keyword.args: Same as with start_keyword.assign: Same as with start_keyword.tags: Same as with start_keyword.source: Same as with start_keyword.lineno: Same as with start_keyword.starttime: Same as with start_keyword.endtime: Keyword execution end time.elapsedtime: Total execution time in milliseconds as an integerstatus: Keyword status as string PASS, FAIL, SKIP or NOT RUN. SKIP and NOT RUN are new in RF 4.0.Called when an executed keyword writes a log message.
message is a dictionary with the following contents:
message: The content of the message.level: Log level used in logging the message.timestamp: Message creation time in format YYYY-MM-DD hh:mm:ss.mil.html: String yes or no denoting whether the message should be interpreted as HTML or not.Not called if the message level is below the current threshold level.
message messageCalled when the framework itself writes a syslog message.
message is a dictionary with the same contents as with log_message method.
Called when a library has been imported.
name is the name of the imported library. If the library has been given a custom name when imported it using AS, name is the specified alias.
Contents of the attribute dictionary:
args: Arguments passed to the library as a list.originalname: The original library name if the library has been given an alias using AS, otherwise same as name.source: An absolute path to the library source. An empty string if getting the source of the library failed for some reason.importer: An absolute path to the file importing the library. None when BuiltIn is imported as well as when using the Import Library keyword.Called when a resource file has been imported.
name is the name of the imported resource file without the file extension.
Contents of the attribute dictionary:
source: An absolute path to the imported resource file.importer: An absolute path to the file importing the resource file. None when using the Import Resource keyword.Called when a variable file has been imported.
name is the name of the imported variable file with the file extension.
Contents of the attribute dictionary:
args: Arguments passed to the variable file as a list.source: An absolute path to the imported variable file.importer: An absolute path to the file importing the resource file. None when using the Import Variables keyword.Called when writing to an output file is ready.
path is an absolute path to the file as a string or a string None if creating the output file is disabled.
Called when writing to a log file is ready.
path is an absolute path to the file as a string. Not called if creating the log file is disabled.
Called when writing to a report file is ready.
path is an absolute path to the file as a string. Not called if creating the report file is disabled.
Called when writing to an xunit file is ready.
path is an absolute path to the file as a string. Only called if creating the xunit file is enabled.
Called when writing to a debug file is ready.
path is an absolute path to the file as a string. Only called if creating the debug file is enabled.
Called when the whole test execution ends.
With library listeners called when the library goes out of scope.
Listener version 3Listener version 3 has mostly the same methods as listener version 2, but arguments of the methods related to test execution are different. These methods get actual running and result model objects that used by Robot Framework itself, and listeners can both query information they need and change the model objects on the fly.
Listener version 3 was enhanced heavily in Robot Framework 7.0 when it got methods related to keywords and control structures. It was enhanced further in Robot Framework 7.1 when it got methods related to library, resource file and variable file imports.
Listener version 3 has separate methods for library keywords, user keywords and all control structures. If there is a need to listen to all keyword related events, it is possible to implement start_keyword and end_keyword. In addition to that, start_body_item and end_body_item can be implemented to get notifications related to all keywords and control structures. These higher level listener methods are not called if more specific methods like start_library_keyword or end_if are implemented.
Listener methods in the API version 3 are listed in the following table and in the API docs of the optional ListenerV3 base class.
Methods in the listener API 3 Method Arguments Documentation start_suite data, resultCalled when a test suite starts.
data and result are model objects representing the executed test suite and its execution results, respectively.
Called when a test suite ends.
Same arguments as with start_suite.
Called when a test case starts.
data and result are model objects representing the executed test case and its execution results, respectively.
Called when a test case ends.
Same arguments as with start_test.
Called when a keyword starts.
data and result are model objects representing the executed keyword call and its execution results, respectively.
This method is called, by default, with user keywords, library keywords and when a keyword call is invalid. It is not called if a more specific start_user_keyword, start_library_keyword or start_invalid_keyword method is implemented.
Called when a keyword ends.
Same arguments and other semantics as with start_keyword.
Called when a user keyword starts.
data and result are the same as with start_keyword and implementation is the actually executed user keyword.
If this method is implemented, start_keyword is not called with user keywords.
Called when a user keyword ends.
Same arguments and other semantics as with start_user_keyword.
Called when a library keyword starts.
data and result are the same as with start_keyword and implementation represents the executed library keyword.
If this method is implemented, start_keyword is not called with library keywords.
Called when a library keyword ends.
Same arguments and other semantics as with start_library_keyword.
Called when an invalid keyword call starts.
data and result are the same as with start_keyword and implementation represents the invalid keyword call. Keyword may not have been found, there could have been multiple matches, or the keyword call itself could have been invalid.
If this method is implemented, start_keyword is not called with invalid keyword calls.
Called when an invalid keyword call ends.
Same arguments and other semantics as with start_invalid_keyword.
Called when control structures start.
See the documentation and type hints of the optional ListenerV3 base class for more information.
end_for, end_for_iteration, end_while, end_while_iteration, end_if, end_if_branch, end_try, end_try_branch, end_group, end_var, end_continue, end_break, end_return data, resultCalled when control structures end.
See the documentation and type hints of the optional ListenerV3 base class for more information.
start_error data, result Called when invalid syntax starts. end_error data, result Called when invalid syntax ends. start_body_item data, result Called when a keyword or a control structure starts, unless a more specific method such asstart_keyword or start_if is implemented. end_body_item data, result Called when a keyword or a control structure ends, unless a more specific method such as end_keyword or end_if is implemented. log_message message
Called when an executed keyword writes a log message. message is a model object representing the logged message.
This method is not called if the message has level below the current threshold level.
message messageCalled when the framework itself writes a syslog message.
message is same object as with log_message.
Called after a library has been imported.
library represents the imported library. It can be inspected and also modified. importer contains information about the location where the library was imported.
resource_import resource, importerCalled after a resource file has been imported.
resource represents the imported resource file. It can be inspected and also modified. importer contains information about the location where the resource was imported.
variables_import attrs, importerCalled after a variable file has been imported.
attrs contains information about the imported variable file as a dictionary. It can be inspected, but modifications to it have no effect. importer contains information about the location where the variable file was imported.
This method will be changed in the future so that the attrs dictionary is replaced with an object representing the imported variable file.
Called when writing to an output file is ready.
path is an absolute path to the file as a pathlib.Path object or the None object if creating the output file is disabled.
Called when writing to a log file is ready.
path is an absolute path to the file as a pathlib.Path object. Not called if creating the log file is disabled.
Called when writing to a report file is ready.
path is an absolute path to the file as a pathlib.Path object. Not called if creating the report file is disabled.
Called when writing to an xunit file is ready.
path is an absolute path to the file as a pathlib.Path object. Only called if creating the xunit file is enabled.
Called when writing to a debug file is ready.
path is an absolute path to the file as a pathlib.Path object. Only called if creating the debug file is enabled.
Called when the whole test execution ends.
With library listeners called when the library goes out of scope.
Note
Methods related to keywords and control structures are new in Robot Framework 7.0.
Note
Methods related to library, resource file and variable file imports are new in Robot Framework 7.1.
Note
Prior to Robot Framework 7.0, paths passed to result file related listener version 3 methods were strings.
4.3.3 Taking listeners into use Registering listeners from command lineListeners that need to be active during the whole execution must be taken into use from the command line. That is done using the --listener option so that the name of the listener is given to it as an argument. The listener name is got from the name of the class or module implementing the listener, similarly as library name is got from the class or module implementing the library. The specified listeners must be in the same module search path where test libraries are searched from when they are imported. In addition to registering a listener by using a name, it is possible to give an absolute or a relative path to the listener file similarly as with test libraries. It is possible to take multiple listeners into use by using this option several times:
robot --listener MyListener tests.robot robot --listener path/to/MyListener.py tests.robot robot --listener module.Listener --listener AnotherListener tests.robot
It is also possible to give arguments to listener classes from the command line. Arguments are specified after the listener name (or path) using a colon (:) as a separator. If a listener is given as an absolute Windows path, the colon after the drive letter is not considered a separator. Additionally, it is possible to use a semicolon (;) as an alternative argument separator. This is useful if listener arguments themselves contain colons, but requires surrounding the whole value with quotes on UNIX-like operating systems:
robot --listener listener.py:arg1:arg2 tests.robot robot --listener "listener.py;arg:with:colons" tests.robot robot --listener c:\path\listener.py;d:\first\arg;e:\second\arg tests.robot
In addition to passing arguments one-by-one as positional arguments, it is possible to pass them using the named argument syntax similarly as when using keywords:
robot --listener listener.py:name=value tests.robot robot --listener "listener.py;name=value:with:colons;second=argument" tests.robot
Listener arguments are automatically converted using same rules as with keywords based on type hints and default values. For example, this listener
class Listener:
def __init__(self, port: int, log=True):
self.port = post
self.log = log
could be used like
robot --listener Listener:8270:false
and the first argument would be converted to an integer based on the type hint and the second to a Boolean based on the default value.
Note
Both the named argument syntax and argument conversion are new in Robot Framework 4.0.
Libraries as listenersSometimes it is useful also for test libraries to get notifications about test execution. This allows them, for example, to perform certain clean-up activities automatically when a test suite or the whole test execution ends.
Registering listenerA test library can register a listener by using the ROBOT_LIBRARY_LISTENER attribute. The value of this attribute should be an instance of the listener to use. It may be a totally independent listener or the library itself can act as a listener. To avoid listener methods to be exposed as keywords in the latter case, it is possible to prefix them with an underscore. For example, instead of using end_suite it is possible to use _end_suite.
Following examples illustrates using an external listener as well as a library acting as a listener itself:
from listener import Listener
class LibraryWithExternalListener:
ROBOT_LIBRARY_SCOPE = 'GLOBAL'
ROBOT_LIBRARY_LISTENER = Listener()
def example_keyword(self):
...
class LibraryItselfAsListener:
ROBOT_LIBRARY_SCOPE = 'SUITE'
ROBOT_LISTENER_API_VERSION = 2
def __init__(self):
self.ROBOT_LIBRARY_LISTENER = self
# Use the '_' prefix to avoid listener method becoming a keyword.
def _end_suite(self, name, attrs):
print(f"Suite '{name}' ending with status {attrs['id']}.")
def example_keyword(self):
...
As the second example above already demonstrated, library listeners can specify listener interface versions using the ROBOT_LISTENER_API_VERSION attribute exactly like any other listener.
Starting from Robot Framework 7.0, a listener can register itself to be a listener also by using a string SELF (case-insensitive) as a listener. This is especially convenient when using the @library decorator:
from robot.api.deco import keyword, library
@library(scope='SUITE', listener='SELF')
class LibraryItselfAsListener:
# Listener version is not specified, so uses the listener version 3 by default.
# When using the @library decorator, keywords must use the @keyword decorator,
# so there is no need to use the '_' prefix here.
def end_suite(self, data, result):
print(f"Suite '{data.name}' ending with status {result.status}.")
@keyword
def example_keyword(self):
...
It is also possible to specify multiple listeners for a single library by giving ROBOT_LIBRARY_LISTENER a value as a list:
from listeners import Listener1, Listener2, Listener3
class LibraryWithMultipleListeners:
ROBOT_LIBRARY_LISTENER = [Listener1(), Listener2(), Listener3()]
def example_keyword(self):
...
Called listener methods
Library listeners get notifications about all events in suites where libraries using them are imported. In practice this means that suite, test, keyword, control structure and log message related methods are called. In addition to them, the close method is called when the library goes out of the scope.
If library creates a new listener instance every time when the library itself is instantiated, the actual listener instance to use will change according to the library scope.
4.3.4 Listener calling orderBy default, listeners are called in the order they are taken into use so that listeners registered from the command line are called before library listeners. It is, however, possible to control the calling order by setting the special ROBOT_LISTENER_PRIORITY attribute to an integer or a floating point value. The bigger the number, the higher precedence the listener has and the earlier it is called. The number can be positive or negative and it is zero by default.
The custom order does not affect the close method of library listeners, though. That method is always called when the library goes out of its scope.
Note
Controlling listener calling order is new in Robot Framework 7.1.
4.3.5 Listener examplesThis section contains examples using the listener interface. First examples illustrate getting notifications during execution and latter examples modify executed tests and created results.
Getting informationThe first example is implemented as a Python module. It uses the listener version 2, but could equally well be implemented by using the listener version 3.
"""Listener that stops execution if a test fails."""
ROBOT_LISTENER_API_VERSION = 2
def end_test(name, attrs):
if attrs['status'] == 'FAIL':
print(f"Test '{name}'" failed: {attrs['message']}")
input("Press enter to continue.")
If the above example would be saved to, for example, PauseExecution.py file, it could be used from the command line like this:
robot --listener path/to/PauseExecution.py tests.robot
The next example, which still uses the listener version 2, is slightly more complicated. It writes all the information it gets into a text file in a temporary directory without much formatting. The filename may be given from the command line, but it also has a default value. Note that in real usage, the debug file functionality available through the command line option --debugfile is probably more useful than this example.
import os.path
import tempfile
class Example:
ROBOT_LISTENER_API_VERSION = 2
def __init__(self, file_name='listen.txt'):
path = os.path.join(tempfile.gettempdir(), file_name)
self.file = open(path, 'w')
def start_suite(self, name, attrs):
self.file.write("%s '%s'\n" % (name, attrs['doc']))
def start_test(self, name, attrs):
tags = ' '.join(attrs['tags'])
self.file.write("- %s '%s' [ %s ] :: " % (name, attrs['doc'], tags))
def end_test(self, name, attrs):
if attrs['status'] == 'PASS':
self.file.write('PASS\n')
else:
self.file.write('FAIL: %s\n' % attrs['message'])
def end_suite(self, name, attrs):
self.file.write('%s\n%s\n' % (attrs['status'], attrs['message']))
def close(self):
self.file.close()
Modifying data and results
The following examples illustrate how to modify the executed tests and suites as well as the execution results. All these examples require using the listener version 3.
Modifying executed suites and testsChanging what is executed is as easy as modifying the model objects representing executed data passed to listener methods. This is illustrated by the example below that adds a new test to each executed suite and a new keyword call to each test.
def start_suite(data, result):
data.tests.create(name='New test')
def start_test(data, result):
data.body.create_keyword(name='Log', args=['Keyword added by listener!'])
This API is very similar to the pre-run modifier API that can be used to modify suites and tests before the whole test execution starts. The main benefit of using the listener API is that modifications can be done dynamically based on execution results or otherwise. This allows, for example, interesting possibilities for model based testing.
Although the listener interface is not built on top of Robot Framework's internal visitor interface similarly as the pre-run modifier API, listeners can still use the visitors interface themselves. For example, the SelectEveryXthTest visitor used in pre-run modifier examples could be used like this:
from SelectEveryXthTest import SelectEveryXthTest
def start_suite(suite, result):
selector = SelectEveryXthTest(x=2)
suite.visit(selector)
Accessing library or resource file
It is possible to get more information about the actually executed keyword and the library or resource file it belongs to:
from robot.running import Keyword as KeywordData, LibraryKeyword
from robot.result import Keyword as KeywordResult
def start_library_keyword(data: KeywordData,
implementation: LibraryKeyword,
result: KeywordResult):
library = implementation.owner
print(f"Keyword '{implementation.name}' is implemented in library "
f"'{library.name}' at '{implementation.source}' on line "
f"{implementation.lineno}. The library has {library.scope.name} "
f"scope and the current instance is {library.instance}.")
As the above example illustrates, it is possible to get an access to the actual library instance. This means that listeners can inspect the library state and also modify it. With user keywords it is even possible to modify the keyword itself or, via the owner resource file, any other keyword in the resource file.
Test execution results can be altered by modifying the result objects passed to listener methods. This is demonstrated by the following listener that is implemented as a class and also uses type hints:
from robot import result, running
class ResultModifier:
def __init__(self, max_seconds: float = 10.0):
self.max_seconds = max_seconds
def start_suite(self, data: running.TestSuite, result: result.TestSuite):
result.doc = 'Documentation set by listener.'
# Information about tests only available via data at this point.
smoke_tests = [test for test in data.tests if 'smoke' in test.tags]
result.metadata['Smoke tests'] = len(smoke_tests)
def end_test(self, data: running.TestCase, result: result.TestCase):
elapsed_seconds = result.elapsed_time.total_seconds()
if result.status == 'PASS' and elapsed_seconds > self.max_seconds:
result.status = 'FAIL'
result.message = 'Test execution took too long.'
def log_message(self, msg: result.Message):
if msg.level == 'WARN' and not msg.html:
msg.message = f'<b style="font-size: 1.5em">{msg.message}</b>'
msg.html = True
if self._message_is_not_relevant(msg.message):
msg.message = None
def _message_is_not_relevant(self, message: str) -> bool:
...
A limitation is that modifying the name of the current test suite or test case is not possible because it has already been written to the output.xml file when listeners are called. Due to the same reason modifying already finished tests in the end_suite method has no effect either.
When modifying logged messages, it is possible to remove a message altogether by setting message to None as the above example demonstrates. This can be used for removing sensitive or non-relevant messages so that there is nothing visible in the log file.
This API is very similar to the pre-Rebot modifier API that can be used to modify results before report and log are generated. The main difference is that listeners modify also the created output.xml file.
Note
Removing messages altogether by setting them to None is new in Robot Framework 7.2.
Listeners can also affect the execution flow by changing statuses of the executed keywords and control structures. For example, if a listener changes the status of a passed keyword to FAIL, the keyword is considered failed exactly as if it had failed normally. Similarly, it is possible to change the status of a passed or failed keyword to SKIP to get the keyword and the whole test skipped. It is also possible to silence failures by changing the status to PASS, but this should be done only in special cases and with great care to avoid hiding real failures.
The following example demonstrates changing the status by failing keywords that take too long time to execute. The previous example had similar logic with tests, but this listener also stops the execution immediately if there is a keyword that is too slow. As the example shows, listeners can also change the error message, not only the status.
from robot import result, running
class KeywordPerformanceMonitor:
def __init__(self, max_seconds: float = 0.1):
self.max_seconds = max_seconds
def end_keyword(self, data: running.Keyword, result: result.Keyword):
elapsed_seconds = result.elapsed_time.total_seconds()
if result.status == 'PASS' and elapsed_seconds > self.max_seconds:
result.status = 'FAIL'
result.message = 'Keyword execution took too long.'
Note
Changes to status only affect the execution flow starting from Robot Framework 7.1.
More examplesKeyword and control structure related listener version 3 methods are so versatile that covering them fully here in the User Guide is not possible. For more examples, you can see the acceptance tests using theses methods in various ways.
4.4 Parser interfaceRobot Framework supports external parsers that can handle custom data formats or even override Robot Framework's own parser.
Note
Custom parsers are new in Robot Framework 6.1.
4.4.1 Taking parsers into useParsers are taken into use from the command line with the --parser option using exactly the same semantics as with listeners. This includes specifying parsers as names or paths, giving arguments to parser classes, and so on:
robot --parser MyParser tests.custom robot --parser path/to/MyParser.py tests.custom robot --parser Parser1:arg --parser Parser2:a1:a2 path/to/tests4.4.2 Parser API
Parsers can be implemented both as modules and classes. This section explains what attributes and methods they must contain.
EXTENSION or extension attribute
This attribute specifies what file extension or extensions the parser supports. Both EXTENSION and extension names are accepted, and the former has precedence if both exist. The attribute can be either a string or a sequence of strings. Extensions are case-insensitive and can be specified with or without the leading dot. If a parser is implemented as a class, it is possible to set this attribute either as a class attribute or as an instance attribute.
Also extensions containing multiple parts like .example.ext or .robot.zip are supported.
Note
If a parser supports the .robot extension, it will be used for parsing these files instead of the standard parser.
parse method
The mandatory parse method is responsible for parsing suite files. It is called with each parsed file that has an extension that the parser supports. The method must return a TestSuite object.
In simple cases parse can be implemented so that it accepts just a single argument that is a pathlib.Path object pointing to the file to parse. If the parser is interested in defaults for Test Setup, Test Teardown, Test Tags and Test Timeout set in higher level suite initialization files, the parse method must accept two arguments. In that case the second argument is a TestDefaults object.
parse_init method
The optional parse_init method is responsible for parsing suite initialization files i.e. files in format __init__.ext where .ext is an extension supported by the parser. The method must return a TestSuite object representing the whole directory. Suites created from child suite files and directories will be added to its child suites.
Also parse_init can be implemented so that it accepts one or two arguments, depending on is it interested in test related default values or not. If it accepts defaults, it can manipulate the passed TestDefaults object and changes are seen when parsing child suite files.
This method is only needed if a parser needs to support suite initialization files.
Optional base classParsers do not need to implement any explicit interface, but it may be helpful to extend the optional Parser base class. The main benefit is that the base class has documentation and type hints. It also works as a bit more formal API specification.
4.4.3 Examples Parser implemented as moduleThe first example demonstrates a simple parser implemented as a module and supporting one hard-coded extension. It just creates a dummy suite and does not actually parse anything.
from robot.api import TestSuite
EXTENSION = '.example'
def parse(source):
suite = TestSuite(name='Example', source=source)
test = suite.tests.create(name='Test')
test.body.create_keyword(name='Log', args=['Hello!'])
return suite
Parser implemented as class
The second parser is implemented as a class that accepts the extension to use as an argument. The parser reads the given source file and creates dummy tests from each line it contains.
from pathlib import Path
from robot.api import TestSuite
class ExampleParser:
def __init__(self, extension: str):
self.extension = extension
def parse(self, source: Path) -> TestSuite:
name = TestSuite.name_from_source(source, self.extension)
suite = TestSuite(name, source=source)
for line in source.read_text().splitlines():
test = suite.tests.create(name=line)
test.body.create_keyword(name='Log', args=['Hello!'])
return suite
Parser extending optional base class
This parser extends the optional Parser base class. It supports parsing suite initialization files, uses TestDefaults and registers multiple extensions.
from pathlib import Path
from robot.api import TestSuite
from robot.api.interfaces import Parser, TestDefaults
class ExampleParser(Parser):
extension = ('example', 'another')
def parse(self, source: Path, defaults: TestDefaults) -> TestSuite:
"""Create a suite and set possible defaults from init files to tests."""
suite = TestSuite(TestSuite.name_from_source(source), source=source)
for line in source.read_text().splitlines():
test = suite.tests.create(name=line, doc='Example')
test.body.create_keyword(name='Log', args=['Hello!'])
defaults.set_to(test)
return suite
def parse_init(self, source: Path, defaults: TestDefaults) -> TestSuite:
"""Create a dummy suite and set some defaults.
This method is called only if there is an initialization file with
a supported extension.
"""
defaults.tags = ('tags', 'from init')
defaults.setup = {'name': 'Log', 'args': ['Hello from init!']}
return TestSuite(TestSuite.name_from_source(source.parent), doc='Example',
source=source, metadata={'Example': 'Value'})
Parser as preprocessor
The final example parser acts as a preprocessor for Robot Framework data files that supports headers in format === Test Cases === in addition to *** Test Cases ***. In this kind of usage it is convenient to use TestSuite.from_string, TestSuite.from_model and TestSuite.from_file_system factory methods for constructing the returned suite.
from pathlib import Path
from robot.running import TestDefaults, TestSuite
class RobotPreprocessor:
extension = '.robot'
def parse(self, source: Path, defaults: TestDefaults) -> TestSuite:
data = source.read_text()
for header in 'Settings', 'Variables', 'Test Cases', 'Keywords':
data = data.replace(f'=== {header} ===', f'*** {header} ***')
suite = TestSuite.from_string(data, defaults=defaults)
return suite.config(name=TestSuite.name_from_source(source), source=source)
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4