aboutsummaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorRichard Henderson <richard.henderson@linaro.org>2021-11-08 18:50:09 +0100
committerRichard Henderson <richard.henderson@linaro.org>2021-11-08 18:50:09 +0100
commit114f3c8cc427333dbae331dfd2ecae64676b087e (patch)
tree6b609b516b20585ab8ad70cd86055c37132fd169 /docs
parent5e10ccc2706768bcba608f5d174d4079944c270a (diff)
parentb94d00898a9ee708b4d4b79432c2272905a56b06 (diff)
Merge remote-tracking branch 'remotes/philmd/tags/avocado-20211108' into staging
Integration testing patches - Rename tests/acceptance/ -> tests/avocado/ - Rename avocado_qemu.Test -> avocado_qemu.QemuSystemTest - Introduce QemuUserTest class - Add the first linux-user test, covering the bFLT loader # gpg: Signature made Mon 08 Nov 2021 05:16:46 PM CET # gpg: using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE # gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full] * remotes/philmd/tags/avocado-20211108: tests/avocado: Remove p7zip binary availability check tests/avocado: Rename avocado_qemu.Test -> QemuSystemTest tests/avocado: Add bFLT loader linux-user test tests/avocado: Share useful helpers from virtiofs_submounts test tests/avocado: Introduce QemuUserTest base class tests/avocado: Make pick_default_qemu_bin() more generic tests/avocado: Extract QemuBaseTest from Test tests/acceptance: rename tests acceptance to tests avocado tests/acceptance: introduce new check-avocado target Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Diffstat (limited to 'docs')
-rw-r--r--docs/devel/build-system.rst2
-rw-r--r--docs/devel/ci-definitions.rst.inc2
-rw-r--r--docs/devel/testing.rst57
-rw-r--r--docs/system/arm/orangepi.rst8
4 files changed, 34 insertions, 35 deletions
diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst
index 84282ca770..7a83f5fc0d 100644
--- a/docs/devel/build-system.rst
+++ b/docs/devel/build-system.rst
@@ -416,7 +416,7 @@ number of dynamically created files listed later.
``tests/Makefile.include``
Rules for external test harnesses. These include the TCG tests,
- ``qemu-iotests`` and the Avocado-based acceptance tests.
+ ``qemu-iotests`` and the Avocado-based integration tests.
``tests/docker/Makefile.include``
Rules for Docker tests. Like tests/Makefile, this file is included
diff --git a/docs/devel/ci-definitions.rst.inc b/docs/devel/ci-definitions.rst.inc
index 32e22ff468..6d5c6fd9f2 100644
--- a/docs/devel/ci-definitions.rst.inc
+++ b/docs/devel/ci-definitions.rst.inc
@@ -59,7 +59,7 @@ to system testing [5]_. Note that, in some cases, system testing may require
interaction with third-party software, like operating system images, databases,
networks, and so on.
-On QEMU, system testing is represented by the 'check-acceptance' target from
+On QEMU, system testing is represented by the 'check-avocado' target from
'make'.
Flaky tests
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 7500f076c2..60c59023e5 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -653,17 +653,16 @@ supported. To start the fuzzer, run
Alternatively, some command different from "qemu-img info" can be tested, by
changing the ``-c`` option.
-Acceptance tests using the Avocado Framework
---------------------------------------------
+Integration tests using the Avocado Framework
+---------------------------------------------
-The ``tests/acceptance`` directory hosts functional tests, also known
-as acceptance level tests. They're usually higher level tests, and
-may interact with external resources and with various guest operating
-systems.
+The ``tests/avocado`` directory hosts integration tests. They're usually
+higher level tests, and may interact with external resources and with
+various guest operating systems.
These tests are written using the Avocado Testing Framework (which must
be installed separately) in conjunction with a the ``avocado_qemu.Test``
-class, implemented at ``tests/acceptance/avocado_qemu``.
+class, implemented at ``tests/avocado/avocado_qemu``.
Tests based on ``avocado_qemu.Test`` can easily:
@@ -695,11 +694,11 @@ Tests based on ``avocado_qemu.Test`` can easily:
Running tests
~~~~~~~~~~~~~
-You can run the acceptance tests simply by executing:
+You can run the avocado tests simply by executing:
.. code::
- make check-acceptance
+ make check-avocado
This involves the automatic creation of Python virtual environment
within the build tree (at ``tests/venv``) which will have all the
@@ -714,12 +713,12 @@ specific version, they may be on packages named ``python3-venv`` and
``python3-pip``.
It is also possible to run tests based on tags using the
-``make check-acceptance`` command and the ``AVOCADO_TAGS`` environment
+``make check-avocado`` command and the ``AVOCADO_TAGS`` environment
variable:
.. code::
- make check-acceptance AVOCADO_TAGS=quick
+ make check-avocado AVOCADO_TAGS=quick
Note that tags separated with commas have an AND behavior, while tags
separated by spaces have an OR behavior. For more information on Avocado
@@ -728,31 +727,31 @@ tags, see:
https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/tags.html
To run a single test file, a couple of them, or a test within a file
-using the ``make check-acceptance`` command, set the ``AVOCADO_TESTS``
+using the ``make check-avocado`` command, set the ``AVOCADO_TESTS``
environment variable with the test files or test names. To run all
tests from a single file, use:
.. code::
- make check-acceptance AVOCADO_TESTS=$FILEPATH
+ make check-avocado AVOCADO_TESTS=$FILEPATH
The same is valid to run tests from multiple test files:
.. code::
- make check-acceptance AVOCADO_TESTS='$FILEPATH1 $FILEPATH2'
+ make check-avocado AVOCADO_TESTS='$FILEPATH1 $FILEPATH2'
To run a single test within a file, use:
.. code::
- make check-acceptance AVOCADO_TESTS=$FILEPATH:$TESTCLASS.$TESTNAME
+ make check-avocado AVOCADO_TESTS=$FILEPATH:$TESTCLASS.$TESTNAME
The same is valid to run single tests from multiple test files:
.. code::
- make check-acceptance AVOCADO_TESTS='$FILEPATH1:$TESTCLASS1.$TESTNAME1 $FILEPATH2:$TESTCLASS2.$TESTNAME2'
+ make check-avocado AVOCADO_TESTS='$FILEPATH1:$TESTCLASS1.$TESTNAME1 $FILEPATH2:$TESTCLASS2.$TESTNAME2'
The scripts installed inside the virtual environment may be used
without an "activation". For instance, the Avocado test runner
@@ -760,9 +759,9 @@ may be invoked by running:
.. code::
- tests/venv/bin/avocado run $OPTION1 $OPTION2 tests/acceptance/
+ tests/venv/bin/avocado run $OPTION1 $OPTION2 tests/avocado/
-Note that if ``make check-acceptance`` was not executed before, it is
+Note that if ``make check-avocado`` was not executed before, it is
possible to create the Python virtual environment with the dependencies
needed running:
@@ -775,20 +774,20 @@ a test file. To run tests from a single file within the build tree, use:
.. code::
- tests/venv/bin/avocado run tests/acceptance/$TESTFILE
+ tests/venv/bin/avocado run tests/avocado/$TESTFILE
To run a single test within a test file, use:
.. code::
- tests/venv/bin/avocado run tests/acceptance/$TESTFILE:$TESTCLASS.$TESTNAME
+ tests/venv/bin/avocado run tests/avocado/$TESTFILE:$TESTCLASS.$TESTNAME
Valid test names are visible in the output from any previous execution
-of Avocado or ``make check-acceptance``, and can also be queried using:
+of Avocado or ``make check-avocado``, and can also be queried using:
.. code::
- tests/venv/bin/avocado list tests/acceptance
+ tests/venv/bin/avocado list tests/avocado
Manual Installation
~~~~~~~~~~~~~~~~~~~
@@ -806,16 +805,16 @@ Alternatively, follow the instructions on this link:
Overview
~~~~~~~~
-The ``tests/acceptance/avocado_qemu`` directory provides the
+The ``tests/avocado/avocado_qemu`` directory provides the
``avocado_qemu`` Python module, containing the ``avocado_qemu.Test``
class. Here's a simple usage example:
.. code::
- from avocado_qemu import Test
+ from avocado_qemu import QemuSystemTest
- class Version(Test):
+ class Version(QemuSystemTest):
"""
:avocado: tags=quick
"""
@@ -860,10 +859,10 @@ and hypothetical example follows:
.. code::
- from avocado_qemu import Test
+ from avocado_qemu import QemuSystemTest
- class MultipleMachines(Test):
+ class MultipleMachines(QemuSystemTest):
def test_multiple_machines(self):
first_machine = self.get_vm()
second_machine = self.get_vm()
@@ -913,7 +912,7 @@ like this:
self.ssh_command('some_command_to_be_run_in_the_guest')
Please refer to tests that use ``avocado_qemu.LinuxTest`` under
-``tests/acceptance`` for more examples.
+``tests/avocado`` for more examples.
QEMUMachine
~~~~~~~~~~~
@@ -1204,7 +1203,7 @@ And remove any package you want with::
pip uninstall <package_name>
-If you've used ``make check-acceptance``, the Python virtual environment where
+If you've used ``make check-avocado``, the Python virtual environment where
Avocado is installed will be cleaned up as part of ``make check-clean``.
.. _checktcg-ref:
diff --git a/docs/system/arm/orangepi.rst b/docs/system/arm/orangepi.rst
index 6f23907fb6..c55694dd91 100644
--- a/docs/system/arm/orangepi.rst
+++ b/docs/system/arm/orangepi.rst
@@ -250,14 +250,14 @@ and set the following environment variables before booting:
Optionally you may save the environment variables to SD card with 'saveenv'.
To continue booting simply give the 'boot' command and NetBSD boots.
-Orange Pi PC acceptance tests
-"""""""""""""""""""""""""""""
+Orange Pi PC integration tests
+""""""""""""""""""""""""""""""
-The Orange Pi PC machine has several acceptance tests included.
+The Orange Pi PC machine has several integration tests included.
To run the whole set of tests, build QEMU from source and simply
provide the following command:
.. code-block:: bash
$ AVOCADO_ALLOW_LARGE_STORAGE=yes avocado --show=app,console run \
- -t machine:orangepi-pc tests/acceptance/boot_linux_console.py
+ -t machine:orangepi-pc tests/avocado/boot_linux_console.py