diff options
| author | Thomas Huth <thuth@redhat.com> | 2023-07-10 19:01:55 +0200 |
|---|---|---|
| committer | Thomas Huth <thuth@redhat.com> | 2023-07-18 11:22:51 +0200 |
| commit | a5754847e0fc2bc08a414dd381803009e8bca390 (patch) | |
| tree | e9e5e7ad909fa5b9db26fa2e686239012ae2a12b | |
| parent | ca2a07f6c3630d5d39af24651987256a705ac500 (diff) | |
tests/avocado: Disable the test_sbsaref_edk2_firmware by default
The test fails occasionally, see e.g.:
https://gitlab.com/thuth/qemu/-/jobs/4196177756#L489
https://gitlab.com/thuth/qemu/-/jobs/4623296271#L290
It also fails on my laptop in ca. 50% of all runs. Thus disable it by
default by using the QEMU_TEST_FLAKY_TESTS environment variable to fence
it (which we also already use in flaky qtests). While we're at it, also
document this variable in docs/devel/testing.rst.
Message-Id: <20230710170155.7192-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
| -rw-r--r-- | docs/devel/testing.rst | 5 | ||||
| -rw-r--r-- | tests/avocado/machine_aarch64_sbsaref.py | 2 |
2 files changed, 6 insertions, 1 deletions
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index fb651eae11..b6ad21bed1 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -1384,6 +1384,11 @@ variable as shown on the code snippet below to skip the test: def test(self): do_something() +QEMU_TEST_FLAKY_TESTS +^^^^^^^^^^^^^^^^^^^^^ +Some tests are not working reliably and thus are disabled by default. +Set this environment variable to enable them. + Uninstalling Avocado ~~~~~~~~~~~~~~~~~~~~ diff --git a/tests/avocado/machine_aarch64_sbsaref.py b/tests/avocado/machine_aarch64_sbsaref.py index cce6ef9f65..a794245e7e 100644 --- a/tests/avocado/machine_aarch64_sbsaref.py +++ b/tests/avocado/machine_aarch64_sbsaref.py @@ -8,7 +8,6 @@ import os -from avocado import skip from avocado import skipUnless from avocado.utils import archive @@ -76,6 +75,7 @@ class Aarch64SbsarefMachine(QemuSystemTest): "sbsa-ref", ) + @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is not reliable') def test_sbsaref_edk2_firmware(self): """ :avocado: tags=cpu:cortex-a57 |