aboutsummaryrefslogtreecommitdiff
path: root/tests/requirements.txt
AgeCommit message (Collapse)Author
2023-08-28Revert "tests: Use separate virtual environment for avocado"Paolo Bonzini
This reverts commit e8e4298feadae7924cf7600bb3bcc5b0a8d7cbe9. ensuregroup allows to specify both the acceptable versions of avocado, and a locked version to be used when avocado is not installed as a system pacakge. This lets us install avocado in pyvenv/ using "mkvenv.py" and reuse the distro package on Fedora and CentOS Stream (the only distros where it's available). ensuregroup's usage of "(>=..., <=...)" constraints when evaluating the distro package, and "==" constraints when installing it from PyPI, makes it possible to avoid conflicts between the known-good version and a package plugins included in the distro. This is because package plugins have "==" constraints on the version that is included in the distro, and, using "pip install avocado==88.1" on a venv that includes system packages will result in an error: avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible. avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible. But at the same time, if the venv does not include a system distribution of avocado then we can install a known-good version and stick to LTS releases. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1663 Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-06-06tests: Use separate virtual environment for avocadoPaolo Bonzini
This reverts commits eea2d141179 ("Makefile: remove $(TESTS_PYTHON)", 2023-05-26) and 9c6692db550 ("tests: Use configure-provided pyvenv for tests", 2023-05-18). Right now, there is a conflict between wanting a ">=" constraint when using a distro-provided package and wanting a "==" constraint when installing Avocado from PyPI; this would provide the best of both worlds in terms of resiliency for both distros that have required packages and distros that don't. The conflict is visible also for meson, where we would like to install the latest 0.63.x version but also accept a distro 1.1.x version. But it is worse for avocado, for two reasons: 1) we cannot use an "==" constraint to install avocado if the venv includes a system avocado. The distro will package plugins that have "==" constraints on the version that is included in the distro, and, using "pip install avocado==88.1" on a venv that includes system packages will result in this error: ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible. avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible. make[1]: Leaving directory '/home/berrange/src/virt/qemu/build' 2) we cannot use ">=" either if the venv does _not_ include a system avocado, because that would result in the installation of v101.0 which is the one we've just reverted. So the idea is to encode the dependencies as an (acceptable, locked) tuple, like this hypothetical TOML that would be committed inside python/ and used by mkvenv.py: [meson] meson = { minimum = "0.63.0", install = "0.63.3", canary = "meson" } [docs] # 6.0 drops support for Python 3.7 sphinx = { minimum = "1.6", install = "<6.0", canary = "sphinx-build" } sphinx_rtd_theme = { minimum = "0.5" } [avocado] avocado-framework = { minimum = "88.1", install = "88.1", canary = "avocado" } Once this is implemented, it would also be possible to install avocado in pyvenv/ using "mkvenv.py ensure", thus using the distro package on Fedora and CentOS Stream (the only distros where it's available). But until this is implemented, keep avocado in a separate venv. There is still the benefit of using a single python for meson custom_targets and for sphinx. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-06-06Revert "tests/requirements.txt: bump up avocado-framework version to 101.0"Paolo Bonzini
This reverts commit ec5ffa0056389c3c10ea2de1e78366f66f4e5abc. Bumping avocado to version 101 has two issues. First, there are problems where Avocado is not logging of command lines or terminal output, and not collecting Python logs outside the avocado namespace. Second, the recent changes to Python handling mean that there is a single virtual environment for all the build, instead of a separate one for testing. Requiring a too-new version of avocado causes conflicts with any avocado plugins installed on the host: $ make check-venv make[1]: Entering directory '/home/berrange/src/virt/qemu/build' GIT ui/keycodemapdb tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc VENVPIP install -e /home/berrange/src/virt/qemu/python/ VENVPIP install -r /home/berrange/src/virt/qemu/tests/requirements.txt ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 101.0 which is incompatible. avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 101.0 which is incompatible. make[1]: Leaving directory '/home/berrange/src/virt/qemu/build' To avoid this issue, tests/requirements.txt should use a ">=" constraint and the version of Avocado should be limited to what distros provide in the system packages. Only Fedora has Avocado, and more specifically version 92.0 (though 98.0 is also available as a module). As a first step, this patch reverts the introduction of a too-new Avocado. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-18tests: Use configure-provided pyvenv for testsJohn Snow
This patch changes how the avocado tests are provided, ever so slightly. Instead of creating a new testing venv, use the configure-provided 'pyvenv' instead, and install optional packages into that. Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20230511035435.734312-20-jsnow@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-04-27tests/requirements.txt: bump up avocado-framework version to 101.0Kautuk Consul
Avocado version 101.0 has a fix to re-compute the checksum of an asset file if the algorithm used in the *-CHECKSUM file isn't the same as the one being passed to it by the avocado user (i.e. the avocado_qemu python module). In the earlier avocado versions this fix wasn't there due to which if the checksum wouldn't match the earlier checksum (calculated by a different algorithm), the avocado code would start downloading a fresh image from the internet URL thus making the test-cases take longer to execute. Bump up the avocado-framework version to 101.0. Signed-off-by: Kautuk Consul <kconsul@linux.vnet.ibm.com> Tested-by: Hariharan T S <hariharan.ts@linux.vnet.ibm.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230327115030.3418323-2-kconsul@linux.vnet.ibm.com> Message-Id: <20230424092249.58552-2-alex.bennee@linaro.org>
2022-06-06tests: install "qemu" namespace package into venvJohn Snow
This patch adds the "qemu" namespace package to the $build/tests/venv directory. It does so in "editable" mode, which means that changes to the source python directory will actively be reflected by the venv. This patch also then removes any sys.path hacking from the avocado test scripts directly. By doing this, the environment of where to find these packages is managed entirely by the virtual environment and not by the scripts themselves. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220526000921.1581503-7-jsnow@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-06-01acceptance tests: bump Avocado version to 88.1Willian Rampazzo
Besides some internal changes, new features, and bug fixes, on the QEMU side, this version fixes the following message seen when running the acceptance tests: "Error running method "pre_tests" of plugin "fetchasset": 'bytes' object has no attribute 'encode'". The release notes are available at https://avocado-framework.readthedocs.io/en/latest/releases/88_0.html. Signed-off-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210520204747.210764-2-willianr@redhat.com> Acked-by: Cleber Rosa <crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-02-15Acceptance Tests: bump Avocado version requirement to 85.0Cleber Rosa
This version (and 84.0) contain improvements that address specific QEMU use cases, including: * Being able to download and use Fedora 31 images and thus re-activate the "boot_linux.py" tests * Being able to register local assets via "avocado assets register" and use them in tests Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210211232835.2608059-2-crosa@redhat.com> Acked-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2020-12-18tests/acceptance: Bump avocado requirements to 83.0Wainer dos Santos Moschetta
To use Avocado's testlogs plug-in on CI it is required to use its 83.0 or greater version. Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20201211183827.915232-2-wainersm@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-13Acceptance tests: bump pycdlib version for easier installationCleber Rosa
On with certain versions of "pip", package installations will attempt to create wheels. And, on environments without a "complete" Python installation (as described in the acceptance tests requirements docs), that will fail. pycdlib, starting with version 1.11.0, is now being made available as wheels, so its instalation on those constrained environments is now possible. Buglink: https://bugs.launchpad.net/qemu/+bug/1897783 Reported-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20201009205513.751968-2-crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-10tests: bump avocado versionPavel Dovgalyuk
Reverse debugging test uses gdb remote client of avocado framework. This client was fixed since the currently used version 76. Therefore this patch bumps the version to 81 and fixes command line version compatibility issue. Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Tested-by: Cleber Rosa <crosa@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <159903462803.28509.16851113546106095750.stgit@pasha-ThinkPad-X280> Message-Id: <20200908202352.298506-2-philmd@redhat.com> Message-Id: <20200909112742.25730-9-alex.bennee@linaro.org>
2020-03-17Acceptance test: add "boot_linux" testsCleber Rosa
This acceptance test, validates that a full blown Linux guest can successfully boot in QEMU. In this specific case, the guest chosen is Fedora version 31. * x86_64, pc-i440fx and pc-q35 machine types, with TCG and KVM as accelerators * aarch64 and virt machine type, with TCG and KVM as accelerators * ppc64 and pseries machine type with TCG as accelerator * s390x and s390-ccw-virtio machine type with TCG as accelerator The Avocado vmimage utils library is used to download and cache the Linux guest images, and from those images a snapshot image is created and given to QEMU. If a qemu-img binary is available in the build directory, it's used to create the snapshot image, so that matching qemu-system-* and qemu-img are used in the same test run. If qemu-img is not available in the build tree, one is attempted to be found installed system-wide (in the $PATH). If qemu-img is not found in the build dir or in the $PATH, the test is canceled. The method for checking the successful boot is based on "cloudinit" and its "phone home" feature. The guest is given an ISO image with the location of the phone home server, and the information to post (the instance ID). Upon receiving the correct information, from the guest, the test is considered to have PASSed. This test is currently limited to user mode networking only, and instructs the guest to connect to the "router" address that is hard coded in QEMU. To create the cloudinit ISO image that will be used to configure the guest, the pycdlib library is also required and has been added as requirement to the virtual environment created by "check-venv". The console output is read by a separate thread, by means of the Avocado datadrainer utility module. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Tested-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20200317141654.29355-3-crosa@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2020-03-04tests/acceptance: bump avocado requirements to 76.0Alex Bennée
If we want to use @skipUnless decorations on the class we need a newer version of avocado. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20200303150622.20133-9-alex.bennee@linaro.org>
2019-09-20Acceptance tests: use avocado.utils.ssh for SSH interactionCleber Rosa
This replaces paramiko with avocado.utils.ssh module, which is based on a (open)ssh binary, supposedly more ubiquitous. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Message-Id: <20190919225905.10829-1-crosa@redhat.com> [Cleber: consolidated existing skipUnless from tests to setUp] Signed-off-by: Cleber Rosa <crosa@redhat.com>
2019-08-28tests/requirements.txt: pin paramiko version requirementCleber Rosa
It's a good practice (I'd really say a must) to pin as much as possible of the software versions used during test, so let's apply that to paramiko. According to https://pypi.org/project/paramiko/, 2.4.2 is the latest released version. It's also easily obtainable on systems such as Fedora 30. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20190607152223.9467-3-crosa@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2019-05-26BootLinuxSshTest: Test some userspace commands on MaltaPhilippe Mathieu-Daudé
This tests boot a full VM and check the serial console until the SSH daemon is running, then start a SSH session and run some commands. This test can be run using: $ avocado --show=ssh run -t arch:mips tests/acceptance/linux_ssh_mips_malta.py ssh: Entering interactive session. ssh: # uname -a ssh: Linux debian-mips 3.2.0-4-4kc-malta #1 Debian 3.2.51-1 mips GNU/Linux ssh: # lspci -d 11ab:4620 ssh: 00:00.0 Host bridge: Marvell Technology Group Ltd. GT-64120/64120A/64121A System Controller (rev 10) ssh: # cat /sys/bus/i2c/devices/i2c-0/name ssh: SMBus PIIX4 adapter at 1100 ssh: # cat /proc/mtd ssh: dev: size erasesize name ssh: mtd0: 00100000 00010000 "YAMON" ssh: mtd1: 002e0000 00010000 "User FS" ssh: mtd2: 00020000 00010000 "Board Config" ssh: # md5sum /dev/mtd2ro ssh: 0dfbe8aa4c20b52e1b8bf3cb6cbdf193 /dev/mtd2ro ssh: # poweroff Acked-by: Aleksandar Markovic <amarkovic@wavecomp.com> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Aleksandar Markovic <amarkovic@wavecomp.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Message-Id: <20190523161832.22490-5-f4bug@amsat.org>
2019-02-22Acceptance tests: drop usage of ":avocado: enable"Cleber Rosa
The Avocado test runner attemps to find its INSTRUMENTED (that is, Python based tests) in a manner that is as safe as possible to the user. Different from plain Python unittest, it won't load or execute test code on an operation such as: $ avocado list tests/acceptance/ Before version 68.0, the logic implemented to identify INSTRUMENTED tests would require either the ":avocado: enable" or ":avocado: recursive" statement as a flag for tests that would not inherit directly from "avocado.Test". This is not necessary anymore, and because of that the boiler plate statements can now be removed. Reference: https://avocado-framework.readthedocs.io/en/68.0/release_notes/68_0.html#users-test-writers Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Caio Carrara <ccarrara@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20190218173723.26120-1-crosa@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2018-10-30Acceptance tests: add make rule for running themCleber Rosa
The acceptance (aka functional, aka Avocado-based) tests are Python files located in "tests/acceptance" that need to be run with the Avocado libs and test runner. Let's provide a convenient way for QEMU developers to run them, by making use of the tests-venv with the required setup. Also, while the Avocado test runner will take care of creating a location to save test results to, it was understood that it's better if the results are kept within the build tree. Signed-off-by: Cleber Rosa <crosa@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Acked-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Caio Carrara <ccarrara@redhat.com> Message-Id: <20181018153134.8493-3-crosa@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-10-30Bootstrap Python venv for testsCleber Rosa
A number of QEMU tests are written in Python, and may benefit from an untainted Python venv. By using make rules, tests that depend on specific Python libs can set that rule as a requirement, along with rules that require the presence or installation of specific libraries. The tests/requirements.txt is supposed to contain the Python requirements that should be added to the venv created by check-venv. Signed-off-by: Cleber Rosa <crosa@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Acked-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Caio Carrara <ccarrara@redhat.com> Message-Id: <20181018153134.8493-2-crosa@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>