aboutsummaryrefslogtreecommitdiff
path: root/.gitlab-ci.d/buildtest.yml
AgeCommit message (Collapse)Author
2023-10-11gitlab: shuffle some targets and reduce avocado noiseAlex Bennée
We move a couple of targets out of the avocado runs because there are no tests to run. Tricore already has some coverage. The cris target only really has check-tcg tests but its getting harder to find anything that packages the compiler. To reduce the noise of CANCEL messages we also set AVOCADO_TAGS appropriately so we filter down the number of tests we attempt. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20231009164104.369749-5-alex.bennee@linaro.org>
2023-08-28Revert "tests: Use separate virtual environment for avocado"Paolo Bonzini
This reverts commit e8e4298feadae7924cf7600bb3bcc5b0a8d7cbe9. ensuregroup allows to specify both the acceptable versions of avocado, and a locked version to be used when avocado is not installed as a system pacakge. This lets us install avocado in pyvenv/ using "mkvenv.py" and reuse the distro package on Fedora and CentOS Stream (the only distros where it's available). ensuregroup's usage of "(>=..., <=...)" constraints when evaluating the distro package, and "==" constraints when installing it from PyPI, makes it possible to avoid conflicts between the known-good version and a package plugins included in the distro. This is because package plugins have "==" constraints on the version that is included in the distro, and, using "pip install avocado==88.1" on a venv that includes system packages will result in an error: avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible. avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible. But at the same time, if the venv does not include a system distribution of avocado then we can install a known-good version and stick to LTS releases. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1663 Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-07-03gitlab: reduce testing scope of check-gcovAlex Bennée
This keeps timing out on gitlab due to some qtests taking a long time. As this is just ensuring the gcov machinery is working and not attempting to be comprehensive lets skip qtest in this run. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230630180423.558337-4-alex.bennee@linaro.org>
2023-07-03gitlab: ensure coverage job also publishes meson logDaniel P. Berrangé
The coverage job wants to publish a coverage report on success, but the tests might fail and in that case we need the meson logs for debugging. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20230503145535.91325-3-berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230630180423.558337-3-alex.bennee@linaro.org>
2023-07-03gitlab: explicit set artifacts publishing criteriaDaniel P. Berrangé
If not set explicitly, gitlab assumes 'when: on_success" as the publishing criteria for artifacts. This is reasonable if the artifact is an output deliverable of the job. This is useless if the artifact is a log file to be used for debugging job failures. This change makes the desired criteria explicit for every job that publishes artifacts. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20230503145535.91325-2-berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230630180423.558337-2-alex.bennee@linaro.org>
2023-06-26gitlab: centralize the container tag nameDaniel P. Berrangé
We use a fixed container tag of 'latest' so that contributors' forks don't end up with an ever growing number of containers as they work on throwaway feature branches. This fixed tag causes problems running CI upstream in stable staging branches, however, because the stable staging branch will publish old container content that clashes with that needed by primary staging branch. This makes it impossible to reliably run CI pipelines in parallel in upstream for different staging branches. This introduces $QEMU_CI_CONTAINER_TAG global variable as a way to change which tag container publishing uses. Initially it can be set by contributors as a git push option if they want to override the default use of 'latest' eg git push gitlab <branch> -o ci.variable=QEMU_CONTAINER_TAG=fish this is useful if contributors need to run pipelines for different branches concurrently in their forks. Reviewed-by: Michael Tokarev <mjt@tls.msk.ru> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20230608164018.2520330-2-berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-06-06tests: Use separate virtual environment for avocadoPaolo Bonzini
This reverts commits eea2d141179 ("Makefile: remove $(TESTS_PYTHON)", 2023-05-26) and 9c6692db550 ("tests: Use configure-provided pyvenv for tests", 2023-05-18). Right now, there is a conflict between wanting a ">=" constraint when using a distro-provided package and wanting a "==" constraint when installing Avocado from PyPI; this would provide the best of both worlds in terms of resiliency for both distros that have required packages and distros that don't. The conflict is visible also for meson, where we would like to install the latest 0.63.x version but also accept a distro 1.1.x version. But it is worse for avocado, for two reasons: 1) we cannot use an "==" constraint to install avocado if the venv includes a system avocado. The distro will package plugins that have "==" constraints on the version that is included in the distro, and, using "pip install avocado==88.1" on a venv that includes system packages will result in this error: ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible. avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible. make[1]: Leaving directory '/home/berrange/src/virt/qemu/build' 2) we cannot use ">=" either if the venv does _not_ include a system avocado, because that would result in the installation of v101.0 which is the one we've just reverted. So the idea is to encode the dependencies as an (acceptable, locked) tuple, like this hypothetical TOML that would be committed inside python/ and used by mkvenv.py: [meson] meson = { minimum = "0.63.0", install = "0.63.3", canary = "meson" } [docs] # 6.0 drops support for Python 3.7 sphinx = { minimum = "1.6", install = "<6.0", canary = "sphinx-build" } sphinx_rtd_theme = { minimum = "0.5" } [avocado] avocado-framework = { minimum = "88.1", install = "88.1", canary = "avocado" } Once this is implemented, it would also be possible to install avocado in pyvenv/ using "mkvenv.py ensure", thus using the distro package on Fedora and CentOS Stream (the only distros where it's available). But until this is implemented, keep avocado in a separate venv. There is still the benefit of using a single python for meson custom_targets and for sphinx. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-22.gitlab-ci.d/buildtest.yml: Run full "make check" with --without-default-devicesThomas Huth
Thanks to the fixes from the previous patches, we can now run the full set of "make check" with all targets here. Message-Id: <20230512124033.502654-19-thuth@redhat.com> Acked-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-05-19Revert last two patchesRichard Henderson
Unintentionally pushed. Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-19Raise crash-test-debian timeout to 90 minutesCamilla Conte
When running on the Kubernetes runner, this CI job is timing out. Raise the limit to give the job enough time to run. Signed-off-by: Camilla Conte <cconte@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20230407145252.32955-2-cconte@redhat.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-18tests: Use configure-provided pyvenv for testsJohn Snow
This patch changes how the avocado tests are provided, ever so slightly. Instead of creating a new testing venv, use the configure-provided 'pyvenv' instead, and install optional packages into that. Signed-off-by: John Snow <jsnow@redhat.com> Message-Id: <20230511035435.734312-20-jsnow@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-04-27scripts/device-crash-test: Add a parameter to run with TCG onlyThomas Huth
We're currently facing the problem that the device-crash-test script runs twice as long in the CI when a runner supports KVM - which sometimes results in a timeout of the CI job. To get a more deterministic runtime here, add an option to the script that allows to run it with TCG only. Reported-by: Eldon Stegall <eldon-qemu@eldondev.com> Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-Id: <20230414145845.456145-3-thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230424092249.58552-6-alex.bennee@linaro.org>
2023-04-27gitlab-ci: Avoid to re-run "configure" in the device-crash-test jobsThomas Huth
After "make check-venv" had been added to these jobs, they started to re-run "configure" each time since our logic in the makefile thinks that some files are out of date here. Avoid it with the same trick that we are using in buildtest-template.yml already by disabling the up-to-date check via NINJA=":". Fixes: 1d8cf47e5b ("tests: run 'device-crash-test' from tests/venv") Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <20230414145845.456145-2-thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230424092249.58552-5-alex.bennee@linaro.org>
2023-03-13gitlab-ci.d/buildtest: Rework the target list of build-system-alpineThomas Huth
The target list of the build-system-alpine job is pretty much a copy of the build-system-ubuntu job (apart from "aarch64-softmmu" which has recently been removed from the ubuntu job in commit 6eda5ef5f8f4, but aarch64-softmmu is still also tested in the opensuse jobs, so we don't need to keep it here). Let's stop wasting our CI minutes with such duplications, and focus on testing targets instead that do not have such a great test coverage yet: The "loongarch64-softmmu" target has never been added to our build tests yet since it has been introduced, and the "mips64-softmmu" target is so far only tested in jobs that lack the "avocado" testing stage (only the little endian or 32-bit MIPS variants are tested in jobs with avocado so far). While we're at it, also move the avr-softmmu and mipsel-softmmu targets from the Debian job to the alpine job, since the Debian job (and its following test jobs) has already a long runtime compared to the others jobs. With this movement, the runtimes should be more equally distributed along the parallel running jobs now. Message-Id: <20230309164850.109882-1-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-03-01gitlab: move the majority of artefact handling to a templateAlex Bennée
To avoid lots of copy and paste lets deal with artefacts in a template. This way we can filter out most of the pre-binary object and library files we no longer need as we have the final binaries. build-system-alpine also saved .git-submodule-status so for simplicity we bring that into the template as well. As an example the build-system-ubuntu artefacts before this patch where around 1.3 GB, after dropping the object files it comes to 970 MB. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-Id: <20230228190653.1602033-23-alex.bennee@linaro.org>
2023-03-01testing: update ubuntu2004 to ubuntu2204Alex Bennée
The 22.04 LTS release has been out for almost a year now so its time to update all the remaining images to the current LTS. We can also drop some hacks we need for older clang TSAN support. We will keep the ubuntu2004 container around for those who wish to test builds on the currently still supported baseline. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20230228190653.1602033-9-alex.bennee@linaro.org>
2023-03-01tests: don't run benchmarks for the tsan buildAlex Bennée
All we are really doing here is checking that TSAN builds compile and are therefor a tool available to developers. The benchmarks are not representative of QEMU's actual threading behaviour and they burn precious CI time. Indeed switching to check-unit reveals many unaddressed issues which have been logged at: https://gitlab.com/qemu-project/qemu/-/issues/1496 So for now disable the make check and make this a build only test. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20230228190653.1602033-8-alex.bennee@linaro.org>
2023-02-27gitlab-ci.d: Build with --enable-fdt=system by defaultThomas Huth
By using --enable-fdt=system we can make sure that the configure script does not try to check out the "dtc" submodule. This should help to safe some precious CI minutes in the long run. While we're at it, also drop some now-redundant --enable-slirp and --enable-capstone statements. These used to have the "=system" suffix in the past, too, which has been dropped when the their corresponding submodules had been removed. Since these features are auto-enabled anyway now (since the containers have the right libraries installed), we do not need the explicit --enable-... statements anymore. Message-Id: <20230207201447.566661-6-thuth@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-02-27gitlab-ci.d/buildtest: Disintegrate the build-coroutine-sigaltstack jobThomas Huth
We can get rid of the build-coroutine-sigaltstack job by moving the configure flags that should be tested here to other jobs: Move --with-coroutine=sigaltstack to the build-system-debian job (where the coroutines should get some more test coverage with "make check-block", too) and --enable-trace-backends=ftrace to the cross-s390x-kvm-only job. Message-Id: <20230207201447.566661-4-thuth@redhat.com> Reviewed-by: David Woodhouse <dwmw2@infradead.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-02-27gitlab-ci.d/buildtest: Remove aarch64-softmmu from the build-system-ubuntu jobThomas Huth
aarch64-softmmu is also checked on the same version of Ubuntu in the gcov job, so it is redundant to check again in the normal ubuntu job. Message-Id: <20230207201447.566661-3-thuth@redhat.com> Reviewed-by: David Woodhouse <dwmw2@infradead.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-02-14build: deprecate --enable-gprof builds and remove from CIAlex Bennée
As gprof relies on instrumentation you rarely get useful data compared to a real optimised build. Lets deprecate the build option and simplify the CI configuration as a result. Buglink: https://gitlab.com/qemu-project/qemu/-/issues/1338 Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230131094224.861621-1-alex.bennee@linaro.org> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-02-02Merge tag 'pull-jan-omnibus-020223-1' of https://gitlab.com/stsquad/qemu ↵Peter Maydell
into staging Testing, docs, semihosting and plugin updates - update playbooks for custom runners - add section timing support to gitlab - upgrade fedora images to 37 - purge perl from the build system and deps - disable unstable tests in CI - improve intro, emulation and semihosting docs - semihosting bug fix and O_BINARY default - add memory-sve test - fix some races in qht - improve plugin handling of memory helpers - optimise plugin hooks - fix some plugin deadlocks - reduce win64-cross build time by dropping some targets # -----BEGIN PGP SIGNATURE----- # # iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmPb3fgACgkQ+9DbCVqe # KkQbXAf9Eoc+PdNvafbqzH/blPjvd9ve8pJ+GcPDukNXwxP8OF/jFEJUQ1E7l9O7 # y0qV4akKCdIqVice4R5bK2CAq44Y3aut8SDf56C8E3Riha2zA2RbQWOv/zCvA3OP # LFF+OaXZyg4JTR48HUKzh9ei2bd1+ccBSUe+xlRi59XaV5K8+5bmcZj10QKUR0lD # 0HC5auEWWpayvd5D7Da15C7+oVY3LMCFxSdpHwbuIPPan/TRo5yqMI6ChYDKB8QD # gdwMCL8znj2ADCTBftyBDYDAtjKVyLQidf7KdQHiSF+nmXYopS6SbsPCOMtJqCMH # tXcKAIxs/MEntPrWTKTdtdnzotJVKw== # =AtfN # -----END PGP SIGNATURE----- # gpg: Signature made Thu 02 Feb 2023 15:59:52 GMT # gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44 # gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full] # Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44 * tag 'pull-jan-omnibus-020223-1' of https://gitlab.com/stsquad/qemu: (36 commits) gitlab: cut even more from cross-win64-system build plugins: Iterate on cb_lists in qemu_plugin_user_exit cpu-exec: assert that plugin_mem_cbs is NULL after execution tcg: exclude non-memory effecting helpers from instrumentation translator: always pair plugin_gen_insn_{start, end} calls plugins: fix optimization in plugin_gen_disable_mem_helpers plugins: make qemu_plugin_user_exit's locking order consistent with fork_start's util/qht: use striped locks under TSAN thread: de-const qemu_spin_destroy util/qht: add missing atomic_set(hashes[i]) cpu: free cpu->tb_jmp_cache with RCU tests/tcg: add memory-sve test for aarch64 semihosting: add O_BINARY flag in host_open for NT compatibility semihosting: Write back semihosting data before completion callback docs: add an introduction to the system docs semihosting: add semihosting section to the docs docs: add a new section to outline emulation support docs: add hotlinks to about preface text MAINTAINERS: Fix the entry for tests/tcg/nios2 gitlab: wrap up test results for custom runners ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2023-02-02gitlab: add lsan suppression file to workaround tcmalloc issuesAlex Bennée
The up-coming upgrade to Fedora 37 will bring in libtcmalloc as a dependency of libglusterfs which confuses our fuzz run. Rather than disable the build lets use LSAN's suppression mechanism to prevent the job from failing. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Cc: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20230124180127.1881110-10-alex.bennee@linaro.org>
2023-01-31gitlab-ci.d/buildtest: Merge the --without-default-* jobsThomas Huth
Let's safe some CI minutes by merging these two jobs. We can now also drop "--disable-capstone" since the capstone submodule has been removed a while ago. We should rather test --disable-fdt now to check a compilation without the "dtc" submodule (for this we have to drop i386-softmmu from the target list unfortunately). Additionally, the qtests with s390x and sh4 are not read for "--without-default-devices" yet, so we can only test mips64 and avr here now. Message-Id: <20230130104446.1286773-5-thuth@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Fabiano Rosas <farosas@suse.de> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-01-31gitlab-ci.d/buildtest: Remove ppc-softmmu from the clang-system jobThomas Huth
We are also compile-testing ppc64-softmmu with clang in the "tsan-build" job, and ppc64-softmmu covers pretty much the same code as ppc-softmmu, so we should not lose much test coverage here by removing ppc-softmmu from the "clang-system" job. Message-Id: <20230130104446.1286773-2-thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-12-23gitlab-ci: Disable docs and GUIs for the build-tci and build-tcg-disabled jobsThomas Huth
These jobs use their own "script:" section and thus do not profit from the global "--disable-docs" from the template. While we're at it, disable also some GUI front ends here since we do not gain any additional test coverage by compiling those here again. Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <20221208135945.99975-1-thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-Id: <20221221090411.1995037-7-alex.bennee@linaro.org>
2022-12-23tests/docker: use prebuilt toolchain for debian-hexagon-crossMukilan Thiyagarajan
The current docker image for cross compiling hexagon guests is manually built since it takes >2 hours to build from source. This patch: 1. Solves the above issue by using the prebuilt clang toolchain hosted on CodeLinaro [1] and maintained by QUIC [2]. 2. The dockerfile is also switched from multi-stage to single stage build to allow the CI docker engine to reuse the layer cache. 3. Re-enables the hexagon-cross-container job to be always run in CI and makes it a non-optional dependency for the build-user-hexagon job. The changes for 1 & 2 together bring down the build time to ~3 minutes in GitLab CI when cache is reused and ~9 minutes when cache cannot be reused. [1]: https://github.com/CodeLinaro/hexagon-builder [2]: https://github.com/quic/toolchain_for_hexagon/releases/ Signed-off-by: Mukilan Thiyagarajan <quic_mthiyaga@quicinc.com> [AJB: also tweak MAINTAINERS, remove QEMU_JOB_ONLY_FORKS and comment] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20221219144354.11659-1-quic_mthiyaga@quicinc.com> Message-Id: <20221221090411.1995037-6-alex.bennee@linaro.org>
2022-11-22gitlab: integrate coverage reportAlex Bennée
This should hopefully give is nice coverage information about what our tests (or at least the subset we are running) have hit. Ideally we would want a way to trigger coverage on tests likely to be affected by the current commit. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20221117172532.538149-12-alex.bennee@linaro.org>
2022-11-06gitlab-ci: increase clang-user timeoutStefan Hajnoczi
The clang-user test exceeds the 1 hour timeout occassionally. Philippe Mathieu-Daudé has pointed out that the number of tcg tests has increased since QEMU 7.1. The execution time therefore probably reflects a legitimate increase in tests rather than a performance regression. Bump the timeout to prevent CI failures. Suggested-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20221104113659.427690-1-stefanha@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-09-26Remove the slirp submodule (i.e. compile only with an external libslirp)Thomas Huth
Since QEMU 7.1 we don't support Ubuntu 18.04 anymore, so the last big important Linux distro that did not have a pre-packaged libslirp has been dismissed. All other major distros seem to have a libslirp package in their distribution already - according to repology.org: Fedora 35: 4.6.1 CentOS 8 (RHEL-8): 4.4.0 Debian 11: 4.4.0 OpenSUSE Leap 15.3: 4.3.1 Ubuntu LTS 20.04: 4.1.0 FreeBSD Ports: 4.7.0 NetBSD pkgsrc: 4.7.0 Homebrew: 4.7.0 MSYS2 mingw: 4.7.0 The only one that was still missing a libslirp package is OpenBSD - but the next version (OpenBSD 7.2 which will be shipped in October) is going to include a libslirp package. Since QEMU 7.2 will be published after OpenBSD 7.2, we should be fine there, too. So there is no real urgent need for keeping the slirp submodule in the QEMU tree anymore. Thus let's drop the slirp submodule now and rely on the libslirp packages from the distributions instead. Message-Id: <20220824151122.704946-7-thuth@redhat.com> Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-07-13configure: Restrict TCG to emulationPhilippe Mathieu-Daudé
If we don't need to emulate any target, we certainly don't need TCG. This should also help to compile again with ".../configure --enable-tools --disable-system --disable-user" on systems that do not have a TCG backend. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> [thuth: Re-arranged the code, remove check-softfloat from buildtest.yml] Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20220706153816.768143-1-thuth@redhat.com>
2022-06-15vfio-user: build libraryJagannathan Raman
add the libvfio-user library as a submodule. build it as a meson subproject. libvfio-user is distributed with BSD 3-Clause license and json-c with MIT (Expat) license Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com> Signed-off-by: John G Johnson <john.g.johnson@oracle.com> Signed-off-by: Jagannathan Raman <jag.raman@oracle.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: c2adec87958b081d1dc8775d4aa05c897912f025.1655151679.git.jag.raman@oracle.com [Changed submodule URL to QEMU's libvfio-user mirror on GitLab. The QEMU project mirrors its dependencies so that it can provide full source code even in the event that its dependencies become unavailable. Note that the mirror repo is manually updated, so please contact me to make newer libvfio-user commits available. If I become a bottleneck we can set up a cronjob. Updated scripts/meson-buildoptions.sh to match the meson_options.txt change. Failure to do so can result in scripts/meson-buildoptions.sh being modified by the build system later on and you end up with a dirty working tree. --Stefan] Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-06-14gitlab-ci: Fix the build-cfi-aarch64 and build-cfi-ppc64-s390x jobsThomas Huth
The job definitions recently got a second "variables:" section by accident and thus are failing now if one tries to run them. Merge the two sections into one again to fix the issue. And while we're at it, bump the timeout here (70 minutes are currently not enough for the aarch64 job). The jobs are marked as manual anyway, so if the user starts them, they want to see their result for sure and then it's annoying if the job timeouts too early. Fixes: e312d1fdbb ("gitlab: convert build/container jobs to .base_job_template") Signed-off-by: Thomas Huth <thuth@redhat.com> Acked-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20220603124809.70794-1-thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20220613171258.1905715-4-alex.bennee@linaro.org>
2022-06-06tests: run 'device-crash-test' from tests/venvJohn Snow
Remove the sys.path hacking from device-crash-test, and add in a little user-friendly message for anyone who was used to running this script directly from the source tree. Modify the GitLab job recipes to create the tests/venv first, then run device-crash-test from that venv. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20220526000921.1581503-10-jsnow@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-01gitlab: convert build/container jobs to .base_job_templateDaniel P. Berrangé
This converts the main build and container jobs to use the base job rules, defining the following new variables - QEMU_JOB_SKIPPED - jobs that are known to be currently broken and should not be run. Can still be manually launched if desired. - QEMU_JOB_AVOCADO - jobs that run the Avocado integration test harness. - QEMU_JOB_PUBLISH - jobs that publish content after the branch is merged upstream As build-tools-and-docs runs on master we declare the requirement of building amd64-debian-container optional as it should already exits once we merge. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20220526110705.59952-5-berrange@redhat.com> [AJB: fix upstream typo, mention optional container req] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20220527153603.887929-32-alex.bennee@linaro.org>
2022-05-18capstone: Remove the capstone submoduleThomas Huth
Now that we allow compiling with Capstone v3.0.5 again, all our supported build hosts should provide at least this version of the disassembler library, so we do not need to ship this as a submodule anymore. Message-Id: <20220516145823.148450-4-thuth@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-05-18capstone: Allow version 3.0.5 againThomas Huth
According to https://lore.kernel.org/qemu-devel/20200921174118.39352-1-richard.henderson@linaro.org/ there was an issue with Capstone 3.0.4 from Ubuntu 18, which was the reason for bumping our minimum Capstone requirement to version 4.0. And indeed, compiling with that version 3.0.4 from Ubuntu 18.04 still fails (after allowing it with a hack in meson.build). But now that we've dropped support for Ubuntu 18.04, that issue is not relevant anymore. Compiling with Capstone version 3.0.5 (e.g. used in Ubuntu 20.04) seems to work fine, so let's allow that version again. Message-Id: <20220516145823.148450-3-thuth@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-03-15gitlab-ci: do not run tests with address sanitizerPaolo Bonzini
This is flaky and sometimes fails or hangs unexplicably. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-02-09linux-user: Remove the deprecated ppc64abi32 targetThomas Huth
It's likely broken, and nobody cared for picking it up again during the deprecation phase, so let's remove this now. Since this is the last entry in deprecated_targets_list, remove the related code in the configure script, too. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Acked-by: Cédric Le Goater <clg@kaod.org> Acked-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20211215084958.185214-1-thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20220112112722.3641051-32-alex.bennee@linaro.org>
2022-01-18ci: explicitly skip I/O tests on alpineDaniel P. Berrangé
The block I/O tests don't work on Alpine because their alternative libc impl emits different strings for errnos, which breaks the expected output matching. e.g. === IO: pattern 102 wrote 512/512 bytes at offset 512 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: Input/output error +qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: I/O error 4 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824 Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=0 Currently the I/O tests are skipped as a side effect of the Alpine image containing a minimal busybox 'sed' binary, rather than GNU sed. This is a fragile assumption that will be invalidated when the dockerfile is changed to be autogenerated from a standardized package list that includes GNU sed. Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20211215141949.3512719-6-berrange@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20220105135009.1584676-6-alex.bennee@linaro.org>
2022-01-05gitlab-ci: Enable docs in the centos jobThomas Huth
We just ran into a problem that the docs don't build on RHEL8 / CentOS 8 anymore. Seems like these distros are using one of the oldest Sphinx versions that we still have to support. Thus enable the docs build in the CI on CentOS so that such bugs don't slip in so easily again. Message-Id: <20220104091240.160867-1-thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-12-15gitlab-ci.d/buildtest: Add jobs that run the device-crash-testThomas Huth
The device-crash-test script has been quite neglected in the past, so that it bit-rot quite often. Let's add CI jobs that run this script for at least some targets, so that this script does not regress that easily anymore. Message-Id: <20211126162724.1162049-1-thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-11-08tests/acceptance: rename tests acceptance to tests avocadoWillian Rampazzo
In the discussion about renaming the `tests/acceptance` [1], the conclusion was that the folders inside `tests` are related to the framework running the tests and not directly related to the type of the tests. This changes the folder to `tests/avocado` and adjusts the MAKEFILE, the CI related files and the documentation. [1] https://lists.gnu.org/archive/html/qemu-devel/2021-05/msg06553.html Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20211105155354.154864-3-willianr@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-11-02microvm: add device tree support.Gerd Hoffmann
Allows edk2 detect virtio-mmio devices and pcie ecam. See comment in hw/i386/microvm-dt.c for more details. Signed-off-by: Gerd Hoffmann <kraxel@redhat.com> Reviewed-by: Sergio Lopez <slp@redhat.com> Message-Id: <20211014193617.2475578-1-kraxel@redhat.com>
2021-09-06gitlab-ci: Don't try to use the system libfdt in the debian jobThomas Huth
libfdt in Debian is too old to be usable for QEMU. So far we were silently falling back to the internal dtc submodule, but since this is wrong, let's remove the --enable-fdt=system switch here now. Message-Id: <20210827151718.178988-1-thuth@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Acked-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-09-03Merge remote-tracking branch 'remotes/stsquad/tags/pull-for-6.2-020921-1' ↵Peter Maydell
into staging Testing and plugin updates: - fix typo in execlog plugin - clean-up and document gitlab FOO_RUNNER_AVAILABLE vars - fix plugin build issue on OSX and modules - add multi-core support to cache modelling plugin - clean-ups for plugin arg=FOO handling # gpg: Signature made Thu 02 Sep 2021 11:33:02 BST # gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44 # gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full] # Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44 * remotes/stsquad/tags/pull-for-6.2-020921-1: (22 commits) docs/devel: be consistent about example plugin names docs/deprecated: deprecate passing plugin args through `arg=` tests/plugins/syscalls: adhere to new arg-passing scheme tests/plugins/mem: introduce "track" arg and make args not positional tests/plugins/insn: made arg inline not positional and parse it as bool tests/plugins/bb: adapt to the new arg passing scheme docs/tcg-plugins: new passing parameters scheme for cache docs plugins/howvec: adapting to the new argument passing scheme plugins/hwprofile: adapt to the new plugin arguments scheme plugins/lockstep: make socket path not positional & parse bool arg plugins/hotblocks: Added correct boolean argument parsing plugins/hotpages: introduce sortby arg and parsed bool args correctly plugins/api: added a boolean parsing plugin api plugins: allow plugin arguments to be passed directly docs/devel/tcg-plugins: added cores arg to cache plugin plugins: sort exported symbol list plugins/cache: supported multicore cache modelling plugins: do not limit exported symbols if modules are active gitlab-ci: Fix ..._RUNNER_AVAILABLE variables and document them gitlab-ci: Remove superfluous "dnf install" statement ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-09-02gitlab-ci: Remove superfluous "dnf install" statementThomas Huth
The container already features meson and ninja, so there is no need to try to install it with dnf again. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210730143809.717079-3-thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20210806141015.2487502-4-alex.bennee@linaro.org>
2021-09-02gitlab-ci: Merge "build-disabled" with "build-without-default-features"Thomas Huth
Both jobs are testing more or less the same thing (building QEMU with features disabled), so we are wasting precious CI cycles here by doing this twice. Merge the jobs by using --without-default-features by default and just adding some additional --disable-... switches which are not covered by the generic switch (yet). And while we're at it, also test compilation with "--disable-fdt" (which forces us to change the list of targets in this job, though, since some targets do not work without fdt). Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210730143809.717079-2-thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20210806141015.2487502-3-alex.bennee@linaro.org>
2021-09-01iotests: move 222 to tests/image-fleecingVladimir Sementsov-Ogievskiy
Give a good name to test file. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210824083856.17408-29-vsementsov@virtuozzo.com> [hreitz: Adjust .gitlab-ci.d/buildtest.yml] Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-07-29gitlab-ci.d/buildtest: Mark the aarch64 and ppc64-s390x CFI jobs as manualThomas Huth
These two jobs are currently failing very often - the linker seems to get killed due to out-of-memory problems. Since apparently nobody has currently an idea how to fix that nicely, let's mark the jobs as manual for the time being until someone comes up with a proper fix. Message-Id: <20210728075141.400816-1-thuth@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>