Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* -smp cleanpus
* Hyper-V englightenment functionality
* Documentation cleanups

# gpg: Signature made Fri 01 Oct 2021 01:11:00 PM EDT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]

* remotes/bonzini/tags/for-upstream: (29 commits)
  docs: reorganize testing.rst
  docs: move gcov section at the end of testing.rst
  docs: reorganize tcg-plugins.rst
  docs: reorganize qgraph.rst
  docs: put "make" information together in build-system.rst
  docs: move notes inside the body of the document
  docs: name included files ".rst.inc"
  i386: Change the default Hyper-V version to match WS2016
  i386: Make Hyper-V version id configurable
  i386: Implement pseudo 'hv-avic' ('hv-apicv') enlightenment
  i386: Move HV_APIC_ACCESS_RECOMMENDED bit setting to hyperv_fill_cpuids()
  i386: Support KVM_CAP_HYPERV_ENFORCE_CPUID
  i386: Support KVM_CAP_ENFORCE_PV_FEATURE_CPUID
  machine: Put all sanity-check in the generic SMP parser
  machine: Use g_autoptr in machine_set_smp
  machine: Move smp_prefer_sockets to struct SMPCompatProps
  machine: Remove smp_parse callback from MachineClass
  machine: Make smp_parse generic enough for all arches
  machine: Tweak the order of topology members in struct CpuTopology
  machine: Use ms instead of global current_machine in sanity-check
  ...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>

9 files changed, 231 insertions, 223 deletions

diff --git a/docs/devel/build-system.rst b/docs/devel/build-system.rst
index 3baec158f2..0f636d620e 100644
--- a/docs/devel/build-system.rst
+++ b/docs/devel/build-system.rst
@@ -380,6 +380,16 @@ phony target, while benchmarks are run with ``make bench``.  Meson test
 suites such as ``unit`` can be ran with ``make check-unit`` too.  It is also
 possible to run tests defined in meson.build with ``meson test``.
 
+Useful make targets
+-------------------
+
+``help``
+  Print a help message for the most common build targets.
+
+``print-VAR``
+  Print the value of the variable VAR. Useful for debugging the build
+  system.
+
 Important files for the build system
 ====================================
 
@@ -473,14 +483,3 @@ Built by Makefile:
   meson.build.  The rules are produced from Meson's JSON description of
   tests (obtained with "meson introspect --tests") through the script
   scripts/mtest2make.py.
-
-
-Useful make targets
--------------------
-
-``help``
-  Print a help message for the most common build targets.
-
-``print-VAR``
-  Print the value of the variable VAR. Useful for debugging the build
-  system.
diff --git a/docs/devel/ci-definitions.rst b/docs/devel/ci-definitions.rst.inc
index 32e22ff468..32e22ff468 100644
--- a/docs/devel/ci-definitions.rst
+++ b/docs/devel/ci-definitions.rst.inc
diff --git a/docs/devel/ci-jobs.rst b/docs/devel/ci-jobs.rst.inc
index 277975e4ad..277975e4ad 100644
--- a/docs/devel/ci-jobs.rst
+++ b/docs/devel/ci-jobs.rst.inc
diff --git a/docs/devel/ci-runners.rst b/docs/devel/ci-runners.rst.inc
index 7817001fb2..7817001fb2 100644
--- a/docs/devel/ci-runners.rst
+++ b/docs/devel/ci-runners.rst.inc
diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
index 8d95247188..d106610096 100644
--- a/docs/devel/ci.rst
+++ b/docs/devel/ci.rst
@@ -8,6 +8,6 @@ found at::
 
    https://wiki.qemu.org/Testing/CI
 
-.. include:: ci-definitions.rst
-.. include:: ci-jobs.rst
-.. include:: ci-runners.rst
+.. include:: ci-definitions.rst.inc
+.. include:: ci-jobs.rst.inc
+.. include:: ci-runners.rst.inc
diff --git a/docs/devel/multi-process.rst b/docs/devel/multi-process.rst
index 69699329d6..e5758a79ab 100644
--- a/docs/devel/multi-process.rst
+++ b/docs/devel/multi-process.rst
@@ -1,15 +1,17 @@
-This is the design document for multi-process QEMU. It does not
-necessarily reflect the status of the current implementation, which
-may lack features or be considerably different from what is described
-in this document. This document is still useful as a description of
-the goals and general direction of this feature.
-
-Please refer to the following wiki for latest details:
-https://wiki.qemu.org/Features/MultiProcessQEMU
-
 Multi-process QEMU
 ===================
 
+.. note::
+
+  This is the design document for multi-process QEMU. It does not
+  necessarily reflect the status of the current implementation, which
+  may lack features or be considerably different from what is described
+  in this document. This document is still useful as a description of
+  the goals and general direction of this feature.
+
+  Please refer to the following wiki for latest details:
+  https://wiki.qemu.org/Features/MultiProcessQEMU
+
 QEMU is often used as the hypervisor for virtual machines running in the
 Oracle cloud. Since one of the advantages of cloud computing is the
 ability to run many VMs from different tenants in the same cloud
diff --git a/docs/devel/qgraph.rst b/docs/devel/qgraph.rst
index c2882c3a33..db44d71002 100644
--- a/docs/devel/qgraph.rst
+++ b/docs/devel/qgraph.rst
@@ -1,8 +1,7 @@
 .. _qgraph:
 
-========================================
 Qtest Driver Framework
-========================================
+======================
 
 In order to test a specific driver, plain libqos tests need to
 take care of booting QEMU with the right machine and devices.
@@ -31,13 +30,15 @@ so the sdhci-test should only care of linking its qgraph node with
 that interface. In this way, if the command line of a sdhci driver
 is changed, only the respective qgraph driver node has to be adjusted.
 
+QGraph concepts
+---------------
+
 The graph is composed by nodes that represent machines, drivers, tests
 and edges that define the relationships between them (``CONSUMES``, ``PRODUCES``, and
 ``CONTAINS``).
 
-
 Nodes
-^^^^^^
+~~~~~
 
 A node can be of four types:
 
@@ -64,7 +65,7 @@ Notes for the nodes:
   drivers name, otherwise they won't be discovered
 
 Edges
-^^^^^^
+~~~~~
 
 An edge relation between two nodes (drivers or machines) ``X`` and ``Y`` can be:
 
@@ -73,7 +74,7 @@ An edge relation between two nodes (drivers or machines) ``X`` and ``Y`` can be:
 - ``X CONTAINS Y``: ``Y`` is part of ``X`` component
 
 Execution steps
-^^^^^^^^^^^^^^^
+~~~~~~~~~~~~~~~
 
 The basic framework steps are the following:
 
@@ -92,8 +93,64 @@ The basic framework steps are the following:
 Depending on the QEMU binary used, only some drivers/machines will be
 available and only test that are reached by them will be executed.
 
+Command line
+~~~~~~~~~~~~
+
+Command line is built by using node names and optional arguments
+passed by the user when building the edges.
+
+There are three types of command line arguments:
+
+- ``in node``      : created from the node name. For example, machines will
+  have ``-M <machine>`` to its command line, while devices
+  ``-device <device>``. It is automatically done by the framework.
+- ``after node``   : added as additional argument to the node name.
+  This argument is added optionally when creating edges,
+  by setting the parameter ``after_cmd_line`` and
+  ``extra_edge_opts`` in ``QOSGraphEdgeOptions``.
+  The framework automatically adds
+  a comma before ``extra_edge_opts``,
+  because it is going to add attributes
+  after the destination node pointed by
+  the edge containing these options, and automatically
+  adds a space before ``after_cmd_line``, because it
+  adds an additional device, not an attribute.
+- ``before node``  : added as additional argument to the node name.
+  This argument is added optionally when creating edges,
+  by setting the parameter ``before_cmd_line`` in
+  ``QOSGraphEdgeOptions``. This attribute
+  is going to add attributes before the destination node
+  pointed by the edge containing these options. It is
+  helpful to commands that are not node-representable,
+  such as ``-fdsev`` or ``-netdev``.
+
+While adding command line in edges is always used, not all nodes names are
+used in every path walk: this is because the contained or produced ones
+are already added by QEMU, so only nodes that "consumes" will be used to
+build the command line. Also, nodes that will have ``{ "abstract" : true }``
+as QMP attribute will loose their command line, since they are not proper
+devices to be added in QEMU.
+
+Example::
+
+    QOSGraphEdgeOptions opts = {
+        .before_cmd_line = "-drive id=drv0,if=none,file=null-co://,"
+                           "file.read-zeroes=on,format=raw",
+        .after_cmd_line = "-device scsi-hd,bus=vs0.0,drive=drv0",
+
+        opts.extra_device_opts = "id=vs0";
+    };
+
+    qos_node_create_driver("virtio-scsi-device",
+                            virtio_scsi_device_create);
+    qos_node_consumes("virtio-scsi-device", "virtio-bus", &opts);
+
+Will produce the following command line:
+``-drive id=drv0,if=none,file=null-co://, -device virtio-scsi-device,id=vs0 -device scsi-hd,bus=vs0.0,drive=drv0``
+
 Troubleshooting unavailable tests
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
 If there is no path from an available machine to a test then that test will be
 unavailable and won't execute. This can happen if a test or driver did not set
 up its qgraph node correctly. It can also happen if the necessary machine type
@@ -151,7 +208,7 @@ Typically this is because the QEMU binary lacks support for the necessary
 machine type or device.
 
 Creating a new driver and its interface
-"""""""""""""""""""""""""""""""""""""""""
+---------------------------------------
 
 Here we continue the ``sdhci`` use case, with the following scenario:
 
@@ -489,7 +546,7 @@ or inverting the consumes edge in consumed_by::
             arm/raspi2b --contains--> generic-sdhci
 
 Adding a new test
-"""""""""""""""""
+-----------------
 
 Given the above setup, adding a new test is very simple.
 ``sdhci-test``, taken from ``tests/qtest/sdhci-test.c``::
@@ -565,62 +622,7 @@ and for the binary ``QTEST_QEMU_BINARY=./qemu-system-arm``:
 
 Additional examples are also in ``test-qgraph.c``
 
-Command line:
-""""""""""""""
-
-Command line is built by using node names and optional arguments
-passed by the user when building the edges.
-
-There are three types of command line arguments:
-
-- ``in node``      : created from the node name. For example, machines will
-  have ``-M <machine>`` to its command line, while devices
-  ``-device <device>``. It is automatically done by the framework.
-- ``after node``   : added as additional argument to the node name.
-  This argument is added optionally when creating edges,
-  by setting the parameter ``after_cmd_line`` and
-  ``extra_edge_opts`` in ``QOSGraphEdgeOptions``.
-  The framework automatically adds
-  a comma before ``extra_edge_opts``,
-  because it is going to add attributes
-  after the destination node pointed by
-  the edge containing these options, and automatically
-  adds a space before ``after_cmd_line``, because it
-  adds an additional device, not an attribute.
-- ``before node``  : added as additional argument to the node name.
-  This argument is added optionally when creating edges,
-  by setting the parameter ``before_cmd_line`` in
-  ``QOSGraphEdgeOptions``. This attribute
-  is going to add attributes before the destination node
-  pointed by the edge containing these options. It is
-  helpful to commands that are not node-representable,
-  such as ``-fdsev`` or ``-netdev``.
-
-While adding command line in edges is always used, not all nodes names are
-used in every path walk: this is because the contained or produced ones
-are already added by QEMU, so only nodes that "consumes" will be used to
-build the command line. Also, nodes that will have ``{ "abstract" : true }``
-as QMP attribute will loose their command line, since they are not proper
-devices to be added in QEMU.
-
-Example::
-
-    QOSGraphEdgeOptions opts = {
-        .before_cmd_line = "-drive id=drv0,if=none,file=null-co://,"
-                           "file.read-zeroes=on,format=raw",
-        .after_cmd_line = "-device scsi-hd,bus=vs0.0,drive=drv0",
-
-        opts.extra_device_opts = "id=vs0";
-    };
-
-    qos_node_create_driver("virtio-scsi-device",
-                            virtio_scsi_device_create);
-    qos_node_consumes("virtio-scsi-device", "virtio-bus", &opts);
-
-Will produce the following command line:
-``-drive id=drv0,if=none,file=null-co://, -device virtio-scsi-device,id=vs0 -device scsi-hd,bus=vs0.0,drive=drv0``
-
 Qgraph API reference
-^^^^^^^^^^^^^^^^^^^^
+--------------------
 
 .. kernel-doc:: tests/qtest/libqos/qgraph.h
diff --git a/docs/devel/tcg-plugins.rst b/docs/devel/tcg-plugins.rst
index dac5101a3c..842ae01a4c 100644
--- a/docs/devel/tcg-plugins.rst
+++ b/docs/devel/tcg-plugins.rst
@@ -3,7 +3,6 @@
    Copyright (c) 2019, Linaro Limited
    Written by Emilio Cota and Alex Bennée
 
-================
 QEMU TCG Plugins
 ================
 
@@ -16,8 +15,30 @@ only monitor it passively. However they can do this down to an
 individual instruction granularity including potentially subscribing
 to all load and store operations.
 
-API Stability
-=============
+Usage
+-----
+
+Any QEMU binary with TCG support has plugins enabled by default.
+Earlier releases needed to be explicitly enabled with::
+
+  configure --enable-plugins
+
+Once built a program can be run with multiple plugins loaded each with
+their own arguments::
+
+  $QEMU $OTHER_QEMU_ARGS \
+      -plugin tests/plugin/libhowvec.so,inline=on,count=hint \
+      -plugin tests/plugin/libhotblocks.so
+
+Arguments are plugin specific and can be used to modify their
+behaviour. In this case the howvec plugin is being asked to use inline
+ops to count and break down the hint instructions by type.
+
+Writing plugins
+---------------
+
+API versioning
+~~~~~~~~~~~~~~
 
 This is a new feature for QEMU and it does allow people to develop
 out-of-tree plugins that can be dynamically linked into a running QEMU
@@ -25,9 +46,6 @@ process. However the project reserves the right to change or break the
 API should it need to do so. The best way to avoid this is to submit
 your plugin upstream so they can be updated if/when the API changes.
 
-API versioning
---------------
-
 All plugins need to declare a symbol which exports the plugin API
 version they were built against. This can be done simply by::
 
@@ -43,18 +61,8 @@ current API versions supported by QEMU. The API version will be
 incremented if new APIs are added. The minimum API version will be
 incremented if existing APIs are changed or removed.
 
-Exposure of QEMU internals
---------------------------
-
-The plugin architecture actively avoids leaking implementation details
-about how QEMU's translation works to the plugins. While there are
-conceptions such as translation time and translation blocks the
-details are opaque to plugins. The plugin is able to query select
-details of instructions and system configuration only through the
-exported *qemu_plugin* functions.
-
-Query Handle Lifetime
----------------------
+Lifetime of the query handle
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Each callback provides an opaque anonymous information handle which
 can usually be further queried to find out information about a
@@ -63,32 +71,8 @@ valid during the lifetime of the callback so it is important that any
 information that is needed is extracted during the callback and saved
 by the plugin.
 
-API
-===
-
-.. kernel-doc:: include/qemu/qemu-plugin.h
-
-Usage
-=====
-
-Any QEMU binary with TCG support has plugins enabled by default.
-Earlier releases needed to be explicitly enabled with::
-
-  configure --enable-plugins
-
-Once built a program can be run with multiple plugins loaded each with
-their own arguments::
-
-  $QEMU $OTHER_QEMU_ARGS \
-      -plugin tests/plugin/libhowvec.so,inline=on,count=hint \
-      -plugin tests/plugin/libhotblocks.so
-
-Arguments are plugin specific and can be used to modify their
-behaviour. In this case the howvec plugin is being asked to use inline
-ops to count and break down the hint instructions by type.
-
-Plugin Life cycle
-=================
+Plugin life cycle
+~~~~~~~~~~~~~~~~~
 
 First the plugin is loaded and the public qemu_plugin_install function
 is called. The plugin will then register callbacks for various plugin
@@ -111,11 +95,26 @@ callback which can then ensure atomicity itself.
 Finally when QEMU exits all the registered *atexit* callbacks are
 invoked.
 
+Exposure of QEMU internals
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The plugin architecture actively avoids leaking implementation details
+about how QEMU's translation works to the plugins. While there are
+conceptions such as translation time and translation blocks the
+details are opaque to plugins. The plugin is able to query select
+details of instructions and system configuration only through the
+exported *qemu_plugin* functions.
+
+API
+~~~
+
+.. kernel-doc:: include/qemu/qemu-plugin.h
+
 Internals
-=========
+---------
 
 Locking
--------
+~~~~~~~
 
 We have to ensure we cannot deadlock, particularly under MTTCG. For
 this we acquire a lock when called from plugin code. We also keep the
@@ -142,7 +141,7 @@ requested. The plugin isn't completely uninstalled until the safe work
 has executed while all vCPUs are quiescent.
 
 Example Plugins
-===============
+---------------
 
 There are a number of plugins included with QEMU and you are
 encouraged to contribute your own plugins plugins upstream. There is a
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 64c9744795..7500f076c2 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -1,11 +1,10 @@
-===============
 Testing in QEMU
 ===============
 
 This document describes the testing infrastructure in QEMU.
 
 Testing with "make check"
-=========================
+-------------------------
 
 The "make check" testing family includes most of the C based tests in QEMU. For
 a quick help, run ``make check-help`` from the source tree.
@@ -24,7 +23,7 @@ expect the executables to exist and will fail with obscure messages if they
 cannot find them.
 
 Unit tests
-----------
+~~~~~~~~~~
 
 Unit tests, which can be invoked with ``make check-unit``, are simple C tests
 that typically link to individual QEMU object files and exercise them by
@@ -67,7 +66,7 @@ and copy the actual command line which executes the unit test, then run
 it from the command line.
 
 QTest
------
+~~~~~
 
 QTest is a device emulation testing framework.  It can be very useful to test
 device models; it could also control certain aspects of QEMU (such as virtual
@@ -81,7 +80,7 @@ QTest cases can be executed with
    make check-qtest
 
 QAPI schema tests
------------------
+~~~~~~~~~~~~~~~~~
 
 The QAPI schema tests validate the QAPI parser used by QMP, by feeding
 predefined input to the parser and comparing the result with the reference
@@ -108,33 +107,14 @@ parser (either fixing a bug or extending/modifying the syntax). To do this:
   ``qapi-schema += foo.json``
 
 check-block
------------
+~~~~~~~~~~~
 
 ``make check-block`` runs a subset of the block layer iotests (the tests that
 are in the "auto" group).
 See the "QEMU iotests" section below for more information.
 
-GCC gcov support
-----------------
-
-``gcov`` is a GCC tool to analyze the testing coverage by
-instrumenting the tested code. To use it, configure QEMU with
-``--enable-gcov`` option and build. Then run ``make check`` as usual.
-
-If you want to gather coverage information on a single test the ``make
-clean-gcda`` target can be used to delete any existing coverage
-information before running a single test.
-
-You can generate a HTML coverage report by executing ``make
-coverage-html`` which will create
-``meson-logs/coveragereport/index.html``.
-
-Further analysis can be conducted by running the ``gcov`` command
-directly on the various .gcda output files. Please read the ``gcov``
-documentation for more information.
-
 QEMU iotests
-============
+------------
 
 QEMU iotests, under the directory ``tests/qemu-iotests``, is the testing
 framework widely used to test block layer related features. It is higher level
@@ -171,7 +151,7 @@ More options are supported by the ``./check`` script, run ``./check -h`` for
 help.
 
 Writing a new test case
------------------------
+~~~~~~~~~~~~~~~~~~~~~~~
 
 Consider writing a tests case when you are making any changes to the block
 layer. An iotest case is usually the choice for that. There are already many
@@ -225,7 +205,8 @@ test failure.  If using such devices are explicitly desired, consider adding
 ``locking=off`` option to disable image locking.
 
 Debugging a test case
------------------------
+~~~~~~~~~~~~~~~~~~~~~
+
 The following options to the ``check`` script can be useful when debugging
 a failing test:
 
@@ -254,7 +235,7 @@ a failing test:
   ``$TEST_DIR/qemu-machine-<random_string>``.
 
 Test case groups
-----------------
+~~~~~~~~~~~~~~~~
 
 "Tests may belong to one or more test groups, which are defined in the form
 of a comment in the test source file. By convention, test groups are listed
@@ -304,10 +285,10 @@ Note that the following group names have a special meaning:
 .. _container-ref:
 
 Container based tests
-=====================
+---------------------
 
 Introduction
-------------
+~~~~~~~~~~~~
 
 The container testing framework in QEMU utilizes public images to
 build and test QEMU in predefined and widely accessible Linux
@@ -322,7 +303,7 @@ The container images are also used to augment the generation of tests
 for testing TCG. See :ref:`checktcg-ref` for more details.
 
 Docker Prerequisites
---------------------
+~~~~~~~~~~~~~~~~~~~~
 
 Install "docker" with the system package manager and start the Docker service
 on your development machine, then make sure you have the privilege to run
@@ -353,7 +334,7 @@ exploit the whole host with Docker bind mounting or other privileged
 operations.  So only do it on development machines.
 
 Podman Prerequisites
---------------------
+~~~~~~~~~~~~~~~~~~~~
 
 Install "podman" with the system package manager.
 
@@ -365,7 +346,7 @@ Install "podman" with the system package manager.
 The last command should print an empty table, to verify the system is ready.
 
 Quickstart
-----------
+~~~~~~~~~~
 
 From source tree, type ``make docker-help`` to see the help. Testing
 can be started without configuring or building QEMU (``configure`` and
@@ -381,7 +362,7 @@ is downloaded and initialized automatically), in which the ``test-build`` job
 is executed.
 
 Registry
---------
+~~~~~~~~
 
 The QEMU project has a container registry hosted by GitLab at
 ``registry.gitlab.com/qemu-project/qemu`` which will automatically be
@@ -395,7 +376,7 @@ locally by using the ``NOCACHE`` build option:
    make docker-image-debian10 NOCACHE=1
 
 Images
-------
+~~~~~~
 
 Along with many other images, the ``centos8`` image is defined in a Dockerfile
 in ``tests/docker/dockerfiles/``, called ``centos8.docker``. ``make docker-help``
@@ -410,7 +391,7 @@ mainly used to do necessary host side setup. One such setup is ``binfmt_misc``,
 for example, to make qemu-user powered cross build containers work.
 
 Tests
------
+~~~~~
 
 Different tests are added to cover various configurations to build and test
 QEMU.  Docker tests are the executables under ``tests/docker`` named
@@ -421,7 +402,7 @@ source and build it.
 The full list of tests is printed in the ``make docker-help`` help.
 
 Debugging a Docker test failure
--------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 When CI tasks, maintainers or yourself report a Docker test failure, follow the
 below steps to debug it:
@@ -438,7 +419,7 @@ below steps to debug it:
    the prompt for debug.
 
 Options
--------
+~~~~~~~
 
 Various options can be used to affect how Docker tests are done. The full
 list is in the ``make docker`` help text. The frequently used ones are:
@@ -452,7 +433,7 @@ list is in the ``make docker`` help text. The frequently used ones are:
   failure" section.
 
 Thread Sanitizer
-================
+----------------
 
 Thread Sanitizer (TSan) is a tool which can detect data races.  QEMU supports
 building and testing with this tool.
@@ -462,7 +443,7 @@ For more information on TSan:
 https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual
 
 Thread Sanitizer in Docker
----------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~
 TSan is currently supported in the ubuntu2004 docker.
 
 The test-tsan test will build using TSan and then run make check.
@@ -477,7 +458,7 @@ We recommend using DEBUG=1 to allow launching the test from inside the docker,
 and to allow review of the warnings generated by TSan.
 
 Building and Testing with TSan
-------------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 It is possible to build and test with TSan, with a few additional steps.
 These steps are normally done automatically in the docker.
@@ -516,7 +497,7 @@ This allows for running the test and then checking the warnings afterwards.
 If you want TSan to stop and exit with error on warnings, use exitcode=66.
 
 TSan Suppressions
------------------
+~~~~~~~~~~~~~~~~~
 Keep in mind that for any data race warning, although there might be a data race
 detected by TSan, there might be no actual bug here.  TSan provides several
 different mechanisms for suppressing warnings.  In general it is recommended
@@ -542,7 +523,7 @@ More information on the file format can be found here under "Blacklist Format":
 https://github.com/google/sanitizers/wiki/ThreadSanitizerFlags
 
 TSan Annotations
-----------------
+~~~~~~~~~~~~~~~~
 include/qemu/tsan.h defines annotations.  See this file for more descriptions
 of the annotations themselves.  Annotations can be used to suppress
 TSan warnings or give TSan more information so that it can detect proper
@@ -559,14 +540,14 @@ The full set of annotations can be found here:
 https://github.com/llvm/llvm-project/blob/master/compiler-rt/lib/tsan/rtl/tsan_interface_ann.cpp
 
 VM testing
-==========
+----------
 
 This test suite contains scripts that bootstrap various guest images that have
 necessary packages to build QEMU. The basic usage is documented in ``Makefile``
 help which is displayed with ``make vm-help``.
 
 Quickstart
-----------
+~~~~~~~~~~
 
 Run ``make vm-help`` to list available make targets. Invoke a specific make
 command to run build test in an image. For example, ``make vm-build-freebsd``
@@ -581,7 +562,7 @@ concerned about attackers taking control of the guest and potentially
 exploiting a QEMU security bug to compromise the host.
 
 QEMU binaries
--------------
+~~~~~~~~~~~~~
 
 By default, qemu-system-x86_64 is searched in $PATH to run the guest. If there
 isn't one, or if it is older than 2.10, the test won't work. In this case,
@@ -590,20 +571,20 @@ provide the QEMU binary in env var: ``QEMU=/path/to/qemu-2.10+``.
 Likewise the path to qemu-img can be set in QEMU_IMG environment variable.
 
 Make jobs
----------
+~~~~~~~~~
 
 The ``-j$X`` option in the make command line is not propagated into the VM,
 specify ``J=$X`` to control the make jobs in the guest.
 
 Debugging
----------
+~~~~~~~~~
 
 Add ``DEBUG=1`` and/or ``V=1`` to the make command to allow interactive
 debugging and verbose output. If this is not enough, see the next section.
 ``V=1`` will be propagated down into the make jobs in the guest.
 
 Manual invocation
------------------
+~~~~~~~~~~~~~~~~~
 
 Each guest script is an executable script with the same command line options.
 For example to work with the netbsd guest, use ``$QEMU_SRC/tests/vm/netbsd``:
@@ -627,7 +608,7 @@ For example to work with the netbsd guest, use ``$QEMU_SRC/tests/vm/netbsd``:
     $ ./netbsd --interactive --image /var/tmp/netbsd.img sh
 
 Adding new guests
------------------
+~~~~~~~~~~~~~~~~~
 
 Please look at existing guest scripts for how to add new guests.
 
@@ -660,7 +641,7 @@ the script's ``main()``.
   recommended.
 
 Image fuzzer testing
-====================
+--------------------
 
 An image fuzzer was added to exercise format drivers. Currently only qcow2 is
 supported. To start the fuzzer, run
@@ -673,7 +654,7 @@ Alternatively, some command different from "qemu-img info" can be tested, by
 changing the ``-c`` option.
 
 Acceptance tests using the Avocado Framework
-============================================
+--------------------------------------------
 
 The ``tests/acceptance`` directory hosts functional tests, also known
 as acceptance level tests.  They're usually higher level tests, and
@@ -712,7 +693,7 @@ Tests based on ``avocado_qemu.Test`` can easily:
    - http://avocado-framework.readthedocs.io/en/latest/api/utils/avocado.utils.html
 
 Running tests
--------------
+~~~~~~~~~~~~~
 
 You can run the acceptance tests simply by executing:
 
@@ -810,7 +791,7 @@ of Avocado or ``make check-acceptance``, and can also be queried using:
   tests/venv/bin/avocado list tests/acceptance
 
 Manual Installation
--------------------
+~~~~~~~~~~~~~~~~~~~
 
 To manually install Avocado and its dependencies, run:
 
@@ -823,7 +804,7 @@ Alternatively, follow the instructions on this link:
   https://avocado-framework.readthedocs.io/en/latest/guides/user/chapters/installing.html
 
 Overview
---------
+~~~~~~~~
 
 The ``tests/acceptance/avocado_qemu`` directory provides the
 ``avocado_qemu`` Python module, containing the ``avocado_qemu.Test``
@@ -859,7 +840,7 @@ in the current directory, tagged as "quick", run:
   avocado run -t quick .
 
 The ``avocado_qemu.Test`` base test class
------------------------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 The ``avocado_qemu.Test`` class has a number of characteristics that
 are worth being mentioned right away.
@@ -909,7 +890,7 @@ At test "tear down", ``avocado_qemu.Test`` handles all the QEMUMachines
 shutdown.
 
 The ``avocado_qemu.LinuxTest`` base test class
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 The ``avocado_qemu.LinuxTest`` is further specialization of the
 ``avocado_qemu.Test`` class, so it contains all the characteristics of
@@ -952,7 +933,7 @@ execution of a QEMU binary, giving its users:
    a more succinct and intuitive way
 
 QEMU binary selection
-~~~~~~~~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^^^^^^^^
 
 The QEMU binary used for the ``self.vm`` QEMUMachine instance will
 primarily depend on the value of the ``qemu_bin`` parameter.  If it's
@@ -973,20 +954,23 @@ The resulting ``qemu_bin`` value will be preserved in the
 ``avocado_qemu.Test`` as an attribute with the same name.
 
 Attribute reference
--------------------
+~~~~~~~~~~~~~~~~~~~
+
+Test
+^^^^
 
 Besides the attributes and methods that are part of the base
 ``avocado.Test`` class, the following attributes are available on any
 ``avocado_qemu.Test`` instance.
 
 vm
-~~
+''
 
 A QEMUMachine instance, initially configured according to the given
 ``qemu_bin`` parameter.
 
 arch
-~~~~
+''''
 
 The architecture can be used on different levels of the stack, e.g. by
 the framework or by the test itself.  At the framework level, it will
@@ -1003,7 +987,7 @@ name.  If one is not given explicitly, it will either be set to
 ``:avocado: tags=arch:VALUE`` tag, it will be set to ``VALUE``.
 
 cpu
-~~~
+'''
 
 The cpu model that will be set to all QEMUMachine instances created
 by the test.
@@ -1014,7 +998,7 @@ name. If one is not given explicitly, it will either be set to
 ``:avocado: tags=cpu:VALUE`` tag, it will be set to ``VALUE``.
 
 machine
-~~~~~~~
+'''''''
 
 The machine type that will be set to all QEMUMachine instances created
 by the test.
@@ -1025,20 +1009,20 @@ name.  If one is not given explicitly, it will either be set to
 ``:avocado: tags=machine:VALUE`` tag, it will be set to ``VALUE``.
 
 qemu_bin
-~~~~~~~~
+''''''''
 
 The preserved value of the ``qemu_bin`` parameter or the result of the
 dynamic probe for a QEMU binary in the current working directory or
 source tree.
 
 LinuxTest
-~~~~~~~~~
+^^^^^^^^^
 
 Besides the attributes present on the ``avocado_qemu.Test`` base
 class, the ``avocado_qemu.LinuxTest`` adds the following attributes:
 
 distro
-......
+''''''
 
 The name of the Linux distribution used as the guest image for the
 test.  The name should match the **Provider** column on the list
@@ -1047,7 +1031,7 @@ of images supported by the avocado.utils.vmimage library:
 https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images
 
 distro_version
-..............
+''''''''''''''
 
 The version of the Linux distribution as the guest image for the
 test.  The name should match the **Version** column on the list
@@ -1056,7 +1040,7 @@ of images supported by the avocado.utils.vmimage library:
 https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images
 
 distro_checksum
-...............
+'''''''''''''''
 
 The sha256 hash of the guest image file used for the test.
 
@@ -1065,7 +1049,7 @@ same name), no validation on the integrity of the image will be
 performed.
 
 Parameter reference
--------------------
+~~~~~~~~~~~~~~~~~~~
 
 To understand how Avocado parameters are accessed by tests, and how
 they can be passed to tests, please refer to::
@@ -1079,8 +1063,11 @@ like the following:
 
   PARAMS (key=qemu_bin, path=*, default=./qemu-system-x86_64) => './qemu-system-x86_64
 
+Test
+^^^^
+
 arch
-~~~~
+''''
 
 The architecture that will influence the selection of a QEMU binary
 (when one is not explicitly given).
@@ -1093,31 +1080,30 @@ This parameter has a direct relation with the ``arch`` attribute.  If
 not given, it will default to None.
 
 cpu
-~~~
+'''
 
 The cpu model that will be set to all QEMUMachine instances created
 by the test.
 
 machine
-~~~~~~~
+'''''''
 
 The machine type that will be set to all QEMUMachine instances created
 by the test.
 
-
 qemu_bin
-~~~~~~~~
+''''''''
 
 The exact QEMU binary to be used on QEMUMachine.
 
 LinuxTest
-~~~~~~~~~
+^^^^^^^^^
 
 Besides the parameters present on the ``avocado_qemu.Test`` base
 class, the ``avocado_qemu.LinuxTest`` adds the following parameters:
 
 distro
-......
+''''''
 
 The name of the Linux distribution used as the guest image for the
 test.  The name should match the **Provider** column on the list
@@ -1126,7 +1112,7 @@ of images supported by the avocado.utils.vmimage library:
 https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images
 
 distro_version
-..............
+''''''''''''''
 
 The version of the Linux distribution as the guest image for the
 test.  The name should match the **Version** column on the list
@@ -1135,7 +1121,7 @@ of images supported by the avocado.utils.vmimage library:
 https://avocado-framework.readthedocs.io/en/latest/guides/writer/libs/vmimage.html#supported-images
 
 distro_checksum
-...............
+'''''''''''''''
 
 The sha256 hash of the guest image file used for the test.
 
@@ -1143,7 +1129,8 @@ If this value is not set in the code or by this parameter no
 validation on the integrity of the image will be performed.
 
 Skipping tests
---------------
+~~~~~~~~~~~~~~
+
 The Avocado framework provides Python decorators which allow for easily skip
 tests running under certain conditions. For example, on the lack of a binary
 on the test system or when the running environment is a CI system. For further
@@ -1158,7 +1145,7 @@ environment variables became a kind of standard way to enable/disable tests.
 Here is a list of the most used variables:
 
 AVOCADO_ALLOW_LARGE_STORAGE
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
 Tests which are going to fetch or produce assets considered *large* are not
 going to run unless that ``AVOCADO_ALLOW_LARGE_STORAGE=1`` is exported on
 the environment.
@@ -1167,7 +1154,7 @@ The definition of *large* is a bit arbitrary here, but it usually means an
 asset which occupies at least 1GB of size on disk when uncompressed.
 
 AVOCADO_ALLOW_UNTRUSTED_CODE
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 There are tests which will boot a kernel image or firmware that can be
 considered not safe to run on the developer's workstation, thus they are
 skipped by default. The definition of *not safe* is also arbitrary but
@@ -1178,7 +1165,7 @@ You should export ``AVOCADO_ALLOW_UNTRUSTED_CODE=1`` on the environment in
 order to allow tests which make use of those kind of assets.
 
 AVOCADO_TIMEOUT_EXPECTED
-~~~~~~~~~~~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^^^^^^^^^^^
 The Avocado framework has a timeout mechanism which interrupts tests to avoid the
 test suite of getting stuck. The timeout value can be set via test parameter or
 property defined in the test class, for further details::
@@ -1192,7 +1179,7 @@ compiled with debug flags. Therefore, the ``AVOCADO_TIMEOUT_EXPECTED`` variable
 has been used to determine whether those tests should run or not.
 
 GITLAB_CI
-~~~~~~~~~
+^^^^^^^^^
 A number of tests are flagged to not run on the GitLab CI. Usually because
 they proved to the flaky or there are constraints on the CI environment which
 would make them fail. If you encounter a similar situation then use that
@@ -1205,7 +1192,7 @@ variable as shown on the code snippet below to skip the test:
       do_something()
 
 Uninstalling Avocado
---------------------
+~~~~~~~~~~~~~~~~~~~~
 
 If you've followed the manual installation instructions above, you can
 easily uninstall Avocado.  Start by listing the packages you have
@@ -1223,7 +1210,7 @@ Avocado is installed will be cleaned up as part of ``make check-clean``.
 .. _checktcg-ref:
 
 Testing with "make check-tcg"
-=============================
+-----------------------------
 
 The check-tcg tests are intended for simple smoke tests of both
 linux-user and softmmu TCG functionality. However to build test
@@ -1256,7 +1243,7 @@ itself.
 See :ref:`container-ref` for more details.
 
 Running subset of tests
------------------------
+~~~~~~~~~~~~~~~~~~~~~~~
 
 You can build the tests for one architecture::
 
@@ -1270,7 +1257,7 @@ Adding ``V=1`` to the invocation will show the details of how to
 invoke QEMU for the test which is useful for debugging tests.
 
 TCG test dependencies
----------------------
+~~~~~~~~~~~~~~~~~~~~~
 
 The TCG tests are deliberately very light on dependencies and are
 either totally bare with minimal gcc lib support (for softmmu tests)
@@ -1302,3 +1289,22 @@ exercise as many corner cases as possible. It is a useful test suite
 to run to exercise QEMU's linux-user code::
 
   https://linux-test-project.github.io/
+
+GCC gcov support
+----------------
+
+``gcov`` is a GCC tool to analyze the testing coverage by
+instrumenting the tested code. To use it, configure QEMU with
+``--enable-gcov`` option and build. Then run the tests as usual.
+
+If you want to gather coverage information on a single test the ``make
+clean-gcda`` target can be used to delete any existing coverage
+information before running a single test.
+
+You can generate a HTML coverage report by executing ``make
+coverage-html`` which will create
+``meson-logs/coveragereport/index.html``.
+
+Further analysis can be conducted by running the ``gcov`` command
+directly on the various .gcda output files. Please read the ``gcov``
+documentation for more information.