aboutsummaryrefslogtreecommitdiff
path: root/tests
AgeCommit message (Collapse)Author
2016-08-02qht: do not segfault when gathering stats from an uninitialized qhtEmilio G. Cota
So far, QHT functions assume that the passed qht has previously been initialized--otherwise they segfault. This patch makes an exception for qht_statistics_init, with the goal of simplifying calling code. For instance, qht_statistics_init is called from the 'info jit' dump, and given that under KVM the TB qht is never initialized, we get a segfault. Thus, instead of complicating the 'info jit' code with additional checks, let's allow passing an uninitialized qht to qht_statistics_init. While at it, add a test for this to test-qht. Before the patch (for $ qemu -enable-kvm [...]): (qemu) info jit [...] direct jump count 0 (0%) (2 jumps=0 0%) Program received signal SIGSEGV, Segmentation fault. After the patch the "TB hash buckets", "TB hash occupancy" and "TB hash avg chain" lines are omitted. (qemu) info jit [...] direct jump count 0 (0%) (2 jumps=0 0%) TB hash buckets 0/0 (-nan% head buckets used) TB hash occupancy nan% avg chain occ. Histogram: (null) TB hash avg chain nan buckets. Histogram: (null) [...] Reported by: Changlong Xie <xiecl.fnst@cn.fujitsu.com> Signed-off-by: Emilio G. Cota <cota@braap.org> Message-Id: <1469205390-14369-1-git-send-email-cota@braap.org> [Extract printing statistics to an entirely separate function. - Paolo] Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-07-29Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-2.7-20160729' ↵Peter Maydell
into staging ppc patch queue 2016-07-29 Here are the current pending ppc and spapr related patches for qemu-2.7. Given the freeze status, these are all bugfixes, with two exceptions: * There's some final rework of the vcpu hotplug model. Specifically we add spapr specific code on the generic basis Igor established to make cpu_index stable for pseries-2.7 and later machine types. - This allows us to remove the limitation that cpu cores had to be inserted in linear order, and removed in LIFO order. - This is worth merging this late in 2.7 because it will avoid considerable future grief with management layers needing to discover whether out-of-order hotplug is possible, amongst other things. - For now we do add a constraint that the initial cpu cannot be unplugged. * We add two extra testcases to make check, for postcopy and drive_del on ppc64. - Not strictly bugfixes, but safe, because they don't affect the actual code, and increase test coverage. # gpg: Signature made Fri 29 Jul 2016 05:50:02 BST # gpg: using RSA key 0x6C38CACA20D9B392 # gpg: Good signature from "David Gibson <david@gibson.dropbear.id.au>" # gpg: aka "David Gibson (Red Hat) <dgibson@redhat.com>" # gpg: aka "David Gibson (ozlabs.org) <dgibson@ozlabs.org>" # gpg: WARNING: This key is not certified with sufficiently trusted signatures! # gpg: It is not certain that the signature belongs to the owner. # Primary key fingerprint: 75F4 6586 AE61 A66C C44E 87DC 6C38 CACA 20D9 B392 * remotes/dgibson/tags/ppc-for-2.7-20160729: tests: add drive_del-test to ppc/ppc64 spapr: Prevent boot CPU core removal ppc: Fix fault PC reporting for lve*/stve* VMX instructions test: port postcopy test to ppc64 Revert "spapr: Ensure CPU cores are added contiguously and removed in LIFO order" spapr: init CPUState->cpu_index with index relative to core-id Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-07-29Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into stagingPeter Maydell
pc, pci, virtio: cleanups, fixes a bunch of bugfixes and a couple of cleanups making these easier and/or making debugging easier Signed-off-by: Michael S. Tsirkin <mst@redhat.com> # gpg: Signature made Fri 29 Jul 2016 04:11:01 BST # gpg: using RSA key 0x281F0DB8D28D5469 # gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>" # gpg: aka "Michael S. Tsirkin <mst@redhat.com>" # Primary key fingerprint: 0270 606B 6F3C DF3D 0B17 0970 C350 3912 AFBE 8E67 # Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA 8A0D 281F 0DB8 D28D 5469 * remotes/mst/tags/for_upstream: (41 commits) mptsas: Fix a migration compatible issue vhost: do not update last avail idx on get_vring_base() failure vhost: add vhost_net_set_backend() vhost-user: add error report in vhost_user_write() tests: fix vhost-user-test leak tests: plug some leaks in virtio-net-test vhost-user: wait until backend init is completed char: add and use tcp_chr_wait_connected char: add chr_wait_connected callback vhost: add assert() to check runtime behaviour vhost-net: vhost_migration_done is vhost-user specific Revert "vhost-net: do not crash if backend is not present" vhost-user: add get_vhost_net() assertions vhost-user: keep vhost_net after a disconnection vhost-user: check vhost_user_{read,write}() return value vhost-user: check qemu_chr_fe_set_msgfds() return value vhost-user: call set_msgfds unconditionally qemu-char: fix qemu_chr_fe_set_msgfds() crash when disconnected vhost: use error_report() instead of fprintf(stderr,...) vhost: add missing VHOST_OPS_DEBUG ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-07-29tests: add drive_del-test to ppc/ppc64Laurent Vivier
Signed-off-by: Laurent Vivier <lvivier@redhat.com> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2016-07-29test: port postcopy test to ppc64lvivier@redhat.com
As userfaultfd syscall is available on powerpc, migration postcopy can be used. This patch adds the support needed to test this on powerpc, instead of using a bootsector to run code to modify memory, we use a FORTH script in "boot-command" property. As spapr machine doesn't support "-prom-env" argument (the nvram is initialized by SLOF and not by QEMU), "boot-command" is provided to SLOF via a file mapped nvram (with "-drive file=...,if=pflash") Signed-off-by: Laurent Vivier <lvivier@redhat.com> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2016-07-29tests: fix vhost-user-test leakMarc-André Lureau
Spotted by valgrind. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2016-07-29tests: plug some leaks in virtio-net-testMarc-André Lureau
Found thanks to valgrind. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2016-07-26iotest: fix python based IO testsDaniel P. Berrange
The previous commit refactoring iotests.py: commit 66613974468fb6e1609fb3eabf55981b1ee436cf Author: Daniel P. Berrange <berrange@redhat.com> Date: Wed Jul 20 14:23:10 2016 +0100 scripts: refactor the VM class in iotests for reuse was not properly tested and included a number of broken bits. - The 'event_match' method was not moved into qemu.py - The 'self._args' list parameter in QEMUMachine needs to be copied otherwise modifications will affect the global 'qemu_opts' variable in iotests.py - The QEMUQtestMachine class methods had inverted parameter order for the super() calls - The QEMUQtestMachine class forgot to add '-machine accel=qtest' - The QEMUQtestMachine class constructor needs to set a default 'name' value before using it as it may be None - The QEMUQtestMachine class constructor needs to use named parameters when calling the super constructor as it is leaving out some positional parameters. - The 'qemu_prog' variable should be a string not a list in iotests.py - The VM classs constructor needs to use named parameters when calling the super constructor as it is leaving out some positional parameters. - The path to the socket-scm-helper needs to be passed into the QEMUMachine class Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-id: 1469549767-27249-1-git-send-email-berrange@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2016-07-25tests: use static qga config fileMarc-André Lureau
Do not create a leaking temporary file, but use a static file instead. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reported-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
2016-07-22tests: introduce a framework for testing migration performanceDaniel P. Berrange
This introduces a moderately general purpose framework for testing performance of migration. The initial guest workload is provided by the included 'stress' program, which is configured to spawn one thread per guest CPU and run a maximally memory intensive workload. It will loop over GB of memory, xor'ing each byte with data from a 4k array of random bytes. This ensures heavy read and write load across all of guest memory to stress the migration performance. While running the 'stress' program will record how long it takes to xor each GB of memory and print this data for later reporting. The test engine will spawn a pair of QEMU processes, either on the same host, or with the target on a remote host via ssh, using the host kernel and a custom initrd built with 'stress' as the /init binary. Kernel command line args are set to ensure a fast kernel boot time (< 1 second) between launching QEMU and the stress program starting execution. None the less, the test engine will initially wait N seconds for the guest workload to stablize, before starting the migration operation. When migration is running, the engine will use pause, post-copy, autoconverge, xbzrle compression and multithread compression features, as well as downtime & bandwidth tuning to encourage completion. If migration completes, the test engine will wait N seconds again for the guest workooad to stablize on the target host. If migration does not complete after a preset number of iterations, it will be aborted. While the QEMU process is running on the source host, the test engine will sample the host CPU usage of QEMU as a whole, and each vCPU thread. While migration is running, it will record all the stats reported by 'query-migration'. Finally, it will capture the output of the stress program running in the guest. All the data produced from a single test execution is recorded in a structured JSON file. A separate program is then able to create interactive charts using the "plotly" python + javascript libraries, showing the characteristics of the migration. The data output provides visualization of the effect on guest vCPU workloads from the migration process, the corresponding vCPU utilization on the host, and the overall CPU hit from QEMU on the host. This is correlated from statistics from the migration process, such as downtime, vCPU throttling and iteration number. While the tests can be run individually with arbitrary parameters, there is also a facility for producing batch reports for a number of pre-defined scenarios / comparisons, in order to be able to get standardized results across different hardware configurations (eg TCP vs RDMA, or comparing different VCPU counts / memory sizes, etc). To use this, first you must build the initrd image $ make tests/migration/initrd-stress.img To run a a one-shot test with all default parameters $ ./tests/migration/guestperf.py > result.json This has many command line args for varying its behaviour. For example, to increase the RAM size and CPU count and bind it to specific host NUMA nodes $ ./tests/migration/guestperf.py \ --mem 4 --cpus 2 \ --src-mem-bind 0 --src-cpu-bind 0,1 \ --dst-mem-bind 1 --dst-cpu-bind 2,3 \ > result.json Using mem + cpu binding is strongly recommended on NUMA machines, otherwise the guest performance results will vary wildly between runs of the test due to lucky/unlucky NUMA placement, making sensible data analysis impossible. To make it run across separate hosts: $ ./tests/migration/guestperf.py \ --dst-host somehostname > result.json To request that post-copy is enabled, with switchover after 5 iterations $ ./tests/migration/guestperf.py \ --post-copy --post-copy-iters 5 > result.json Once a result.json file is created, a graph of the data can be generated, showing guest workload performance per thread and the migration iteration points: $ ./tests/migration/guestperf-plot.py --output result.html \ --migration-iters --split-guest-cpu result.json To further include host vCPU utilization and overall QEMU utilization $ ./tests/migration/guestperf-plot.py --output result.html \ --migration-iters --split-guest-cpu \ --qemu-cpu --vcpu-cpu result.json NB, the 'guestperf-plot.py' command requires that you have the plotly python library installed. eg you must do $ pip install --user plotly Viewing the result.html file requires that you have the plotly.min.js file in the same directory as the HTML output. This js file is installed as part of the plotly python library, so can be found in $HOME/.local/lib/python2.7/site-packages/plotly/offline/plotly.min.js The guestperf-plot.py program can accept multiple json files to plot, enabling results from different configurations to be compared. Finally, to run the entire standardized set of comparisons $ ./tests/migration/guestperf-batch.py \ --dst-host somehost \ --mem 4 --cpus 2 \ --src-mem-bind 0 --src-cpu-bind 0,1 \ --dst-mem-bind 1 --dst-cpu-bind 2,3 --output tcp-somehost-4gb-2cpu will store JSON files from all scenarios in the directory named tcp-somehost-4gb-2cpu Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <1469020993-29426-7-git-send-email-berrange@redhat.com> Signed-off-by: Amit Shah <amit.shah@redhat.com>
2016-07-22scripts: refactor the VM class in iotests for reuseDaniel P. Berrange
The iotests module has a python class for controlling QEMU processes. Pull the generic functionality out of this file and create a scripts/qemu.py module containing a QEMUMachine class. Put the QTest integration support into a subclass QEMUQtestMachine. Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <1469020993-29426-4-git-send-email-berrange@redhat.com> Signed-off-by: Amit Shah <amit.shah@redhat.com>
2016-07-21Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into stagingPeter Maydell
pc, pci, virtio: new features, cleanups, fixes - interrupt remapping for intel iommus - a bunch of virtio cleanups - fixes all over the place Signed-off-by: Michael S. Tsirkin <mst@redhat.com> # gpg: Signature made Thu 21 Jul 2016 18:49:30 BST # gpg: using RSA key 0x281F0DB8D28D5469 # gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>" # gpg: aka "Michael S. Tsirkin <mst@redhat.com>" # Primary key fingerprint: 0270 606B 6F3C DF3D 0B17 0970 C350 3912 AFBE 8E67 # Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA 8A0D 281F 0DB8 D28D 5469 * remotes/mst/tags/for_upstream: (57 commits) intel_iommu: avoid unnamed fields virtio: Update migration docs virtio-gpu: Wrap in vmstate virtio-gpu: Use migrate_add_blocker for virgl migration blocking virtio-input: Wrap in vmstate 9pfs: Wrap in vmstate virtio-serial: Wrap in vmstate virtio-net: Wrap in vmstate virtio-balloon: Wrap in vmstate virtio-rng: Wrap in vmstate virtio-blk: Wrap in vmstate virtio-scsi: Wrap in vmstate virtio: Migration helper function and macro virtio-serial: Remove old migration version support virtio-net: Remove old migration version support virtio-scsi: Replace HandleOutput typedef Revert "mirror: Workaround for unexpected iohandler events during completion" virtio-scsi: Call virtio_add_queue_aio virtio-blk: Call virtio_add_queue_aio virtio: Introduce virtio_add_queue_aio ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-07-20Merge remote-tracking branch 'remotes/famz/tags/docker-pull-request' into ↵Peter Maydell
staging # gpg: Signature made Wed 20 Jul 2016 12:19:56 BST # gpg: using RSA key 0xCA35624C6A9171C6 # gpg: Good signature from "Fam Zheng <famz@redhat.com>" # gpg: WARNING: This key is not certified with a trusted signature! # gpg: There is no indication that the signature belongs to the owner. # Primary key fingerprint: 5003 7CB7 9706 0F76 F021 AD56 CA35 624C 6A91 71C6 * remotes/famz/tags/docker-pull-request: docker: pass EXECUTABLE to build script docker: Don't start a container that doesn't exist docker: Add "images" subcommand to docker.py docker: Fix exit code if $CMD failed docker: More sensible run script tests/docker/docker.py: add update operation tests/docker/dockerfiles: new debian-bootstrap.docker tests/docker/docker.py: check and run .pre script tests/docker/docker.py: support --include-executable tests/docker/docker.py: docker_dir outside build Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-07-20tests/prom-env-test: increase the test timeoutMarcel Apfelbaum
On a slower machine the test can take more than 30 seconds. Increase the timeout to 100 seconds. Signed-off-by: Marcel Apfelbaum <marcel@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Tested-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
2016-07-20Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2016-07-19' into ↵Peter Maydell
staging QAPI patches for 2016-07-19 # gpg: Signature made Tue 19 Jul 2016 19:35:27 BST # gpg: using RSA key 0x3870B400EB918653 # gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" # gpg: aka "Markus Armbruster <armbru@pond.sub.org>" # Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653 * remotes/armbru/tags/pull-qapi-2016-07-19: net: Use correct type for bool flag qapi: Change Netdev into a flat union block: Simplify drive-mirror block: Simplify block_set_io_throttle qapi: Implement boxed types for commands/events qapi: Plumb in 'boxed' to qapi generator lower levels qapi-event: Simplify visit of non-implicit data qapi: Drop useless gen_err_check() qapi: Add type.is_empty() helper qapi: Hide tag_name data member of variants qapi: Special case c_name() for empty type qapi: Require all branches of flat union enum to be covered net: use Netdev instead of NetClientOptions in client init qapi: change QmpInputVisitor to QSLIST qapi: change QmpOutputVisitor to QSLIST Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-07-20docker: pass EXECUTABLE to build scriptAlex Bennée
To build a docker image with which needs qemu linux-user emulation we need to pass --include-executable to the build script. Using the same mechanism as for other container controls we enable the option is EXECUTABLE is set on the make command line e.g: make docker-image-debian-bootstrap V=1 J=9 DEB_ARCH=armhf \ DEB_TYPE=stable EXECUTABLE=./arm-linux-user/qemu-arm Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-11-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-20docker: Don't start a container that doesn't existFam Zheng
Image building targets are dependencies of test running targets, so when a docker image doesn't exist, it means it's skipped (due to dependency checks in pre script). Therefore, skip the test too. Signed-off-by: Fam Zheng <famz@redhat.com> Message-id: 1468934445-32183-10-git-send-email-famz@redhat.com
2016-07-20docker: Add "images" subcommand to docker.pyFam Zheng
This is a wrapper for the 'docker images' command. Signed-off-by: Fam Zheng <famz@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-9-git-send-email-famz@redhat.com
2016-07-20docker: Fix exit code if $CMD failedFam Zheng
Signed-off-by: Fam Zheng <famz@redhat.com> Message-id: 1468934445-32183-8-git-send-email-famz@redhat.com
2016-07-20docker: More sensible run scriptFam Zheng
It is very easy to figure out current directory and bash option from the execution, so do less in the Makefile invocation command line, and figure both options in the script. This makes the next patch easier. Signed-off-by: Fam Zheng <famz@redhat.com> Message-id: 1468934445-32183-7-git-send-email-famz@redhat.com
2016-07-20tests/docker/docker.py: add update operationAlex Bennée
This adds a new operation to the docker script to allow updating of binaries in an existing container. This is because it would be inefficient to re-build the whole container just for an update to the QEMU binary. To update the executable run: ./tests/docker/docker.py update \ debian:armhf ./arm-linux-user/qemu-arm Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-6-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-20tests/docker/dockerfiles: new debian-bootstrap.dockerAlex Bennée
Together with the debian-bootstrap.pre script can now build an arbitrary architecture of Debian using debootstrap. This allows debootstrap to set up its first stage before the container is built. To build a container you need a command line like: DEB_ARCH=armhf DEB_TYPE=testing \ ./tests/docker/docker.py build \ --include-executable=arm-linux-user/qemu-arm debian:armhf \ ./tests/docker/dockerfiles/debian-bootstrap.docker Although a number of non-debian systems package the debootstrap script it is fairly portable in itself. Assuming we have some sort of fakeroot implementation we can just clone the upstream repository and use the script from there. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-5-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-20tests/docker/docker.py: check and run .pre scriptAlex Bennée
The docker script will now search for an associated $dockerfile.pre script which gets run in the same build context as the dockerfile will be. This is to support pre-seeding the build context before running the docker build. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-4-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-20tests/docker/docker.py: support --include-executableAlex Bennée
When passed the path to a binary we copy it and any linked libraries (if it is dynamically linked) into the docker build context. These can then be included by a dockerfile with the line: # Copy all of context into container ADD . / This is mainly intended for setting up foreign architecture docker images which use qemu-$arch to do cross-architecture linux-user execution. It also relies on the host and guest file-system following reasonable multi-arch layouts so the copied libraries don't clash with the guest ones. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-3-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-20tests/docker/docker.py: docker_dir outside buildAlex Bennée
Instead of letting the build_image create the temporary working dir we move the creation to the build command. This is preparation for the later patches where additional files can be added to the build context before the build step is run. We also ensure we remove the build context after we are done (mkdtemp doesn't do this automatically for you). Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Fam Zheng <famz@redhat.com> Message-id: 1468934445-32183-2-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-19qapi: Implement boxed types for commands/eventsEric Blake
Turn on the ability to pass command and event arguments in a single boxed parameter, which must name a non-empty type (although the type can be a struct with all optional members). For structs, it makes it possible to pass a single qapi type instead of a breakout of all struct members (useful if the arguments are already in a struct or if the number of members is large); for other complex types, it is now possible to use a union or alternate as the data for a command or event. The empty type may be technically feasible if needed down the road, but it's easier to forbid it now and relax things to allow it later, than it is to allow it now and have to special case how the generated 'q_empty' type is handled (see commit 7ce106a9 for reasons why nothing is generated for the empty type). An alternate type is never considered empty, but now that a boxed type can be either an object or an alternate, we have to provide a trivial QAPISchemaAlternateType.is_empty(). The new call to arg_type.is_empty() during QAPISchemaCommand.check() requires that we first check the type in question; but there is no chance of introducing a cycle since objects do not refer back to commands. We still have a split in syntax checking between ad-hoc parsing up front (merely validates that 'boxed' has a sane value) and during .check() methods (if 'boxed' is set, then 'data' must name a non-empty user-defined type). Generated code is unchanged, as long as no client uses the new feature. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1468468228-27827-10-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Test files renamed to *-boxed-*] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-19qapi: Plumb in 'boxed' to qapi generator lower levelsEric Blake
The next patch will add support for passing a qapi union type as the 'data' of a command. But to do that, the user function for implementing the command, as called by the generated marshal command, must take the corresponding C struct as a single boxed pointer, rather than a breakdown into one parameter per member. Even without a union, being able to use a C struct rather than a list of parameters can make it much easier to handle coding with QAPI. This patch adds the internal plumbing of a 'boxed' flag associated with each command and event. In several cases, this means adding indentation, with one new dead branch and the remaining branch being the original code more deeply nested; this was done so that the new implementation in the next patch is easier to review without also being mixed with indentation changes. For this patch, no behavior or generated output changes, other than the testsuite outputting the value of the new flag (always False for now). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1468468228-27827-9-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Identifier box renamed to boxed in two places] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-19qapi: Hide tag_name data member of variantsEric Blake
Clean up the only remaining external use of the tag_name field of QAPISchemaObjectTypeVariants, by explicitly listing the generated 'type' tag for all variants in the testsuite (you can still tell simple unions by the -wrapper types). Then we can mark the tag_name field as private by adding a leading underscore to prevent any further use. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1468468228-27827-5-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-19qapi: Require all branches of flat union enum to be coveredEric Blake
We were previously enforcing that all flat union branches were found in the corresponding enum, but not that all enum values were covered by branches. The resulting generated code would abort() if the user passes the uncovered enum value. We don't automatically treat non-present branches in a flat union as empty types, for symmetry with simple unions (there, the enum type is generated from the list of all branches, so there is no way to omit a branch but still have it be part of the union). A later patch will add shorthand so that branches that are empty in flat unions can be declared as 'branch':{} instead of 'branch':'Empty', to avoid the need for an otherwise useless explicit empty type. [Such shorthand for simple unions is a bit harder to justify, since we would still have to generate a wrapper type that parses 'data':{}, rather than truly being an empty branch with no additional siblings to the 'type' member.] Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1468468228-27827-3-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-18block: ignore flush requests when storage is cleanEvgeny Yakovlev
Some guests (win2008 server for example) do a lot of unnecessary flushing when underlying media has not changed. This adds additional overhead on host when calling fsync/fdatasync. This change introduces a write generation scheme in BlockDriverState. Current write generation is checked against last flushed generation to avoid unnessesary flushes. The problem with excessive flushing was found by a performance test which does parallel directory tree creation (from 2 processes). Results improved from 0.424 loops/sec to 0.432 loops/sec. Each loop creates 10^3 directories with 10 files in each. This affected some blkdebug testcases that were expecting error logs from failure-injected flushes which are now skipped entirely (tests 026 071 089). This also affects the performance of block jobs and thus BLOCK_JOB_READY events for driver-mirror and active block-commit commands now arrives faster, before QMP send successfully returns to caller (tests 141 144). Signed-off-by: Evgeny Yakovlev <eyakovlev@virtuozzo.com> Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 1468870792-7411-5-git-send-email-den@openvz.org CC: Kevin Wolf <kwolf@redhat.com> CC: Max Reitz <mreitz@redhat.com> CC: Stefan Hajnoczi <stefanha@redhat.com> CC: Fam Zheng <famz@redhat.com> CC: John Snow <jsnow@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2016-07-18tests: in IDE and AHCI tests perform DMA write before flushingEvgeny Yakovlev
Due to changes in flush behaviour clean disks stopped generating flush_to_disk events and IDE and AHCI tests that test flush commands started to fail. This change adds additional DMA writes to affected tests before sending flush commands so that bdrv_flush actually generates flush_to_disk event. Signed-off-by: Evgeny Yakovlev <eyakovlev@virtuozzo.com> Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 1468870792-7411-4-git-send-email-den@openvz.org CC: Kevin Wolf <kwolf@redhat.com> CC: Max Reitz <mreitz@redhat.com> CC: Stefan Hajnoczi <stefanha@redhat.com> CC: Fam Zheng <famz@redhat.com> CC: John Snow <jsnow@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2016-07-13iotests: Make 157 actually format-agnosticMax Reitz
iotest 157 pretends not to care about the image format used, but in fact it does due to the format name not being filtered in its output. This patch adds filtering and changes the reference output accordingly. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20160711132246.3152-1-mreitz@redhat.com Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2016-07-13qemu-iotests: Test naming of throttling groupsAlberto Garcia
Throttling groups are named using the 'group' parameter of the block_set_io_throttle command and the throttling.group command-line option. If that parameter is unspecified the groups get the name of the block device. This patch adds a new test to check the naming of throttling groups. Signed-off-by: Alberto Garcia <berto@igalia.com> Message-id: d87d02823a6b91609509d8bb18e2f5dbd9a6102c.1467986342.git.berto@igalia.com Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2016-07-13qemu-iotests: Test setting WCE with qdevKevin Wolf
Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com>
2016-07-13coroutine: move entry argument to qemu_coroutine_createPaolo Bonzini
In practice the entry argument is always known at creation time, and it is confusing that sometimes qemu_coroutine_enter is used with a non-NULL argument to re-enter a coroutine (this happens in block/sheepdog.c and tests/test-coroutine.c). So pass the opaque value at creation time, for consistency with e.g. aio_bh_new. Mostly done with the following semantic patch: @ entry1 @ expression entry, arg, co; @@ - co = qemu_coroutine_create(entry); + co = qemu_coroutine_create(entry, arg); ... - qemu_coroutine_enter(co, arg); + qemu_coroutine_enter(co); @ entry2 @ expression entry, arg; identifier co; @@ - Coroutine *co = qemu_coroutine_create(entry); + Coroutine *co = qemu_coroutine_create(entry, arg); ... - qemu_coroutine_enter(co, arg); + qemu_coroutine_enter(co); @ entry3 @ expression entry, arg; @@ - qemu_coroutine_enter(qemu_coroutine_create(entry), arg); + qemu_coroutine_enter(qemu_coroutine_create(entry, arg)); @ reentry @ expression co; @@ - qemu_coroutine_enter(co, NULL); + qemu_coroutine_enter(co); except for the aforementioned few places where the semantic patch stumbled (as expected) and for test_co_queue, which would otherwise produce an uninitialized variable warning. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2016-07-13test-coroutine: prepare for the next patchPaolo Bonzini
The next patch moves the coroutine argument from first-enter to creation time. In this case, coroutine has not been initialized yet when the coroutine is created, so change to a pointer. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2016-07-13blockjob: Add 'job_id' parameter to block_job_create()Alberto Garcia
When a new job is created, the job ID is taken from the device name of the BDS. This patch adds a new 'job_id' parameter to let the caller provide one instead. This patch also verifies that the ID is always unique and well-formed. This causes problems in a couple of places where no ID is being set, because the BDS does not have a device name. In the case of test_block_job_start() (from test-blockjob-txn.c) we can simply use this new 'job_id' parameter to set the missing ID. In the case of img_commit() (from qemu-img.c) we still don't have the API to make commit_active_start() set the job ID, so we solve it by setting a default value. We'll get rid of this as soon as we extend the API. Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2016-07-12Clean up ill-advised or unusual header guardsMarkus Armbruster
Cleaned up with scripts/clean-header-guards.pl. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Richard Henderson <rth@twiddle.net>
2016-07-12Use #include "..." for our own headers, <...> for othersMarkus Armbruster
Tracked down with an ugly, brittle and probably buggy Perl script. Also move includes converted to <...> up so they get included before ours where that's obviously okay. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Tested-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard Henderson <rth@twiddle.net>
2016-07-06qapi: Add new clone visitorEric Blake
We have a couple places in the code base that want to deep-clone one QAPI object into another, and they were resorting to serializing the struct out to QObject then reparsing it. A much more efficient version can be done by adding a new clone visitor. Since cloning is still relatively uncommon, expose the use of the new visitor via a QAPI_CLONE() macro that takes care of type-punning the underlying function pointer, rather than generating lots of unused functions for types that won't be cloned. And yes, we're relying on the compiler treating all pointers equally, even though a strict C program cannot portably do so - but we're not the first one in the qemu code base to expect it to work (hello, glib!). The choice of adding a fourth visitor type deserves some explanation. On the surface, the clone visitor is mostly an input visitor (it takes arbitrary input - in this case, another QAPI object - and creates a new QAPI object during the course of the visit). But ever since commit da72ab0 consolidated enum visits based on the visitor type, using VISITOR_INPUT would cause us to run visit_type_str(), even though for cloning there is nothing to do (we just copy the enum value across, without regards to its mapping to strings). Also, since our input happens to be a QAPI object, we can also satisfy the internal checks for VISITOR_OUTPUT. So in the end, I settled with a new VISITOR_CLONE, and chose its value such that many internal checks can use 'v->type & mask', sticking to 'v->type == value' where the difference matters. Note that we can only clone objects (including alternates) and lists, not built-ins or enums. The visitor core hides integer width from the actual visitor (since commit 04e070d), and as long as that's the case, we can't clone top-level integers. Then again, those can always be cloned by direct copy, since they are not objects with deep pointers, so it's no real loss. And restricting cloning to just objects and lists is cleaner than restricting it to non-integers. As such, I documented that the clone visitor is for direct use only by code internal to QAPI, and should not be used on incomplete objects (other than a hack to work around the fact that we allow NULL in place of "" in visit_type_str() in other output visitors). Note that as written, the clone visitor will never fail on a complete object. Scalars (including enums) not at the root of the clone copy just fine with no additional effort while visiting the scalar, by virtue of a g_memdup() each time we push another struct onto the stack. Cloning a string requires deduplication of a pointer, which means it can also provide the guarantee of an input visitor of never producing NULL even when still accepting NULL in place of "" the way the QMP output visitor does. Cloning an 'any' type could be possible by incrementing the QObject refcnt, but it's not obvious whether that is better than implementing a QObject deep clone. So for now, we document it as unsupported, and intentionally omit the .type_any() callback to let a developer know their usage needs implementation. Add testsuite coverage for several different clone situations, to ensure that the code is working. I also tested that valgrind was happy with the test. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-14-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06qapi: Add new visit_complete() functionEric Blake
Making each output visitor provide its own output collection function was the only remaining reason for exposing visitor sub-types to the rest of the code base. Add a polymorphic visit_complete() function which is a no-op for input visitors, and which populates an opaque pointer for output visitors. For maximum type-safety, also add a parameter to the output visitor constructors with a type-correct version of the output pointer, and assert that the two uses match. This approach was considered superior to either passing the output parameter only during construction (action at a distance during visit_free() feels awkward) or only during visit_complete() (defeating type safety makes it easier to use incorrectly). Most callers were function-local, and therefore a mechanical conversion; the testsuite was a bit trickier, but the previous cleanup patch minimized the churn here. The visit_complete() function may be called at most once; doing so lets us use transfer semantics rather than duplication or ref-count semantics to get the just-built output back to the caller, even though it means our behavior is not idempotent. Generated code is simplified as follows for events: |@@ -26,7 +26,7 @@ void qapi_event_send_acpi_device_ost(ACP | QDict *qmp; | Error *err = NULL; | QMPEventFuncEmit emit; |- QmpOutputVisitor *qov; |+ QObject *obj; | Visitor *v; | q_obj_ACPI_DEVICE_OST_arg param = { | info |@@ -39,8 +39,7 @@ void qapi_event_send_acpi_device_ost(ACP | | qmp = qmp_event_build_dict("ACPI_DEVICE_OST"); | |- qov = qmp_output_visitor_new(); |- v = qmp_output_get_visitor(qov); |+ v = qmp_output_visitor_new(&obj); | | visit_start_struct(v, "ACPI_DEVICE_OST", NULL, 0, &err); | if (err) { |@@ -55,7 +54,8 @@ void qapi_event_send_acpi_device_ost(ACP | goto out; | } | |- qdict_put_obj(qmp, "data", qmp_output_get_qobject(qov)); |+ visit_complete(v, &obj); |+ qdict_put_obj(qmp, "data", obj); | emit(QAPI_EVENT_ACPI_DEVICE_OST, qmp, &err); and for commands: | { | Error *err = NULL; |- QmpOutputVisitor *qov = qmp_output_visitor_new(); | Visitor *v; | |- v = qmp_output_get_visitor(qov); |+ v = qmp_output_visitor_new(ret_out); | visit_type_AddfdInfo(v, "unused", &ret_in, &err); |- if (err) { |- goto out; |+ if (!err) { |+ visit_complete(v, ret_out); | } |- *ret_out = qmp_output_get_qobject(qov); |- |-out: | error_propagate(errp, err); Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-13-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06tests: Factor out common code in qapi output testsEric Blake
Create a new visitor_get() function to capture common actions taken in collecting output from an output visitor, to make it easier to refactor the output visitors in a later patch. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-12-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06tests: Clean up test-string-output-visitorEric Blake
Use &error_abort and error_free_or_abort() in more places, use the generated qapi_free_intList() instead of open-coding it, reduce the scope of some variables, avoid code duplication during test setup with visitor_output_setup_internal(), and copy the visitor_reset() concept from the qmp-output test to the string-output test. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-11-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06qmp-output-visitor: Favor new visit_free() functionEric Blake
Now that we have a polymorphic visit_free(), we no longer need qmp_output_visitor_cleanup(); however, we still need to expose the subtype for qmp_output_get_qobject(). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-10-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06string-output-visitor: Favor new visit_free() functionEric Blake
Now that we have a polymorphic visit_free(), we no longer need string_output_visitor_cleanup(); however, we still need to expose the subtype for string_output_get_string(). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-9-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06qmp-input-visitor: Favor new visit_free() functionEric Blake
Now that we have a polymorphic visit_free(), we no longer need qmp_input_visitor_cleanup(); which in turn means we no longer need to return a subtype from qmp_input_visitor_new() nor a public upcast function. Generated code changes to qmp-marshal.c look like: |@@ -52,11 +52,10 @@ void qmp_marshal_add_fd(QDict *args, QOb | { | Error *err = NULL; | AddfdInfo *retval; |- QmpInputVisitor *qiv = qmp_input_visitor_new(QOBJECT(args), true); | Visitor *v; | q_obj_add_fd_arg arg = {0}; | |- v = qmp_input_get_visitor(qiv); |+ v = qmp_input_visitor_new(QOBJECT(args), true); | visit_start_struct(v, NULL, NULL, 0, &err); | if (err) { | goto out; Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-8-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06string-input-visitor: Favor new visit_free() functionEric Blake
Now that we have a polymorphic visit_free(), we no longer need string_input_visitor_cleanup(); which in turn means we no longer need to return a subtype from string_input_visitor_new() nor a public upcast function. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-7-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06opts-visitor: Favor new visit_free() functionEric Blake
Now that we have a polymorphic visit_free(), we no longer need opts_visitor_cleanup(); which in turn means we no longer need to return a subtype from opts_visitor_new() nor a public upcast function. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-6-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06qapi: Add new visit_free() functionEric Blake
Making each visitor provide its own (awkwardly-named) FOO_cleanup() is unusual, when we can instead have a polymorphic visit_free() interface. Over the next few patches, we can use the polymorphic functions to eliminate the need for a FOO_get_visitor() function for accessing specific visitor functionality, once everything can be accessed directly through the Visitor* interfaces. The dealloc visitor is the first one converted to completely use the new entry point, since qapi_dealloc_visitor_cleanup() was the only reason that qapi_dealloc_get_visitor() existed, and only generated and testsuite code was even using it. With the new visit_free() entry point in place, we no longer need to expose the QapiDeallocVisitor subtype through qapi_dealloc_visitor_new(), and can get by with less generated code, with diffs that look like: | void qapi_free_ACPIOSTInfo(ACPIOSTInfo *obj) | { |- QapiDeallocVisitor *qdv; | Visitor *v; | | if (!obj) { | return; | } | |- qdv = qapi_dealloc_visitor_new(); |- v = qapi_dealloc_get_visitor(qdv); |+ v = qapi_dealloc_visitor_new(); | visit_type_ACPIOSTInfo(v, NULL, &obj, NULL); |- qapi_dealloc_visitor_cleanup(qdv); |+ visit_free(v); |} Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-5-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06qapi: Add parameter to visit_end_*Eric Blake
Rather than making the dealloc visitor track of stack of pointers remembered during visit_start_* in order to free them during visit_end_*, it's a lot easier to just make all callers pass the same pointer to visit_end_*. The generated code has access to the same pointer, while all other users are doing virtual walks and can pass NULL. The dealloc visitor is then greatly simplified. All three visit_end_*() functions intentionally take a void**, even though the visit_start_*() functions differ between void**, GenericList**, and GenericAlternate**. This is done for several reasons: when doing a virtual walk, passing NULL doesn't care what the type is, but when doing a generated walk, we already have to cast the caller's specific FOO* to call visit_start, while using void** lets us use visit_end without a cast. Also, an upcoming patch will add a clone visitor that wants to use the same implementation for all three visit_end callbacks, which is made easier if all three share the same signature. For visitors with already track per-object state (the QMP visitors via a stack, and the string visitors which do not allow nesting), add an assertion that the caller is indeed passing the same pointer to paired calls. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-4-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
generated by cgit v1.2.3 (git 2.26.2) at 2024-12-19 22:00:25 +0000