aboutsummaryrefslogtreecommitdiff
path: root/tests
AgeCommit message (Collapse)Author
2016-07-20Merge remote-tracking branch 'remotes/famz/tags/docker-pull-request' into ↵Peter Maydell
staging # gpg: Signature made Wed 20 Jul 2016 12:19:56 BST # gpg: using RSA key 0xCA35624C6A9171C6 # gpg: Good signature from "Fam Zheng <famz@redhat.com>" # gpg: WARNING: This key is not certified with a trusted signature! # gpg: There is no indication that the signature belongs to the owner. # Primary key fingerprint: 5003 7CB7 9706 0F76 F021 AD56 CA35 624C 6A91 71C6 * remotes/famz/tags/docker-pull-request: docker: pass EXECUTABLE to build script docker: Don't start a container that doesn't exist docker: Add "images" subcommand to docker.py docker: Fix exit code if $CMD failed docker: More sensible run script tests/docker/docker.py: add update operation tests/docker/dockerfiles: new debian-bootstrap.docker tests/docker/docker.py: check and run .pre script tests/docker/docker.py: support --include-executable tests/docker/docker.py: docker_dir outside build Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-07-20Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2016-07-19' into ↵Peter Maydell
staging QAPI patches for 2016-07-19 # gpg: Signature made Tue 19 Jul 2016 19:35:27 BST # gpg: using RSA key 0x3870B400EB918653 # gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" # gpg: aka "Markus Armbruster <armbru@pond.sub.org>" # Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653 * remotes/armbru/tags/pull-qapi-2016-07-19: net: Use correct type for bool flag qapi: Change Netdev into a flat union block: Simplify drive-mirror block: Simplify block_set_io_throttle qapi: Implement boxed types for commands/events qapi: Plumb in 'boxed' to qapi generator lower levels qapi-event: Simplify visit of non-implicit data qapi: Drop useless gen_err_check() qapi: Add type.is_empty() helper qapi: Hide tag_name data member of variants qapi: Special case c_name() for empty type qapi: Require all branches of flat union enum to be covered net: use Netdev instead of NetClientOptions in client init qapi: change QmpInputVisitor to QSLIST qapi: change QmpOutputVisitor to QSLIST Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-07-20docker: pass EXECUTABLE to build scriptAlex Bennée
To build a docker image with which needs qemu linux-user emulation we need to pass --include-executable to the build script. Using the same mechanism as for other container controls we enable the option is EXECUTABLE is set on the make command line e.g: make docker-image-debian-bootstrap V=1 J=9 DEB_ARCH=armhf \ DEB_TYPE=stable EXECUTABLE=./arm-linux-user/qemu-arm Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-11-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-20docker: Don't start a container that doesn't existFam Zheng
Image building targets are dependencies of test running targets, so when a docker image doesn't exist, it means it's skipped (due to dependency checks in pre script). Therefore, skip the test too. Signed-off-by: Fam Zheng <famz@redhat.com> Message-id: 1468934445-32183-10-git-send-email-famz@redhat.com
2016-07-20docker: Add "images" subcommand to docker.pyFam Zheng
This is a wrapper for the 'docker images' command. Signed-off-by: Fam Zheng <famz@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-9-git-send-email-famz@redhat.com
2016-07-20docker: Fix exit code if $CMD failedFam Zheng
Signed-off-by: Fam Zheng <famz@redhat.com> Message-id: 1468934445-32183-8-git-send-email-famz@redhat.com
2016-07-20docker: More sensible run scriptFam Zheng
It is very easy to figure out current directory and bash option from the execution, so do less in the Makefile invocation command line, and figure both options in the script. This makes the next patch easier. Signed-off-by: Fam Zheng <famz@redhat.com> Message-id: 1468934445-32183-7-git-send-email-famz@redhat.com
2016-07-20tests/docker/docker.py: add update operationAlex Bennée
This adds a new operation to the docker script to allow updating of binaries in an existing container. This is because it would be inefficient to re-build the whole container just for an update to the QEMU binary. To update the executable run: ./tests/docker/docker.py update \ debian:armhf ./arm-linux-user/qemu-arm Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-6-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-20tests/docker/dockerfiles: new debian-bootstrap.dockerAlex Bennée
Together with the debian-bootstrap.pre script can now build an arbitrary architecture of Debian using debootstrap. This allows debootstrap to set up its first stage before the container is built. To build a container you need a command line like: DEB_ARCH=armhf DEB_TYPE=testing \ ./tests/docker/docker.py build \ --include-executable=arm-linux-user/qemu-arm debian:armhf \ ./tests/docker/dockerfiles/debian-bootstrap.docker Although a number of non-debian systems package the debootstrap script it is fairly portable in itself. Assuming we have some sort of fakeroot implementation we can just clone the upstream repository and use the script from there. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-5-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-20tests/docker/docker.py: check and run .pre scriptAlex Bennée
The docker script will now search for an associated $dockerfile.pre script which gets run in the same build context as the dockerfile will be. This is to support pre-seeding the build context before running the docker build. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-4-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-20tests/docker/docker.py: support --include-executableAlex Bennée
When passed the path to a binary we copy it and any linked libraries (if it is dynamically linked) into the docker build context. These can then be included by a dockerfile with the line: # Copy all of context into container ADD . / This is mainly intended for setting up foreign architecture docker images which use qemu-$arch to do cross-architecture linux-user execution. It also relies on the host and guest file-system following reasonable multi-arch layouts so the copied libraries don't clash with the guest ones. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-id: 1468934445-32183-3-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-20tests/docker/docker.py: docker_dir outside buildAlex Bennée
Instead of letting the build_image create the temporary working dir we move the creation to the build command. This is preparation for the later patches where additional files can be added to the build context before the build step is run. We also ensure we remove the build context after we are done (mkdtemp doesn't do this automatically for you). Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Fam Zheng <famz@redhat.com> Message-id: 1468934445-32183-2-git-send-email-famz@redhat.com Signed-off-by: Fam Zheng <famz@redhat.com>
2016-07-19qapi: Implement boxed types for commands/eventsEric Blake
Turn on the ability to pass command and event arguments in a single boxed parameter, which must name a non-empty type (although the type can be a struct with all optional members). For structs, it makes it possible to pass a single qapi type instead of a breakout of all struct members (useful if the arguments are already in a struct or if the number of members is large); for other complex types, it is now possible to use a union or alternate as the data for a command or event. The empty type may be technically feasible if needed down the road, but it's easier to forbid it now and relax things to allow it later, than it is to allow it now and have to special case how the generated 'q_empty' type is handled (see commit 7ce106a9 for reasons why nothing is generated for the empty type). An alternate type is never considered empty, but now that a boxed type can be either an object or an alternate, we have to provide a trivial QAPISchemaAlternateType.is_empty(). The new call to arg_type.is_empty() during QAPISchemaCommand.check() requires that we first check the type in question; but there is no chance of introducing a cycle since objects do not refer back to commands. We still have a split in syntax checking between ad-hoc parsing up front (merely validates that 'boxed' has a sane value) and during .check() methods (if 'boxed' is set, then 'data' must name a non-empty user-defined type). Generated code is unchanged, as long as no client uses the new feature. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1468468228-27827-10-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Test files renamed to *-boxed-*] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-19qapi: Plumb in 'boxed' to qapi generator lower levelsEric Blake
The next patch will add support for passing a qapi union type as the 'data' of a command. But to do that, the user function for implementing the command, as called by the generated marshal command, must take the corresponding C struct as a single boxed pointer, rather than a breakdown into one parameter per member. Even without a union, being able to use a C struct rather than a list of parameters can make it much easier to handle coding with QAPI. This patch adds the internal plumbing of a 'boxed' flag associated with each command and event. In several cases, this means adding indentation, with one new dead branch and the remaining branch being the original code more deeply nested; this was done so that the new implementation in the next patch is easier to review without also being mixed with indentation changes. For this patch, no behavior or generated output changes, other than the testsuite outputting the value of the new flag (always False for now). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1468468228-27827-9-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Identifier box renamed to boxed in two places] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-19qapi: Hide tag_name data member of variantsEric Blake
Clean up the only remaining external use of the tag_name field of QAPISchemaObjectTypeVariants, by explicitly listing the generated 'type' tag for all variants in the testsuite (you can still tell simple unions by the -wrapper types). Then we can mark the tag_name field as private by adding a leading underscore to prevent any further use. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1468468228-27827-5-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-19qapi: Require all branches of flat union enum to be coveredEric Blake
We were previously enforcing that all flat union branches were found in the corresponding enum, but not that all enum values were covered by branches. The resulting generated code would abort() if the user passes the uncovered enum value. We don't automatically treat non-present branches in a flat union as empty types, for symmetry with simple unions (there, the enum type is generated from the list of all branches, so there is no way to omit a branch but still have it be part of the union). A later patch will add shorthand so that branches that are empty in flat unions can be declared as 'branch':{} instead of 'branch':'Empty', to avoid the need for an otherwise useless explicit empty type. [Such shorthand for simple unions is a bit harder to justify, since we would still have to generate a wrapper type that parses 'data':{}, rather than truly being an empty branch with no additional siblings to the 'type' member.] Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1468468228-27827-3-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-18block: ignore flush requests when storage is cleanEvgeny Yakovlev
Some guests (win2008 server for example) do a lot of unnecessary flushing when underlying media has not changed. This adds additional overhead on host when calling fsync/fdatasync. This change introduces a write generation scheme in BlockDriverState. Current write generation is checked against last flushed generation to avoid unnessesary flushes. The problem with excessive flushing was found by a performance test which does parallel directory tree creation (from 2 processes). Results improved from 0.424 loops/sec to 0.432 loops/sec. Each loop creates 10^3 directories with 10 files in each. This affected some blkdebug testcases that were expecting error logs from failure-injected flushes which are now skipped entirely (tests 026 071 089). This also affects the performance of block jobs and thus BLOCK_JOB_READY events for driver-mirror and active block-commit commands now arrives faster, before QMP send successfully returns to caller (tests 141 144). Signed-off-by: Evgeny Yakovlev <eyakovlev@virtuozzo.com> Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 1468870792-7411-5-git-send-email-den@openvz.org CC: Kevin Wolf <kwolf@redhat.com> CC: Max Reitz <mreitz@redhat.com> CC: Stefan Hajnoczi <stefanha@redhat.com> CC: Fam Zheng <famz@redhat.com> CC: John Snow <jsnow@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2016-07-18tests: in IDE and AHCI tests perform DMA write before flushingEvgeny Yakovlev
Due to changes in flush behaviour clean disks stopped generating flush_to_disk events and IDE and AHCI tests that test flush commands started to fail. This change adds additional DMA writes to affected tests before sending flush commands so that bdrv_flush actually generates flush_to_disk event. Signed-off-by: Evgeny Yakovlev <eyakovlev@virtuozzo.com> Signed-off-by: Denis V. Lunev <den@openvz.org> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 1468870792-7411-4-git-send-email-den@openvz.org CC: Kevin Wolf <kwolf@redhat.com> CC: Max Reitz <mreitz@redhat.com> CC: Stefan Hajnoczi <stefanha@redhat.com> CC: Fam Zheng <famz@redhat.com> CC: John Snow <jsnow@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2016-07-13iotests: Make 157 actually format-agnosticMax Reitz
iotest 157 pretends not to care about the image format used, but in fact it does due to the format name not being filtered in its output. This patch adds filtering and changes the reference output accordingly. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20160711132246.3152-1-mreitz@redhat.com Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2016-07-13qemu-iotests: Test naming of throttling groupsAlberto Garcia
Throttling groups are named using the 'group' parameter of the block_set_io_throttle command and the throttling.group command-line option. If that parameter is unspecified the groups get the name of the block device. This patch adds a new test to check the naming of throttling groups. Signed-off-by: Alberto Garcia <berto@igalia.com> Message-id: d87d02823a6b91609509d8bb18e2f5dbd9a6102c.1467986342.git.berto@igalia.com Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2016-07-13qemu-iotests: Test setting WCE with qdevKevin Wolf
Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com>
2016-07-13coroutine: move entry argument to qemu_coroutine_createPaolo Bonzini
In practice the entry argument is always known at creation time, and it is confusing that sometimes qemu_coroutine_enter is used with a non-NULL argument to re-enter a coroutine (this happens in block/sheepdog.c and tests/test-coroutine.c). So pass the opaque value at creation time, for consistency with e.g. aio_bh_new. Mostly done with the following semantic patch: @ entry1 @ expression entry, arg, co; @@ - co = qemu_coroutine_create(entry); + co = qemu_coroutine_create(entry, arg); ... - qemu_coroutine_enter(co, arg); + qemu_coroutine_enter(co); @ entry2 @ expression entry, arg; identifier co; @@ - Coroutine *co = qemu_coroutine_create(entry); + Coroutine *co = qemu_coroutine_create(entry, arg); ... - qemu_coroutine_enter(co, arg); + qemu_coroutine_enter(co); @ entry3 @ expression entry, arg; @@ - qemu_coroutine_enter(qemu_coroutine_create(entry), arg); + qemu_coroutine_enter(qemu_coroutine_create(entry, arg)); @ reentry @ expression co; @@ - qemu_coroutine_enter(co, NULL); + qemu_coroutine_enter(co); except for the aforementioned few places where the semantic patch stumbled (as expected) and for test_co_queue, which would otherwise produce an uninitialized variable warning. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2016-07-13test-coroutine: prepare for the next patchPaolo Bonzini
The next patch moves the coroutine argument from first-enter to creation time. In this case, coroutine has not been initialized yet when the coroutine is created, so change to a pointer. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2016-07-13blockjob: Add 'job_id' parameter to block_job_create()Alberto Garcia
When a new job is created, the job ID is taken from the device name of the BDS. This patch adds a new 'job_id' parameter to let the caller provide one instead. This patch also verifies that the ID is always unique and well-formed. This causes problems in a couple of places where no ID is being set, because the BDS does not have a device name. In the case of test_block_job_start() (from test-blockjob-txn.c) we can simply use this new 'job_id' parameter to set the missing ID. In the case of img_commit() (from qemu-img.c) we still don't have the API to make commit_active_start() set the job ID, so we solve it by setting a default value. We'll get rid of this as soon as we extend the API. Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2016-07-12Clean up ill-advised or unusual header guardsMarkus Armbruster
Cleaned up with scripts/clean-header-guards.pl. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Richard Henderson <rth@twiddle.net>
2016-07-12Use #include "..." for our own headers, <...> for othersMarkus Armbruster
Tracked down with an ugly, brittle and probably buggy Perl script. Also move includes converted to <...> up so they get included before ours where that's obviously okay. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Tested-by: Eric Blake <eblake@redhat.com> Reviewed-by: Richard Henderson <rth@twiddle.net>
2016-07-06qapi: Add new clone visitorEric Blake
We have a couple places in the code base that want to deep-clone one QAPI object into another, and they were resorting to serializing the struct out to QObject then reparsing it. A much more efficient version can be done by adding a new clone visitor. Since cloning is still relatively uncommon, expose the use of the new visitor via a QAPI_CLONE() macro that takes care of type-punning the underlying function pointer, rather than generating lots of unused functions for types that won't be cloned. And yes, we're relying on the compiler treating all pointers equally, even though a strict C program cannot portably do so - but we're not the first one in the qemu code base to expect it to work (hello, glib!). The choice of adding a fourth visitor type deserves some explanation. On the surface, the clone visitor is mostly an input visitor (it takes arbitrary input - in this case, another QAPI object - and creates a new QAPI object during the course of the visit). But ever since commit da72ab0 consolidated enum visits based on the visitor type, using VISITOR_INPUT would cause us to run visit_type_str(), even though for cloning there is nothing to do (we just copy the enum value across, without regards to its mapping to strings). Also, since our input happens to be a QAPI object, we can also satisfy the internal checks for VISITOR_OUTPUT. So in the end, I settled with a new VISITOR_CLONE, and chose its value such that many internal checks can use 'v->type & mask', sticking to 'v->type == value' where the difference matters. Note that we can only clone objects (including alternates) and lists, not built-ins or enums. The visitor core hides integer width from the actual visitor (since commit 04e070d), and as long as that's the case, we can't clone top-level integers. Then again, those can always be cloned by direct copy, since they are not objects with deep pointers, so it's no real loss. And restricting cloning to just objects and lists is cleaner than restricting it to non-integers. As such, I documented that the clone visitor is for direct use only by code internal to QAPI, and should not be used on incomplete objects (other than a hack to work around the fact that we allow NULL in place of "" in visit_type_str() in other output visitors). Note that as written, the clone visitor will never fail on a complete object. Scalars (including enums) not at the root of the clone copy just fine with no additional effort while visiting the scalar, by virtue of a g_memdup() each time we push another struct onto the stack. Cloning a string requires deduplication of a pointer, which means it can also provide the guarantee of an input visitor of never producing NULL even when still accepting NULL in place of "" the way the QMP output visitor does. Cloning an 'any' type could be possible by incrementing the QObject refcnt, but it's not obvious whether that is better than implementing a QObject deep clone. So for now, we document it as unsupported, and intentionally omit the .type_any() callback to let a developer know their usage needs implementation. Add testsuite coverage for several different clone situations, to ensure that the code is working. I also tested that valgrind was happy with the test. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-14-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06qapi: Add new visit_complete() functionEric Blake
Making each output visitor provide its own output collection function was the only remaining reason for exposing visitor sub-types to the rest of the code base. Add a polymorphic visit_complete() function which is a no-op for input visitors, and which populates an opaque pointer for output visitors. For maximum type-safety, also add a parameter to the output visitor constructors with a type-correct version of the output pointer, and assert that the two uses match. This approach was considered superior to either passing the output parameter only during construction (action at a distance during visit_free() feels awkward) or only during visit_complete() (defeating type safety makes it easier to use incorrectly). Most callers were function-local, and therefore a mechanical conversion; the testsuite was a bit trickier, but the previous cleanup patch minimized the churn here. The visit_complete() function may be called at most once; doing so lets us use transfer semantics rather than duplication or ref-count semantics to get the just-built output back to the caller, even though it means our behavior is not idempotent. Generated code is simplified as follows for events: |@@ -26,7 +26,7 @@ void qapi_event_send_acpi_device_ost(ACP | QDict *qmp; | Error *err = NULL; | QMPEventFuncEmit emit; |- QmpOutputVisitor *qov; |+ QObject *obj; | Visitor *v; | q_obj_ACPI_DEVICE_OST_arg param = { | info |@@ -39,8 +39,7 @@ void qapi_event_send_acpi_device_ost(ACP | | qmp = qmp_event_build_dict("ACPI_DEVICE_OST"); | |- qov = qmp_output_visitor_new(); |- v = qmp_output_get_visitor(qov); |+ v = qmp_output_visitor_new(&obj); | | visit_start_struct(v, "ACPI_DEVICE_OST", NULL, 0, &err); | if (err) { |@@ -55,7 +54,8 @@ void qapi_event_send_acpi_device_ost(ACP | goto out; | } | |- qdict_put_obj(qmp, "data", qmp_output_get_qobject(qov)); |+ visit_complete(v, &obj); |+ qdict_put_obj(qmp, "data", obj); | emit(QAPI_EVENT_ACPI_DEVICE_OST, qmp, &err); and for commands: | { | Error *err = NULL; |- QmpOutputVisitor *qov = qmp_output_visitor_new(); | Visitor *v; | |- v = qmp_output_get_visitor(qov); |+ v = qmp_output_visitor_new(ret_out); | visit_type_AddfdInfo(v, "unused", &ret_in, &err); |- if (err) { |- goto out; |+ if (!err) { |+ visit_complete(v, ret_out); | } |- *ret_out = qmp_output_get_qobject(qov); |- |-out: | error_propagate(errp, err); Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-13-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06tests: Factor out common code in qapi output testsEric Blake
Create a new visitor_get() function to capture common actions taken in collecting output from an output visitor, to make it easier to refactor the output visitors in a later patch. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-12-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06tests: Clean up test-string-output-visitorEric Blake
Use &error_abort and error_free_or_abort() in more places, use the generated qapi_free_intList() instead of open-coding it, reduce the scope of some variables, avoid code duplication during test setup with visitor_output_setup_internal(), and copy the visitor_reset() concept from the qmp-output test to the string-output test. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-11-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06qmp-output-visitor: Favor new visit_free() functionEric Blake
Now that we have a polymorphic visit_free(), we no longer need qmp_output_visitor_cleanup(); however, we still need to expose the subtype for qmp_output_get_qobject(). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-10-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06string-output-visitor: Favor new visit_free() functionEric Blake
Now that we have a polymorphic visit_free(), we no longer need string_output_visitor_cleanup(); however, we still need to expose the subtype for string_output_get_string(). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-9-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06qmp-input-visitor: Favor new visit_free() functionEric Blake
Now that we have a polymorphic visit_free(), we no longer need qmp_input_visitor_cleanup(); which in turn means we no longer need to return a subtype from qmp_input_visitor_new() nor a public upcast function. Generated code changes to qmp-marshal.c look like: |@@ -52,11 +52,10 @@ void qmp_marshal_add_fd(QDict *args, QOb | { | Error *err = NULL; | AddfdInfo *retval; |- QmpInputVisitor *qiv = qmp_input_visitor_new(QOBJECT(args), true); | Visitor *v; | q_obj_add_fd_arg arg = {0}; | |- v = qmp_input_get_visitor(qiv); |+ v = qmp_input_visitor_new(QOBJECT(args), true); | visit_start_struct(v, NULL, NULL, 0, &err); | if (err) { | goto out; Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-8-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06string-input-visitor: Favor new visit_free() functionEric Blake
Now that we have a polymorphic visit_free(), we no longer need string_input_visitor_cleanup(); which in turn means we no longer need to return a subtype from string_input_visitor_new() nor a public upcast function. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-7-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06opts-visitor: Favor new visit_free() functionEric Blake
Now that we have a polymorphic visit_free(), we no longer need opts_visitor_cleanup(); which in turn means we no longer need to return a subtype from opts_visitor_new() nor a public upcast function. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-6-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06qapi: Add new visit_free() functionEric Blake
Making each visitor provide its own (awkwardly-named) FOO_cleanup() is unusual, when we can instead have a polymorphic visit_free() interface. Over the next few patches, we can use the polymorphic functions to eliminate the need for a FOO_get_visitor() function for accessing specific visitor functionality, once everything can be accessed directly through the Visitor* interfaces. The dealloc visitor is the first one converted to completely use the new entry point, since qapi_dealloc_visitor_cleanup() was the only reason that qapi_dealloc_get_visitor() existed, and only generated and testsuite code was even using it. With the new visit_free() entry point in place, we no longer need to expose the QapiDeallocVisitor subtype through qapi_dealloc_visitor_new(), and can get by with less generated code, with diffs that look like: | void qapi_free_ACPIOSTInfo(ACPIOSTInfo *obj) | { |- QapiDeallocVisitor *qdv; | Visitor *v; | | if (!obj) { | return; | } | |- qdv = qapi_dealloc_visitor_new(); |- v = qapi_dealloc_get_visitor(qdv); |+ v = qapi_dealloc_visitor_new(); | visit_type_ACPIOSTInfo(v, NULL, &obj, NULL); |- qapi_dealloc_visitor_cleanup(qdv); |+ visit_free(v); |} Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-5-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06qapi: Add parameter to visit_end_*Eric Blake
Rather than making the dealloc visitor track of stack of pointers remembered during visit_start_* in order to free them during visit_end_*, it's a lot easier to just make all callers pass the same pointer to visit_end_*. The generated code has access to the same pointer, while all other users are doing virtual walks and can pass NULL. The dealloc visitor is then greatly simplified. All three visit_end_*() functions intentionally take a void**, even though the visit_start_*() functions differ between void**, GenericList**, and GenericAlternate**. This is done for several reasons: when doing a virtual walk, passing NULL doesn't care what the type is, but when doing a generated walk, we already have to cast the caller's specific FOO* to call visit_start, while using void** lets us use visit_end without a cast. Also, an upcoming patch will add a clone visitor that wants to use the same implementation for all three visit_end callbacks, which is made easier if all three share the same signature. For visitors with already track per-object state (the QMP visitors via a stack, and the string visitors which do not allow nesting), add an assertion that the caller is indeed passing the same pointer to paired calls. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-4-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-06qapi: Improve use of qmp/types.hEric Blake
'qjson.h' is not a QObject subtype; include this file directly in .c files that are using it, rather than abusing qmp/types.h for that purpose. Meanwhile, for files that include a list of individual QObject subtypes, it's easier to just use qmp/types.h for that purpose. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1465490926-28625-2-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-07-05Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into stagingPeter Maydell
pc, pci, virtio: new features, cleanups, fixes iommus can not be added with -device. cleanups and fixes all over the place Signed-off-by: Michael S. Tsirkin <mst@redhat.com> # gpg: Signature made Tue 05 Jul 2016 11:18:32 BST # gpg: using RSA key 0x281F0DB8D28D5469 # gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>" # gpg: aka "Michael S. Tsirkin <mst@redhat.com>" # Primary key fingerprint: 0270 606B 6F3C DF3D 0B17 0970 C350 3912 AFBE 8E67 # Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA 8A0D 281F 0DB8 D28D 5469 * remotes/mst/tags/for_upstream: (30 commits) vmw_pvscsi: remove unnecessary internal msi state flag e1000e: remove unnecessary internal msi state flag vmxnet3: remove unnecessary internal msi state flag mptsas: remove unnecessary internal msi state flag megasas: remove unnecessary megasas_use_msi() pci: Convert msi_init() to Error and fix callers to check it pci bridge dev: change msi property type megasas: change msi/msix property type mptsas: change msi property type intel-hda: change msi property type usb xhci: change msi/msix property type change pvscsi_init_msi() type to void tests: add APIC.cphp and DSDT.cphp blobs tests: acpi: add CPU hotplug testcase log: Permit -dfilter 0..0xffffffffffffffff range: Replace internal representation of Range range: Eliminate direct Range member access log: Clean up misuse of Range for -dfilter pci_register_bar: cleanup Revert "virtio-net: unbreak self announcement and guest offloads after migration" ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-07-04tests: add APIC.cphp and DSDT.cphp blobsIgor Mammedov
Signed-off-by: Igor Mammedov <imammedo@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2016-07-04crypto: implement sha224, sha384, sha512 and ripemd160 hashesDaniel P. Berrange
Wire up the nettle and gcrypt hash backends so that they can support the sha224, sha384, sha512 and ripemd160 hash algorithms. Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2016-07-04tests: acpi: add CPU hotplug testcaseIgor Mammedov
Test with: -smp 2,cores=3,sockets=2,maxcpus=6 to capture sparse APIC ID values that default AMD CPU has in above configuration. Signed-off-by: Igor Mammedov <imammedo@redhat.com> Reviewed-by: Marcel Apfelbaum <marcel@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2016-07-04log: Permit -dfilter 0..0xffffffffffffffffMarkus Armbruster
Works fine since the previous commit fixed the underlying range data type. Of course it filters out nothing, but so does 0..1,2..0xffffffffffffffff, and we don't bother rejecting that either. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2016-07-04log: Clean up misuse of Range for -dfilterMarkus Armbruster
Range encodes an integer interval [a,b] as { begin = a, end = b + 1 }, where a \in [0,2^64-1] and b \in [1,2^64]. Thus, zero end is to be interpreted as 2^64. The implementation of -dfilter (commit 3514552) uses Range differently: it encodes [a,b] as { begin = a, end = b }. The code works, but it contradicts the specification of Range in range.h. Switch to the specified representation. Since it can't represent [0,UINT64_MAX], we have to reject that now. Add a test for it. While we're rejecting anyway: observe that we reject -dfilter LOB..UPB where LOB > UPB when UPB is zero, but happily create an empty Range when it isn't. Reject it then, too, and add a test for it. While there, add a positive test for the problematic upper bound UINT64_MAX. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2016-07-04crypto: switch hash code to use nettle/gcrypt directlyDaniel P. Berrange
Currently the internal hash code is using the gnutls hash APIs. GNUTLS in turn is wrapping either nettle or gcrypt. Not only were the GNUTLS hash APIs not added until GNUTLS 2.9.10, but they don't expose support for all the algorithms QEMU needs to use with LUKS. Address this by directly wrapping nettle/gcrypt in QEMU and avoiding GNUTLS's extra layer of indirection. This gives us support for hash functions on a much wider range of platforms and opens up ability to support more hash functions. It also avoids a GNUTLS bug which would not correctly handle hashing of large data blocks if int != size_t. Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2016-07-04crypto: rename OUT to out in xts test to avoid clash on MinGWDaniel P. Berrange
On MinGW one of the system headers already has "OUT" defined which causes a compile failure of the test suite. Rename the test suite var to 'out' to avoid this clash Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2016-07-04crypto: fix handling of iv generator hash defaultsDaniel P. Berrange
When opening an existing LUKS volume, if the iv generator is essiv, then the iv hash algorithm is mandatory to provide. We must report an error if it is omitted in the cipher mode spec, not silently default to hash 0 (md5). If the iv generator is not essiv, then we explicitly ignore any iv hash algorithm, rather than report an error, for compatibility with dm-crypt. When creating a new LUKS volume, if the iv generator is essiv and no iv hsah algorithm is provided, we should default to using the sha256 hash. Reported-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2016-06-30qapi: Fix crash on missing alternate member of QAPI structEric Blake
If a QAPI struct has a mandatory alternate member which is not present on input, the input visitor reports an error for the missing alternate without setting the discriminator, but the cleanup code for the struct still tries to use the dealloc visitor to clean up the alternate. Commit dbf11922 changed visit_start_alternate to set *obj to NULL when an error occurs, where it was previously left untouched. Thus, before the patch, the dealloc visitor is blindly trying to cleanup whatever branch corresponds to (*obj)->type == 0 (that is, QTYPE_NONE, because *obj still pointed to zeroed memory), which selects the default branch of the switch and sets an error, but this second error is ignored by the way the dealloc visitor is used; but after the patch, the attempt to switch dereferences NULL. When cleaning up after a partial object parse, we specifically check for !*obj after visit_start_struct() (see gen_visit_object()); doing the same for alternates fixes the crash. Enhance the testsuite to give coverage for both missing struct and missing alternate members. Also add an abort - we expect visit_start_alternate() to either set an error or to set (*obj)->type to a valid QType that corresponds to actual user input, and QTYPE_NONE should never be reachable from valid input. Had the abort() been in place earlier, we might have noticed the dealloc visitor dereferencing bogus zeroed memory prior to when commit dbf11922 forced our hand by setting *obj to NULL and causing a fault. Test case: {'execute':'blockdev-add', 'arguments':{'options':{'driver':'raw'}}} The choice of 'driver':'raw' selects a BlockdevOptionsGenericFormat struct, which has a mandatory 'file':'BlockdevRef' in QAPI. Since 'file' is missing as a sibling of 'driver', this should report a graceful error rather than fault. After this patch, we are back to: {"error": {"class": "GenericError", "desc": "Parameter 'file' is missing"}} Generated code in qapi-visit.c changes as: |@@ -2444,6 +2444,9 @@ void visit_type_BlockdevRef(Visitor *v, | if (err) { | goto out; | } |+ if (!*obj) { |+ goto out_obj; |+ } | switch ((*obj)->type) { | case QTYPE_QDICT: | visit_start_struct(v, name, NULL, 0, &err); |@@ -2459,10 +2462,13 @@ void visit_type_BlockdevRef(Visitor *v, | case QTYPE_QSTRING: | visit_type_str(v, name, &(*obj)->u.reference, &err); | break; |+ case QTYPE_NONE: |+ abort(); | default: | error_setg(&err, QERR_INVALID_PARAMETER_TYPE, name ? name : "null", | "BlockdevRef"); | } |+out_obj: | visit_end_alternate(v); Reported by Kashyap Chamarthy <kchamart@redhat.com> CC: qemu-stable@nongnu.org Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1466012271-5204-1-git-send-email-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Tested-by: Kashyap Chamarthy <kchamart@redhat.com> [Commit message tweaked] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2016-06-29Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into stagingPeter Maydell
* serial port fixes (Paolo) * Q35 modeling improvements (Paolo, Vasily) * chardev cleanup improvements (Marc-André) * iscsi bugfix (Peter L.) * cpu_exec patch from multi-arch patches (Peter C.) * pci-assign tweak (Lin Ma) # gpg: Signature made Wed 29 Jun 2016 15:56:30 BST # gpg: using RSA key 0xBFFBD25F78C7AE83 # gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" # gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" # Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1 # Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83 * remotes/bonzini/tags/for-upstream: (35 commits) socket: unlink unix socket on remove socket: add listen feature char: clean up remaining chardevs when leaving vhost-user: disable chardev handlers on close vhost-user-test: fix g_cond_wait_until compat implementation vl: smp_parse: fix regression ich9: implement SCI_IRQ_SEL register ich9: implement ACPI_EN register serial: reinstate watch after migration serial: remove watch on reset char: change qemu_chr_fe_add_watch to return unsigned serial: separate serial_xmit and serial_watch_cb serial: simplify tsr_retry reset serial: make tsr_retry unsigned iscsi: fix assertion in is_sector_request_lun_aligned target-*: Don't redefine cpu_exec() pci-assign: Move "Invalid ROM" error message to pci-assign-load-rom.c vnc: generalize "VNC server running on ..." message scsi: esp: fix migration MC146818 RTC: add GPIO access to output IRQ ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-06-29socket: unlink unix socket on removeMarc-André Lureau
qemu leaves unix socket files behind when removing a listening chardev or leaving. qemu could clean that up, even if doing so isn't race-free. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1347077 Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <1466105332-10285-4-git-send-email-marcandre.lureau@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>