aboutsummaryrefslogtreecommitdiff
path: root/scripts
AgeCommit message (Collapse)Author
2015-11-10qapi: Provide nicer array names in introspectionEric Blake
For the sake of humans reading introspection output, it is nice to have the name of implicit array types be recognizable as arrays of the underlying type. However, while this patch allows humans to skip from a command with return type "[123]" straight to the definition of type "123" without having to first inspect type "[123]", document that this shortcut should not be taken by client apps. This makes the resulting introspection string slightly larger by default (just over 200 bytes), but it's in the noise (less than 0.3% of the overall 70k size of 'query-qmp-capabilities'). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1446791754-23823-12-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-10qapi: Test failure in middle of array parseEric Blake
Our generated list visitors have the same problem as has been mentioned elsewhere (see commit 2f52e20): they allocate data even on failure. An upcoming patch will correct things to provide saner guarantees, but first we need to expose the behavior in the testsuite to ensure we aren't introducing any memory usage bugs. There are more test cases throughout the test-qmp-input-* tests that already deal with partial allocation; a later commit will clean up all visit_type_FOO(), without marking all of the tests with FIXME at this time. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1446791754-23823-10-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-05Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into stagingPeter Maydell
* Guest ABI fixes for PC machines (hw_version) * Fixes for recent Perl * John Snow's configure fixes * file-backed RAM improvements (Igor, Pavel) * -Werror=clobbered fixes (Stefan) * Kill -d ioport * Fix qemu-system-s390x * Performance improvement for kvmclock migration # gpg: Signature made Thu 05 Nov 2015 13:42:27 GMT using RSA key ID 78C7AE83 # gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" # gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" * remotes/bonzini/tags/for-upstream: iscsi: Translate scsi sense into error code Revert "Introduce cpu_clean_all_dirty" kvmclock: add a new function to update env->tsc. configure: disable FORTIFY_SOURCE under clang backends/hostmem-file: Allow to specify full pathname for backing file configure: disallow ccache during compile tests cpu-exec: Fix compiler warning (-Werror=clobbered) memory: call begin, log_start and commit when registering a new listener megasas: Use qemu_hw_version() instead of QEMU_VERSION osdep: Rename qemu_{get, set}_version() to qemu_{, set_}hw_version() pc: Set hw_version on all machine classes qemu-log: remove -d ioport ioport: do not use CPU_LOG_IOPORT target-i386: fix pcmpxstrx equal-ordered (strstr) mode scripts/text2pod.pl: Escape left brace file_ram_alloc: propagate error to caller instead of terminating QEMU Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-11-04migration: fix analyze-migration.py scriptMark Cave-Ayland
Commit 61964 "Add configuration section" broke the analyze-migration.py script which terminates due to the unrecognised section. Fix the script by parsing the contents of the configuration section directly into a new ConfigurationSection object (although nothing is done with it yet). Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk> Reviewed-by: Juan Quintela <quintela@redhat.com>al3 Signed-off-by: Juan Quintela <quintela@redhat.com>al3
2015-11-02scripts/text2pod.pl: Escape left braceFam Zheng
Latest perl now deprecates "{" literal in regex and print warnings like "unescaped left brace in regex is deprecated". Add escapes to keep it happy. Signed-off-by: Fam Zheng <famz@redhat.com> Message-Id: <1445326726-16031-1-git-send-email-famz@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-11-02qapi: Simplify gen_struct_field()Eric Blake
Rather than having all callers pass a name, type, and optional flag, have them instead pass a QAPISchemaObjectTypeMember which already has all that information. No change to generated code. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1445898903-12082-25-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-02qapi: Reserve 'u' member nameEric Blake
Now that we have separated union tag values from colliding with non-variant C names, by naming the union 'u', we should reserve this name for our use. Note that we want to forbid 'u' even in a struct with no variants, because it is possible for a future qemu release to extend QMP in a backwards-compatible manner while converting from a struct to a flat union. Fortunately, no existing clients were using this member name. If we ever find the need for QMP to have a member 'u', we could at that time relax things, perhaps by having c_name() munge the QMP member to 'q_u'. Note that we cannot forbid 'u' everywhere (by adding the rejection code to check_name()), because the existing QKeyCode enum already uses it; therefore we only reserve it as a struct type member name. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1445898903-12082-24-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-02qapi: Finish converting to new qapi union layoutEric Blake
We have two issues with our qapi union layout: 1) Even though the QMP wire format spells the tag 'type', the C code spells it 'kind', requiring some hacks in the generator. 2) The C struct uses an anonymous union, which places all tag values in the same namespace as all non-variant members. This leads to spurious collisions if a tag value matches a non-variant member's name. This patch is the back end for a series that converts to a saner qapi union layout. Now that all clients have been converted to use 'type' and 'obj->u.value', we can drop the temporary parallel support for 'kind' and 'obj->value'. Given a simple union qapi type: { 'union':'Foo', 'data': { 'a':'int', 'b':'bool' } } this is the overall effect, when compared to the state before this series of patches: | struct Foo { |- FooKind kind; |- union { /* union tag is @kind */ |+ FooKind type; |+ union { /* union tag is @type */ | void *data; | int64_t a; | bool b; |- }; |+ } u; | }; The testsuite still contains some examples of artificial restrictions (see flat-union-clash-type.json, for example) that are no longer technically necessary, now that there is no longer a collision between enum tag values and non-variant member names; but fixing this will be done in later patches, in part because some further changes are required to keep QAPISchema*.check() from asserting. Also, a later patch will add a reservation for the member name 'u' to avoid a collision between a user's non-variant names and our internal choice of C union name. Note, however, that we do not rename the generated enum, which is still 'FooKind'. A further patch could generate implicit enums as 'FooType', but while the generator already reserved the '*Kind' namespace (commit 4dc2e69), there are already QMP constructs with '*Type' naming, which means changing our reservation namespace would have lots of churn to C code to deal with a forced name change. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1445898903-12082-23-git-send-email-eblake@redhat.com> [Commit message tweaked] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-02qapi-visit: Convert to new qapi union layoutEric Blake
We have two issues with our qapi union layout: 1) Even though the QMP wire format spells the tag 'type', the C code spells it 'kind', requiring some hacks in the generator. 2) The C struct uses an anonymous union, which places all tag values in the same namespace as all non-variant members. This leads to spurious collisions if a tag value matches a non-variant member's name. Make the conversion to the new layout for qapi-visit.py. Generated code changes look like: |@@ -4912,16 +4912,16 @@ void visit_type_MemoryDeviceInfo(Visitor | if (!*obj) { | goto out_obj; | } |- visit_type_MemoryDeviceInfoKind(v, &(*obj)->kind, "type", &err); |+ visit_type_MemoryDeviceInfoKind(v, &(*obj)->type, "type", &err); | if (err) { | goto out_obj; | } |- if (!visit_start_union(v, !!(*obj)->data, &err) || err) { |+ if (!visit_start_union(v, !!(*obj)->u.data, &err) || err) { | goto out_obj; | } |- switch ((*obj)->kind) { |+ switch ((*obj)->type) { | case MEMORY_DEVICE_INFO_KIND_DIMM: |- visit_type_PCDIMMDeviceInfo(v, &(*obj)->dimm, "data", &err); |+ visit_type_PCDIMMDeviceInfo(v, &(*obj)->u.dimm, "data", &err); | break; | default: | abort(); |@@ -4930,7 +4930,7 @@ out_obj: | error_propagate(errp, err); | err = NULL; | if (*obj) { |- visit_end_union(v, !!(*obj)->data, &err); |+ visit_end_union(v, !!(*obj)->u.data, &err); | } | error_propagate(errp, err); | err = NULL; Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1445898903-12082-14-git-send-email-eblake@redhat.com> [Commit message tweaked slightly] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-02qapi: Start converting to new qapi union layoutEric Blake
We have two issues with our qapi union layout: 1) Even though the QMP wire format spells the tag 'type', the C code spells it 'kind', requiring some hacks in the generator. 2) The C struct uses an anonymous union, which places all tag values in the same namespace as all non-variant members. This leads to spurious collisions if a tag value matches a non-variant member's name. This patch is the front end for a series that converts to a saner qapi union layout. By the end of the series, we will no longer have the type/kind mismatch, and all tag values will be under a named union, which requires clients to access 'obj->u.value' instead of 'obj->value'. But since the conversion touches a number of files, it is easiest if we temporarily support BOTH layouts simultaneously. Given a simple union qapi type: { 'union':'Foo', 'data': { 'a':'int', 'b':'bool' } } make the following changes in generated qapi-types.h: | struct Foo { |- FooKind kind; |- union { /* union tag is @kind */ |+ union { |+ FooKind kind; |+ FooKind type; |+ }; |+ union { /* union tag is @type */ | void *data; | int64_t a; | bool b; |+ union { /* union tag is @type */ |+ void *data; |+ int64_t a; |+ bool b; |+ } u; | }; | }; Flat unions do not need the anonymous union for the tag member, as we already fixed that to use the member name instead of 'kind' back in commit 0f61af3e. One additional change is needed in qapi.py: check_union() now needs to check for collisions with 'type' in addition to those with 'kind'. Later, when the conversions are complete, we will remove the duplication hacks, and also drop the check_union() restrictions. Note, however, that we do not rename the generated enum, which is still 'FooKind'. A further patch could generate implicit enums as 'FooType', but while the generator already reserved the '*Kind' namespace (commit 4dc2e69), there are already QMP constructs with '*Type' naming, which means changing our reservation namespace would have lots of churn to C code to deal with a forced name change. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1445898903-12082-13-git-send-email-eblake@redhat.com> [Commit message tweaked slightly] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-02qapi-visit: Remove redundant functions for flat union baseEric Blake
The code for visiting the base class of a child struct created visit_type_Base_fields() which covers all fields of Base; while the code for visiting the base class of a flat union created visit_type_Union_fields() covering all fields of the base except the discriminator. But since the base class includes the discriminator of a flat union, we can just visit the entire base, without needing a separate visit of the discriminator. Not only is consistently visiting all fields easier to understand, it lets us share code. The generated code in qapi-visit.c loses several now-unused visit_type_UNION_fields(), along with changes like: |@@ -1654,11 +1557,7 @@ void visit_type_BlockdevOptions(Visitor | if (!*obj) { | goto out_obj; | } |- visit_type_BlockdevOptions_fields(v, obj, &err); |- if (err) { |- goto out_obj; |- } |- visit_type_BlockdevDriver(v, &(*obj)->driver, "driver", &err); |+ visit_type_BlockdevOptionsBase_fields(v, (BlockdevOptionsBase **)obj, &err); | if (err) { | goto out_obj; | } and forward declarations where needed. Note that the cast of obj to BASE ** is necessary to call visit_type_BASE_fields() (and we can't use our upcast wrappers, because those work on pointers while we have a pointer-to-pointer). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1445898903-12082-12-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-02qapi: Unbox base membersEric Blake
Rather than storing a base class as a pointer to a box, just store the fields of that base class in the same order, so that a child struct can be directly cast to its parent. This gives less malloc overhead, less pointer dereferencing, and even less generated code. Compare to the earlier commit 1e6c1616a "qapi: Generate a nicer struct for flat unions" (although that patch had fewer places to change, as less of qemu was directly using qapi structs for flat unions). It also allows us to turn on automatic type-safe wrappers for upcasting to the base class of a struct. Changes to the generated code look like this in qapi-types.h: | struct SpiceChannel { |- SpiceBasicInfo *base; |+ /* Members inherited from SpiceBasicInfo: */ |+ char *host; |+ char *port; |+ NetworkAddressFamily family; |+ /* Own members: */ | int64_t connection_id; as well as additional upcast functions like qapi_SpiceChannel_base(). Meanwhile, changes to qapi-visit.c look like: | static void visit_type_SpiceChannel_fields(Visitor *v, SpiceChannel **obj, Error **errp) | { | Error *err = NULL; | |- visit_type_implicit_SpiceBasicInfo(v, &(*obj)->base, &err); |+ visit_type_SpiceBasicInfo_fields(v, (SpiceBasicInfo **)obj, &err); | if (err) { (the cast is necessary, since our upcast wrappers only deal with a single pointer, not pointer-to-pointer); plus the wholesale elimination of some now-unused visit_type_implicit_FOO() functions. Without boxing, the corner case of one empty struct having another empty struct as its base type now requires inserting a dummy member (previously, the 'Base *base' member sufficed). And now that we no longer consume a 'base' member in the generated C struct, we can delete the former negative struct-base-clash-base test. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1445898903-12082-11-git-send-email-eblake@redhat.com> [Commit message tweaked slightly] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-02qapi: Prefer typesafe upcasts to qapi base classesEric Blake
A previous patch (commit 1e6c1616) made it possible to directly cast from a qapi flat union type to its base type. However, it requires the use of a C cast, which turns off compiler type-safety checks. Fortunately, no such casts exist, just yet. Regardless, add inline type-safe wrappers named qapi_FOO_base() for any union type FOO that has a base, which can be used for a safer upcast, and enhance the testsuite to cover the new functionality. A future patch will extend the upcast support to structs, where such conversions do exist already. Note that C makes const-correct upcasts annoying because it lacks overloads; these functions cast away const so that they can accept user pointers whether const or not, and the result in turn can be assigned to normal or const pointers. Alternatively, this could have been done with macros, but type-safe macros are hairy, and not worthwhile here. This patch just adds upcasts. None of our code needed to downcast from a base qapi class to a child. Also, in the case of grandchildren (such as BlockdevOptionsQcow2), the caller will need to call two functions to get to the inner base (although it wouldn't be too hard to generate a qapi_FOO_base_base() if desired). If a user changes qapi to alter the base class hierarchy, such as going from 'A -> C' to 'A -> B -> C', it will change the type of 'qapi_C_base()', and the compiler will point out the places that are affected by the new base. One alternative was proposed, but was deemed too ugly to use in practice: the generators could output redundant information using anonymous types: | struct Child { | union { | struct { | Type1 parent_member1; | Type2 parent_member2; | }; | Parent base; | }; | }; With that ugly proposal, for a given qapi type, obj->member and obj->base.member would refer to the same storage; allowing convenience in working with members without needing 'base.' allowing typesafe upcast without needing a C cast by accessing '&obj->base', and allowing downcasts from the parent back to the child possible through container_of(obj, Child, base). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1445898903-12082-10-git-send-email-eblake@redhat.com> [Commit message tweaked] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-02qapi-types: Refactor base fields outputEric Blake
Move code from gen_union() into gen_struct_fields() in order for a later patch to share code when enumerating inherited fields for struct types. No change to generated code. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1445898903-12082-9-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-02qapi-visit: Split off visit_type_FOO_fields forward declEric Blake
We generate a static visit_type_FOO_fields() for every type FOO. However, sometimes we need a forward declaration. Split the code to generate the forward declaration out of gen_visit_implicit_struct() into a new gen_visit_fields_decl(), and also prepare for a forward declaration to be emitted during gen_visit_struct(), so that a future patch can switch from using visit_type_FOO_implicit() to the simpler visit_type_FOO_fields() as part of unboxing the base class of a struct. No change to generated code. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1445898903-12082-8-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-02qapi: Reserve 'q_*' and 'has_*' member namesEric Blake
c_name() produces names starting with 'q_' when protecting a dictionary member name that would fail to directly compile, but in doing so can cause clashes with any member name already beginning with 'q-' or 'q_'. Likewise, we create a C name 'has_' for any optional member that can clash with any member name beginning with 'has-' or 'has_'. Technically, rather than blindly reserving the namespace, we could try to complain about user names only when an actual collision occurs, or even teach c_name() how to munge names to avoid collisions. But it is not trivial, especially when collisions can occur across multiple types (such as via inheritance or flat unions). Besides, no existing .json files are trying to use these names. So it's easier to just outright forbid the potential for collision. We can always relax things in the future if a real need arises for QMP to express member names that have been forbidden here. 'has_' only has to be reserved for struct/union member names, while 'q_' is reserved everywhere (matching the fact that only members can be optional, while we use c_name() for munging both members and entities). Note that we could relax 'q_' restrictions on entities independently from member names; for example, c_name('qmp_' + 'unix') would result in a different function name than our current 'qmp_' + c_name('unix'). Update and add tests to cover the new error messages. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1445898903-12082-6-git-send-email-eblake@redhat.com> [Consistently pass protect=False to c_name(); commit message tweaked slightly] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-02qapi: Reserve '*List' type names for list typesEric Blake
Type names ending in 'List' can clash with qapi list types in generated C. We don't currently use such names. It is easier to outlaw them now than to worry about how to resolve such a clash in the future. For precedence, see commit 4dc2e69, which did the same for names ending in 'Kind' versus implicit enum types for qapi unions. Update the testsuite to match. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1445898903-12082-5-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-02qapi: More robust conditions for when labels are neededEric Blake
We were using regular expressions to see if ret included any earlier text that emitted a 'goto out;' line, to decide whether we needed to output an 'out:' label. But this is fragile, if the ret text can possibly combine more than one generated function body, where the first function used a goto but the second does not. Change the code to just check for the known conditions which cause an error check to be needed. Besides, it's slightly more efficient to use plain checks than regular expression searching. No change to generated code. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1445898903-12082-4-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-11-02qapi: More idiomatic string operationsEric Blake
Rather than slicing the end of a string, we can use python's endswith(). And rather than creating a set of characters, we can search for a character within a string. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1445898903-12082-3-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-29gdb command: qemu handlersDr. David Alan Gilbert
A new gdb commands are added: qemu handlers That dumps an AioContext list (by default qemu_aio_context) possibly including a backtrace for cases it knows about (with the verbose option). Intended to help find why something is hanging waiting for IO. Use 'qemu handlers --verbose iohandler_ctx' to find out why your incoming migration is stuck. Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-id: 1445951385-11924-1-git-send-email-dgilbert@redhat.com V2: Merge into one command with optional handlers arg, and only do backtrace in verbose mode (gdb) qemu handlers ---- {pfd = {fd = 6, events = 25, revents = 0}, io_read = 0x55869656ffd0 <event_notifier_dummy_cb>, io_write = 0x0, deleted = 0, opaque = 0x558698c4ce08, node = {le_next = 0x0, le_prev = 0x558698c4cdc0}} (gdb) qemu handlers iohandler_ctx ---- {pfd = {fd = 9, events = 25, revents = 0}, io_read = 0x558696581380 <fd_coroutine_enter>, io_write = 0x0, deleted = 0, opaque = 0x558698dc99d0, node = {le_next = 0x558698c4cca0, le_prev = 0x558698c4c1d0}} ---- {pfd = {fd = 4, events = 25, revents = 0}, io_read = 0x55869657b330 <sigfd_handler>, io_write = 0x0, deleted = 0, opaque = 0x4, node = {le_next = 0x558698c4c260, le_prev = 0x558699f72508}} ---- {pfd = {fd = 5, events = 25, revents = 0}, io_read = 0x55869656ffd0 <event_notifier_dummy_cb>, io_write = 0x0, deleted = 0, opaque = 0x558698c4c218, node = {le_next = 0x0, le_prev = 0x558698c4ccc8}} ---- (gdb) qemu handlers --verbose iohandler_ctx ---- {pfd = {fd = 9, events = 25, revents = 0}, io_read = 0x558696581380 <fd_coroutine_enter>, io_write = 0x0, deleted = 0, opaque = 0x558698dc99d0, node = {le_next = 0x558698c4cca0, le_prev = 0x558698c4c1d0}} #0 0x0000558696581820 in qemu_coroutine_switch (from_=from_@entry=0x558698cb3cf0, to_=to_@entry=0x7f421c37eac8, action=action@entry=COROUTINE_YIELD) at /home/dgilbert/git/qemu/coroutine-ucontext.c:177 #1 0x0000558696580c00 in qemu_coroutine_yield () at /home/dgilbert/git/qemu/qemu-coroutine.c:145 #2 0x00005586965814f5 in yield_until_fd_readable (fd=9) at /home/dgilbert/git/qemu/qemu-coroutine-io.c:90 #3 0x0000558696523937 in socket_get_buffer (opaque=0x55869a3dc620, buf=0x558698c505a0 "", pos=<optimized out>, size=32768) at /home/dgilbert/git/qemu/migration/qemu-file-unix.c:101 #4 0x0000558696521fac in qemu_fill_buffer (f=0x558698c50570) at /home/dgilbert/git/qemu/migration/qemu-file.c:227 #5 0x0000558696522989 in qemu_peek_byte (f=0x558698c50570, offset=0) at /home/dgilbert/git/qemu/migration/qemu-file.c:507 #6 0x0000558696522bf4 in qemu_get_be32 (f=0x558698c50570) at /home/dgilbert/git/qemu/migration/qemu-file.c:520 #7 0x0000558696522bf4 in qemu_get_be32 (f=f@entry=0x558698c50570) at /home/dgilbert/git/qemu/migration/qemu-file.c:604 #8 0x0000558696347e5c in qemu_loadvm_state (f=f@entry=0x558698c50570) at /home/dgilbert/git/qemu/migration/savevm.c:1821 #9 0x000055869651de8c in process_incoming_migration_co (opaque=0x558698c50570) at /home/dgilbert/git/qemu/migration/migration.c:336 #10 0x000055869658188a in coroutine_trampoline (i0=<optimized out>, i1=<optimized out>) at /home/dgilbert/git/qemu/coroutine-ucontext.c:80 #11 0x00007f420f05df10 in __start_context () at /lib64/libc.so.6 #12 0x00007ffc40815f50 in () #13 0x0000000000000000 in () ---- Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2015-10-29qemu-gdb: add $qemu_coroutine_sp and $qemu_coroutine_pcPaolo Bonzini
These can be useful to manually get a stack trace of a coroutine inside a core dump. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 1444636974-19950-4-git-send-email-pbonzini@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2015-10-29qemu-gdb: extract parts of "qemu coroutine" implementationPaolo Bonzini
Provide useful Python functions to reach and decipher a jmpbuf. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 1444636974-19950-3-git-send-email-pbonzini@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2015-10-29qemu-gdb: allow using glibc_pointer_guard() on core dumpsPaolo Bonzini
get_fs_base() cannot be run on a core dump, because it uses the arch_prctl system call. The fs base is the value that is returned by pthread_self(), and it would be nice to just glean it from the "info threads" output: * 1 Thread 0x7f16a3fff700 (LWP 33642) pthread_cond_wait@@GLIBC_2.3.2 () ^^^^^^^^^^^^^^ but unfortunately the gdb API does not provide that. Instead, we can look for the "arg" argument of the start_thread function if glibc debug information are available. If not, fall back to the old mechanism. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 1444636974-19950-2-git-send-email-pbonzini@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2015-10-22Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into stagingPeter Maydell
vhost, pc, virtio features, fixes, cleanups New features: VT-d support for devices behind a bridge vhost-user migration support Signed-off-by: Michael S. Tsirkin <mst@redhat.com> # gpg: Signature made Thu 22 Oct 2015 12:39:19 BST using RSA key ID D28D5469 # gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>" # gpg: aka "Michael S. Tsirkin <mst@redhat.com>" * remotes/mst/tags/for_upstream: (37 commits) hw/isa/lpc_ich9: inject the SMI on the VCPU that is writing to APM_CNT i386: keep cpu_model field in MachineState uptodate vhost: set the correct queue index in case of migration with multiqueue piix: fix resource leak reported by Coverity seccomp: add memfd_create to whitelist vhost-user-test: check ownership during migration vhost-user-test: add live-migration test vhost-user-test: learn to tweak various qemu arguments vhost-user-test: wrap server in TestServer struct vhost-user-test: remove useless static check vhost-user-test: move wait_for_fds() out vhost: add migration block if memfd failed vhost-user: use an enum helper for features mask vhost user: add rarp sending after live migration for legacy guest vhost user: add support of live migration net: add trace_vhost_user_event vhost-user: document migration log vhost: use a function for each call vhost-user: add a migration blocker vhost-user: send log shm fd along with log_base ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-10-22linux-headers: add unistd.hMarc-André Lureau
New syscalls are not yet widely distributed. Add them to qemu linux-headers include directory. Update based on v4.3-rc3 kernel headers. Exclude mips for now, which is more problematic due to extra header inclusion and probably unnecessary here. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Tested-by: Thibaut Collet <thibaut.collet@6wind.com>
2015-10-19Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into stagingPeter Maydell
* KVM page size fix for PPC * Support for Linux 4.4's new Hyper-V features * Eliminate g_slice from areas I maintain * checkpatch fix * Peter's cpu_reload_memory_map() cleanups * More changes to MAINTAINERS * Require Python 2.6 * chardev creation fixes * PCI requester id for ARM KVM * cleanups and doc fixes * Allow customization of the Hyper-V vendor id # gpg: Signature made Mon 19 Oct 2015 09:13:10 BST using RSA key ID 78C7AE83 # gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" # gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" * remotes/bonzini/tags/for-upstream: (49 commits) kvm: Allow the Hyper-V vendor ID to be specified kvm: Move x86-specific functions into target-i386/kvm.c kvm: Pass PCI device pointer to MSI routing functions hw/pci: Introduce pci_requester_id() kvm: Make KVM_CAP_SIGNAL_MSI globally available doc/rcu: fix g_free_rcu() usage example qemu-char: cleanup after completed conversion to cd->create qemu-char: convert ringbuf backend to data-driven creation qemu-char: convert vc backend to data-driven creation qemu-char: convert spice backend to data-driven creation qemu-char: convert console backend to data-driven creation qemu-char: convert stdio backend to data-driven creation qemu-char: convert testdev backend to data-driven creation qemu-char: convert braille backend to data-driven creation qemu-char: convert msmouse backend to data-driven creation qemu-char: convert mux backend to data-driven creation qemu-char: convert null backend to data-driven creation qemu-char: convert pty backend to data-driven creation qemu-char: convert UDP backend to data-driven creation qemu-char: convert socket backend to data-driven creation ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-10-15qapi: Track location that created an implicit typeEric Blake
A future patch will move some error checking from the parser to the various QAPISchema*.check() methods, which run only after parsing completes. It will thus be possible to create a python instance representing an implicit QAPI type that parses fine but will fail validation during check(). Since all errors have to have an associated 'info' location, we need a location to be associated with those implicit types. The intuitive info to use is the location of the enclosing entity that caused the creation of the implicit type. Note that we do not anticipate builtin types being used in an error message (as they are not part of the user's QAPI input, the user can't cause a semantic error in their behavior), so we exempt those types from requiring info, by setting a flag to track the completion of _def_predefineds(), and tracking that flag in _def_entity(). No change to the generated code. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1444710158-8723-13-git-send-email-eblake@redhat.com> [Missing QAPISchemaArrayType.is_implicit() supplied] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-15qapi: Create simple union type member earlierEric Blake
For simple unions, we were creating the implicit 'type' tag member during the QAPISchemaObjectTypeVariants constructor. This is different from every other implicit QAPISchemaEntity object, which get created by QAPISchema methods. Hoist the creation to the caller (renaming _make_tag_enum() to _make_implicit_tag()), and pass the entity rather than the string name, so that we have the nice property that no entities are created as a side effect within a different entity. A later patch will then have an easier time of associating location info with each entity creation. No change to generated code. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1444710158-8723-10-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-15qapi: Lazy creation of array typesEric Blake
Commit ac88219a had several TODO markers about whether we needed to automatically create the corresponding array type alongside any other type. It turns out that most of the time, we don't! There are a few exceptions: 1) We have a few situations where we use an array type in internal code but do not expose that type through QMP; fix it by declaring a dummy type that forces the generator to see that we want to use the array type. 2) The builtin arrays (such as intList for QAPI ['int']) must always be generated, because of the way our QAPI_TYPES_BUILTIN compile guard works: we have situations (at the very least tests/test-qmp-output-visitor.c) that include both top-level "qapi-types.h" (via "error.h") and a secondary "test-qapi-types.h". If we were to only emit the builtin types when used locally, then the first .h file would not include all types, but the second .h does not declare anything at all because the first .h set QAPI_TYPES_BUILTIN, and we would end up with compilation error due to things like unknown type 'int8List'. Actually, we may need to revisit how we do type guards, and change from a single QAPI_TYPES_BUILTIN over to a different usage pattern that does one #ifdef per qapi type - right now, the only types that are declared multiple times between two qapi .json files for inclusion by a single .c file happen to be the builtin arrays. But now that we have QAPI 'include' statements, it is logical to assume that we will soon reach a point where we want to reuse non-builtin types (yes, I'm thinking about what it will take to add introspection to QGA, where we will want to reuse the SchemaInfo type and friends). One #ifdef per type will help ensure that generating the same qapi type into more than one qapi-types.h won't cause collisions when both are included in the same .c file; but we also have to solve how to avoid creating duplicate qapi-types.c entry points. So that is a problem left for another day. Generated code for qapi-types and qapi-visit is drastically reduced; less than a third of the arrays that were blindly created were actually needed (a quick grep shows we dropped from 219 to 69 *List types), and the .o files lost more than 30% of their bulk. [For best results, diff the generated files with 'git diff --patience --no-index pre post'.] Interestingly, the introspection output is unchanged - this is because we already cull all types that are not indirectly reachable from a command or event, so introspection was already using only a subset of array types. The subset of types introspected is now a much larger percentage of the overall set of array types emitted in qapi-types.h (since the larger set shrunk), but still not 100% (evidence that the array types emitted for our new Dummy structs, and the new struct itself, don't affect QMP). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1444710158-8723-9-git-send-email-eblake@redhat.com> [Moved array info tracking to a later patch] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-15qapi: Don't use info as witness of implicit object typeEric Blake
A future patch will enable error reporting from the various QAPISchema*.check() methods. But to report an error related to an implicit type, we'll need to associate a location with the type (the same location as the top-level entity that is causing the creation of the implicit type), and once we do that, keying off of whether foo.info exists is no longer a viable way to determine if foo is an implicit type. Instead, add an is_implicit() method to QAPISchemaEntity, and use it. It can be overridden later for ObjectType and EnumType, when implicit instances of those classes gain info. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1444710158-8723-8-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-15qapi: Drop redundant returns-int testEric Blake
qapi-schema-test was already testing that we could have a command returning int, but burned a command name in the whitelist. Merge the redundant positive test returns-int, and pick a name that reduces the whitelist size. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1444710158-8723-6-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-15qapi: Prepare for errors during check()Eric Blake
The next few patches will start migrating error checking from ad hoc parse methods into the QAPISchema*.check() methods. But for an error message to display, we first have to fix the overall 'try' to catch those errors. We also want to enable a few more assertions, such as making sure every attempt to raise a semantic error is passed a valid location info, or that various preconditions hold. The general approach for moving error checking will then be to relax an assertion into an if that raises an exception if the condition does not hold, and removing the counterpart ad hoc check done during the parse phase. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1444710158-8723-3-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-15qapi: Use predicate callback to determine visit filteringEric Blake
Previously, qapi-types and qapi-visit filtered out implicit objects during visit_object_type() by using 'info' (works since implicit objects do not [yet] have associated info); meanwhile qapi-introspect filtered out all schema types on the first pass by returning a python type from visit_begin(), which was then used at a distance in QAPISchema.visit() to do the filtering. Rather than keeping these ad hoc approaches, add a new visitor callback visit_needed() which returns False to skip a given entity, and which defaults to True unless overridden. Use the new mechanism to simplify all three filtering visitors. No change to the generated code. Suggested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1444710158-8723-2-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-15qapi: Fix regression with '-netdev help'Eric Blake
Commit e36c714e causes 'qemu -netdev help' to dump core, because the call to visit_end_union() is no longer conditional on whether *obj was allocated. Reported by Marc-André Lureau <marcandre.lureau@gmail.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1444861825-19256-1-git-send-email-eblake@redhat.com> [Commit message tweaked to say 'help' instead of '?'] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-12qapi: Simplify gen_visit_fields() error handlingEric Blake
Since we have consolidated all generated code to use 'err' as the name of the local variable for error detection, we can simplify the decision on whether to skip error detection (useful for deallocation paths) to be a boolean. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1443565276-4535-18-git-send-email-eblake@redhat.com> [Change to gen_visit_fields() simplified] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-12qapi: Share gen_visit_fields()Eric Blake
Consolidate the code between visit, command marshalling, and event generation that iterates over the members of a struct. It reduces code duplication in the generator, so that a future patch can reduce the size of generated code while touching only one instead of three locations. There are no changes to the generated marshal code. The visitor code becomes slightly more verbose, but remains semantically equivalent, and is actually easier to read as it follows a more common idiom: | visit_optional(v, &(*obj)->has_device, "device", &err); |- if (!err && (*obj)->has_device) { |- visit_type_str(v, &(*obj)->device, "device", &err); |- } | if (err) { | goto out; | } |+ if ((*obj)->has_device) { |+ visit_type_str(v, &(*obj)->device, "device", &err); |+ if (err) { |+ goto out; |+ } |+ } The event code becomes slightly more verbose, but this is arguably a bug fix: although the visitors are not well documented, use of an optional member should not be attempted unless guarded by a prior call to visit_optional(). Works only because the output qmp visitor has a no-op visit_optional(): |+ visit_optional(v, &has_offset, "offset", &err); |+ if (err) { |+ goto out; |+ } | if (has_offset) { | visit_type_int(v, &offset, "offset", &err); Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1443565276-4535-17-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-12qapi: Share gen_err_check()Eric Blake
qapi-commands has a nice helper gen_err_check(), but did not use it everywhere. In fact, using it in more places makes it easier to reduce the lines of code used for generating error checks. This in turn will make it easier for later patches to consolidate another common pattern among the generators. The generated code has fewer blank lines in qapi-event.c functions, but has no semantic difference. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1443565276-4535-16-git-send-email-eblake@redhat.com> [Drop another blank line for symmetry] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-12qapi: Consistent generated code: minimize push_indent() usageEric Blake
We had some pointless differences in the generated code for visit, command marshalling, and events; unifying them makes it easier for future patches to consolidate to common helper functions. This is one patch of a series to clean up these differences. This patch reduces the number of push_indent()/pop_indent() pairs so that generated code is typically already at its natural output indentation in the python files. It is easier to reason about generated code if the reader does not have to track how much spacing will be inserted alongside the code, and moreso when all of the generators use the same patterns (qapi-type and qapi-event were already using in-place indentation). Arguably, the resulting python may be a bit harder to read with C code at the same indentation as python; on the other hand, not having to think about push_indent() is a win, and most decent editors provide syntax highlighting that makes it easier to visually distinguish python code from string literals that will become C code. There is no change to the generated output. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1443565276-4535-15-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-12qapi: Consistent generated code: prefer common indentationEric Blake
We had some pointless differences in the generated code for visit, command marshalling, and events; unifying them makes it easier for future patches to consolidate to common helper functions. This is one patch of a series to clean up these differences. This patch adjusts gen_visit_union() to use the same indentation as other functions, namely, by jumping early to the error label if the object was not set rather than placing the rest of the body inside an if for when it is set. No change in semantics to the generated code. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1443565276-4535-14-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-12qapi: Consistent generated code: prefer common labelsEric Blake
We had some pointless differences in the generated code for visit, command marshalling, and events; unifying them makes it easier for future patches to consolidate to common helper functions. This is one patch of a series to clean up these differences. This patch names the goto labels 'out' (not 'clean') and 'out_obj' (not 'out_end'). Additionally, the generator was inconsistent on whether labels had a leading space [our HACKING is silent; while emacs 'gnu' style adds the space to avoid littering column 1]. For minimal churn, prefer no leading space; this also matches the style that is more prevalent in current qemu.git. No change in semantics to the generated code. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1443565276-4535-13-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-12qapi: Consistent generated code: prefer visitor 'v'Eric Blake
We had some pointless differences in the generated code for visit, command marshalling, and events; unifying them makes it easier for future patches to consolidate to common helper functions. This is one patch of a series to clean up these differences. This patch names the local visitor variable 'v' rather than 'm'. Related objects, such as 'QapiDeallocVisitor', are also named by their initials instead of an unrelated leading m. No change in semantics to the generated code. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1443565276-4535-12-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-12qapi: Consistent generated code: prefer error 'err'Eric Blake
We had some pointless differences in the generated code for visit, command marshalling, and events; unifying them makes it easier for future patches to consolidate to common helper functions. This is one patch of a series to clean up these differences. This patch consistently names the local error variable 'err' rather than 'local_err'. No change in semantics to the generated code. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1443565276-4535-11-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-12qapi: Reuse code for flat union base validationEric Blake
Rather than open-code the check for a valid base type, we should reuse the common functionality. This allows for consistent error messages, and also makes it easier for a later patch to turn on support for inline anonymous base structures. Test flat-union-inline is updated to test only one feature (anonymous branch dictionaries), which can be implemented independently (test flat-union-bad-base already covers the idea of an anonymous base dictionary). Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1443565276-4535-10-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-12qapi: Avoid assertion failure on union 'type' collisionEric Blake
The previous commit added two tests that triggered an assertion failure. It's fairly straightforward to avoid the failure by just outright forbidding the collision between a union's tag values and its discriminator name (including the implicit name 'kind' supplied for simple unions [*]). Ultimately, we'd like to move the collision detection into QAPISchema*.check(), but for now it is easier just to enhance the existing checks. [*] Of course, down the road, we have plans to rename the simple union tag name to 'type' to match the QMP wire name, but the idea of the collision will still be present even then. Technically, we could avoid the collision by naming the C union members representing each enum value as '_case_value' rather than 'value'; but until we have an actual qapi client (and not just our testsuite) that has a legitimate reason to match a case label to the name of a QMP key and needs the name munging to satisfy the compiler, it's easier to just reject the qapi as invalid. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1443565276-4535-7-git-send-email-eblake@redhat.com> [Polished a few comments] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-12qapi: Clean up qapi.py per pep8Eric Blake
Silence pep8, and make pylint a bit happier. Just style cleanups, plus killing a useless comment in camel_to_upper(); no semantic changes. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1443565276-4535-5-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-12qapi: Invoke exception superclass initializerEric Blake
pylint recommends that every exception class should explicitly invoke the superclass __init__, even though things seem to work fine without it. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1443565276-4535-4-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-12qapi: Improve 'include' error messageEric Blake
Use of '"...%s" % include' to print non-strings can lead to ugly messages, such as this (if the .json change is applied without the qapi.py change): Expected a file name (string), got: OrderedDict() Better is to just omit the actual non-string value in the message. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <1443565276-4535-3-git-send-email-eblake@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-10-12checkpatch: port fix from kernel "## is not a valid modifier"Andy Whitcroft
checkpatch currently loops on fpu/softfloat.c Turns out this is fixed in the Linux version of checkpatch. So this is a port of Andy Whitcrofts fix from Linux, Original commit was commit 89a883530fe7 ("checkpatch: ## is not a valid modifier") As suggested by Peter Maydell for the QEMU version we drop the last "|" as there seems to be no need for that. (FWIW, the kernel discusion about that dried out: http://www.spinics.net/lists/kernel/msg1944421.html ) Cc: Andy Whitcroft <apw@canonical.com> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com> Message-Id: <1444291524-66569-1-git-send-email-borntraeger@de.ibm.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-10-12checkpatch: allow open braces on typedef linesPaolo Bonzini
The style here seems to be split according to the maintainer, but traditionally open braces were placed on typedef lines. Suggested-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-10-09update-linux-headers: Rename SW_MAX to SW_MAX_Markus Armbruster
The next commit will compile hw/input/virtio-input.c and hw/input/virtio-input-hid.c even when CONFIG_LINUX is off. These files include both "include/standard-headers/linux/input.h" and <windows.h> then. Doesn't work, because both define SW_MAX. We don't actually use it. Patch input.h to define SW_MAX_ instead. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <1444320700-26260-2-git-send-email-armbru@redhat.com> Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>