aboutsummaryrefslogtreecommitdiff
path: root/qapi
AgeCommit message (Collapse)Author
2020-08-25qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bitAlberto Garcia
Now that the implementation of subclusters is complete we can finally add the necessary options to create and read images with this feature, which we call "extended L2 entries". Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <6476caaa73216bd05b7bb2d504a20415e1665176.1594396418.git.berto@igalia.com> [mreitz: %s/5\.1/5.2/; fixed 302's and 303's reference output] Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-08-22Merge remote-tracking branch 'remotes/ericb/tags/pull-bitmaps-2020-08-21' ↵Peter Maydell
into staging bitmaps patches for 2020-08-21 - Andrey Shinkevich: Enhance qcow2.py for iotest inspection of qcow2 images - Max Reitz: Add block-bitmap-mapping migration parameter # gpg: Signature made Fri 21 Aug 2020 15:05:07 BST # gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A # gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full] # gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full] # gpg: aka "[jpeg image of size 6874]" [full] # Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A * remotes/ericb/tags/pull-bitmaps-2020-08-21: iotests: Test node/bitmap aliases during migration iotests.py: Let wait_migration() return on failure migration: Add block-bitmap-mapping parameter iotests: dump QCOW2 header in JSON in #303 qcow2_format.py: support dumping metadata in JSON format qcow2_format.py: collect fields to dump in JSON format qcow2.py: Introduce '-j' key to dump in JSON format qcow2_format.py: Dump bitmap table serialized entries qcow2_format.py: pass cluster size to substructures qcow2_format.py: Dump bitmap directory information qcow2_format.py: dump bitmap flags in human readable way. qcow2_format.py: change Qcow2BitmapExt initialization method qcow2_format.py: make printable data an extension class member iotests: add test for QCOW2 header dump Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-08-22Merge remote-tracking branch ↵Peter Maydell
'remotes/ehabkost/tags/machine-next-pull-request' into staging Machine queue 2020-08-19 Regular post-release changes: * hw: add compat machines for 5.2 (Cornelia Huck) Features: * qmp: Expose MachineClass::default_ram_id (Michal Privoznik) Cleanups: * qdev: Document qdev_prop_set_drive_err() return value (Philippe Mathieu-Daudé) # gpg: Signature made Wed 19 Aug 2020 16:21:55 BST # gpg: using RSA key 5A322FD5ABC4D3DBACCFD1AA2807936F984DC5A6 # gpg: issuer "ehabkost@redhat.com" # gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" [full] # Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF D1AA 2807 936F 984D C5A6 * remotes/ehabkost/tags/machine-next-pull-request: hw: add compat machines for 5.2 qmp: Expose MachineClass::default_ram_id qdev: Document qdev_prop_set_drive_err() return value Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-08-21migration: Add block-bitmap-mapping parameterMax Reitz
This migration parameter allows mapping block node names and bitmap names to aliases for the purpose of block dirty bitmap migration. This way, management tools can use different node and bitmap names on the source and destination and pass the mapping of how bitmaps are to be transferred to qemu (on the source, the destination, or even both with arbitrary aliases in the migration stream). While touching this code, fix a bug where bitmap names longer than 255 bytes would fail an assertion in qemu_put_counted_string(). Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200820150725.68687-2-mreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-21meson: convert qapi-specific to mesonMarc-André Lureau
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-08-21libqemuutil, qapi, trace: convert to mesonPaolo Bonzini
This shows how to do some "computations" in meson.build using its array and dictionary data structures, and also a basic usage of the sourceset module for conditional compilation. Notice the new "if have_system" part of util/meson.build, which fixes a bug in the old build system was buggy: util/dbus.c was built even for non-softmmu builds, but the dependency on -lgio was lost when the linking was done through libqemuutil.a. Because all of its users required gio otherwise, the bug was hidden. Meson instead propagates libqemuutil's dependencies down to its users, and shows the problem. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-08-21trace: switch position of headers to what Meson requiresPaolo Bonzini
Meson doesn't enjoy the same flexibility we have with Make in choosing the include path. In particular the tracing headers are using $(build_root)/$(<D). In order to keep the include directives unchanged, the simplest solution is to generate headers with patterns like "trace/trace-audio.h" and place forwarding headers in the source tree such that for example "audio/trace.h" includes "trace/trace-audio.h". This patch is too ugly to be applied to the Makefiles now. It's only a way to separate the changes to the tracing header files from the Meson rewrite of the tracing logic. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-08-19qmp: Expose MachineClass::default_ram_idMichal Privoznik
If a management application (like Libvirt) want's to preserve migration ability and switch to '-machine memory-backend' it needs to set exactly the same RAM id as QEMU would. Since the id is machine type dependant, expose it under 'query-machines' result. Some machine types don't have the attribute set (riscv family for example), therefore the QMP attribute must be optional. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Message-Id: <9384422f63fe594a54d801f9cb4539b1d2ce9b67.1590481402.git.mprivozn@redhat.com> Reviewed-by: Igor Mammedov <imammedo@redhat.com> [ehabkost: updated doc to "since 5.2"] Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2020-08-03schemas: Add vim modelineAndrea Bolognani
The various schemas included in QEMU use a JSON-based format which is, however, strictly speaking not valid JSON. As a consequence, when vim tries to apply syntax highlight rules for JSON (as guessed from the file name), the result is an unreadable mess which mostly consist of red markers pointing out supposed errors in, well, pretty much everything. Using Python syntax highlighting produces much better results, and in fact these files already start with specially-formatted comments that instruct Emacs to process them as if they were Python files. This commit adds the equivalent special comments for vim. Signed-off-by: Andrea Bolognani <abologna@redhat.com> Message-Id: <20200729185024.121766-1-abologna@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-08-03qapi: Delete unwanted indentation of top-level expressionsMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20200730091656.2633334-1-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [One more line de-indented]
2020-08-03qapi/machine.json: Fix missing newline in doc commentPeter Maydell
In commit 176d2cda0dee9f4 we added the @die-id field to the CpuInstanceProperties struct, but in the process accidentally removed the newline between the doc-comment lines for @core-id and @thread-id. Put the newline back in; this fixes a misformatting in the generated HTML QMP reference manual. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Message-Id: <20200729191019.19168-1-peter.maydell@linaro.org> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-07-17Remove VXHS block deviceMarc-André Lureau
The vxhs code doesn't compile since v2.12.0. There's no point in fixing and then adding CI for a config that our users have demonstrated that they do not use; better to just remove it. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20200711065926.2204721-1-marcandre.lureau@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-14file-posix: Mitigate file fragmentation with extent size hintsKevin Wolf
Especially when O_DIRECT is used with image files so that the page cache indirection can't cause a merge of allocating requests, the file will fragment on the file system layer, with a potentially very small fragment size (this depends on the requests the guest sent). On Linux, fragmentation can be reduced by setting an extent size hint when creating the file (at least on XFS, it can't be set any more after the first extent has been allocated), basically giving raw files a "cluster size" for allocation. This adds a create option to set the extent size hint, and changes the default from not setting a hint to setting it to 1 MB. The main reason why qcow2 defaults to smaller cluster sizes is that COW becomes more expensive, which is not an issue with raw files, so we can choose a larger size. The tradeoff here is only potentially wasted disk space. For qcow2 (or other image formats) over file-posix, the advantage should even be greater because they grow sequentially without leaving holes, so there won't be wasted space. Setting even larger extent size hints for such images may make sense. This can be done with the new option, but let's keep the default conservative for now. The effect is very visible with a test that intentionally creates a badly fragmented file with qemu-img bench (the time difference while creating the file is already remarkable) and then looks at the number of extents and the time a simple "qemu-img map" takes. Without an extent size hint: $ ./qemu-img create -f raw -o extent_size_hint=0 ~/tmp/test.raw 10G Formatting '/home/kwolf/tmp/test.raw', fmt=raw size=10737418240 extent_size_hint=0 $ ./qemu-img bench -f raw -t none -n -w ~/tmp/test.raw -c 1000000 -S 8192 -o 0 Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 0, step size 8192) Run completed in 25.848 seconds. $ ./qemu-img bench -f raw -t none -n -w ~/tmp/test.raw -c 1000000 -S 8192 -o 4096 Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 4096, step size 8192) Run completed in 19.616 seconds. $ filefrag ~/tmp/test.raw /home/kwolf/tmp/test.raw: 2000000 extents found $ time ./qemu-img map ~/tmp/test.raw Offset Length Mapped to File 0 0x1e8480000 0 /home/kwolf/tmp/test.raw real 0m1,279s user 0m0,043s sys 0m1,226s With the new default extent size hint of 1 MB: $ ./qemu-img create -f raw -o extent_size_hint=1M ~/tmp/test.raw 10G Formatting '/home/kwolf/tmp/test.raw', fmt=raw size=10737418240 extent_size_hint=1048576 $ ./qemu-img bench -f raw -t none -n -w ~/tmp/test.raw -c 1000000 -S 8192 -o 0 Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 0, step size 8192) Run completed in 11.833 seconds. $ ./qemu-img bench -f raw -t none -n -w ~/tmp/test.raw -c 1000000 -S 8192 -o 4096 Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 4096, step size 8192) Run completed in 10.155 seconds. $ filefrag ~/tmp/test.raw /home/kwolf/tmp/test.raw: 178 extents found $ time ./qemu-img map ~/tmp/test.raw Offset Length Mapped to File 0 0x1e8480000 0 /home/kwolf/tmp/test.raw real 0m0,061s user 0m0,040s sys 0m0,014s Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20200707142329.48303-1-kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-11target/avr: Register AVR support with the rest of QEMUMichael Rolnik
Add AVR related definitions into QEMU, make AVR support buildable. [AM: Remove word 'Atmel' from filenames and all elements of code] Suggested-by: Aleksandar Markovic <aleksandar.m.mail@gmail.com> Signed-off-by: Michael Rolnik <mrolnik@gmail.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Aleksandar Markovic <aleksandar.m.mail@gmail.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com> Signed-off-by: Thomas Huth <huth@tuxfamily.org> Message-Id: <20200705140315.260514-23-huth@tuxfamily.org> [PMD: Fixed @avr tag in qapi/machine.json] Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2020-07-10qapi: Purge error_propagate() from QAPI coreMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200707160613.848843-42-armbru@redhat.com>
2020-07-10error: Eliminate error_propagate() with Coccinelle, part 1Markus Armbruster
When all we do with an Error we receive into a local variable is propagating to somewhere else, we can just as well receive it there right away. Convert if (!foo(..., &err)) { ... error_propagate(errp, err); ... return ... } to if (!foo(..., errp)) { ... ... return ... } where nothing else needs @err. Coccinelle script: @rule1 forall@ identifier fun, err, errp, lbl; expression list args, args2; binary operator op; constant c1, c2; symbol false; @@ if ( ( - fun(args, &err, args2) + fun(args, errp, args2) | - !fun(args, &err, args2) + !fun(args, errp, args2) | - fun(args, &err, args2) op c1 + fun(args, errp, args2) op c1 ) ) { ... when != err when != lbl: when strict - error_propagate(errp, err); ... when != err ( return; | return c2; | return false; ) } @rule2 forall@ identifier fun, err, errp, lbl; expression list args, args2; expression var; binary operator op; constant c1, c2; symbol false; @@ - var = fun(args, &err, args2); + var = fun(args, errp, args2); ... when != err if ( ( var | !var | var op c1 ) ) { ... when != err when != lbl: when strict - error_propagate(errp, err); ... when != err ( return; | return c2; | return false; | return var; ) } @depends on rule1 || rule2@ identifier err; @@ - Error *err = NULL; ... when != err Not exactly elegant, I'm afraid. The "when != lbl:" is necessary to avoid transforming if (fun(args, &err)) { goto out } ... out: error_propagate(errp, err); even though other paths to label out still need the error_propagate(). For an actual example, see sclp_realize(). Without the "when strict", Coccinelle transforms vfio_msix_setup(), incorrectly. I don't know what exactly "when strict" does, only that it helps here. The match of return is narrower than what I want, but I can't figure out how to express "return where the operand doesn't use @err". For an example where it's too narrow, see vfio_intx_enable(). Silently fails to convert hw/arm/armsse.c, because Coccinelle gets confused by ARMSSE being used both as typedef and function-like macro there. Converted manually. Line breaks tidied up manually. One nested declaration of @local_err deleted manually. Preexisting unwanted blank line dropped in hw/riscv/sifive_e.c. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200707160613.848843-35-armbru@redhat.com>
2020-07-10qapi: Make visitor functions taking Error ** return bool, not voidMarkus Armbruster
See recent commit "error: Document Error API usage rules" for rationale. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20200707160613.848843-18-armbru@redhat.com>
2020-07-10qemu-option: Use returned bool to check for failureMarkus Armbruster
The previous commit enables conversion of foo(..., &err); if (err) { ... } to if (!foo(..., &err)) { ... } for QemuOpts functions that now return true / false on success / error. Coccinelle script: @@ identifier fun = { opts_do_parse, parse_option_bool, parse_option_number, parse_option_size, qemu_opt_parse, qemu_opt_rename, qemu_opt_set, qemu_opt_set_bool, qemu_opt_set_number, qemu_opts_absorb_qdict, qemu_opts_do_parse, qemu_opts_from_qdict_entry, qemu_opts_set, qemu_opts_validate }; expression list args, args2; typedef Error; Error *err; @@ - fun(args, &err, args2); - if (err) + if (!fun(args, &err, args2)) { ... } A few line breaks tidied up manually. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20200707160613.848843-15-armbru@redhat.com> [Conflict with commit 0b6786a9c1 "block/amend: refactor qcow2 amend options" resolved by rerunning Coccinelle on master's version]
2020-07-07Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-07-06' ↵Peter Maydell
into staging Block patches for 5.1: - LUKS keyslot amendment (+ patches to make the iotests pass on non-Linux systems, and to keep the tests passing for qcow v1, and to skip LUKS tests (including qcow2 LUKS) when the built qemu does not support it) - Refactoring in the block layer: Drop the basically unnecessary unallocated_blocks_are_zero field from BlockDriverInfo - Fix qcow2 preallocation when the image size is not a multiple of the cluster size - Fix in block-copy code # gpg: Signature made Mon 06 Jul 2020 11:02:53 BST # gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40 # gpg: issuer "mreitz@redhat.com" # gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full] # Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40 * remotes/maxreitz/tags/pull-block-2020-07-06: (31 commits) qed: Simplify backing reads block: drop unallocated_blocks_are_zero block/vhdx: drop unallocated_blocks_are_zero block/file-posix: drop unallocated_blocks_are_zero block/iscsi: drop unallocated_blocks_are_zero block/crypto: drop unallocated_blocks_are_zero block/vpc: return ZERO block-status when appropriate block/vdi: return ZERO block-status when appropriate block: inline bdrv_unallocated_blocks_are_zero() qemu-img: convert: don't use unallocated_blocks_are_zero iotests: add tests for blockdev-amend block/qcow2: implement blockdev-amend block/crypto: implement blockdev-amend block/core: add generic infrastructure for x-blockdev-amend qmp command iotests: qemu-img tests for luks key management block/qcow2: extend qemu-img amend interface with crypto options block/crypto: implement the encryption key management block/crypto: rename two functions block/amend: refactor qcow2 amend options block/amend: separate amend and create options for qemu-img ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-07-07vhost-vdpa: introduce vhost-vdpa net clientCindy Lu
This patch set introduces a new net client type: vhost-vdpa. vhost-vdpa net client will set up a vDPA device which is specified by a "vhostdev" parameter. Signed-off-by: Lingshan Zhu <lingshan.zhu@intel.com> Signed-off-by: Tiwei Bie <tiwei.bie@intel.com> Signed-off-by: Cindy Lu <lulu@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com> Message-Id: <20200701145538.22333-15-lulu@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Acked-by: Jason Wang <jasowang@redhat.com>
2020-07-06block/qcow2: implement blockdev-amendMaxim Levitsky
Currently the implementation only supports amending the encryption options, unlike the qemu-img version Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200608094030.670121-14-mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-07-06block/crypto: implement blockdev-amendMaxim Levitsky
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200608094030.670121-13-mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-07-06block/core: add generic infrastructure for x-blockdev-amend qmp commandMaxim Levitsky
blockdev-amend will be used similiar to blockdev-create to allow on the fly changes of the structure of the format based block devices. Current plan is to first support encryption keyslot management for luks based formats (raw and embedded in qcow2) Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200608094030.670121-12-mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-07-06qcrypto/luks: implement encryption key managementMaxim Levitsky
Next few patches will expose that functionality to the user. Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200608094030.670121-3-mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-07-06qcrypto/core: add generic infrastructure for crypto options amendmentMaxim Levitsky
This will be used first to implement luks keyslot management. block_crypto_amend_opts_init will be used to convert qemu-img cmdline to QCryptoBlockAmendOptions Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200608094030.670121-2-mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-07-03virtio-pci: Send qapi events when the virtio-mem size changesDavid Hildenbrand
Let's register the notifier and trigger the qapi event with the right device id. MEMORY_DEVICE_SIZE_CHANGE is similar to BALLOON_CHANGE, however on a memory device level. Don't unregister the notifier (we neither have finalize() nor unrealize() for VirtIOPCIProxy, so it's not that simple to do it) - both devices are expected to vanish at the same time. Cc: "Michael S. Tsirkin" <mst@redhat.com> Cc: Markus Armbruster <armbru@redhat.com> Cc: "Dr. David Alan Gilbert" <dgilbert@redhat.com> Cc: Eric Blake <eblake@redhat.com> Cc: Igor Mammedov <imammedo@redhat.com> Signed-off-by: David Hildenbrand <david@redhat.com> Message-Id: <20200626072248.78761-18-david@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2020-07-03virtio-mem: Paravirtualized memory hot(un)plugDavid Hildenbrand
This is the very basic/initial version of virtio-mem. An introduction to virtio-mem can be found in the Linux kernel driver [1]. While it can be used in the current state for hotplug of a smaller amount of memory, it will heavily benefit from resizeable memory regions in the future. Each virtio-mem device manages a memory region (provided via a memory backend). After requested by the hypervisor ("requested-size"), the guest can try to plug/unplug blocks of memory within that region, in order to reach the requested size. Initially, and after a reboot, all memory is unplugged (except in special cases - reboot during postcopy). The guest may only try to plug/unplug blocks of memory within the usable region size. The usable region size is a little bigger than the requested size, to give the device driver some flexibility. The usable region size will only grow, except on reboots or when all memory is requested to get unplugged. The guest can never plug more memory than requested. Unplugged memory will get zapped/discarded, similar to in a balloon device. The block size is variable, however, it is always chosen in a way such that THP splits are avoided (e.g., 2MB). The state of each block (plugged/unplugged) is tracked in a bitmap. As virtio-mem devices (e.g., virtio-mem-pci) will be memory devices, we now expose "VirtioMEMDeviceInfo" via "query-memory-devices". -------------------------------------------------------------------------- There are two important follow-up items that are in the works: 1. Resizeable memory regions: Use resizeable allocations/RAM blocks to grow/shrink along with the usable region size. This avoids creating initially very big VMAs, RAM blocks, and KVM slots. 2. Protection of unplugged memory: Make sure the gust cannot actually make use of unplugged memory. Other follow-up items that are in the works: 1. Exclude unplugged memory during migration (via precopy notifier). 2. Handle remapping of memory. 3. Support for other architectures. -------------------------------------------------------------------------- Example usage (virtio-mem-pci is introduced in follow-up patches): Start QEMU with two virtio-mem devices (one per NUMA node): $ qemu-system-x86_64 -m 4G,maxmem=20G \ -smp sockets=2,cores=2 \ -numa node,nodeid=0,cpus=0-1 -numa node,nodeid=1,cpus=2-3 \ [...] -object memory-backend-ram,id=mem0,size=8G \ -device virtio-mem-pci,id=vm0,memdev=mem0,node=0,requested-size=0M \ -object memory-backend-ram,id=mem1,size=8G \ -device virtio-mem-pci,id=vm1,memdev=mem1,node=1,requested-size=1G Query the configuration: (qemu) info memory-devices Memory device [virtio-mem]: "vm0" memaddr: 0x140000000 node: 0 requested-size: 0 size: 0 max-size: 8589934592 block-size: 2097152 memdev: /objects/mem0 Memory device [virtio-mem]: "vm1" memaddr: 0x340000000 node: 1 requested-size: 1073741824 size: 1073741824 max-size: 8589934592 block-size: 2097152 memdev: /objects/mem1 Add some memory to node 0: (qemu) qom-set vm0 requested-size 500M Remove some memory from node 1: (qemu) qom-set vm1 requested-size 200M Query the configuration again: (qemu) info memory-devices Memory device [virtio-mem]: "vm0" memaddr: 0x140000000 node: 0 requested-size: 524288000 size: 524288000 max-size: 8589934592 block-size: 2097152 memdev: /objects/mem0 Memory device [virtio-mem]: "vm1" memaddr: 0x340000000 node: 1 requested-size: 209715200 size: 209715200 max-size: 8589934592 block-size: 2097152 memdev: /objects/mem1 [1] https://lkml.kernel.org/r/20200311171422.10484-1-david@redhat.com Cc: "Michael S. Tsirkin" <mst@redhat.com> Cc: Eric Blake <eblake@redhat.com> Cc: Markus Armbruster <armbru@redhat.com> Cc: "Dr. David Alan Gilbert" <dgilbert@redhat.com> Cc: Igor Mammedov <imammedo@redhat.com> Signed-off-by: David Hildenbrand <david@redhat.com> Message-Id: <20200626072248.78761-11-david@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2020-06-18net: Drop the NetLegacy structure, always use Netdev insteadThomas Huth
Now that the "name" parameter is gone, there is hardly any difference between NetLegacy and Netdev anymore, so we can drop NetLegacy and always use Netdev to simplify the code quite a bit. The only two differences that were really left between Netdev and NetLegacy: 1) NetLegacy does not allow a "hubport" type. We can continue to block this with a simple check in net_client_init1() for this type. 2) The "id" parameter was optional in NetLegacy (and an internal id was chosen via assign_name() during initialization), but it is mandatory for Netdev. To avoid that the visitor code bails out here, we have to add an internal id to the QemuOpts already earlier now. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2020-06-18net: Drop the legacy "name" parameter from the -net optionThomas Huth
It's been deprecated since QEMU v3.1, so it's time to finally remove it. The "id" parameter can simply be used instead. Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2020-06-12pci: Display PCI IRQ pin in "info pci"Peter Xu
Sometimes it would be good to be able to read the pin number along with the IRQ number allocated. Since we'll dump the IRQ number, no reason to not dump the pin information. For example, the vfio-pci device will overwrite the pin with the hardware pin number. It would be nice to know the pin number of one assigned device from QMP/HMP. CC: Dr. David Alan Gilbert <dgilbert@redhat.com> CC: Alex Williamson <alex.williamson@redhat.com> CC: Michael S. Tsirkin <mst@redhat.com> CC: Marcel Apfelbaum <marcel.apfelbaum@gmail.com> CC: Julia Suvorova <jusual@redhat.com> CC: Markus Armbruster <armbru@redhat.com> Signed-off-by: Peter Xu <peterx@redhat.com> Message-Id: <20200317195908.283800-1-peterx@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Acked-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Acked-by: Markus Armbruster <armbru@redhat.com>
2020-05-28qcow2: Expose bitmaps' size during measureEric Blake
It's useful to know how much space can be occupied by qcow2 persistent bitmaps, even though such metadata is unrelated to the guest-visible data. Report this value as an additional QMP field, present when measuring an existing image and output format that both support bitmaps. Update iotest 178 and 190 to updated output, as well as new coverage in 190 demonstrating non-zero values made possible with the recently-added qemu-img bitmap command (see 3b51ab4b). The new 'bitmaps size:' field is displayed automatically as part of 'qemu-img measure' any time it is present in QMP (that is, any time both the source image being measured and destination format support bitmaps, even if the measurement is 0 because there are no bitmaps present). If the field is absent, it means that no bitmaps can be copied (source, destination, or both lack bitmaps, including when measuring based on size rather than on a source image). This behavior is compatible with an upcoming patch adding 'qemu-img convert --bitmaps': that command will fail in the same situations where this patch omits the field. The addition of a new field demonstrates why we should always zero-initialize qapi C structs; while the qcow2 driver still fully populates all fields, the raw and crypto drivers had to be tweaked to avoid uninitialized data. Consideration was also given towards having a 'qemu-img measure --bitmaps' which errors out when bitmaps are not possible, and otherwise sums the bitmaps into the existing allocation totals rather than displaying as a separate field, as a potential convenience factor. But this was ultimately decided to be more complexity than necessary when the QMP interface was sufficient enough with bitmaps remaining a separate field. See also: https://bugzilla.redhat.com/1779904 Reported-by: Nir Soffer <nsoffer@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200521192137.1120211-3-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2020-05-25audio/jack: add JACK client audiodevGeoffrey McRae
This commit adds a new audiodev backend to allow QEMU to use JACK as both an audio sink and source. Signed-off-by: Geoffrey McRae <geoff@hostfission.com> Message-Id: <20200512101603.E3DB73A038E@moya.office.hostfission.com> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2020-05-20qemu-sockets: add abstract UNIX domain socket supportxiaoqiang zhao
unix_listen/connect_saddr now support abstract address types two aditional BOOL switches are introduced: tight: whether to set @addrlen to the minimal string length, or the maximum sun_path length. default is TRUE abstract: whether we use abstract address. default is FALSE cli example: -monitor unix:/tmp/unix.socket,abstract,tight=off OR -chardev socket,path=/tmp/unix.socket,id=unix1,abstract,tight=on Signed-off-by: xiaoqiang zhao <zxq_yx_007@163.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-05-13qcow2: add zstd cluster compressionDenis Plotnikov
zstd significantly reduces cluster compression time. It provides better compression performance maintaining the same level of the compression ratio in comparison with zlib, which, at the moment, is the only compression method available. The performance test results: Test compresses and decompresses qemu qcow2 image with just installed rhel-7.6 guest. Image cluster size: 64K. Image on disk size: 2.2G The test was conducted with brd disk to reduce the influence of disk subsystem to the test results. The results is given in seconds. compress cmd: time ./qemu-img convert -O qcow2 -c -o compression_type=[zlib|zstd] src.img [zlib|zstd]_compressed.img decompress cmd time ./qemu-img convert -O qcow2 [zlib|zstd]_compressed.img uncompressed.img compression decompression zlib zstd zlib zstd ------------------------------------------------------------ real 65.5 16.3 (-75 %) 1.9 1.6 (-16 %) user 65.0 15.8 5.3 2.5 sys 3.3 0.2 2.0 2.0 Both ZLIB and ZSTD gave the same compression ratio: 1.57 compressed image size in both cases: 1.4G Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> QAPI part: Acked-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20200507082521.29210-4-dplotnikov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-05-13qcow2: introduce compression type featureDenis Plotnikov
The patch adds some preparation parts for incompatible compression type feature to qcow2 allowing the use different compression methods for image clusters (de)compressing. It is implied that the compression type is set on the image creation and can be changed only later by image conversion, thus compression type defines the only compression algorithm used for the image, and thus, for all image clusters. The goal of the feature is to add support of other compression methods to qcow2. For example, ZSTD which is more effective on compression than ZLIB. The default compression is ZLIB. Images created with ZLIB compression type are backward compatible with older qemu versions. Adding of the compression type breaks a number of tests because now the compression type is reported on image creation and there are some changes in the qcow2 header in size and offsets. The tests are fixed in the following ways: * filter out compression_type for many tests * fix header size, feature table size and backing file offset affected tests: 031, 036, 061, 080 header_size +=8: 1 byte compression type 7 bytes padding feature_table += 48: incompatible feature compression type backing_file_offset += 56 (8 + 48 -> header_change + feature_table_change) * add "compression type" for test output matching when it isn't filtered affected tests: 049, 060, 061, 065, 082, 085, 144, 182, 185, 198, 206, 242, 255, 274, 280 Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> QAPI part: Acked-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20200507082521.29210-2-dplotnikov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-05-07migration/xbzrle: add encoding rateWei Wang
Users may need to check the xbzrle encoding rate to know if the guest memory is xbzrle encoding-friendly, and dynamically turn off the encoding if the encoding rate is low. Signed-off-by: Yi Sun <yi.y.sun@intel.com> Signed-off-by: Wei Wang <wei.w.wang@intel.com> Message-Id: <1588208375-19556-1-git-send-email-wei.w.wang@intel.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-05-07migration/throttle: Add cpu-throttle-tailslow migration parameterKeqian Zhu
At the tail stage of throttling, the Guest is very sensitive to CPU percentage while the @cpu-throttle-increment is excessive usually at tail stage. If this parameter is true, we will compute the ideal CPU percentage used by the Guest, which may exactly make the dirty rate match the dirty rate threshold. Then we will choose a smaller throttle increment between the one specified by @cpu-throttle-increment and the one generated by ideal CPU percentage. Therefore, it is compatible to traditional throttling, meanwhile the throttle increment won't be excessive at tail stage. This may make migration time longer, and is disabled by default. Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com> Message-Id: <20200413101508.54793-1-zhukeqian1@huawei.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-04-30qapi: Assert non-input visitors see only valid narrow integersMarkus Armbruster
visit_type_intN() and visit_type_uintN() fail when the value is out of bounds. This is appropriate with an input visitor: the value comes from input, and input may be bad. It should never happen with the other visitors: the value comes from the caller, and callers must keep it within bounds. Assert that. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200424084338.26803-10-armbru@redhat.com>
2020-04-30qapi: Assert output visitors see only valid enum valuesMarkus Armbruster
output_type_enum() fails when *obj is not a valid value of the enum type. Should not happen. Drop the check, along with its unit tests. This unmasks qapi_enum_lookup()'s assertion. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200424084338.26803-9-armbru@redhat.com> [Commit message tweaked]
2020-04-30qapi: Fix Visitor contract for start_alternate()Markus Armbruster
The contract demands v->start_alternate() for input and dealloc visitors, but visit_start_alternate() actually requires it for input and clone visitors. Fix the contract, and delete superfluous qapi_dealloc_start_alternate(). Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200424084338.26803-8-armbru@redhat.com>
2020-04-30qapi: Assert incomplete object occurs only in dealloc visitorMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200424084338.26803-7-armbru@redhat.com>
2020-04-30qobject: Eliminate qdict_iter(), use qdict_first(), qdict_next()Markus Armbruster
qdict_iter() has just three uses and no test coverage. Replace by qdict_first(), qdict_next() for more concise code and less type punning. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20200415083048.14339-5-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2020-04-02qmp: fix leak on callbacks that return both value and errorMarc-André Lureau
Direct leak of 4120 byte(s) in 1 object(s) allocated from: #0 0x7fa114931887 in __interceptor_calloc (/lib64/libasan.so.6+0xb0887) #1 0x7fa1144ad8f0 in g_malloc0 (/lib64/libglib-2.0.so.0+0x588f0) #2 0x561e3c9c8897 in qmp_object_add /home/elmarco/src/qemu/qom/qom-qmp-cmds.c:291 #3 0x561e3cf48736 in qmp_dispatch /home/elmarco/src/qemu/qapi/qmp-dispatch.c:155 #4 0x561e3c8efb36 in monitor_qmp_dispatch /home/elmarco/src/qemu/monitor/qmp.c:145 #5 0x561e3c8f09ed in monitor_qmp_bh_dispatcher /home/elmarco/src/qemu/monitor/qmp.c:234 #6 0x561e3d08c993 in aio_bh_call /home/elmarco/src/qemu/util/async.c:136 #7 0x561e3d08d0a5 in aio_bh_poll /home/elmarco/src/qemu/util/async.c:164 #8 0x561e3d0a535a in aio_dispatch /home/elmarco/src/qemu/util/aio-posix.c:380 #9 0x561e3d08e3ca in aio_ctx_dispatch /home/elmarco/src/qemu/util/async.c:298 #10 0x7fa1144a776e in g_main_context_dispatch (/lib64/libglib-2.0.so.0+0x5276e) Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20200325184723.2029630-3-marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-03-19Add rx-softmmuYoshinori Sato
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Yoshinori Sato <ysato@users.sourceforge.jp> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> [PMD: Squashed patches from Richard Henderson modifying qapi/common.json and tests/machine-none-test.c] Message-Id: <20200224141923.82118-21-ysato@users.sourceforge.jp> [PMD: Added @since 5.0 tag in SysEmuTarget] Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2020-03-17net: Complete qapi-fication of netdev_addEric Blake
We've had all the required pieces for doing a type-safe representation of netdev_add as a flat union for quite some time now (since 0e55c381f6 in v2.7.0, released in 2016), but did not make the final switch to using it because of concern about whether a command-line regression in accepting "1" in place of 1 for integer arguments would be problematic. Back then, we did not have the deprecation cycle to allow us to make progress. But now that we have waited so long, other problems have crept in: for example, our desire to add qemu-storage-daemon is hampered by the inability to express net objects, and we are unable to introspect what we actually accept. Additionally, our round-trip through QemuOpts silently eats any argument that expands to an array, rendering dnssearch, hostfwd, and guestfwd useless through QMP: {"execute": "netdev_add", "arguments": { "id": "netdev0", "type": "user", "dnssearch": [ { "str": "8.8.8.8" }, { "str": "8.8.4.4" } ]}} So without further ado, let's turn on proper QAPI. netdev_add() was a trivial wrapper around net_client_init(), which did a few steps prior to calling net_client_init1(); with this patch, we now skip directly to net_client_init1(). In addition to fixing array parameters, the following additional differences occur: - {"execute": "netdev_add", "arguments": {"type": "help"}} no longer attempts to print help to stdout and exit. Bug fix, broken in 547203ead4 'net: List available netdevs with "-netdev help"', v2.12.0. - {"execute": "netdev_add", "arguments': {... "ipv6-net": "..." }} no longer attempts to desugar the undocumented ipv6-net magic string into the proper "ipv6-prefix" and "ipv6-prefixlen". Undocumented misfeature, introduced in commit 7aac531ef2 "qapi-schema, qemu-options & slirp: Adding Qemu options for IPv6 addresses", v2.6.0. - {'execute':'netdev_add', 'arguments':{'id':'net2', 'type':'hubport', 'hubid':"2"}} {"error": {"class": "GenericError", "desc": "Invalid parameter type for 'hubid', expected: integer"}} Used to succeed: since our command line treats everything as strings, our not-so-round-trip conversion from QAPI -> QemuOpts -> QAPI lost the original typing and turned everything into a string; now that we skip the QemuOpts, the JSON input has to match the exact QAPI type. But this stricter QMP is desirable, and introspection is sufficient for any affected applications to make sure they use it correctly. In qmp_netdev_add(), we still have to create a QemuOpts object so that qmp_netdev_del() will be able to remove a hotplugged network device; but the opts->head remains empty since we now manage all parsing through the QAPI object rather than QemuOpts; a separate patch will address the abuse of QemuOpts as a witness for whether a NetClientState is a netdev. In the meantime, our argument that we are okay requires auditing all uses of option group "netdev": - qemu_netdev_opts: option group definition, empty .desc[] - CLI (CLI netdev parsing ends before monitors start, so while monitors can mess with CLI netdevs, CLI cannot mess with monitor netdevs): - main() case QEMU_OPTION_netdev: store CLI definition - main() case QEMU_OPTION_readconfig, case QEMU_OPTION_writeconfig: similar, dealing only with CLI - net_init_clients(): Pass CLI to net_client_init() - Monitor: - hmp_netdev_add(): straightforward parse into net_client_init() - qmp_netdev_add(): subject of this patch, used to add full object to option group, now just adds bare-bones id - qmp_netdev_del(), netdev_del_completion(): check the option group solely for id, as a 'is this a netdev' predicate Reported-by: Alex Kirillov <lekiravi@yandex-team.ru> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200317201711.322764-2-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Commit message typo fixed] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-03-17qmp: constify QmpCommand and listMarc-André Lureau
Since 0b69f6f72ce47a37a749b056b6d5ec64c61f11e8 "qapi: remove qmp_unregister_command()", the command list can be declared const. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Damien Hedde <damien.hedde@greensocs.com> Message-Id: <20200316171824.2319695-1-marcandre.lureau@redhat.com> [Rebased] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-03-17qapi: Mark deprecated QMP parts with feature 'deprecated'Markus Armbruster
Add feature 'deprecated' to the deprecated QMP commands, so their deprecation becomes visible in output of query-qmp-schema. Looks like this: {"name": "query-cpus", "ret-type": "[164]", "meta-type": "command", "arg-type": "0", ---> "features": ["deprecated"]} Management applications could conceivably use this for static checking. The deprecated commands are change, cpu-add, migrate-set-cache-size, migrate_set_downtime, migrate_set_speed, query-cpus, query-events, query-migrate-cache-size. The deprecated command arguments are block-commit arguments @base and @top, and block_set_io_throttle, blockdev-change-medium, blockdev-close-tray, blockdev-open-tray, eject argument @device. The deprecated command results are query-cpus-fast result @arch, query-block result @dirty-bitmaps, query-named-block-nodes result @encryption_key_missing and result @dirty-bitmaps's member @status. Same for query-block result @inserted, which mirrors query-named-block-nodes. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20200317115459.31821-27-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-17qapi: Replace qmp_dispatch()'s TODO comment by an explanationMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20200317115459.31821-25-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-17qapi: Simplify how qmp_dispatch() gets the request IDMarkus Armbruster
We convert the request object to a QDict twice: first in qmp_dispatch() to get the request ID, and then again in qmp_dispatch_check_obj(), which converts to QDict, then checks and returns it. We can't get the request ID from the latter, because it's null when the qdict flunks the checks. Move the checked conversion to QDict from qmp_dispatch_check_obj() to qmp_dispatch(), and drop the duplicate there. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20200317115459.31821-24-armbru@redhat.com>
2020-03-17qapi: Simplify how qmp_dispatch() deals with QCO_NO_SUCCESS_RESPMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20200317115459.31821-23-armbru@redhat.com>
generated by cgit v1.2.3 (git 2.26.2) at 2024-12-03 13:17:26 +0000