aboutsummaryrefslogtreecommitdiff
path: root/qapi
AgeCommit message (Collapse)Author
2020-07-14file-posix: Mitigate file fragmentation with extent size hintsKevin Wolf
Especially when O_DIRECT is used with image files so that the page cache indirection can't cause a merge of allocating requests, the file will fragment on the file system layer, with a potentially very small fragment size (this depends on the requests the guest sent). On Linux, fragmentation can be reduced by setting an extent size hint when creating the file (at least on XFS, it can't be set any more after the first extent has been allocated), basically giving raw files a "cluster size" for allocation. This adds a create option to set the extent size hint, and changes the default from not setting a hint to setting it to 1 MB. The main reason why qcow2 defaults to smaller cluster sizes is that COW becomes more expensive, which is not an issue with raw files, so we can choose a larger size. The tradeoff here is only potentially wasted disk space. For qcow2 (or other image formats) over file-posix, the advantage should even be greater because they grow sequentially without leaving holes, so there won't be wasted space. Setting even larger extent size hints for such images may make sense. This can be done with the new option, but let's keep the default conservative for now. The effect is very visible with a test that intentionally creates a badly fragmented file with qemu-img bench (the time difference while creating the file is already remarkable) and then looks at the number of extents and the time a simple "qemu-img map" takes. Without an extent size hint: $ ./qemu-img create -f raw -o extent_size_hint=0 ~/tmp/test.raw 10G Formatting '/home/kwolf/tmp/test.raw', fmt=raw size=10737418240 extent_size_hint=0 $ ./qemu-img bench -f raw -t none -n -w ~/tmp/test.raw -c 1000000 -S 8192 -o 0 Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 0, step size 8192) Run completed in 25.848 seconds. $ ./qemu-img bench -f raw -t none -n -w ~/tmp/test.raw -c 1000000 -S 8192 -o 4096 Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 4096, step size 8192) Run completed in 19.616 seconds. $ filefrag ~/tmp/test.raw /home/kwolf/tmp/test.raw: 2000000 extents found $ time ./qemu-img map ~/tmp/test.raw Offset Length Mapped to File 0 0x1e8480000 0 /home/kwolf/tmp/test.raw real 0m1,279s user 0m0,043s sys 0m1,226s With the new default extent size hint of 1 MB: $ ./qemu-img create -f raw -o extent_size_hint=1M ~/tmp/test.raw 10G Formatting '/home/kwolf/tmp/test.raw', fmt=raw size=10737418240 extent_size_hint=1048576 $ ./qemu-img bench -f raw -t none -n -w ~/tmp/test.raw -c 1000000 -S 8192 -o 0 Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 0, step size 8192) Run completed in 11.833 seconds. $ ./qemu-img bench -f raw -t none -n -w ~/tmp/test.raw -c 1000000 -S 8192 -o 4096 Sending 1000000 write requests, 4096 bytes each, 64 in parallel (starting at offset 4096, step size 8192) Run completed in 10.155 seconds. $ filefrag ~/tmp/test.raw /home/kwolf/tmp/test.raw: 178 extents found $ time ./qemu-img map ~/tmp/test.raw Offset Length Mapped to File 0 0x1e8480000 0 /home/kwolf/tmp/test.raw real 0m0,061s user 0m0,040s sys 0m0,014s Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20200707142329.48303-1-kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-11target/avr: Register AVR support with the rest of QEMUMichael Rolnik
Add AVR related definitions into QEMU, make AVR support buildable. [AM: Remove word 'Atmel' from filenames and all elements of code] Suggested-by: Aleksandar Markovic <aleksandar.m.mail@gmail.com> Signed-off-by: Michael Rolnik <mrolnik@gmail.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Aleksandar Markovic <aleksandar.m.mail@gmail.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com> Signed-off-by: Thomas Huth <huth@tuxfamily.org> Message-Id: <20200705140315.260514-23-huth@tuxfamily.org> [PMD: Fixed @avr tag in qapi/machine.json] Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2020-07-10qapi: Purge error_propagate() from QAPI coreMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200707160613.848843-42-armbru@redhat.com>
2020-07-10error: Eliminate error_propagate() with Coccinelle, part 1Markus Armbruster
When all we do with an Error we receive into a local variable is propagating to somewhere else, we can just as well receive it there right away. Convert if (!foo(..., &err)) { ... error_propagate(errp, err); ... return ... } to if (!foo(..., errp)) { ... ... return ... } where nothing else needs @err. Coccinelle script: @rule1 forall@ identifier fun, err, errp, lbl; expression list args, args2; binary operator op; constant c1, c2; symbol false; @@ if ( ( - fun(args, &err, args2) + fun(args, errp, args2) | - !fun(args, &err, args2) + !fun(args, errp, args2) | - fun(args, &err, args2) op c1 + fun(args, errp, args2) op c1 ) ) { ... when != err when != lbl: when strict - error_propagate(errp, err); ... when != err ( return; | return c2; | return false; ) } @rule2 forall@ identifier fun, err, errp, lbl; expression list args, args2; expression var; binary operator op; constant c1, c2; symbol false; @@ - var = fun(args, &err, args2); + var = fun(args, errp, args2); ... when != err if ( ( var | !var | var op c1 ) ) { ... when != err when != lbl: when strict - error_propagate(errp, err); ... when != err ( return; | return c2; | return false; | return var; ) } @depends on rule1 || rule2@ identifier err; @@ - Error *err = NULL; ... when != err Not exactly elegant, I'm afraid. The "when != lbl:" is necessary to avoid transforming if (fun(args, &err)) { goto out } ... out: error_propagate(errp, err); even though other paths to label out still need the error_propagate(). For an actual example, see sclp_realize(). Without the "when strict", Coccinelle transforms vfio_msix_setup(), incorrectly. I don't know what exactly "when strict" does, only that it helps here. The match of return is narrower than what I want, but I can't figure out how to express "return where the operand doesn't use @err". For an example where it's too narrow, see vfio_intx_enable(). Silently fails to convert hw/arm/armsse.c, because Coccinelle gets confused by ARMSSE being used both as typedef and function-like macro there. Converted manually. Line breaks tidied up manually. One nested declaration of @local_err deleted manually. Preexisting unwanted blank line dropped in hw/riscv/sifive_e.c. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200707160613.848843-35-armbru@redhat.com>
2020-07-10qapi: Make visitor functions taking Error ** return bool, not voidMarkus Armbruster
See recent commit "error: Document Error API usage rules" for rationale. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20200707160613.848843-18-armbru@redhat.com>
2020-07-10qemu-option: Use returned bool to check for failureMarkus Armbruster
The previous commit enables conversion of foo(..., &err); if (err) { ... } to if (!foo(..., &err)) { ... } for QemuOpts functions that now return true / false on success / error. Coccinelle script: @@ identifier fun = { opts_do_parse, parse_option_bool, parse_option_number, parse_option_size, qemu_opt_parse, qemu_opt_rename, qemu_opt_set, qemu_opt_set_bool, qemu_opt_set_number, qemu_opts_absorb_qdict, qemu_opts_do_parse, qemu_opts_from_qdict_entry, qemu_opts_set, qemu_opts_validate }; expression list args, args2; typedef Error; Error *err; @@ - fun(args, &err, args2); - if (err) + if (!fun(args, &err, args2)) { ... } A few line breaks tidied up manually. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20200707160613.848843-15-armbru@redhat.com> [Conflict with commit 0b6786a9c1 "block/amend: refactor qcow2 amend options" resolved by rerunning Coccinelle on master's version]
2020-07-07Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-07-06' ↵Peter Maydell
into staging Block patches for 5.1: - LUKS keyslot amendment (+ patches to make the iotests pass on non-Linux systems, and to keep the tests passing for qcow v1, and to skip LUKS tests (including qcow2 LUKS) when the built qemu does not support it) - Refactoring in the block layer: Drop the basically unnecessary unallocated_blocks_are_zero field from BlockDriverInfo - Fix qcow2 preallocation when the image size is not a multiple of the cluster size - Fix in block-copy code # gpg: Signature made Mon 06 Jul 2020 11:02:53 BST # gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40 # gpg: issuer "mreitz@redhat.com" # gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full] # Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40 * remotes/maxreitz/tags/pull-block-2020-07-06: (31 commits) qed: Simplify backing reads block: drop unallocated_blocks_are_zero block/vhdx: drop unallocated_blocks_are_zero block/file-posix: drop unallocated_blocks_are_zero block/iscsi: drop unallocated_blocks_are_zero block/crypto: drop unallocated_blocks_are_zero block/vpc: return ZERO block-status when appropriate block/vdi: return ZERO block-status when appropriate block: inline bdrv_unallocated_blocks_are_zero() qemu-img: convert: don't use unallocated_blocks_are_zero iotests: add tests for blockdev-amend block/qcow2: implement blockdev-amend block/crypto: implement blockdev-amend block/core: add generic infrastructure for x-blockdev-amend qmp command iotests: qemu-img tests for luks key management block/qcow2: extend qemu-img amend interface with crypto options block/crypto: implement the encryption key management block/crypto: rename two functions block/amend: refactor qcow2 amend options block/amend: separate amend and create options for qemu-img ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-07-07vhost-vdpa: introduce vhost-vdpa net clientCindy Lu
This patch set introduces a new net client type: vhost-vdpa. vhost-vdpa net client will set up a vDPA device which is specified by a "vhostdev" parameter. Signed-off-by: Lingshan Zhu <lingshan.zhu@intel.com> Signed-off-by: Tiwei Bie <tiwei.bie@intel.com> Signed-off-by: Cindy Lu <lulu@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com> Message-Id: <20200701145538.22333-15-lulu@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Acked-by: Jason Wang <jasowang@redhat.com>
2020-07-06block/qcow2: implement blockdev-amendMaxim Levitsky
Currently the implementation only supports amending the encryption options, unlike the qemu-img version Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200608094030.670121-14-mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-07-06block/crypto: implement blockdev-amendMaxim Levitsky
Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200608094030.670121-13-mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-07-06block/core: add generic infrastructure for x-blockdev-amend qmp commandMaxim Levitsky
blockdev-amend will be used similiar to blockdev-create to allow on the fly changes of the structure of the format based block devices. Current plan is to first support encryption keyslot management for luks based formats (raw and embedded in qcow2) Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200608094030.670121-12-mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-07-06qcrypto/luks: implement encryption key managementMaxim Levitsky
Next few patches will expose that functionality to the user. Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200608094030.670121-3-mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-07-06qcrypto/core: add generic infrastructure for crypto options amendmentMaxim Levitsky
This will be used first to implement luks keyslot management. block_crypto_amend_opts_init will be used to convert qemu-img cmdline to QCryptoBlockAmendOptions Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200608094030.670121-2-mlevitsk@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-07-03virtio-pci: Send qapi events when the virtio-mem size changesDavid Hildenbrand
Let's register the notifier and trigger the qapi event with the right device id. MEMORY_DEVICE_SIZE_CHANGE is similar to BALLOON_CHANGE, however on a memory device level. Don't unregister the notifier (we neither have finalize() nor unrealize() for VirtIOPCIProxy, so it's not that simple to do it) - both devices are expected to vanish at the same time. Cc: "Michael S. Tsirkin" <mst@redhat.com> Cc: Markus Armbruster <armbru@redhat.com> Cc: "Dr. David Alan Gilbert" <dgilbert@redhat.com> Cc: Eric Blake <eblake@redhat.com> Cc: Igor Mammedov <imammedo@redhat.com> Signed-off-by: David Hildenbrand <david@redhat.com> Message-Id: <20200626072248.78761-18-david@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2020-07-03virtio-mem: Paravirtualized memory hot(un)plugDavid Hildenbrand
This is the very basic/initial version of virtio-mem. An introduction to virtio-mem can be found in the Linux kernel driver [1]. While it can be used in the current state for hotplug of a smaller amount of memory, it will heavily benefit from resizeable memory regions in the future. Each virtio-mem device manages a memory region (provided via a memory backend). After requested by the hypervisor ("requested-size"), the guest can try to plug/unplug blocks of memory within that region, in order to reach the requested size. Initially, and after a reboot, all memory is unplugged (except in special cases - reboot during postcopy). The guest may only try to plug/unplug blocks of memory within the usable region size. The usable region size is a little bigger than the requested size, to give the device driver some flexibility. The usable region size will only grow, except on reboots or when all memory is requested to get unplugged. The guest can never plug more memory than requested. Unplugged memory will get zapped/discarded, similar to in a balloon device. The block size is variable, however, it is always chosen in a way such that THP splits are avoided (e.g., 2MB). The state of each block (plugged/unplugged) is tracked in a bitmap. As virtio-mem devices (e.g., virtio-mem-pci) will be memory devices, we now expose "VirtioMEMDeviceInfo" via "query-memory-devices". -------------------------------------------------------------------------- There are two important follow-up items that are in the works: 1. Resizeable memory regions: Use resizeable allocations/RAM blocks to grow/shrink along with the usable region size. This avoids creating initially very big VMAs, RAM blocks, and KVM slots. 2. Protection of unplugged memory: Make sure the gust cannot actually make use of unplugged memory. Other follow-up items that are in the works: 1. Exclude unplugged memory during migration (via precopy notifier). 2. Handle remapping of memory. 3. Support for other architectures. -------------------------------------------------------------------------- Example usage (virtio-mem-pci is introduced in follow-up patches): Start QEMU with two virtio-mem devices (one per NUMA node): $ qemu-system-x86_64 -m 4G,maxmem=20G \ -smp sockets=2,cores=2 \ -numa node,nodeid=0,cpus=0-1 -numa node,nodeid=1,cpus=2-3 \ [...] -object memory-backend-ram,id=mem0,size=8G \ -device virtio-mem-pci,id=vm0,memdev=mem0,node=0,requested-size=0M \ -object memory-backend-ram,id=mem1,size=8G \ -device virtio-mem-pci,id=vm1,memdev=mem1,node=1,requested-size=1G Query the configuration: (qemu) info memory-devices Memory device [virtio-mem]: "vm0" memaddr: 0x140000000 node: 0 requested-size: 0 size: 0 max-size: 8589934592 block-size: 2097152 memdev: /objects/mem0 Memory device [virtio-mem]: "vm1" memaddr: 0x340000000 node: 1 requested-size: 1073741824 size: 1073741824 max-size: 8589934592 block-size: 2097152 memdev: /objects/mem1 Add some memory to node 0: (qemu) qom-set vm0 requested-size 500M Remove some memory from node 1: (qemu) qom-set vm1 requested-size 200M Query the configuration again: (qemu) info memory-devices Memory device [virtio-mem]: "vm0" memaddr: 0x140000000 node: 0 requested-size: 524288000 size: 524288000 max-size: 8589934592 block-size: 2097152 memdev: /objects/mem0 Memory device [virtio-mem]: "vm1" memaddr: 0x340000000 node: 1 requested-size: 209715200 size: 209715200 max-size: 8589934592 block-size: 2097152 memdev: /objects/mem1 [1] https://lkml.kernel.org/r/20200311171422.10484-1-david@redhat.com Cc: "Michael S. Tsirkin" <mst@redhat.com> Cc: Eric Blake <eblake@redhat.com> Cc: Markus Armbruster <armbru@redhat.com> Cc: "Dr. David Alan Gilbert" <dgilbert@redhat.com> Cc: Igor Mammedov <imammedo@redhat.com> Signed-off-by: David Hildenbrand <david@redhat.com> Message-Id: <20200626072248.78761-11-david@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2020-06-18net: Drop the NetLegacy structure, always use Netdev insteadThomas Huth
Now that the "name" parameter is gone, there is hardly any difference between NetLegacy and Netdev anymore, so we can drop NetLegacy and always use Netdev to simplify the code quite a bit. The only two differences that were really left between Netdev and NetLegacy: 1) NetLegacy does not allow a "hubport" type. We can continue to block this with a simple check in net_client_init1() for this type. 2) The "id" parameter was optional in NetLegacy (and an internal id was chosen via assign_name() during initialization), but it is mandatory for Netdev. To avoid that the visitor code bails out here, we have to add an internal id to the QemuOpts already earlier now. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2020-06-18net: Drop the legacy "name" parameter from the -net optionThomas Huth
It's been deprecated since QEMU v3.1, so it's time to finally remove it. The "id" parameter can simply be used instead. Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2020-06-12pci: Display PCI IRQ pin in "info pci"Peter Xu
Sometimes it would be good to be able to read the pin number along with the IRQ number allocated. Since we'll dump the IRQ number, no reason to not dump the pin information. For example, the vfio-pci device will overwrite the pin with the hardware pin number. It would be nice to know the pin number of one assigned device from QMP/HMP. CC: Dr. David Alan Gilbert <dgilbert@redhat.com> CC: Alex Williamson <alex.williamson@redhat.com> CC: Michael S. Tsirkin <mst@redhat.com> CC: Marcel Apfelbaum <marcel.apfelbaum@gmail.com> CC: Julia Suvorova <jusual@redhat.com> CC: Markus Armbruster <armbru@redhat.com> Signed-off-by: Peter Xu <peterx@redhat.com> Message-Id: <20200317195908.283800-1-peterx@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Acked-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Acked-by: Markus Armbruster <armbru@redhat.com>
2020-05-28qcow2: Expose bitmaps' size during measureEric Blake
It's useful to know how much space can be occupied by qcow2 persistent bitmaps, even though such metadata is unrelated to the guest-visible data. Report this value as an additional QMP field, present when measuring an existing image and output format that both support bitmaps. Update iotest 178 and 190 to updated output, as well as new coverage in 190 demonstrating non-zero values made possible with the recently-added qemu-img bitmap command (see 3b51ab4b). The new 'bitmaps size:' field is displayed automatically as part of 'qemu-img measure' any time it is present in QMP (that is, any time both the source image being measured and destination format support bitmaps, even if the measurement is 0 because there are no bitmaps present). If the field is absent, it means that no bitmaps can be copied (source, destination, or both lack bitmaps, including when measuring based on size rather than on a source image). This behavior is compatible with an upcoming patch adding 'qemu-img convert --bitmaps': that command will fail in the same situations where this patch omits the field. The addition of a new field demonstrates why we should always zero-initialize qapi C structs; while the qcow2 driver still fully populates all fields, the raw and crypto drivers had to be tweaked to avoid uninitialized data. Consideration was also given towards having a 'qemu-img measure --bitmaps' which errors out when bitmaps are not possible, and otherwise sums the bitmaps into the existing allocation totals rather than displaying as a separate field, as a potential convenience factor. But this was ultimately decided to be more complexity than necessary when the QMP interface was sufficient enough with bitmaps remaining a separate field. See also: https://bugzilla.redhat.com/1779904 Reported-by: Nir Soffer <nsoffer@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200521192137.1120211-3-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2020-05-25audio/jack: add JACK client audiodevGeoffrey McRae
This commit adds a new audiodev backend to allow QEMU to use JACK as both an audio sink and source. Signed-off-by: Geoffrey McRae <geoff@hostfission.com> Message-Id: <20200512101603.E3DB73A038E@moya.office.hostfission.com> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2020-05-20qemu-sockets: add abstract UNIX domain socket supportxiaoqiang zhao
unix_listen/connect_saddr now support abstract address types two aditional BOOL switches are introduced: tight: whether to set @addrlen to the minimal string length, or the maximum sun_path length. default is TRUE abstract: whether we use abstract address. default is FALSE cli example: -monitor unix:/tmp/unix.socket,abstract,tight=off OR -chardev socket,path=/tmp/unix.socket,id=unix1,abstract,tight=on Signed-off-by: xiaoqiang zhao <zxq_yx_007@163.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2020-05-13qcow2: add zstd cluster compressionDenis Plotnikov
zstd significantly reduces cluster compression time. It provides better compression performance maintaining the same level of the compression ratio in comparison with zlib, which, at the moment, is the only compression method available. The performance test results: Test compresses and decompresses qemu qcow2 image with just installed rhel-7.6 guest. Image cluster size: 64K. Image on disk size: 2.2G The test was conducted with brd disk to reduce the influence of disk subsystem to the test results. The results is given in seconds. compress cmd: time ./qemu-img convert -O qcow2 -c -o compression_type=[zlib|zstd] src.img [zlib|zstd]_compressed.img decompress cmd time ./qemu-img convert -O qcow2 [zlib|zstd]_compressed.img uncompressed.img compression decompression zlib zstd zlib zstd ------------------------------------------------------------ real 65.5 16.3 (-75 %) 1.9 1.6 (-16 %) user 65.0 15.8 5.3 2.5 sys 3.3 0.2 2.0 2.0 Both ZLIB and ZSTD gave the same compression ratio: 1.57 compressed image size in both cases: 1.4G Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> QAPI part: Acked-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20200507082521.29210-4-dplotnikov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-05-13qcow2: introduce compression type featureDenis Plotnikov
The patch adds some preparation parts for incompatible compression type feature to qcow2 allowing the use different compression methods for image clusters (de)compressing. It is implied that the compression type is set on the image creation and can be changed only later by image conversion, thus compression type defines the only compression algorithm used for the image, and thus, for all image clusters. The goal of the feature is to add support of other compression methods to qcow2. For example, ZSTD which is more effective on compression than ZLIB. The default compression is ZLIB. Images created with ZLIB compression type are backward compatible with older qemu versions. Adding of the compression type breaks a number of tests because now the compression type is reported on image creation and there are some changes in the qcow2 header in size and offsets. The tests are fixed in the following ways: * filter out compression_type for many tests * fix header size, feature table size and backing file offset affected tests: 031, 036, 061, 080 header_size +=8: 1 byte compression type 7 bytes padding feature_table += 48: incompatible feature compression type backing_file_offset += 56 (8 + 48 -> header_change + feature_table_change) * add "compression type" for test output matching when it isn't filtered affected tests: 049, 060, 061, 065, 082, 085, 144, 182, 185, 198, 206, 242, 255, 274, 280 Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> QAPI part: Acked-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20200507082521.29210-2-dplotnikov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-05-07migration/xbzrle: add encoding rateWei Wang
Users may need to check the xbzrle encoding rate to know if the guest memory is xbzrle encoding-friendly, and dynamically turn off the encoding if the encoding rate is low. Signed-off-by: Yi Sun <yi.y.sun@intel.com> Signed-off-by: Wei Wang <wei.w.wang@intel.com> Message-Id: <1588208375-19556-1-git-send-email-wei.w.wang@intel.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-05-07migration/throttle: Add cpu-throttle-tailslow migration parameterKeqian Zhu
At the tail stage of throttling, the Guest is very sensitive to CPU percentage while the @cpu-throttle-increment is excessive usually at tail stage. If this parameter is true, we will compute the ideal CPU percentage used by the Guest, which may exactly make the dirty rate match the dirty rate threshold. Then we will choose a smaller throttle increment between the one specified by @cpu-throttle-increment and the one generated by ideal CPU percentage. Therefore, it is compatible to traditional throttling, meanwhile the throttle increment won't be excessive at tail stage. This may make migration time longer, and is disabled by default. Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com> Message-Id: <20200413101508.54793-1-zhukeqian1@huawei.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-04-30qapi: Assert non-input visitors see only valid narrow integersMarkus Armbruster
visit_type_intN() and visit_type_uintN() fail when the value is out of bounds. This is appropriate with an input visitor: the value comes from input, and input may be bad. It should never happen with the other visitors: the value comes from the caller, and callers must keep it within bounds. Assert that. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200424084338.26803-10-armbru@redhat.com>
2020-04-30qapi: Assert output visitors see only valid enum valuesMarkus Armbruster
output_type_enum() fails when *obj is not a valid value of the enum type. Should not happen. Drop the check, along with its unit tests. This unmasks qapi_enum_lookup()'s assertion. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200424084338.26803-9-armbru@redhat.com> [Commit message tweaked]
2020-04-30qapi: Fix Visitor contract for start_alternate()Markus Armbruster
The contract demands v->start_alternate() for input and dealloc visitors, but visit_start_alternate() actually requires it for input and clone visitors. Fix the contract, and delete superfluous qapi_dealloc_start_alternate(). Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200424084338.26803-8-armbru@redhat.com>
2020-04-30qapi: Assert incomplete object occurs only in dealloc visitorMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200424084338.26803-7-armbru@redhat.com>
2020-04-30qobject: Eliminate qdict_iter(), use qdict_first(), qdict_next()Markus Armbruster
qdict_iter() has just three uses and no test coverage. Replace by qdict_first(), qdict_next() for more concise code and less type punning. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20200415083048.14339-5-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2020-04-02qmp: fix leak on callbacks that return both value and errorMarc-André Lureau
Direct leak of 4120 byte(s) in 1 object(s) allocated from: #0 0x7fa114931887 in __interceptor_calloc (/lib64/libasan.so.6+0xb0887) #1 0x7fa1144ad8f0 in g_malloc0 (/lib64/libglib-2.0.so.0+0x588f0) #2 0x561e3c9c8897 in qmp_object_add /home/elmarco/src/qemu/qom/qom-qmp-cmds.c:291 #3 0x561e3cf48736 in qmp_dispatch /home/elmarco/src/qemu/qapi/qmp-dispatch.c:155 #4 0x561e3c8efb36 in monitor_qmp_dispatch /home/elmarco/src/qemu/monitor/qmp.c:145 #5 0x561e3c8f09ed in monitor_qmp_bh_dispatcher /home/elmarco/src/qemu/monitor/qmp.c:234 #6 0x561e3d08c993 in aio_bh_call /home/elmarco/src/qemu/util/async.c:136 #7 0x561e3d08d0a5 in aio_bh_poll /home/elmarco/src/qemu/util/async.c:164 #8 0x561e3d0a535a in aio_dispatch /home/elmarco/src/qemu/util/aio-posix.c:380 #9 0x561e3d08e3ca in aio_ctx_dispatch /home/elmarco/src/qemu/util/async.c:298 #10 0x7fa1144a776e in g_main_context_dispatch (/lib64/libglib-2.0.so.0+0x5276e) Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20200325184723.2029630-3-marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-03-19Add rx-softmmuYoshinori Sato
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Signed-off-by: Yoshinori Sato <ysato@users.sourceforge.jp> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> [PMD: Squashed patches from Richard Henderson modifying qapi/common.json and tests/machine-none-test.c] Message-Id: <20200224141923.82118-21-ysato@users.sourceforge.jp> [PMD: Added @since 5.0 tag in SysEmuTarget] Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2020-03-17net: Complete qapi-fication of netdev_addEric Blake
We've had all the required pieces for doing a type-safe representation of netdev_add as a flat union for quite some time now (since 0e55c381f6 in v2.7.0, released in 2016), but did not make the final switch to using it because of concern about whether a command-line regression in accepting "1" in place of 1 for integer arguments would be problematic. Back then, we did not have the deprecation cycle to allow us to make progress. But now that we have waited so long, other problems have crept in: for example, our desire to add qemu-storage-daemon is hampered by the inability to express net objects, and we are unable to introspect what we actually accept. Additionally, our round-trip through QemuOpts silently eats any argument that expands to an array, rendering dnssearch, hostfwd, and guestfwd useless through QMP: {"execute": "netdev_add", "arguments": { "id": "netdev0", "type": "user", "dnssearch": [ { "str": "8.8.8.8" }, { "str": "8.8.4.4" } ]}} So without further ado, let's turn on proper QAPI. netdev_add() was a trivial wrapper around net_client_init(), which did a few steps prior to calling net_client_init1(); with this patch, we now skip directly to net_client_init1(). In addition to fixing array parameters, the following additional differences occur: - {"execute": "netdev_add", "arguments": {"type": "help"}} no longer attempts to print help to stdout and exit. Bug fix, broken in 547203ead4 'net: List available netdevs with "-netdev help"', v2.12.0. - {"execute": "netdev_add", "arguments': {... "ipv6-net": "..." }} no longer attempts to desugar the undocumented ipv6-net magic string into the proper "ipv6-prefix" and "ipv6-prefixlen". Undocumented misfeature, introduced in commit 7aac531ef2 "qapi-schema, qemu-options & slirp: Adding Qemu options for IPv6 addresses", v2.6.0. - {'execute':'netdev_add', 'arguments':{'id':'net2', 'type':'hubport', 'hubid':"2"}} {"error": {"class": "GenericError", "desc": "Invalid parameter type for 'hubid', expected: integer"}} Used to succeed: since our command line treats everything as strings, our not-so-round-trip conversion from QAPI -> QemuOpts -> QAPI lost the original typing and turned everything into a string; now that we skip the QemuOpts, the JSON input has to match the exact QAPI type. But this stricter QMP is desirable, and introspection is sufficient for any affected applications to make sure they use it correctly. In qmp_netdev_add(), we still have to create a QemuOpts object so that qmp_netdev_del() will be able to remove a hotplugged network device; but the opts->head remains empty since we now manage all parsing through the QAPI object rather than QemuOpts; a separate patch will address the abuse of QemuOpts as a witness for whether a NetClientState is a netdev. In the meantime, our argument that we are okay requires auditing all uses of option group "netdev": - qemu_netdev_opts: option group definition, empty .desc[] - CLI (CLI netdev parsing ends before monitors start, so while monitors can mess with CLI netdevs, CLI cannot mess with monitor netdevs): - main() case QEMU_OPTION_netdev: store CLI definition - main() case QEMU_OPTION_readconfig, case QEMU_OPTION_writeconfig: similar, dealing only with CLI - net_init_clients(): Pass CLI to net_client_init() - Monitor: - hmp_netdev_add(): straightforward parse into net_client_init() - qmp_netdev_add(): subject of this patch, used to add full object to option group, now just adds bare-bones id - qmp_netdev_del(), netdev_del_completion(): check the option group solely for id, as a 'is this a netdev' predicate Reported-by: Alex Kirillov <lekiravi@yandex-team.ru> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200317201711.322764-2-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Commit message typo fixed] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-03-17qmp: constify QmpCommand and listMarc-André Lureau
Since 0b69f6f72ce47a37a749b056b6d5ec64c61f11e8 "qapi: remove qmp_unregister_command()", the command list can be declared const. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Damien Hedde <damien.hedde@greensocs.com> Message-Id: <20200316171824.2319695-1-marcandre.lureau@redhat.com> [Rebased] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-03-17qapi: Mark deprecated QMP parts with feature 'deprecated'Markus Armbruster
Add feature 'deprecated' to the deprecated QMP commands, so their deprecation becomes visible in output of query-qmp-schema. Looks like this: {"name": "query-cpus", "ret-type": "[164]", "meta-type": "command", "arg-type": "0", ---> "features": ["deprecated"]} Management applications could conceivably use this for static checking. The deprecated commands are change, cpu-add, migrate-set-cache-size, migrate_set_downtime, migrate_set_speed, query-cpus, query-events, query-migrate-cache-size. The deprecated command arguments are block-commit arguments @base and @top, and block_set_io_throttle, blockdev-change-medium, blockdev-close-tray, blockdev-open-tray, eject argument @device. The deprecated command results are query-cpus-fast result @arch, query-block result @dirty-bitmaps, query-named-block-nodes result @encryption_key_missing and result @dirty-bitmaps's member @status. Same for query-block result @inserted, which mirrors query-named-block-nodes. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20200317115459.31821-27-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-17qapi: Replace qmp_dispatch()'s TODO comment by an explanationMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20200317115459.31821-25-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-17qapi: Simplify how qmp_dispatch() gets the request IDMarkus Armbruster
We convert the request object to a QDict twice: first in qmp_dispatch() to get the request ID, and then again in qmp_dispatch_check_obj(), which converts to QDict, then checks and returns it. We can't get the request ID from the latter, because it's null when the qdict flunks the checks. Move the checked conversion to QDict from qmp_dispatch_check_obj() to qmp_dispatch(), and drop the duplicate there. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20200317115459.31821-24-armbru@redhat.com>
2020-03-17qapi: Simplify how qmp_dispatch() deals with QCO_NO_SUCCESS_RESPMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20200317115459.31821-23-armbru@redhat.com>
2020-03-17qapi: Inline do_qmp_dispatch() into qmp_dispatch()Markus Armbruster
Both functions check @request is a QDict, and both have code for QCO_NO_SUCCESS_RESP. This wasn't the case back when they were created. It's a sign of muddled responsibilities. Inline. The next commits will clean up some more. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20200317115459.31821-22-armbru@redhat.com>
2020-03-17qapi: Add feature flags to struct membersMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20200317115459.31821-21-armbru@redhat.com>
2020-03-17qapi: Add feature flags to remaining definitionsMarkus Armbruster
In v4.1.0, we added feature flags just to struct types (commit 6a8c0b5102^..f3ed93d545), to satisfy an immediate need (commit c9d4070991 "file-posix: Add dynamic-auto-read-only QAPI feature"). In v4.2.0, we added them to commands (commit 23394b4c39 "qapi: Add feature flags to commands") to satisfy another immediate need (commit d76744e65e "qapi: Allow introspecting fix for savevm's cooperation with blockdev"). Add them to the remaining definitions: enumeration types, union types, alternate types, and events. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200317115459.31821-13-armbru@redhat.com>
2020-03-17qapi: Belatedly update doc comment for @wait deprecationMarkus Armbruster
Commit a9b305ba29 "socket: allow wait=false for client socket" deprecated use of @wait for client socket chardevs, but neglected to update char.json's doc comment. Make up for that. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200317115459.31821-3-armbru@redhat.com>
2020-03-16qapi/audio: add documentation for AudioFormatVolker Rümelin
The review for patch ed2a4a7941 "audio: proper support for float samples in mixeng" suggested this would be a good idea. Acked-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Volker Rümelin <vr_qemu@t-online.de> Tested-by: John Arbuckle <programmingkidx@gmail.com> Message-id: 20200308193321.20668-1-vr_qemu@t-online.de Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2020-03-13migration/throttle: Add throttle-trig-thres migration parameterKeqian Zhu
Currently, if the bytes_dirty_period is more than the 50% of bytes_xfer_period, we start or increase throttling. If we make this percentage higher, then we can tolerate higher dirty rate during migration, which means less impact on guest. The side effect of higher percentage is longer migration time. We can make this parameter configurable to switch between mig- ration time first or guest performance first. The default value is 50 and valid range is 1 to 100. Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com> Message-Id: <20200224023142.39360-1-zhukeqian1@huawei.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-03-11qapi: Add '@allow-write-only-overlay' feature for 'blockdev-snapshot'Peter Krempa
Anounce that 'blockdev-snapshot' command's permissions allow changing of the backing file if the 'consistent_read' permission is not required. This is useful for libvirt to allow late opening of the backing chain during a blockdev-mirror. Signed-off-by: Peter Krempa <pkrempa@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20200310113831.27293-8-kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-03-06block/rbd: Add support for ceph namespacesFlorian Florensa
Starting from ceph Nautilus, RBD has support for namespaces, allowing for finer grain ACLs on images inside a pool, and tenant isolation. In the rbd cli tool documentation, the new image-spec and snap-spec are : - [pool-name/[namespace-name/]]image-name - [pool-name/[namespace-name/]]image-name@snap-name When using an non namespace's enabled qemu, it complains about not finding the image called namespace-name/image-name, thus we only need to parse the image once again to find if there is a '/' in its name, and if there is, use what is before it as the name of the namespace to later pass it to rados_ioctx_set_namespace. rados_ioctx_set_namespace if called with en empty string or a null pointer as the namespace parameters pretty much does nothing, as it then defaults to the default namespace. The namespace is extracted inside qemu_rbd_parse_filename, stored in the qdict, and used in qemu_rbd_connect to make it work with both qemu-img, and qemu itself. Signed-off-by: Florian Florensa <fflorensa@online.net> Message-Id: <20200110111513.321728-2-fflorensa@online.net> Reviewed-by: Jason Dillaman <dillaman@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-03-06qemu-storage-daemon: Add --monitor optionKevin Wolf
This adds and parses the --monitor option, so that a QMP monitor can be used in the storage daemon. The monitor offers commands defined in the QAPI schema at storage-daemon/qapi/qapi-schema.json. The --monitor options currently allows to create multiple monitors with the same ID. This part of the interface is considered unstable. We will reject such configurations as soon as we have a design for the monitor subsystem to perform these checks. (In the system emulator, we depend on QemuOpts rejecting duplicate IDs.) Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20200224143008.13362-21-kwolf@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-03-06monitor: Add allow_hmp parameter to monitor_init()Kevin Wolf
Add a new parameter allow_hmp to monitor_init() so that the storage daemon can disable HMP. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20200224143008.13362-20-kwolf@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-03-06monitor: Create QAPIfied monitor_init()Kevin Wolf
This adds a new QAPI-based monitor_init() function. The existing monitor_init_opts() is rewritten to simply put its QemuOpts parameter into a visitor and pass the resulting QAPI object to monitor_init(). This will cause some change in those error messages for the monitor options in the system emulator that are now generated by the visitor rather than explicitly checked in monitor_init_opts(). Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20200224143008.13362-17-kwolf@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-03-06qapi: Create 'pragma' moduleKevin Wolf
We want to share the whitelists between the system emulator schema and the storage daemon schema, so move all the pragmas from the main schema file into a separate file that can be included from both. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20200224143008.13362-16-kwolf@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
generated by cgit v1.2.3 (git 2.26.2) at 2024-12-22 12:53:55 +0000