aboutsummaryrefslogtreecommitdiff
path: root/qemu-img.c
AgeCommit message (Collapse)Author
2019-08-27block: define .*_part io handlers in BlockDriverVladimir Sementsov-Ogievskiy
Add handlers supporting qiov_offset parameter: bdrv_co_preadv_part bdrv_co_pwritev_part bdrv_co_pwritev_compressed_part This is used to reduce need of defining local_qiovs and hd_qiovs in all corners of block layer code. The following patches will increase usage of this new API part by part. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20190604161514.262241-5-vsementsov@virtuozzo.com Message-Id: <20190604161514.262241-5-vsementsov@virtuozzo.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2019-08-19qemu-img: Fix bdrv_has_zero_init() use in convertMax Reitz
bdrv_has_zero_init() only has meaning for newly created images or image areas. If qemu-img convert did not create the image itself, it cannot rely on bdrv_has_zero_init()'s result to carry any meaning. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20190724171239.8764-2-mreitz@redhat.com Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-08-16Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into stagingPeter Maydell
Block layer patches: - file-posix: Fix O_DIRECT alignment detection - Fixes for concurrent block jobs - block-backend: Queue requests while drained (fix IDE vs. job crashes) - qemu-img convert: Deprecate using -n and -o together - iotests: Migration tests with filter nodes - iotests: More media change tests # gpg: Signature made Fri 16 Aug 2019 10:29:18 BST # gpg: using RSA key 7F09B272C88F2FD6 # gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full] # Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6 * remotes/kevin/tags/for-upstream: file-posix: Handle undetectable alignment qemu-img convert: Deprecate using -n and -o together block-backend: Queue requests while drained mirror: Keep mirror_top_bs drained after dropping permissions block: Remove blk_pread_unthrottled() iotests: Add test for concurrent stream/commit tests: Test mid-drain bdrv_replace_child_noperm() tests: Test polling in bdrv_drop_intermediate() block: Reduce (un)drains when replacing a child block: Keep subtree drained in drop_intermediate block: Simplify bdrv_filter_default_perms() iotests: Test migration with all kinds of filter nodes iotests: Move migration helpers to iotests.py iotests/118: Add -blockdev based tests iotests/118: Create test classes dynamically iotests/118: Test media change for scsi-cd Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-08-16Clean up inclusion of sysemu/sysemu.hMarkus Armbruster
In my "build everything" tree, changing sysemu/sysemu.h triggers a recompile of some 5400 out of 6600 objects (not counting tests and objects that don't depend on qemu/osdep.h). Almost a third of its inclusions are actually superfluous. Delete them. Downgrade two more to qapi/qapi-types-run-state.h, and move one from char/serial.h to char/serial.c. hw/semihosting/config.c, monitor/monitor.c, qdev-monitor.c, and stubs/semihost.c define variables declared in sysemu/sysemu.h without including it. The compiler is cool with that, but include it anyway. This doesn't reduce actual use much, as it's still included into widely included headers. The next commit will tackle that. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-Id: <20190812052359.30071-27-armbru@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
2019-08-16Include qemu/main-loop.h lessMarkus Armbruster
In my "build everything" tree, changing qemu/main-loop.h triggers a recompile of some 5600 out of 6600 objects (not counting tests and objects that don't depend on qemu/osdep.h). It includes block/aio.h, which in turn includes qemu/event_notifier.h, qemu/notify.h, qemu/processor.h, qemu/qsp.h, qemu/queue.h, qemu/thread-posix.h, qemu/thread.h, qemu/timer.h, and a few more. Include qemu/main-loop.h only where it's needed. Touching it now recompiles only some 1700 objects. For block/aio.h and qemu/event_notifier.h, these numbers drop from 5600 to 2800. For the others, they shrink only slightly. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190812052359.30071-21-armbru@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-08-16qemu-img convert: Deprecate using -n and -o togetherKevin Wolf
bdrv_create options specified with -o have no effect when skipping image creation with -n, so this doesn't make sense. Warn against the misuse and deprecate the combination so we can make it a hard error later. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2019-07-02block: include base when checking image chain for block allocationAndrey Shinkevich
This patch is used in the 'block/stream: introduce a bottom node' that is following. Instead of the base node, the caller may pass the node that has the base as its backing image to the function bdrv_is_allocated_above() with a new parameter include_base = true and get rid of the dependency on the base that may change during commit/stream parallel jobs. Now, if the specified base is not found in the backing image chain, the QEMU will abort. Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Message-id: 1559152576-281803-2-git-send-email-andrey.shinkevich@virtuozzo.com [mreitz: Squashed in the following as a rebase on conflicting patches:] Message-id: e3cf99ae-62e9-8b6e-5a06-d3c8b9363b85@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-06-14qemu-img: Add salvaging mode to convertMax Reitz
This adds a salvaging mode (--salvage) to qemu-img convert which ignores read errors and treats the respective areas as containing only zeroes. This can be used for instance to at least partially recover the data from terminally corrupted qcow2 images. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20190507203508.18026-3-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-06-14qemu-img: Move quiet into ImgConvertStateMax Reitz
Move img_convert()'s quiet flag into the ImgConvertState so it is accessible by nested functions. -q dictates that it suppresses anything but errors, so if those functions want to emit warnings, they need to query this flag first. (There currently are no such warnings, but there will be as of the next patch.) Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20190507203508.18026-2-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-06-14qemu-img: Fix options leakage in img_rebase()Max Reitz
img_rebase() can leak a QDict in two occasions. Fix it. Coverity: CID 1401416 Fixes: d16699b64671466b42079c45b89127aeea1ca565 Fixes: 330c72957196e0ae382abcaa97ebf4eb9bc8574f Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20190528195338.12376-1-mreitz@redhat.com Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-06-12Include qemu-common.h exactly where neededMarkus Armbruster
No header includes qemu-common.h after this commit, as prescribed by qemu-common.h's file comment. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190523143508.25387-5-armbru@redhat.com> [Rebased with conflicts resolved automatically, except for include/hw/arm/xlnx-zynqmp.h hw/arm/nrf51_soc.c hw/arm/msf2-soc.c block/qcow2-refcount.c block/qcow2-cluster.c block/qcow2-cache.c target/arm/cpu.h target/lm32/cpu.h target/m68k/cpu.h target/mips/cpu.h target/moxie/cpu.h target/nios2/cpu.h target/openrisc/cpu.h target/riscv/cpu.h target/tilegx/cpu.h target/tricore/cpu.h target/unicore32/cpu.h target/xtensa/cpu.h; bsd-user/main.c and net/tap-bsd.c fixed up]
2019-06-12Include qemu/module.h where needed, drop it from qemu-common.hMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190523143508.25387-4-armbru@redhat.com> [Rebased with conflicts resolved automatically, except for hw/usb/dev-hub.c hw/misc/exynos4210_rng.c hw/misc/bcm2835_rng.c hw/misc/aspeed_scu.c hw/display/virtio-vga.c hw/arm/stm32f205_soc.c; ui/cocoa.m fixed up]
2019-06-04block: Add BlockBackend.ctxKevin Wolf
This adds a new parameter to blk_new() which requires its callers to declare from which AioContext this BlockBackend is going to be used (or the locks of which AioContext need to be taken anyway). The given context is only stored and kept up to date when changing AioContexts. Actually applying the stored AioContext to the root node is saved for another commit. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-05-28qemu-img: rebase: Reuse in-chain BlockDriverStateSam Eiderman
If a chain was detected, don't open a new BlockBackend from the target backing file which will create a new BlockDriverState. Instead, create an empty BlockBackend and attach the already open BlockDriverState. Permissions for blk_new() were copied from blk_new_open() when flags = 0. Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com> Reviewed-by: Eyal Moscovici <eyal.moscovici@oracle.com> Signed-off-by: Sagi Amit <sagi.amit@oracle.com> Co-developed-by: Sagi Amit <sagi.amit@oracle.com> Signed-off-by: Sam Eiderman <shmuel.eiderman@oracle.com> Message-id: 20190523163337.4497-4-shmuel.eiderman@oracle.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28qemu-img: rebase: Reduce reads on in-chain rebaseSam Eiderman
In the following case: (base) A <- B <- C (tip) when running: qemu-img rebase -b A C QEMU would read all sectors not allocated in the file being rebased (C) and compare them to the new base image (A), regardless of whether they were changed or even allocated anywhere along the chain between the new base and the top image (B). This causes many unneeded reads when rebasing an image which represents a small diff of a large disk, as it would read most of the disk's sectors. Instead, use bdrv_is_allocated_above() to reduce the number of unnecessary reads. Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com> Signed-off-by: Sam Eiderman <shmuel.eiderman@oracle.com> Signed-off-by: Eyal Moscovici <eyal.moscovici@oracle.com> Message-id: 20190523163337.4497-3-shmuel.eiderman@oracle.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28qemu-img: rebase: Reuse parent BlockDriverStateSam Eiderman
In safe mode we open the entire chain, including the parent backing file of the rebased file. Do not open a new BlockBackend for the parent backing file, which saves opening the rest of the chain twice, which for long chains saves many "pricy" bdrv_open() calls. Permissions for blk_new() were copied from blk_new_open() when flags = 0. Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com> Reviewed-by: Eyal Moscovici <eyal.moscovici@oracle.com> Signed-off-by: Sagi Amit <sagi.amit@oracle.com> Co-developed-by: Sagi Amit <sagi.amit@oracle.com> Signed-off-by: Sam Eiderman <shmuel.eiderman@oracle.com> Message-id: 20190523163337.4497-2-shmuel.eiderman@oracle.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-10qemu-img: Use zero writes after source backing EOFMax Reitz
Past the end of the source backing file, we memset() buf_old to zero, so it is clearly easy to use blk_pwrite_zeroes() instead of blk_pwrite() then. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-05-10qemu-img: Allow rebase with no input baseMax Reitz
Currently, without -u, you cannot add a backing file to an image when it currently has none: $ qemu-img rebase -b base.qcow2 foo.qcow2 qemu-img: Could not open old backing file '': The 'file' block driver requires a file name It is really simple to allow this, though (effectively by setting old_backing_size to 0), so this patch does just that. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-05-10qemu-img: Use IEC binary prefixes for size constantsStefano Garzarella
Using IEC binary prefixes in order to make the code more readable. Signed-off-by: Stefano Garzarella <sgarzare@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-30qemu-img: use buffer-based ioVladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-30qemu-img: Make create hint at protocol optionsMax Reitz
qemu-img create allows giving just a format and "-o help" to get a list of the options supported by that format. Users may not realize that the protocol level may offer even more options, which they only get to see by specifying a filename. This patch adds a note to hint at that fact. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-18block/qapi: Clean up how we print to monitor or stdoutMarkus Armbruster
bdrv_snapshot_dump(), bdrv_image_info_specific_dump(), bdrv_image_info_dump() and their helpers take an fprintf()-like callback and a FILE * to pass to it. hmp.c passes monitor_printf() cast to fprintf_function and the current monitor cast to FILE *. qemu-img.c and qemu-io-cmds.c pass fprintf and stdout. The type-punning is technically undefined behaviour, but works in practice. Clean up: drop the callback, and call qemu_printf() instead. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-Id: <20190417191805.28198-8-armbru@redhat.com>
2019-04-17qemu-img: Use error_vreport() in error_exit()Markus Armbruster
error_exit() uses low-level error_printf() to report errors. Modernize it to use error_vreport(). Cc: Kevin Wolf <kwolf@redhat.com> Cc: Max Reitz <mreitz@redhat.com> Cc: qemu-block@nongnu.org Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20190417190641.26814-2-armbru@redhat.com>
2019-04-17log: Make glib logging go through QEMUChristophe Fergeau
This commit adds a error_init() helper which calls g_log_set_default_handler() so that glib logs (g_log, g_warning, ...) are handled similarly to other QEMU logs. This means they will get a timestamp if timestamps are enabled, and they will go through the HMP monitor if one is configured. This commit also adds a call to error_init() to the binaries installed by QEMU. Since error_init() also calls error_set_progname(), this means that *-linux-user, *-bsd-user and qemu-pr-helper messages output with error_report, info_report, ... will slightly change: they will be prefixed by the binary name. glib debug messages are enabled through G_MESSAGES_DEBUG similarly to the glib default log handler. At the moment, this change will mostly impact SPICE logging if your spice version is >= 0.14.1. With older spice versions, this is not going to work as expected, but will not have any ill effect, so this call is not conditional on the SPICE version. Signed-off-by: Christophe Fergeau <cfergeau@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20190131164614.19209-3-cfergeau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2019-04-02qemu-img: Enable BDRV_REQ_MAY_UNMAP in convertNir Soffer
With Kevin's "block: Fix slow pre-zeroing in qemu-img convert"[1] (commit c9fdcf202f, 'qemu-img: Use BDRV_REQ_NO_FALLBACK for pre-zeroing') we skip the pre zero step called like this: blk_make_zero(s->target, BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) And we write zeroes later using: blk_co_pwrite_zeroes(s->target, sector_num << BDRV_SECTOR_BITS, n << BDRV_SECTOR_BITS, 0); Since we use flags=0, this is translated to NBD_CMD_WRITE_ZEROES with NBD_CMD_FLAG_NO_HOLE flag, which cause the NBD server to allocated space instead of punching a hole. Here is an example failure: $ dd if=/dev/urandom of=src.img bs=1M count=5 $ truncate -s 50m src.img $ truncate -s 50m dst.img $ nbdkit -f -v -e '' -U nbd.sock file file=dst.img $ ./qemu-img convert -n src.img nbd:unix:nbd.sock We can see in nbdkit log that it received the NBD_CMD_FLAG_NO_HOLE (may_trim=0): nbdkit: file[1]: debug: newstyle negotiation: flags: export 0x4d nbdkit: file[1]: debug: pwrite count=2097152 offset=0 nbdkit: file[1]: debug: pwrite count=2097152 offset=2097152 nbdkit: file[1]: debug: pwrite count=1048576 offset=4194304 nbdkit: file[1]: debug: zero count=33554432 offset=5242880 may_trim=0 nbdkit: file[1]: debug: zero count=13631488 offset=38797312 may_trim=0 nbdkit: file[1]: debug: flush And the image became fully allocated: $ qemu-img info dst.img virtual size: 50M (52428800 bytes) disk size: 50M With this change we see that nbdkit did not receive the NBD_CMD_FLAG_NO_HOLE (may_trim=1): nbdkit: file[1]: debug: newstyle negotiation: flags: export 0x4d nbdkit: file[1]: debug: pwrite count=2097152 offset=0 nbdkit: file[1]: debug: pwrite count=2097152 offset=2097152 nbdkit: file[1]: debug: pwrite count=1048576 offset=4194304 nbdkit: file[1]: debug: zero count=33554432 offset=5242880 may_trim=1 nbdkit: file[1]: debug: zero count=13631488 offset=38797312 may_trim=1 nbdkit: file[1]: debug: flush And the file is sparse as expected: $ qemu-img info dst.img virtual size: 50M (52428800 bytes) disk size: 5.0M [1] http://lists.nongnu.org/archive/html/qemu-block/2019-03/msg00761.html Signed-off-by: Nir Soffer <nsoffer@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-30qemu-img: Gracefully shutdown when map can't finishEric Blake
Trying 'qemu-img map -f raw nbd://localhost:10809' causes the NBD server to output a scary message: qemu-nbd: Disconnect client, due to: Failed to read request: Unexpected end-of-file before all bytes were read This is because the NBD client, being remote, has no way to expose a human-readable map (the --output=json data is fine, however). But because we exit(1) right after the message, causing the client to bypass all block cleanup, the server sees the abrupt exit and warns, whereas it would be silent had the client had a chance to send NBD_CMD_DISC. Other protocols may have similar cleanup issues, where failure to blk_unref() could cause unintended effects. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20190326184043.7544-1-eblake@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2019-03-30qemu-img: Report bdrv_block_status failuresEric Blake
If bdrv_block_status_above() fails, we are aborting the convert process but failing to print an error message. Broken in commit 690c7301 (v2.4) when rewriting convert's logic. Discovered when teaching nbdkit to support NBD_CMD_BLOCK_STATUS, and accidentally violating the protocol by returning more than one extent in spite of qemu asking for NBD_CMD_FLAG_REQ_ONE. The qemu NBD code should probably handle the server's non-compliance more gracefully than failing with EINVAL, but qemu-img shouldn't be silently squelching any block status failures. It doesn't help that qemu 3.1 masks the qemu-img bug with extra noise that the nbd code is dumping to stderr (that noise was cleaned up in d8b4bad8). Reported-by: Richard W.M. Jones <rjones@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20190323212639.579-2-eblake@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-03-26qemu-img: Use BDRV_REQ_NO_FALLBACK for pre-zeroingKevin Wolf
If qemu-img convert sees that the target image isn't zero-initialised yet, it tries to do an efficient zero write for the whole image first to save the overhead of repeated explicit zero writes during the conversion. Obviously, this provides only an advantage if the pre-zeroing is actually efficient. Otherwise, we can end up writing zeroes slowly while zeroing out the whole image, and then overwrite the same blocks again with real data, potentially doubling the written data. Pass BDRV_REQ_NO_FALLBACK to blk_make_zero() to avoid this case. If we can't efficiently zero out, we'll instead write explicit zeroes only if there is no data to be written to a block. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Acked-by: Eric Blake <eblake@redhat.com>
2019-03-08block: iterate_format with account of whitelistingAndrey Shinkevich
bdrv_iterate_format (which is currently only used for printing out the formats supported by the block layer) doesn't take format whitelisting into account. This creates a problem for tests: they enumerate supported formats to decide which tests to enable, but then discover that QEMU doesn't let them actually use some of those formats. To avoid that, exclude formats that are not whitelisted from enumeration, if whitelisting is in use. Since we have separate whitelists for r/w and r/o, take this a parameter to bdrv_iterate_format, and print two lists of supported formats (r/w and r/o) in main qemu. Signed-off-by: Roman Kagan <rkagan@virtuozzo.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-02-26Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into stagingPeter Maydell
Block layer patches: - Block graph change fixes (avoid loops, cope with non-tree graphs) - bdrv_set_aio_context() related fixes - HMP snapshot commands: Use only tag, not the ID to identify snapshots - qmeu-img, commit: Error path fixes - block/nvme: Build fix for gcc 9 - MAINTAINERS updates - Fix various issues with bdrv_refresh_filename() - Fix various iotests - Include LUKS overhead in qemu-img measure for qcow2 - A fix for vmdk's image creation interface # gpg: Signature made Mon 25 Feb 2019 14:18:15 GMT # gpg: using RSA key 7F09B272C88F2FD6 # gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full] # Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6 * remotes/kevin/tags/for-upstream: (71 commits) iotests: Skip 211 on insufficient memory vmdk: false positive of compat6 with hwversion not set iotests: add LUKS payload overhead to 178 qemu-img measure test qcow2: include LUKS payload overhead in qemu-img measure iotests.py: s/_/-/g on keys in qmp_log() iotests: Let 045 be run concurrently iotests: Filter SSH paths iotests.py: Filter filename in any string value iotests.py: Add is_str() iotests: Fix 207 to use QMP filters for qmp_log iotests: Fix 232 for LUKS iotests: Remove superfluous rm from 232 iotests: Fix 237 for Python 2.x iotests: Re-add filename filters iotests: Test json:{} filenames of internal BDSs block: BDS options may lack the "driver" option block/null: Generate filename even with latency-ns block/curl: Implement bdrv_refresh_filename() block/curl: Harmonize option defaults block/nvme: Fix bdrv_refresh_filename() ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-02-25block: bdrv_get_full_backing_filename_from_...'s ret. val.Max Reitz
Make bdrv_get_full_backing_filename_from_filename() return an allocated string instead of placing the result in a caller-provided buffer. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20190201192935.18394-11-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-02-25block: Use bdrv_refresh_filename() to pullMax Reitz
Before this patch, bdrv_refresh_filename() is used in a pushing manner: Whenever the BDS graph is modified, the parents of the modified edges are supposed to be updated (recursively upwards). However, that is nonviable, considering that we want child changes not to concern parents. Also, in the long run we want a pull model anyway: Here, we would have a bdrv_filename() function which returns a BDS's filename, freshly constructed. This patch is an intermediate step. It adds bdrv_refresh_filename() calls before every place a BDS.filename value is used. The only exceptions are protocol drivers that use their own filename, which clearly would not profit from refreshing that filename before. Also, bdrv_get_encrypted_filename() is removed along the way (as a user of BDS.filename), since it is completely unused. In turn, all of the calls to bdrv_refresh_filename() before this patch are removed, because we no longer have to call this function on graph changes. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20190201192935.18394-2-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-02-25qemu-img: fix error reporting for -objectDaniel P. Berrangé
Error reporting for user_creatable_add_opts_foreach was changed so that it no longer called 'error_report_err' in: commit 7e1e0c11127bde81cff260fc6859690435c509d6 Author: Markus Armbruster <armbru@redhat.com> Date: Wed Oct 17 10:26:43 2018 +0200 qom: Clean up error reporting in user_creatable_add_opts_foreach() Some callers were updated to pass in "&error_fatal" but all the ones in qemu-img were left passing NULL. As a result all errors went to /dev/null instead of being reported to the user. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-02-25block/snapshot: remove bdrv_snapshot_delete_by_id_or_nameDaniel Henrique Barboza
After the previous patch, the only instance of this function left is inside qemu-img.c. qemu-img is using it inside the 'img_snapshot' function to delete snapshots in the SNAPSHOT_DELETE case, based on a "snapshot_name" string that refers to the tag, not ID, of the QEMUSnapshotInfo struct. This can be verified by checking the SNAPSHOT_CREATE case that comes shortly before SNAPSHOT_DELETE. In that case, the same "snapshot_name" variable is being strcpy to the 'name' field of the QEMUSnapshotInfo struct sn: pstrcpy(sn.name, sizeof(sn.name), snapshot_name); Based on that, it is unlikely that "snapshot_name" might contain an "id" in SNAPSHOT_DELETE. This patch changes SNAPSHOT_DELETE to use snapshot_find() and snapshot_delete() instead of bdrv_snapshot_delete_by_id_or_name. After that, there is no instances left of bdrv_snapshot_delete_by_id_or_name in the code, so it is safe to remove it entirely. Suggested-by: Murilo Opsfelder Araujo <muriloo@linux.ibm.com> Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-02-22qemu-img: use qemu_iovec_init_bufVladimir Sementsov-Ogievskiy
Use new qemu_iovec_init_buf() instead of qemu_iovec_init_external( ... , 1), which simplifies the code. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20190218140926.333779-13-vsementsov@virtuozzo.com Message-Id: <20190218140926.333779-13-vsementsov@virtuozzo.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2019-01-31qemu-io: Add generic function for reinitializing optind.Richard W.M. Jones
On FreeBSD 11.2: $ nbdkit memory size=1M --run './qemu-io -f raw -c "aio_write 0 512" $nbd' Parsing error: non-numeric argument, or extraneous/unrecognized suffix -- aio_write After main option parsing, we reinitialize optind so we can parse each command. However reinitializing optind to 0 does not work on FreeBSD. What happens when you do this is optind remains 0 after the option parsing loop, and the result is we try to parse argv[optind] == argv[0] == "aio_write" as if it was the first parameter. The FreeBSD manual page says: In order to use getopt() to evaluate multiple sets of arguments, or to evaluate a single set of arguments multiple times, the variable optreset must be set to 1 before the second and each additional set of calls to getopt(), and the variable optind must be reinitialized. (From the rest of the man page it is clear that optind must be reinitialized to 1). The glibc man page says: A program that scans multiple argument vectors, or rescans the same vector more than once, and wants to make use of GNU extensions such as '+' and '-' at the start of optstring, or changes the value of POSIXLY_CORRECT between scans, must reinitialize getopt() by resetting optind to 0, rather than the traditional value of 1. (Resetting to 0 forces the invocation of an internal initialization routine that rechecks POSIXLY_CORRECT and checks for GNU extensions in optstring.) This commit introduces an OS-portability function called qemu_reset_optind which provides a way of resetting optind that works on FreeBSD and platforms that use optreset, while keeping it the same as now on other platforms. Note that the qemu codebase sets optind in many other places, but in those other places it's setting a local variable and not using getopt. This change is only needed in places where we are using getopt and the associated global variable optind. Signed-off-by: Richard W.M. Jones <rjones@redhat.com> Message-id: 20190118101114.11759-2-rjones@redhat.com Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-11-21qemu-img: Fix leakMax Reitz
create_opts was leaked here. This is not too bad since the process is about to exit anyway, but relying on that does not make the code nicer to read. Fixes: d402b6a21a825a5c07aac9251990860723d49f5d Reported-by: Kevin Wolf <kwolf@redhat.com> Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-11-21qemu-img: Fix typoMax Reitz
Fixes: d402b6a21a825a5c07aac9251990860723d49f5d Reported-by: Kevin Wolf <kwolf@redhat.com> Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-11-12qemu-img: assert block_job_get() does not return NULL in img_commit()Liam Merwick
Although the function block_job_get() can return NULL, it would be a serious bug if it did so (because the job yields before executing anything (if it started successfully); but otherwise, commit_active_start() would have returned an error). However, as a precaution, before dereferencing the 'job' pointer in img_commit() assert it is not NULL. Signed-off-by: Liam Merwick <Liam.Merwick@oracle.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 1541453919-25973-4-git-send-email-Liam.Merwick@oracle.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-11-05option: Make option help nicer to readMax Reitz
This adds some whitespace into the option help (including indentation) and puts angle brackets around the type names. Furthermore, the list name is no longer printed as part of every line, but only once in advance, and only if the caller did not print a caption already. This patch also restores the description alignment we had before commit 9cbef9d68ee1d8d0, just at 24 instead of 16 characters like we used to. This increase is because now we have the type and two spaces of indentation before the description, and with a usual type name length of three chracters, this sums up to eight additional characters -- which means that we now need 24 characters to get the same amount of padding for most options. Also, 24 is a third of 80, which makes it kind of a round number in terminal terms. Finally, this patch amends the reference output of iotest 082 to match the changes (and thus makes it pass again). Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-08-15qemu-img: fix regression copying secrets during convertDaniel P. Berrangé
When the convert command is creating an output file that needs secrets, we need to ensure those secrets are passed to both the blk_new_open and bdrv_create API calls. This is done by qemu-img extracting all opts matching the name suffix "key-secret". Unfortunately the code doing this was run after the call to bdrv_create(), which meant the QemuOpts it was extracting secrets from was now empty. Previously this worked by luks as a bug meant the "key-secret" parameters were not purged from the QemuOpts. This bug was fixed in commit b76b4f604521e59f857d6177bc55f6f2e41fd392 Author: Kevin Wolf <kwolf@redhat.com> Date: Thu Jan 11 16:18:08 2018 +0100 qcow2: Use visitor for options in qcow2_create() Exposing the latent bug in qemu-img. This fix simply moves the copying of secrets to before the bdrv_create() call. Cc: qemu-stable@nongnu.org Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-30qemu-img: Add -C option for convert with copy offloadingFam Zheng
Signed-off-by: Fam Zheng <famz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-23qemu-img: avoid overflow of min_sparse parameterPeter Lieven
the min_sparse convert parameter can overflow (e.g. -S 1024G) in the conversion from int64_t to int resulting in a negative min_sparse parameter. Avoid this by limiting the valid parameters to sane values. In fact anything exceeding the convert buffer size is also pointless. While at it also forbid values that are non multiple of 512 to avoid undesired behaviour. For instance, values between 1 and 511 were legal, but resulted in full allocation. Cc: qemu-stable@nongnu.org Signed-off-by: Peter Lieven <pl@kamp.de> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-12qemu-img: align result of is_allocated_sectorsPeter Lieven
We currently don't enforce that the sparse segments we detect during convert are aligned. This leads to unnecessary and costly read-modify-write cycles either internally in Qemu or in the background on the storage device as nearly all modern filesystems or hardware have a 4k alignment internally. This patch modifies is_allocated_sectors so that its *pnum result will always end at an alignment boundary. This way all requests will end at an alignment boundary. The start of all requests will also be aligned as long as the results of get_block_status do not lead to an unaligned offset. The number of RMW cycles when converting an example image [1] to a raw device that has 4k sector size is about 4600 4k read requests to perform a total of about 15000 write requests. With this path the additional 4600 read requests are eliminated while the number of total write requests stays constant. [1] https://cloud-images.ubuntu.com/releases/16.04/release/ubuntu-16.04-server-cloudimg-amd64-disk1.vmdk Signed-off-by: Peter Lieven <pl@kamp.de> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-10block: split flags in copy_rangeVladimir Sementsov-Ogievskiy
Pass read flags and write flags separately. This is needed to handle coming BDRV_REQ_NO_SERIALISING clearly in following patches. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Fam Zheng <famz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-07-05qemu-img: allow compressed not-in-order writesVladimir Sementsov-Ogievskiy
No reason to forbid them, and they are needed to improve performance with compress-threads in further patches. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-06-15qemu-img: Fix assert when mapping unaligned raw fileEric Blake
Commit a290f085 exposed a latent bug in qemu-img map introduced during the conversion of block status to be byte-based. Earlier in commit 5e344dd8, the internal interface get_block_status() switched to take byte-based parameters, but still called a sector-based block layer function; as such, rounding was added in the lone caller to obey the contract. However, commit 237d78f8 changed get_block_status() to truly be byte-based, at which point rounding to sector boundaries can result in calling bdrv_block_status() with 'bytes == 0' (a coding error) when the boundary between data and a hole falls mid-sector (true for the past-EOF implicit hole present in POSIX files). Fix things by removing the rounding that is now no longer necessary. See also https://bugzilla.redhat.com/1589738 Fixes: 237d78f8 Reported-by: Dan Kenigsberg <danken@redhat.com> Reported-by: Nir Soffer <nsoffer@redhat.com> Reported-by: Maor Lipchuk <mlipchuk@redhat.com> CC: qemu-stable@nongnu.org Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-06-11qemu-img: Remove deprecated -s snapshot_id_or_name optionThomas Huth
It has been marked as deprecated since QEMU v2.0 already, so it is time now to finally remove it. Signed-off-by: Thomas Huth <thuth@redhat.com> Message-id: 1528288551-31641-1-git-send-email-thuth@redhat.com Reviewed-by: Jeff Cody <jcody@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-11qemu-img: Special post-backing convert handlingMax Reitz
Currently, qemu-img convert writes zeroes when it reads zeroes. Sometimes it does not because the target is initialized to zeroes anyway, so we do not need to overwrite (and thus potentially allocate) it. This is never the case for targets with backing files, though. But even they may have an area that is initialized to zeroes, and that is the area past the end of the backing file (if that is shorter than the overlay). So if the target format's unallocated blocks are zero and there is a gap between the target's backing file's end and the target's end, we do not have to explicitly write zeroes there. Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1527898 Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20180501165750.19242-2-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-06-11qemu-img: Resolve relative backing paths in rebaseMax Reitz
Currently, rebase interprets a relative path for the new backing image as follows: (1) Open the new backing image with the given relative path (thus relative to qemu-img's working directory). (2) Write it directly into the overlay's backing path field (thus relative to the overlay). If the overlay is not in qemu-img's working directory, both will be different interpretations, which may either lead to an error somewhere (either rebase fails because it cannot open the new backing image, or your overlay becomes unusable because its backing path does not point to a file), or, even worse, it may result in your rebase being performed for a different backing file than what your overlay will point to after the rebase. Fix this by interpreting the target backing path as relative to the overlay, like qemu-img does everywhere else. Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1569835 Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20180509182002.8044-2-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
generated by cgit v1.2.3 (git 2.26.2) at 2024-12-15 06:03:31 +0000