aboutsummaryrefslogtreecommitdiff
path: root/tests/qemu-iotests/group
AgeCommit message (Collapse)Author
2021-01-27iotests: rewrite check into pythonVladimir Sementsov-Ogievskiy
Just use classes introduced in previous three commits. Behavior difference is described in these three commits. Drop group file, as it becomes unused. Drop common.env: now check is in python, and for tests we use same python interpreter that runs the check itself. Use build environment PYTHON in check-block instead, to keep "make check" use the same python. Checking for virtio-blk moved to iotests.py, as it actually iotests.py dependency. Actually not all python iotests depend on it, so in future it may be refactored to checked only when really needed. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20210125185056.129513-6-vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-01-26iotests: Add test for the regression fixed in c8bf9a9169Alberto Garcia
Signed-off-by: Alberto Garcia <berto@igalia.com> Suggested-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com> Message-Id: <20210112170540.2912-1-berto@igalia.com> [mreitz: Add "# group:" line] Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26iotests: add #310 to test bottom node in COR driverAndrey Shinkevich
The test case #310 is similar to #216 by Max Reitz. The difference is that the test #310 involves a bottom node to the COR filter driver. Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> [vsementsov: detach backing to test reads from top, limit to qcow2] Message-Id: <20201216061703.70908-7-vsementsov@virtuozzo.com> [mreitz: Add "# group:" line] Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-06tests/iotests: drop test 312 from auto groupAlex Bennée
The "auto" documentation states: That means they should run with every QEMU binary (also non-x86) which is not the case as the check-system-fedora build which only includes a rag tag group of rare and deprecated targets doesn't support the virtio device required. Fixes: ef9bba1484b ("quorum: Implement bdrv_co_block_status()") Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20210105100402.12350-1-alex.bennee@linaro.org>
2020-12-18quorum: Implement bdrv_co_block_status()Alberto Garcia
The quorum driver does not implement bdrv_co_block_status() and because of that it always reports to contain data even if all its children are known to be empty. One consequence of this is that if we for example create a quorum with a size of 10GB and we mirror it to a new image the operation will write 10GB of actual zeroes to the destination image wasting a lot of time and disk space. Since a quorum has an arbitrary number of children of potentially different formats there is no way to report all possible allocation status flags in a way that makes sense, so this implementation only reports when a given region is known to contain zeroes (BDRV_BLOCK_ZERO) or not (BDRV_BLOCK_DATA). If all children agree that a region contains zeroes then we can return BDRV_BLOCK_ZERO using the smallest size reported by the children (because all agree that a region of at least that size contains zeroes). If at least one child disagrees we have to return BDRV_BLOCK_DATA. In this case we use the largest of the sizes reported by the children that didn't return BDRV_BLOCK_ZERO (because we know that there won't be an agreement for at least that size). Signed-off-by: Alberto Garcia <berto@igalia.com> Tested-by: Tao Xu <tao3.xu@intel.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <db83149afcf0f793effc8878089d29af4c46ffe1.1605286097.git.berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18iotests: add 298 to test new preallocate filter driverVladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20201021145859.11201-13-vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-11iotests/308: Add test for FUSE exportsMax Reitz
We have good coverage of the normal I/O paths now, but what remains is a test that tests some more special cases: Exporting an image on itself (thus turning a formatted image into a raw one), some error cases, and non-writable and non-growable exports. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20201027190600.192171-21-mreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-10-30nbd: Add 'qemu-nbd -A' to expose allocation depthEric Blake
Allow the server to expose an additional metacontext to be requested by savvy clients. qemu-nbd adds a new option -A to expose the qemu:allocation-depth metacontext through NBD_CMD_BLOCK_STATUS; this can also be set via QMP when using block-export-add. qemu as client is hacked into viewing the key aspects of this new context by abusing the already-experimental x-dirty-bitmap option to collapse all depths greater than 2, which results in a tri-state value visible in the output of 'qemu-img map --output=json' (yes, that means x-dirty-bitmap is now a bit of a misnomer, but I didn't feel like renaming it as it would introduce a needless break of back-compat, even though we make no compat guarantees with x- members): unallocated (depth 0) => "zero":false, "data":true local (depth 1) => "zero":false, "data":false backing (depth 2+) => "zero":true, "data":true libnbd as client is probably a nicer way to get at the information without having to decipher such hacks in qemu as client. ;) Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20201027050556.269064-11-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2020-10-12qemu-iotests, qtest: rewrite test 067 as a qtestPaolo Bonzini
Test 067 from qemu-iotests is executing QMP commands to hotplug and hot-unplug disks, devices and blockdevs. Because the power of the text-based test harness is limited, it is actually limiting the checks that it does, for example by skipping DEVICE_DELETED events. tests/qtest already has a similar test, drive_del-test.c. We can merge them, and even reuse some of the existing code in drive_del-test.c. This will improve the quality of the test by covering DEVICE_DELETED events and testing multiple architectures (therefore covering multiple PCI hotplug mechanisms as well as s390x virtio-ccw). The only difference is that the new test will always use null-co:// for the medium rather than qcow2 or raw, but this should be irrelevant for what the test is covering. For example there are no "qemu-img check" runs in 067 that would check that the file is properly closed. The new tests requires PCI hot-plug support, so drive_del-test is moved from qemu-system-ppc to qemu-system-ppc64. Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-02iotests: Test block-export-* QMP interfaceKevin Wolf
Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20200924152717.287415-32-kwolf@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-09-15qcow2: Fix removal of list members from BDRVQcow2State.cluster_allocsAlberto Garcia
When a write request needs to allocate new clusters (or change the L2 bitmap of existing ones) a QCowL2Meta structure is created so the L2 metadata can be later updated and any copy-on-write can be performed if necessary. A write request can span a region consisting of an arbitrary combination of previously unallocated and allocated clusters, and if the unallocated ones can be put contiguous to the existing ones then QEMU will do so in order to minimize the number of write operations. In practice this means that a write request has not just one but a number of QCowL2Meta structures. All of them are added to the cluster_allocs list that is stored in BDRVQcow2State and is used to detect overlapping requests. After the write request finishes all its associated QCowL2Meta are removed from that list. calculate_l2_meta() takes care of creating and putting those structures in the list, and qcow2_handle_l2meta() takes care of removing them. The problem is that the error path in handle_alloc() also tries to remove an item in that list, a remnant from the time when this was handled there (that code would not even be correct anymore because it only removes one struct and not all the ones from the same write request). This can trigger a double removal of the same item from the list, causing a crash. This is not easy to reproduce in practice because it requires that do_alloc_cluster_offset() fails after a successful previous allocation during the same write request, but it can be reproduced with the included test case. Signed-off-by: Alberto Garcia <berto@igalia.com> Message-Id: <3440a1c4d53c4fe48312b478c96accb338cbef7c.1599150873.git.berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-08-26iotests: Add tests for qcow2 images with extended L2 entriesAlberto Garcia
Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <e6dd0429cafe84ca603179c298a8703bddca2904.1594396418.git.berto@igalia.com> [mreitz: Use env in shebang line] Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-08-21iotests: Test node/bitmap aliases during migrationMax Reitz
Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200820150725.68687-4-mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Tested-by: Eric Blake <eblake@redhat.com> [eblake: fold in python cleanups recommended by Vladimir] Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-21iotests: add test for QCOW2 header dumpAndrey Shinkevich
The simple script creates a QCOW2 image and fills it with some data. Two bitmaps are created as well. Then the script reads the image header with extensions from the disk by running the script qcow2.py and dumps the information to the output. Other entities, such as snapshots, may be added to the test later. Suggested-by: Eric Blake <eblake@redhat.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <1596742557-320265-2-git-send-email-andrey.shinkevich@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-08-11iotests: add test for unaligned granularity bitmap backupStefan Reiter
Start a VM with a 4097 byte image attached, add a 4096 byte granularity dirty bitmap, mark it dirty, and then do a backup. This used to run into an assert and fail, check that it works as expected and also check the created image to ensure that misaligned backups in general work correctly. Signed-off-by: Stefan Reiter <s.reiter@proxmox.com> Message-Id: <20200810095523.15071-2-s.reiter@proxmox.com> [mreitz: Drop bitmap, and do not write past the image's end] Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-07-28Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2020-07-28' into ↵Peter Maydell
staging nbd patches for 2020-07-28 - fix NBD handling of trim/zero requests larger than 2G - allow no-op resizes on NBD (in turn fixing qemu-img convert -c into NBD) - several deadlock fixes when using NBD reconnect # gpg: Signature made Tue 28 Jul 2020 15:59:42 BST # gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A # gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full] # gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full] # gpg: aka "[jpeg image of size 6874]" [full] # Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A * remotes/ericb/tags/pull-nbd-2020-07-28: block/nbd: nbd_co_reconnect_loop(): don't sleep if drained block/nbd: on shutdown terminate connection attempt block/nbd: allow drain during reconnect attempt block/nbd: split nbd_establish_connection out of nbd_client_connect iotests: Test convert to qcow2 compressed to NBD iotests: Add more qemu_img helpers iotests: Make qemu_nbd_popen() a contextmanager block: nbd: Fix convert qcow2 compressed to nbd nbd: Fix large trim/zero requests Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-07-28iotests: Test convert to qcow2 compressed to NBDNir Soffer
Add test for "qemu-img convert -O qcow2 -c" to NBD target. The tests     create a OVA file and write compressed qcow2 disk content directly into the OVA file via qemu-nbd. Signed-off-by: Nir Soffer <nsoffer@redhat.com> Message-Id: <20200727215846.395443-5-nsoffer@redhat.com> Tested-by: Eric Blake <eblake@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2020-07-27iotests: Adjust which migration tests are quickEric Blake
A quick run of './check -qcow2 -g migration' shows that test 169 is NOT quick, but meanwhile several other tests ARE quick. Let's adjust the test designations accordingly. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200727195117.132151-1-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2020-07-17iotests: test shutdown when bitmap is exported through NBDVladimir Sementsov-Ogievskiy
Test shutdown when bitmap is exported through NBD and active client exists. The previous patch fixes a crash, provoked by this scenario. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Tested-by: Eric Blake <eblake@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200714162234.13113-3-vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-14qcow: Tolerate backing_fmt=Eric Blake
qcow has no space in the metadata to store a backing format, and there are existing qcow images backed both by raw or by other formats (usually qcow) images, reliant on probing to tell the difference. On the bright side, because we probe every time, raw files are marked as probed and we thus forbid a commit action into the backing file where guest-controlled contents could change the result of the probe next time around (the iotest added here proves that). Still, allowing the user to specify the backing format during creation, even if we can't record it, is a good thing. This patch blindly allows any value that resolves to a known driver, even if the user's request is a mismatch from what probing finds; then the next patch will further enhance things to verify that the user's request matches what we actually probe. With this and the next patch in place, we will finally be ready to deprecate the creation of images where a backing format was not explicitly specified by the user. Note that this is only for QemuOpts usage; there is no change to the QAPI to allow a format through -blockdev. Add a new iotest 301 just for qcow, to demonstrate the latest behavior, and to make it easier to show the improvements made in the next patch. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20200706203954.341758-6-eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06iotests: add tests for blockdev-amendMaxim Levitsky
This commit adds two tests that cover the new blockdev-amend functionality of luks and qcow2 driver Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> [mreitz: Let 295 verify that LUKS works; drop 295 and 296 from the auto group] Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200625125548.870061-20-mreitz@redhat.com>
2020-07-06iotests: qemu-img tests for luks key managementMaxim Levitsky
This commit adds two tests, which test the new amend interface of both luks raw images and qcow2 luks encrypted images. Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> [mreitz: Let 293 verify that LUKS works; drop $(seq) usage from 293; drop 293 and 294 from the auto group] Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200625125548.870061-16-mreitz@redhat.com>
2020-05-28iotests: Add test 291 to for qemu-img bitmap coverageEric Blake
Add a new test covering the 'qemu-img bitmap' subcommand, as well as 'qemu-img convert --bitmaps', both added in recent patches. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20200521192137.1120211-6-eblake@redhat.com>
2020-05-18iotests: Run pylint and mypy in a testcaseKevin Wolf
We made sure that iotests.py passes pylint. It would be a shame if we allowed new patches in that break this again, so let's just add a meta-test case that runs pylint on it. While we don't pass mypy --strict yet, we can already run it with a few options that would be part of --strict to make sure that we won't regress on these aspects at least until we can enable the full thing. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20200511163529.349329-3-kwolf@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-05-13iotests: 287: add qcow2 compression type testDenis Plotnikov
The test checks fulfilling qcow2 requirements for the compression type feature and zstd compression type operability. Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Tested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200507082521.29210-5-dplotnikov@virtuozzo.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-05-08qcow2: Avoid integer wraparound in qcow2_co_truncate()Alberto Garcia
After commit f01643fb8b47e8a70c04bbf45e0f12a9e5bc54de when an image is extended and BDRV_REQ_ZERO_WRITE is set then the new clusters are zeroized. The code however does not detect correctly situations when the old and the new end of the image are within the same cluster. The problem can be reproduced with these steps: qemu-img create -f qcow2 backing.qcow2 1M qemu-img create -f qcow2 -F qcow2 -b backing.qcow2 top.qcow2 qemu-img resize --shrink top.qcow2 520k qemu-img resize top.qcow2 567k In the last step offset - zero_start causes an integer wraparound. Signed-off-by: Alberto Garcia <berto@igalia.com> Message-Id: <20200504155217.10325-1-berto@igalia.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-04-30iotests: Test committing to short backing fileKevin Wolf
Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20200424125448.63318-10-kwolf@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-04-07qcow2: Forbid discard in qcow2 v2 images with backing filesAlberto Garcia
A discard request deallocates the selected clusters so they read back as zeroes. This is done by clearing the cluster offset field and setting QCOW_OFLAG_ZERO in the L2 entry. This flag is however only supported when qcow_version >= 3. In older images the cluster is simply deallocated, exposing any possible stale data from the backing file. Since discard is an advisory operation it's safer to simply forbid it in this scenario. Note that we are adding this check to qcow2_co_pdiscard() and not to qcow2_cluster_discard() or discard_in_l2_slice() because the last two are also used by qcow2_snapshot_create() to discard the clusters used by the VM state. In this case there's no risk of exposing stale data to the guest and we really want that the clusters are always discarded. Signed-off-by: Alberto Garcia <berto@igalia.com> Message-Id: <20200331114345.29993-1-berto@igalia.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-03-24iotests/026: Move v3-exclusive test to new fileMax Reitz
data_file does not work with v2, and we probably want 026 to keep working for v2 images. Thus, open a new file for v3-exclusive error path test cases. Fixes: 81311255f217859413c94f2cd9cebf2684bbda94 (“iotests/026: Test EIO on allocation in a data-file”) Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200311140707.1243218-1-mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Tested-by: John Snow <jsnow@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-03-12Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into stagingPeter Maydell
Block layer patches: - Relax restrictions for blockdev-snapshot (allows libvirt to do live storage migration with blockdev-mirror) - luks: Delete created files when block_crypto_co_create_opts_luks fails - Fix memleaks in qmp_object_add # gpg: Signature made Wed 11 Mar 2020 15:38:59 GMT # gpg: using RSA key 7F09B272C88F2FD6 # gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full] # Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6 * remotes/kevin/tags/for-upstream: qemu-iotests: adding LUKS cleanup for non-UTF8 secret error crypto.c: cleanup created file when block_crypto_co_create_opts_luks fails block.c: adding bdrv_co_delete_file block: introducing 'bdrv_co_delete_file' interface tests/qemu-iotests: Fix socket_scm_helper build path qapi: Add '@allow-write-only-overlay' feature for 'blockdev-snapshot' iotests: Add iothread cases to 155 block: Fix cross-AioContext blockdev-snapshot iotests: Test mirror with temporarily disabled target backing file iotests: Fix run_job() with use_log=False block: Relax restrictions for blockdev-snapshot block: Make bdrv_get_cumulative_perm() public qom-qmp-cmds: fix two memleaks in qmp_object_add Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-03-11qemu-iotests: adding LUKS cleanup for non-UTF8 secret errorDaniel Henrique Barboza
This patch adds a new test file to exercise the case where qemu-img fails to complete for the LUKS format when a non-UTF8 secret is used. Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com> Message-Id: <20200130213907.2830642-5-danielhb413@gmail.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-03-11iotests: add 288 luks qemu-img measure testStefan Hajnoczi
This test exercises the block/crypto.c "luks" block driver .bdrv_measure() code. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200221112522.1497712-5-stefanha@redhat.com> [mreitz: Renamed test from 282 to 288] Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20iotests: Test snapshot -l field separationMax Reitz
Add a test that all fields in "qemu-img snapshot -l"s output are separated by spaces. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200117105859.241818-3-mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [mreitz: Renamed test from 284 to 286] Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20iotests: Add test for image creation fallbackMax Reitz
Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200122164532.178040-6-mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com> [mreitz: Added a note that NBD does not support resizing, which is why the second case is expected to fail] Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20block: always fill entire LUKS header space with zerosDaniel P. Berrangé
When initializing the LUKS header the size with default encryption parameters will currently be 2068480 bytes. This is rounded up to a multiple of the cluster size, 2081792, with 64k sectors. If the end of the header is not the same as the end of the cluster we fill the extra space with zeros. This was forgetting that not even the space allocated for the header will be fully initialized, as we only write key material for the first key slot. The space left for the other 7 slots is never written to. An optimization to the ref count checking code: commit a5fff8d4b4d928311a5005efa12d0991fe3b66f9 (refs/bisect/bad) Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Date: Wed Feb 27 16:14:30 2019 +0300 qcow2-refcount: avoid eating RAM made the assumption that every cluster which was allocated would have at least some data written to it. This was violated by way the LUKS header is only partially written, with much space simply reserved for future use. Depending on the cluster size this problem was masked by the logic which wrote zeros between the end of the LUKS header and the end of the cluster. $ qemu-img create --object secret,id=cluster_encrypt0,data=123456 \ -f qcow2 -o cluster_size=2k,encrypt.iter-time=1,\ encrypt.format=luks,encrypt.key-secret=cluster_encrypt0 \ cluster_size_check.qcow2 100M Formatting 'cluster_size_check.qcow2', fmt=qcow2 size=104857600 encrypt.format=luks encrypt.key-secret=cluster_encrypt0 encrypt.iter-time=1 cluster_size=2048 lazy_refcounts=off refcount_bits=16 $ qemu-img check --object secret,id=cluster_encrypt0,data=redhat \ 'json:{"driver": "qcow2", "encrypt.format": "luks", \ "encrypt.key-secret": "cluster_encrypt0", \ "file.driver": "file", "file.filename": "cluster_size_check.qcow2"}' ERROR: counting reference for region exceeding the end of the file by one cluster or more: offset 0x2000 size 0x1f9000 Leaked cluster 4 refcount=1 reference=0 ...snip... Leaked cluster 130 refcount=1 reference=0 1 errors were found on the image. Data may be corrupted, or further writes to the image may corrupt it. 127 leaked clusters were found on the image. This means waste of disk space, but no harm to data. Image end offset: 268288 The problem only exists when the disk image is entirely empty. Writing data to the disk image payload will solve the problem by causing the end of the file to be extended further. The change fixes it by ensuring that the entire allocated LUKS header region is fully initialized with zeros. The qemu-img check will still fail for any pre-existing disk images created prior to this change, unless at least 1 byte of the payload is written to. Fully writing zeros to the entire LUKS header is a good idea regardless as it ensures that space has been allocated on the host filesystem (or whatever block storage backend is used). Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200207135520.2669430-1-berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06iotests: add test for backup-top failure on permission activationVladimir Sementsov-Ogievskiy
This test checks that bug is really fixed by previous commit. Cc: qemu-stable@nongnu.org # v4.2.0 Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20200121142802.21467-3-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06iotests: Enable more tests in the 'auto' group to improve test coverageThomas Huth
According to Kevin, tests 030, 040 and 041 are among the most valuable tests that we have, so we should always run them if possible, even if they take a little bit longer. According to Max, it would be good to have a test for iothreads and migration. 127 and 256 seem to be good candidates for iothreads. For migration, let's enable 181 and 203 (which also tests iothreads). (091 would be a good candidate for migration, too, but Alex Bennée reported that this test fails on ZFS file systems, so it can't be included yet) Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com> Message-id: 20200121095205.26323-7-thuth@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-27iotests: Test handling of AioContexts with some blockdev actionsSergio Lopez
Includes the following tests: - Adding a dirty bitmap. * RHBZ: 1782175 - Starting a drive-mirror to an NBD-backed target. * RHBZ: 1746217, 1773517 - Aborting an external snapshot transaction. * RHBZ: 1779036 - Aborting a blockdev backup transaction. * RHBZ: 1782111 For each one of them, a VM with a number of disks running in an IOThread AioContext is used. Signed-off-by: Sergio Lopez <slp@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-12-19iotests: Test external snapshot with VM stateKevin Wolf
This tests creating an external snapshot with VM state (which results in an active overlay over an inactive backing file, which is also the root node of an inactive BlockBackend), re-activating the images and performing some operations to test that the re-activation worked as intended. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-12-18qemu-img: fix info --backing-chain --image-optsStefan Hajnoczi
Only apply --image-opts to the topmost image when listing an entire backing chain. It is incorrect to treat backing filenames as image options. Assuming we have the backing chain t.IMGFMT.base <- t.IMGFMT.mid <- t.IMGFMT, qemu-img info fails as follows: $ qemu-img info --backing-chain --image-opts \ driver=qcow2,file.driver=file,file.filename=t.IMGFMT qemu-img: Could not open 'TEST_DIR/t.IMGFMT.mid': Cannot find device=TEST_DIR/t.IMGFMT.mid nor node_name=TEST_DIR/t.IMGFMT.mid Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-11-18iotests: Test NBD client reconnectionAndrey Shinkevich
The test for an NBD client. The NBD server is disconnected after the client write request. The NBD client should reconnect and complete the write operation. Suggested-by: Denis V. Lunev <den@openvz.org> Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Tested-by: Eric Blake <eblake@redhat.com> Message-Id: <1573529976-815699-1-git-send-email-andrey.shinkevich@virtuozzo.com>
2019-11-18iotests: Test multiple blockdev-snapshot callsKevin Wolf
Test that doing a second blockdev-snapshot doesn't make the first overlay's backing file go away. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com>
2019-11-07iotests: Add test for 4G+ compressed qcow2 writeMax Reitz
Test what qemu-img check says about an image after one has written compressed data to an offset above 4 GB. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20191028161841.1198-3-mreitz@redhat.com Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-31iotests: Remove 130 from the "auto" groupThomas Huth
Peter hit a "Could not open 'TEST_DIR/t.IMGFMT': Failed to get shared 'write' lock - Is another process using the image [TEST_DIR/t.IMGFMT]?" error with 130 already twice. Looks like this test is a little bit shaky, and currently nobody has a real clue what could be causing this issue, so for the time being, let's disable it from the "auto" group so that it does not gate the pull requests. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-10-28iotests: Test qcow2's snapshot table handlingMax Reitz
Add a test how our qcow2 driver handles extra data in snapshot table entries, and how it repairs overly long snapshot tables. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-id: 20191011152814.14791-17-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-22iotests: test nbd reconnectVladimir Sementsov-Ogievskiy
Add test, which starts backup to nbd target and restarts nbd server during backup. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20191009084158.15614-4-vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2019-10-17iotests: add test 260 to check bitmap life after snapshot + commitVladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20190927122355.7344-8-vsementsov@virtuozzo.com [Maintainer edit: removed 260 from auto group per Peter Maydell. --js] Signed-off-by: John Snow <jsnow@redhat.com>
2019-10-14iotests: Test large write request to qcow2 fileMax Reitz
Without HEAD^, the following happens when you attempt a large write request to a qcow2 file such that the number of bytes covered by all clusters involved in a single allocation will exceed INT_MAX: (A) handle_alloc_space() decides to fill the whole area with zeroes and fails because bdrv_co_pwrite_zeroes() fails (the request is too large). (B) If handle_alloc_space() does not do anything, but merge_cow() decides that the requests can be merged, it will create a too long IOV that later cannot be written. (C) Otherwise, all parts will be written separately, so those requests will work. In either B or C, though, qcow2_alloc_cluster_link_l2() will have an overflow: We use an int (i) to iterate over nb_clusters, and then calculate the L2 entry based on "i << s->cluster_bits" -- which will overflow if the range covers more than INT_MAX bytes. This then leads to image corruption because the L2 entry will be wrong (it will be recognized as a compressed cluster). Even if that were not the case, the .cow_end area would be empty (because handle_alloc() will cap avail_bytes and nb_bytes at INT_MAX, so their difference (which is the .cow_end size) will be 0). So this test checks that on such large requests, the image will not be corrupted. Unfortunately, we cannot check whether COW will be handled correctly, because that data is discarded when it is written to null-co (but we have to use null-co, because writing 2 GB of data in a test is not quite reasonable). Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-10-14block: Reject misaligned write requests with BDRV_REQ_NO_FALLBACKAlberto Garcia
The BDRV_REQ_NO_FALLBACK flag means that an operation should only be performed if it can be offloaded or otherwise performed efficiently. However a misaligned write request requires a RMW so we should return an error and let the caller decide how to proceed. This hits an assertion since commit c8bb23cbdb if the required alignment is larger than the cluster size: qemu-img create -f qcow2 -o cluster_size=2k img.qcow2 4G qemu-io -c "open -o driver=qcow2,file.align=4k blkdebug::img.qcow2" \ -c 'write 0 512' qemu-io: block/io.c:1127: bdrv_driver_pwritev: Assertion `!(flags & BDRV_REQ_NO_FALLBACK)' failed. Aborted The reason is that when writing to an unallocated cluster we try to skip the copy-on-write part and zeroize it using BDRV_REQ_NO_FALLBACK instead, resulting in a write request that is too small (2KB cluster size vs 4KB required alignment). Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-10-04iotests: Test internal snapshots with -blockdevKevin Wolf
Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Peter Krempa <pkrempa@redhat.com> Tested-by: Peter Krempa <pkrempa@redhat.com>
generated by cgit v1.2.3 (git 2.26.2) at 2024-12-12 17:06:33 +0000