aboutsummaryrefslogtreecommitdiff
path: root/tests/qemu-iotests
AgeCommit message (Collapse)Author
2019-06-04iotests: Fix duplicated diff output on failureKevin Wolf
Commit 70ff5b07 wanted to move the diff between actual and reference output to the end after printing the test result line. It really only copied it, though, so the diff is now displayed twice. Remove the old one. Fixes: 70ff5b07fcdd378180ad2d5cc0b0d5e67e7ef325 Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-06-04iotests: test big qcow2 shrinkVladimir Sementsov-Ogievskiy
This test checks bug in qcow2_process_discards, fixed by previous commit. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-06-04iotests: Attach new devices to node in non-default iothreadKevin Wolf
This tests that devices refuse to be attached to a node that has already been moved to a different iothread if they can't be or aren't configured to work in the same iothread. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-06-04scsi-disk: Use qdev_prop_drive_iothreadKevin Wolf
This makes use of qdev_prop_drive_iothread for scsi-disk so that the disk can be attached to a node that is already in the target AioContext. We need to check that the HBA actually supports iothreads, otherwise scsi-disk must make sure that the node is already in the main AioContext. This changes the error message for conflicting iothread settings. Previously, virtio-scsi produced the error message, now it comes from blk_set_aio_context(). Update a test case accordingly. Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-06-04nbd-server: Call blk_set_allow_aio_context_change()Kevin Wolf
The NBD server uses an AioContext notifier, so it can tolerate that its BlockBackend is switched to a different AioContext. Before we start actually calling bdrv_try_set_aio_context(), which checks for consistency, outside of test cases, we need to make sure that the NBD server actually allows this. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2019-06-04iotests: Test cancelling a job and closing the VMMax Reitz
This patch adds a test where we cancel a throttled mirror job and immediately close the VM before it can be cancelled. Doing so will invoke bdrv_drain_all() while the mirror job tries to drain the throttled node. When bdrv_drain_all_end() tries to lift its drain on the throttle node, the job will exit and replace the current root node of the BB drive0 (which is the job's filter node) by the throttle node. Before the previous patch, this replacement did not increase drive0's quiesce_counter by a sufficient amount, so when bdrv_parent_drained_end() (invoked by bdrv_do_drained_end(), invoked by bdrv_drain_all_end()) tried to end the drain on all of the throttle node's parents, it decreased drive0's quiesce_counter below 0 -- which fails an assertion. Signed-off-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-06-04block: avoid recursive block_status call if possibleVladimir Sementsov-Ogievskiy
drv_co_block_status digs bs->file for additional, more accurate search for hole inside region, reported as DATA by bs since 5daa74a6ebc. This accuracy is not free: assume we have qcow2 disk. Actually, qcow2 knows, where are holes and where is data. But every block_status request calls lseek additionally. Assume a big disk, full of data, in any iterative copying block job (or img convert) we'll call lseek(HOLE) on every iteration, and each of these lseeks will have to iterate through all metadata up to the end of file. It's obviously ineffective behavior. And for many scenarios we don't need this lseek at all. However, lseek is needed when we have metadata-preallocated image. So, let's detect metadata-preallocation case and don't dig qcow2's protocol file in other cases. The idea is to compare allocation size in POV of filesystem with allocations size in POV of Qcow2 (by refcounts). If allocation in fs is significantly lower, consider it as metadata-preallocation case. 102 iotest changed, as our detector can't detect shrinked file as metadata-preallocation, which don't seem to be wrong, as with metadata preallocation we always have valid file length. Two other iotests have a slight change in their QMP output sequence: Active 'block-commit' returns earlier because the job coroutine yields earlier on a blocking operation. This operation is loading the refcount blocks in qcow2_detect_metadata_preallocation(). Suggested-by: Denis V. Lunev <den@openvz.org> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-06-04iotests: Test commit job start with concurrent I/OKevin Wolf
This tests that concurrent requests are correctly drained before making graph modifications instead of running into assertions in bdrv_replace_node(). Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com>
2019-05-30Merge remote-tracking branch 'remotes/jnsnow/tags/bitmaps-pull-request' into ↵Peter Maydell
staging Pull request # gpg: Signature made Wed 29 May 2019 00:58:33 BST # gpg: using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E # gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full] # Primary key fingerprint: FAEB 9711 A12C F475 812F 18F2 88A9 064D 1835 61EB # Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76 CBD0 7DEF 8106 AAFC 390E * remotes/jnsnow/tags/bitmaps-pull-request: iotests: test external snapshot with bitmap copying qapi: support external bitmaps in block-dirty-bitmap-merge migration/dirty-bitmaps: change bitmap enumeration method Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-05-28iotests: test external snapshot with bitmap copyingVladimir Sementsov-Ogievskiy
This test shows that external snapshots and incremental backups are friends. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20190517152111.206494-3-vsementsov@virtuozzo.com Signed-off-by: John Snow <jsnow@redhat.com>
2019-05-28blockdev: loosen restrictions on drive-backup source nodeJohn Snow
We mandate that the source node must be a root node; but there's no reason I am aware of that it needs to be restricted to such. In some cases, we need to make sure that there's a medium present, but in the general case we can allow the backup job itself to do the graph checking. This patch helps improve the error message when you try to backup from the same node more than once, which is reflected in the change to test 056. For backups with bitmaps, it will also show a better error message that the bitmap is in use instead of giving you something cryptic like "need a root node." Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1707303 Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20190521210053.8864-1-jsnow@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28qcow2: skip writing zero buffers to empty COW areasAnton Nefedov
If COW areas of the newly allocated clusters are zeroes on the backing image, efficient bdrv_write_zeroes(flags=BDRV_REQ_NO_FALLBACK) can be used on the whole cluster instead of writing explicit zero buffers later in perform_cow(). iotest 060: write to the discarded cluster does not trigger COW anymore. Use a backing image instead. Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com> Message-id: 20190516142749.81019-2-anton.nefedov@virtuozzo.com Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28tests/qemu-iotests: re-format output to for make check-blockAlex Bennée
This attempts to clean-up the output to better match the output of the rest of the QEMU check system when called with -makecheck. This includes: - formatting as " TEST iotest-FMT: nnn" - only dumping config on failure (when -makecheck enabled) The non-make check output has been cleaned up as well: - line re-displayed (\r) at the end - fancy colours for pass/fail/skip - timestamps always printed (option removed) Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20190503143904.31211-1-alex.bennee@linaro.org> Tested-by: Thomas Huth <thuth@redhat.com>
2019-05-28tests/qemu-iotests/group: Re-use the "auto" group for tests that can always runThomas Huth
Currently, all tests are in the "auto" group. This is a little bit pointless. OTOH, we need a group for the tests that we can automatically run during "make check" each time, too. Tests in this new group are supposed to run with every possible QEMU configuration, for example they must run with every QEMU binary (also non-x86), without failing when an optional features is missing (but reporting "skip" is ok), and be able to run on all kind of host filesystems and users (i.e. also as "nobody" or "root"). So let's use the "auto" group for this class of tests now. The initial list has been determined by running the iotests with non-x86 QEMU targets and with our CI pipelines on Gitlab, Cirrus-CI and Travis (i.e. including macOS and FreeBSD). Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <20190502084506.8009-7-thuth@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-05-21tests/qemu-iotests: Remove the "_supported_os Linux" line from many testsThomas Huth
A lot of tests run fine on FreeBSD and macOS, too - the limitation to Linux here was likely just copied-and-pasted from other tests. Thus remove the "_supported_os Linux" line from tests that run successful in our CI pipelines on FreeBSD and macOS. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Tested-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Christian Borntraeger <borntraeger@de.ibm.com> Message-Id: <20190502084506.8009-6-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2019-05-21tests/qemu-iotests: Do not hard-code the path to bashThomas Huth
bash is installed in a different directory on non-Linux systems like FreeBSD. Do not hard-code /bin/bash here so that the tests can run there, too. Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Christian Borntraeger <borntraeger@de.ibm.com> Message-Id: <20190502084506.8009-4-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2019-05-21tests/qemu-iotests/check: Pick a default machine if necessaryThomas Huth
qemu-system-arm, qemu-system-aarch64 and qemu-system-tricore do not have a default machine, so when running the qemu-iotests with such a binary, lots of tests are failing. Fix it by picking a default machine in the "check" script instead. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Acked-by: Christian Borntraeger <borntraeger@de.ibm.com> Message-Id: <20190502084506.8009-3-thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2019-05-21tests/qemu-iotests/005: Add a sanity check for large sparse file supportThomas Huth
"check -raw 005" fails when running on certain filesystems - these do not support such large sparse files. Use the same check as in test 220 to skip the test in this case. Suggested-by: Eric Blake <eblake@redhat.com> Message-Id: <20190502084506.8009-2-thuth@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Thomas Huth <thuth@redhat.com>
2019-05-20iotests: Make 245 faster and more reliableMax Reitz
Sometimes, 245 fails for me because some stream job has already finished while the test expects it to still be active. (With -c none, it fails basically every time.) The most reliable way to fix this is to simply set auto_finalize=false so the job will remain in the block graph as long as we need it. This allows us to drop the rate limiting, too, which makes the test faster. The only problem with this is that there is a single place that yields a different error message depending on whether the stream job is still copying data (so COR is enabled) or not (COR has been disabled, but the job still has the WRITE_UNCHANGED permission on the target node). We can easily address that by expecting either error message. Note that we do not need auto_finalize=false (or rate limiting) for the active commit job, because It never completes without an explicit block-job-complete anyway. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-05-20iotests.py: Fix VM.run_jobMax Reitz
log() is in the current module, there is no need to prefix it. In fact, doing so may make VM.run_job() unusable in tests that never use iotests.log() themselves. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-05-20iotests.py: Let assert_qmp() accept an arrayMax Reitz
Sometimes we cannot tell which error message qemu will emit, and we do not care. With this change, we can then just pass an array of all possible messages to assert_qmp() and it will choose the right one. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-05-20iotests: Test unaligned raw images with O_DIRECTMax Reitz
We already have 221 for accesses through the page cache, but it is better to create a new file for O_DIRECT instead of integrating those test cases into 221. This way, we can make use of _supported_cache_modes (and _default_cache_mode) so the test is automatically skipped on filesystems that do not support O_DIRECT. As part of the split, add _supported_cache_modes to 221. With that, it no longer fails when run with -c none or -c directsync. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-05-20iotest: fix 169: do not run qmp_cont in RUN_STATE_FINISH_MIGRATEVladimir Sementsov-Ogievskiy
qmp_cont fails if vm in RUN_STATE_FINISH_MIGRATE, so let's wait for final RUN_STATE_POSTMIGRATE. Also, while being here, check qmp_cont result. Reported-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Tested-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-05-10iotests: Add test for rebase without input baseMax Reitz
This patch adds a test for rebasing an image that currently does not have a backing file. Signed-off-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-05-10qemu-iotests: Fix cleanup for 192Kevin Wolf
Test case 192 calls _launch_qemu, so it also needs to _cleanup_qemu when it's done, otherwise the QMP FIFOs stay around in scratch/. It also creates a temporary NBD socket that needs to be removed as well at the end of the test case. Reported-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Tested-by: Thomas Huth <thuth@redhat.com>
2019-05-09Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-05-07' ↵Peter Maydell
into staging Block patches: - Fixes to qcow2's implementation of qemu-img check - Our SSH driver now supports bdrv_refresh_filename() - Miscellaneous fixes # gpg: Signature made Tue 07 May 2019 16:16:58 BST # gpg: using RSA key F407DB0061D5CF40 # gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full] # Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40 * remotes/maxreitz/tags/pull-block-2019-05-07: iotests: Fix iotests 110 and 126 commit: Use bdrv_append() in commit_start() block: Assert that drv->bdrv_child_perm is set in bdrv_child_perm() block/ssh: Implement .bdrv_dirname() block/ssh: Implement .bdrv_refresh_filename() qcow2: discard bitmap when removed qcow2-refcount: don't mask corruptions under internal errors qcow2-refcount: check_refcounts_l2: don't count fixed cluster as allocated qcow2-refcount: check_refcounts_l2: reduce ignored overlaps qcow2-refcount: avoid eating RAM qcow2-refcount: fix check_oflag_copied Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-05-07iotests: Fix iotests 110 and 126Max Reitz
A recent patch results in qemu-img reporting the backing file format of vmdk images as vmdk. This broke iotests 110 and 126. Fixes: 7502be838e2fb62cc00f9e55f632e0b88ead5e6e Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20190415154129.31021-1-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-07block/ssh: Implement .bdrv_refresh_filename()Max Reitz
This requires some changes to keep iotests 104 and 207 working. qemu-img info in 104 will now return a filename including the user name and the port, which need to be filtered by adjusting REMOTE_TEST_DIR in common.rc. This additional information has to be marked optional, however (which is simple as REMOTE_TEST_DIR is a regex), because otherwise 197 and 215 would fail: They use it (indirectly) to filter qemu-img create output which contains a backing filename they have passed to it -- which probably does not contain a user name or port number. The problem in 207 is a nice one to have: qemu-img info used to return json:{} filenames, but with this patch it returns nice plain ones. We now need to adjust the filtering to hide the user name (and port number while we are at it). The simplest way to do this is to include both in iotests.remote_filename() so that bdrv_refresh_filename() will not change it, and then iotests.img_info_log() will filter it correctly automatically. Signed-off-by: Max Reitz <mreitz@redhat.com> Tested-by: Richard W.M. Jones <rjones@redhat.com> Message-id: 20190225190828.17726-2-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-07qcow2-refcount: avoid eating RAMVladimir Sementsov-Ogievskiy
qcow2_inc_refcounts_imrt() (through realloc_refcount_array()) can eat an unpredictable amount of memory on corrupted table entries, which are referencing regions far beyond the end of file. Prevent this, by skipping such regions from further processing. Interesting that iotest 138 checks exactly the behavior which we fix here. So, change the test appropriately. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20190227131433.197063-3-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-07iotests: Make 182 do without device_addMax Reitz
182 fails if qemu has no support for hotplugging of a virtio-blk device. Using an NBD server instead works just as well for the test, even on qemus without hotplugging support. Fixes: 6d0a4a0fb5c8f10c8eb68b52cfda0082b00ae963 Reported-by: Danilo C. L. de Paula <ddepaula@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20190417153005.30096-1-mreitz@redhat.com> Tested-by: Eric Blake <eblake@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Acked-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2019-05-07iotests: Tweak 221 sizing for different hole granularitiesEric Blake
For some particular configurations of ext4, sizing an image to 84 sectors + 1 byte causes test failures when the size of the hole is rounded to a 4k alignment. Let's instead size things to 128 sectors + 1 byte, as the 64k boundary is more likely to work with various hole granularities. Reported-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20190506172111.31594-1-eblake@redhat.com> Tested-by: Thomas Huth <thuth@redhat.com>
2019-05-07tests/qemu-iotests: Fix more reference output files due to recent qemu-io changeThomas Huth
The output of qemu-io changed recently - most tests have been fixed in commit 36b9986b08787019ef42 ("tests/qemu-iotests: Fix output of qemu-io related tests") already, but a qcow1, vmdk, and nbd test were still missing. Fixes: 99e98d7c9fc1a1639fad ("qemu-io: Use error_[gs]et_progname()") Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <20190501134127.21104-1-thuth@redhat.com> [eblake: squash in NBD 083 fixes] Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Tested-by: Stefano Garzarella <sgarzare@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2019-05-07iotests: Fix 233 for ports other than 10809Max Reitz
233 generally filters the port, but in two cases does not. If some other concurrently running application has already taken port 10809, this will result in an output mismatch. Fix this by applying the filter in these two cases, too. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-Id: <20190506160529.6955-1-mreitz@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2019-04-30iotests: Check that images are in read-only mode after block-commitAlberto Garcia
This tests the fix from the previous patch. Signed-off-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-30qemu-img: Make create hint at protocol optionsMax Reitz
qemu-img create allows giving just a format and "-o help" to get a list of the options supported by that format. Users may not realize that the protocol level may offer even more options, which they only get to see by specifying a filename. This patch adds a note to hint at that fact. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-30iotests: Perform the correct test in 082Max Reitz
In the "amend" section of 082, we perform a single "convert" test (namely "convert -o help"). That does not make sense, especially because we have done exactly that "convert" test earlier in 082 already. Replacing "convert" by "amend" yields an error, which is correct because there is no point in "amend" having a default format. The user has to either specify the format, or give a file for qemu-img to probe. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-30qemu-img: Saner printing of large file sizesEric Blake
Disk sizes close to INT64_MAX cause overflow, for some pretty ridiculous output: $ ./nbdkit -U - memory size=$((2**63 - 512)) --run 'qemu-img info $nbd' image: nbd+unix://?socket=/tmp/nbdkitHSAzNz/socket file format: raw virtual size: -8388607T (9223372036854775296 bytes) disk size: unavailable But there's no reason to have two separate implementations of integer to human-readable abbreviation, where one has overflow and stops at 'T', while the other avoids overflow and goes all the way to 'E'. With this patch, the output now claims 8EiB instead of -8388607T, which really is the correct rounding of largest file size supported by qemu (we could go 511 bytes larger if we used byte-accurate sizing instead of rounding up to the next sector boundary, but that wouldn't change the human-readable result). Quite a few iotests need updates to expected output to match. Reported-by: Richard W.M. Jones <rjones@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Tested-by: Richard W.M. Jones <rjones@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> Tested-by: Max Reitz <mreitz@redhat.com>
2019-04-30tests/qemu-iotests: Fix output of qemu-io related testsThomas Huth
One of the recent commits changed the way qemu-io prints out its errors and warnings - they are now prefixed with the program name. We've got to adapt the iotests accordingly to prevent that they are failing. Fixes: 99e98d7c9fc1a1639fad ("qemu-io: Use error_[gs]et_progname()") Signed-off-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-12iotest: Fix 241 to run in generic directoryEric Blake
Filter the qemu-nbd server output to get rid of a direct reference to my build directory. Fixes: e9dce9cb Reported-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-12iotests: Let 245 pass on tmpfsMax Reitz
tmpfs does not support O_DIRECT. Detect this case, and skip flipping @direct if the filesystem does not support it. Fixes: bf3e50f6239090e63a8ffaaec971671e66d88e07 Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-02tests/qemu-iotests/235: Allow fallback to tcgThomas Huth
iotest 235 currently only works with KVM - this is bad for systems where it is not available, e.g. CI pipelines. The test also works when using "tcg" as accelerator, so we can simply add that to the list of accelerators, too. Signed-off-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-02block: test block-stream with a base node that is used by block-commitAlberto Garcia
The base node of a block-stream operation indicates the first image from the backing chain starting from which no data is copied to the top node. The block-stream job allows others to use that base image, so a second block-stream job could be writing to it at the same time. An important restriction is that the base image must not disappear while the stream job is ongoing. stream_start() freezes the backing chain from top to base with that purpose but it does it too late in the code so there is a race condition there. This bug was fixed in the previous commit, and this patch contains an iotest for this scenario. Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-02tests/qemu-iotests: Remove redundant COPYING fileThomas Huth
The file tests/qemu-iotests/COPYING is the same text as in the COPYING file in the main directory. So as far as I can see, we don't need the duplicate here. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-02iotests: Fix test 200 on s390x without virtio-pciThomas Huth
virtio-pci is optional on s390x, e.g. in downstream RHEL builds, it is disabled. On s390x, virtio-ccw should be used instead. Other tests like 051 or 240 already use virtio-scsi-ccw instead of virtio-scsi-pci on s390x, so let's do the same here and always use virtio-scsi-ccw on s390x. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-04-01nbd/server: Advertise actual minimum block sizeEric Blake
Both NBD_CMD_BLOCK_STATUS and structured NBD_CMD_READ will split their reply according to bdrv_block_status() boundaries. If the block device has a request_alignment smaller than 512, but we advertise a block alignment of 512 to the client, then this can result in the server reply violating client expectations by reporting a smaller region of the export than what the client is permitted to address (although this is less of an issue for qemu 4.0 clients, given recent client patches to overlook our non-compliance at EOF). Since it's always better to be strict in what we send, it is worth advertising the actual minimum block limit rather than blindly rounding it up to 512. Note that this patch is not foolproof - it is still possible to provoke non-compliant server behavior using: $ qemu-nbd --image-opts driver=blkdebug,align=512,image.driver=file,image.filename=/path/to/non-aligned-file That is arguably a bug in the blkdebug driver (it should never pass back block status smaller than its alignment, even if it has to make multiple bdrv_get_status calls and determine the least-common-denominator status among the group to return). It may also be possible to observe issues with a backing layer with smaller alignment than the active layer, although so far I have been unable to write a reliable iotest for that scenario (but again, an issue like that could be argued to be a bug in the block layer, or something where we need a flag to bdrv_block_status() to state whether the result must be aligned to the current layer's limits or can be subdivided for accuracy when chasing backing files). Anyways, as blkdebug is not normally used, and as this patch makes our server more interoperable with qemu 3.1 clients, it is worth applying now, even while we still work on a larger patch series for the 4.1 timeframe to have byte-accurate file lengths. Note that the iotests output changes - for 223 and 233, we can see the server's better granularity advertisement; and for 241, the three test cases have the following effects: - natural alignment: the server's smaller alignment is now advertised, and the hole reported at EOF is now the right result; we've gotten rid of the server's non-compliance - forced server alignment: the server still advertises 512 bytes, but still sends a mid-sector hole. This is still a server compliance bug, which needs to be fixed in the block layer in a later patch; output does not change because the client is already being tolerant of the non-compliance - forced client alignment: the server's smaller alignment means that the client now sees the server's status change mid-sector without any protocol violations, but the fact that the map shows an unaligned mid-sector hole is evidence of the block layer problems with aligned block status, to be fixed in a later patch Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20190329042750.14704-7-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> [eblake: rebase to enhanced iotest 241 coverage]
2019-03-30nbd/client: Report offsets in bdrv_block_statusEric Blake
It is desirable for 'qemu-img map' to have the same output for a file whether it is served over file or nbd protocols. However, ever since we implemented block status for NBD (2.12), the NBD protocol forgot to inform the block layer that as the final layer in the chain, the offset is valid; without an offset, the human-readable form of qemu-img map gives up with the unhelpful: $ nbdkit -U - data data="1" size=512 --run 'qemu-img map $nbd' Offset Length Mapped to File qemu-img: File contains external, encrypted or compressed clusters. The --output=json form always works, because it is reporting the lower-level bdrv_block_status results directly rather than trying to filter out sparse ranges for human consumption - but now it also shows the offset member. With this patch, the human output changes to: Offset Length Mapped to File 0 0x200 0 nbd+unix://?socket=/tmp/nbdkitOxeoLa/socket This change is observable to several iotests. Fixes: 78a33ab5 Reported-by: Richard W.M. Jones <rjones@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20190329042750.14704-4-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-03-30iotests: Add 241 to test NBD on unaligned imagesEric Blake
Add a test for the NBD client workaround in the previous patch. It's not really feasible for an iotest to assume a specific tracing engine, so we can't really probe trace_nbd_parse_blockstatus_compliance to see if the server was fixed vs. whether the client just worked around the server (other than by rearranging order between code patches and this test). But having a successful exchange sure beats the previous state of an error message. Since format probing can change alignment, we can use that as an easy way to test several configurations. Not tested yet, but worth adding to this test in future patches: an NBD server that can advertise a non-sector-aligned size (such as nbdkit) causes qemu as the NBD client to misbehave when it rounds the size up and accesses beyond the advertised size. Qemu as NBD server never advertises a non-sector-aligned size (since bdrv_getlength() currently rounds up to sector boundaries); until qemu can act as such a server, testing that flaw will have to rely on external binaries. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20190329042750.14704-2-eblake@redhat.com> Tested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> [eblake: add forced-512 alignment, and nbdkit reproducer comment]
2019-03-26iotests: add 248: test resume mirror after auto pause on ENOSPCVladimir Sementsov-Ogievskiy
Test that mirror job actually resume on resume command after being automatically paused on ENOSPC error. It's a follow-up test for 8d9648cbf3e "blockjob: fix user pause in block_job_error_action" Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Tested-by: John Snow <jsnow@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-19qemu-iotests: Treat custom TEST_DIR in 051Lukáš Doktor
When custom TEST_DIR is specified the output includes it without leading '/': $ TEST_DIR=/var/tmp ./check -file -qcow2 051 .... -drive0 (NODE_NAME): json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}}, "driver": "qcow2", "file": {"driver": "file", "filename": SNAPSHOT_PATH}} (qcow2, read-only) +drive0 (NODE_NAME): json:{"backing": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/t.qcow2"}}, "driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/vl.ziHfeP"}} (qcow2, read-only) Let's remove it from the sed regexp. Signed-off-by: Lukáš Doktor <ldoktor@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-19qemu-iotests: Fix 232 for non-qcow2Kevin Wolf
232 is marked as generic, but commit 12efe428c9e added code that assumes qcow2. What the new test really needs is backing files and support for updating the backing file link (.bdrv_change_backing_file). Split the non-generic code into a new test case 247 and make it work with qed, too. Signed-off-by: Kevin Wolf <kwolf@redhat.com>