aboutsummaryrefslogtreecommitdiff
path: root/tests
AgeCommit message (Collapse)Author
2017-11-29blockjob: remove clock argument from block_job_sleep_nsPaolo Bonzini
All callers are using QEMU_CLOCK_REALTIME, and it will not be possible to support more than one clock when block_job_sleep_ns switches to a single timer stored in the BlockJob struct. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Tested-By: Jeff Cody <jcody@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2017-11-27iotests: fix 075 and 078John Snow
Both of these tests are for formats which now stipulate that they are read-only. Adjust the tests to match. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Lukáš Doktor <ldoktor@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2017-11-21Merge remote-tracking branch 'remotes/cody/tags/block-pull-request' into stagingPeter Maydell
# gpg: Signature made Tue 21 Nov 2017 17:01:33 GMT # gpg: using RSA key 0xBDBE7B27C0DE3057 # gpg: Good signature from "Jeffrey Cody <jcody@redhat.com>" # gpg: aka "Jeffrey Cody <jeff@codyprime.org>" # gpg: aka "Jeffrey Cody <codyprime@gmail.com>" # Primary key fingerprint: 9957 4B4D 3474 90E7 9D98 D624 BDBE 7B27 C0DE 3057 * remotes/cody/tags/block-pull-request: qemu-iotest: add test for blockjob coroutine race condition qemu-iotests: add option in common.qemu for mismatch only coroutine: abort if we try to schedule or enter a pending coroutine blockjob: do not allow coroutine double entry or entry-after-completion Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2017-11-21qemu-iotest: add test for blockjob coroutine race conditionJeff Cody
Signed-off-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
2017-11-21qemu-iotests: add option in common.qemu for mismatch onlyJeff Cody
Add option to echo response to QMP / HMP command only on mismatch. Useful for ignore all normal responses, but catching things like segfaults. Signed-off-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
2017-11-21iotests: Fix 176 on 32-bit hostEric Blake
The contents of a qcow2 bitmap are rounded up to a size that matches the number of bits available for the granularity, but that granularity differs for 32-bit hosts (our default 64k cluster allows for 2M bitmap coverage per 'long') and 64-bit hosts (4M bitmap per 'long'). If the image is a multiple of 2M but not 4M, then the number of bytes occupied by the array of longs in memory differs between architecture, thus resulting in different SHA256 hashes. Furthermore (but untested by me), if our computation of the SHA256 hash is at all endian-dependent because of how we store data in memory, that's another variable we'd have to account for (ideally, we specified the bitmap stored in qcow2 as fixed-endian on disk, because the same qcow2 file must be usable across any architecture; but that says nothing about how we represent things in memory). But we already have test 165 to validate that bitmaps are stored correctly on disk, while this test is merely testing that the bitmap exists. So for this test, the easiest solution is to filter out the actual hash value. Broken in commit 4096974e. Reported-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-id: 20171117190422.23626-1-eblake@redhat.com Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-21block: Close a BlockDriverState completely even when bs->drv is NULLAlberto Garcia
bdrv_close() skips much of its logic when bs->drv is NULL. This is fine when we're closing a BlockDriverState that has just been created (because e.g the initialization process failed), but it's not enough in other cases. For example, when a valid qcow2 image is found to be corrupted then QEMU marks it as such in the file header and then sets bs->drv to NULL in order to make the BlockDriverState unusable. When that BDS is later closed then many of its data structures are not freed (leaking their memory) and none of its children are detached. This results in bdrv_close_all() failing to close all BDSs and making this assertion fail when QEMU is being shut down: bdrv_close_all: Assertion `QTAILQ_EMPTY(&all_bdrv_states)' failed. This patch makes bdrv_close() do the full uninitialization process in all cases. This fixes the problem with corrupted images and still works fine with freshly created BDSs. Signed-off-by: Alberto Garcia <berto@igalia.com> Message-id: 20171106145345.12038-1-berto@igalia.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17iotests: Make 087 pass without AIO enabledMax Reitz
If AIO has not been enabled in the qemu build that is to be tested, we should skip the "aio=native without O_DIRECT" test instead of failing. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171115180732.31753-1-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17qcow2: Refuse to get unaligned offsets from cacheMax Reitz
Instead of using an assertion, it is better to emit a corruption event here. Checking all offsets for correct alignment can be tedious and it is easily possible to forget to do so. qcow2_cache_do_get() is a function every L2 and refblock access has to go through, so this is a good central point to add such a check. And for good measure, let us also add an assertion that the offset is non-zero. Making this a corruption event is not feasible, because a zero offset usually means something special (such as the cluster is unused), so all callers should be checking this anyway. If they do not, it is their fault, hence the assertion here. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171110203111.7666-6-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17qcow2: Add bounds check to get_refblock_offset()Max Reitz
Reported-by: R. Nageswara Sastry <nasastry@in.ibm.com> Buglink: https://bugs.launchpad.net/qemu/+bug/1728661 Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171110203111.7666-5-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17block: Guard against NULL bs->drvMax Reitz
We currently do not guard everywhere against a NULL bs->drv where we should be doing so. Most of the places fixed here just do not care about that case at all. Some care implicitly, e.g. through a prior function call to bdrv_getlength() which would always fail for an ejected BDS. Add an assert there to make it more obvious. Other places seem to care, but do so insufficiently: Freeing clusters in a qcow2 image is an error-free operation, but it may leave the image in an unusable state anyway. Giving qcow2_free_clusters() an error code is not really viable, it is much easier to note that bs->drv may be NULL even after a successful driver call. This concerns bdrv_co_flush(), and the way the check is added to bdrv_co_pdiscard() (in every iteration instead of only once). Finally, some places employ at least an assert(bs->drv); somewhere, that may be reasonable (such as in the reopen code), but in bdrv_has_zero_init(), it is definitely not. Returning 0 there in case of an ejected BDS saves us much headache instead. Reported-by: R. Nageswara Sastry <nasastry@in.ibm.com> Buglink: https://bugs.launchpad.net/qemu/+bug/1728660 Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171110203111.7666-4-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17qcow2: Unaligned zero cluster in handle_alloc()Max Reitz
We should check whether the cluster offset we are about to use is actually valid; that is, whether it is aligned to cluster boundaries. Reported-by: R. Nageswara Sastry <nasastry@in.ibm.com> Buglink: https://bugs.launchpad.net/qemu/+bug/1728643 Buglink: https://bugs.launchpad.net/qemu/+bug/1728657 Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171110203111.7666-3-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17qcow2: check_errors are fatalMax Reitz
When trying to repair a dirty image, qcow2_check() may apparently succeed (no really fatal error occurred that would prevent the check from continuing), but if check_errors in the result object is non-zero, we cannot trust the image to be usable. Reported-by: R. Nageswara Sastry <nasastry@in.ibm.com> Buglink: https://bugs.launchpad.net/qemu/+bug/1728639 Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171110203111.7666-2-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17iotests: Add test for failing qemu-img commitMax Reitz
Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20170616135847.17726-1-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17tests: Add check-qobject for equality testsMax Reitz
Add a new test file (check-qobject.c) for unit tests that concern QObjects as a whole. Its only purpose for now is to test the qobject_is_equal() function. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171114180128.17076-7-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17iotests: Add test for non-string option reopeningMax Reitz
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-id: 20171114180128.17076-6-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17qapi/qnull: Add own headerMax Reitz
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-id: 20171114180128.17076-2-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17qcow2: fix image corruption on commit with persistent bitmapEric Blake
If an image contains persistent bitmaps, we cannot use the fast path of bdrv_make_empty() to clear the image during qemu-img commit, because that will lose the clusters related to the bitmaps. Also leave a comment in qcow2_read_extensions to remind future feature additions to think about fast-path removal, since we just barely fixed the same bug for LUKS encryption. It's a pain that qemu-img has not yet been taught to manipulate, or even at a very minimum display, information about persistent bitmaps; instead, we have to use QMP commands. It's also a pain that only qeury-block and x-debug-block-dirty-bitmap-sha256 will allow bitmap introspection; but the former requires the node to be hooked to a block device, and the latter is experimental. Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2017-11-17iotests: test clearing unknown autoclear_features by qcow2Vladimir Sementsov-Ogievskiy
Test clearing unknown autoclear_features by qcow2 on incoming migration. [ kwolf: Fixed wait for destination VM startup ] Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com>
2017-11-17qcow2: fix image corruption after committing qcow2 image into baseDaniel P. Berrange
After committing the qcow2 image contents into the base image, qemu-img will call bdrv_make_empty to drop the payload in the layered image. When this is done for qcow2 images, it blows away the LUKS encryption header, making the resulting image unusable. There are two codepaths for emptying a qcow2 image, and the second (slower) codepath leaves the LUKS header intact, so force use of that codepath. Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2017-11-17qemu-iotests: Use -nographic in 182Kevin Wolf
This avoids that random UI frontend error messages end up in the output. In particular, we were seeing this line in CI error logs: +Unable to init server: Could not connect: Connection refused Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Kashyap Chamarthy <kchamart@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com>
2017-11-16tests/bios-tables-test: Fix endianess problems when passing data to iaslThomas Huth
The bios-tables-test was writing out files that we pass to iasl in with the wrong endianness in the header when running on a big endian host. So instead of storing mixed endian information in our structures, let's keep everything in little endian and byte-swap it only when we need a value in the code. Reported-by: Daniel P. Berrange <berrange@redhat.com> Buglink: https://bugs.launchpad.net/qemu/+bug/1724570 Suggested-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com> Tested-by: "Daniel P. Berrange" <berrange@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2017-11-16tests/acpi-test-data: update _CRS in DSDTMichael S. Tsirkin
commit dadf988e81b15065ac1d6dbaf4b87b5b80c7b670 hw/pci-host: Fix x86 Host Bridges 64bit PCI hole Added a 64 bit hole to _CRS of PCI0. Update the expected files accordingly. Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2017-11-14qemu-iotests: update unsupported image formats in 194Jeff Cody
Test 194 checks for 'luks' to exclude as an unsupported format, However, most formats are unsupported, due to migration blockers. Rather than specifying a blacklist of unsupported formats, whitelist supported formats (specifically, qcow2, qed, raw, dmg). Tested-by: Alexey Kardashevskiy <aik@ozlabs.ru> Signed-off-by: Jeff Cody <jcody@redhat.com> Message-id: 23ca18c7f843c86a28b1529ca9ac6db4b35ca0e4.1510059970.git.jcody@redhat.com Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Denis V. Lunev <den@openvz.org> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14iotests: 077: Filter out 'resume' linesFam Zheng
In the "Overlapping multiple requests" cases, the 3rd reqs (the break point B) doesn't wait for the 2nd, and once resumed the I/O will just continue. This is because the 2nd is already waiting for the 1st, and in wait_serialising_requests() there is: /* If the request is already (indirectly) waiting for us, or * will wait for us as soon as it wakes up, then just go on * (instead of producing a deadlock in the former case). */ if (!req->waiting_for) { /* actually break */ ... } Consequently, the following "sleep 100; resume A" command races with the completion of that request, and sometimes results in an unexpected order of output: > @@ -56,9 +56,9 @@ > wrote XXX/XXX bytes at offset XXX > XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > blkdebug: Resuming request 'B' > +blkdebug: Resuming request 'A' > wrote XXX/XXX bytes at offset XXX > XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > -blkdebug: Resuming request 'A' > wrote XXX/XXX bytes at offset XXX > XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > wrote XXX/XXX bytes at offset XXX Filter out the "Resuming request" lines to make the output deterministic. Reported-by: Patchew <no-reply@patchew.org> Signed-off-by: Fam Zheng <famz@redhat.com> Message-id: 20171113150026.4743-1-famz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14qcow2: Check that corrupted images can be repaired in iotest 060Alberto Garcia
We just fixed a few bugs that caused QEMU to crash when trying to write to corrupted qcow2 images, and iotest 060 was expanded to test all those scenarios. In almost all cases the corrupted images can be repaired using qemu-img, so this patch verifies that. Signed-off-by: Alberto Garcia <berto@igalia.com> Message-id: 0b1b95340ecdfbc6927e36adf2fd42ae6198747a.1510143008.git.berto@igalia.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14iotests: Use new-style NBD connectionsEric Blake
Old-style NBD is deprecated upstream (it is documented, but no longer implemented in the reference implementation), and it is severely limited (it cannot support structured replies, which means it cannot support efficient handling of zeroes), when compared to new-style NBD. We are better off having our iotests favor new-style everywhere (although some explicit tests, particularly 83, still cover old-style for back-compat reasons); this is as simple as supplying the empty string as the default export name, as it does not change the URI needed to connect a client to the server. This also gives us more coverage of the just-added structured reply code, when not overriding $QEMU_NBD to intentionally point to an older server. Signed-off-by: Eric Blake <eblake@redhat.com> Message-id: 20171109221216.10248-1-eblake@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14iotests: Make 136 less flakyMax Reitz
136 executes some AIO requests without a final aio_flush; then it advances the virtual clock and thus expects the last access time of the device to be less than the current time when queried (i.e. idle_time_ns to be greater than 0). However, without the aio_flush, some requests may be settled after the clock_step invocation. In that case, idle_time_ns would be 0 and the test fails. Fix this by adding an aio_flush if any AIO request other than some other aio_flush has been executed. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20171109203025.27493-6-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14iotests: Make 083 less flakyMax Reitz
083 has (at least) two issues: 1. By launching the nbd-fault-injector in background, it may not be scheduled until the first grep on its output file is executed. However, until then, that file may not have been created yet -- so it either does not exist yet (thus making the grep emit an error), or it does exist but contains stale data (thus making the rest of the test case work connect to a wrong address). Fix this by explicitly overwriting the output file before executing nbd-fault-injector. 2. The nbd-fault-injector prints things other than "Listening on...". It also prints a "Closing connection" message from time to time. We currently invoke sed on the whole file in the hope of it only containing the "Listening on..." line yet. That hope is sometimes shattered by the brutal reality of race conditions, so make the sed script more robust. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171109203025.27493-5-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14iotests: Make 055 less flakyMax Reitz
First of all, test 055 does a valiant job of invoking pause_drive() sometimes, but that is worth nothing without blkdebug. So the first thing to do is to sprinkle a couple of "blkdebug::" in there -- with the exception of the transaction tests, because the blkdebug break points make the transaction QMP command hang (which is bad). In that case, we can get away with throttling the block job that it effectively is paused. Then, 055 usually does not pause the drive before starting a block job that should be cancelled. This means that the backup job might be completed already before block-job-cancel is invoked; thus making the test either fail (currently) or moot if cancel_and_wait() ignored this condition. Fix this by pausing the drive before starting the job. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20171109203025.27493-4-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14iotests: Add missing 'blkdebug::' in 040Max Reitz
040 tries to invoke pause_drive() on a drive that does not use blkdebug. Good idea, but let's use blkdebug to make it actually work. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20171109203025.27493-3-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14iotests: Make 030 less flakyMax Reitz
This patch fixes two race conditions in 030: 1. The first is in TestENOSPC.test_enospc(). After resuming the job, querying it to confirm it is no longer paused may fail because in the meantime it might have completed already. The same was fixed in TestEIO.test_ignore() already (in commit 2c3b44da07d341557a8203cc509ea07fe3605e11). 2. The second is in TestSetSpeed.test_set_speed_invalid(): Here, a stream job is started on a drive without any break points, with a block-job-set-speed invoked subsequently. However, without any break points, the job might have completed in the meantime (on tmpfs at least); or it might complete before cancel_and_wait() which expects the job to still exist. This can be fixed like everywhere else by pausing the drive (installing break points) before starting the job and letting cancel_and_wait() resume it. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20171109203025.27493-2-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14qcow2: Add iotest for an empty refcount tableAlberto Garcia
This patch adds a simple iotest in which we try to write to an image with an empty refcount table (i.e. with all entries set to 0). This scenario was already handled by the existing consistency checks, but we add an explicit test case for completeness. Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 7e48b0e2ae1a0a18e0ee303b3045f130feec0474.1509718618.git.berto@igalia.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14qcow2: Add iotest for an image with header.refcount_table_offset == 0Alberto Garcia
This patch adds a simple iotest in which we try to write to an image with the refcount table offset set to 0. This scenario was already handled by the existing consistency checks, but we add an explicit test case for completeness. Signed-off-by: Alberto Garcia <berto@igalia.com> Message-id: feeceada92486bb8790b90f303fc9fe82a27391a.1509718618.git.berto@igalia.com Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14qcow2: Don't open images with header.refcount_table_clusters == 0Alberto Garcia
qcow2_do_open() is checking that header.refcount_table_clusters is not too large, but it doesn't check that it's greater than zero. Apart from the fact that an image like that is obviously corrupted, trying to use it crashes QEMU since we end up with a null s->refcount_table after qcow2_refcount_init(). These images can however be repaired, so allow opening them if the BDRV_O_CHECK flag is set. Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: f9750f50c80359babba11062e88f5075a47e8e16.1509718618.git.berto@igalia.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14qcow2: Prevent allocating compressed clusters at offset 0Alberto Garcia
If the refcount data is corrupted then we can end up trying to allocate a new compressed cluster at offset 0 in the image, triggering an assertion in qcow2_alloc_bytes() that would crash QEMU: qcow2_alloc_bytes: Assertion `offset' failed. This patch adds an explicit check for this scenario and a new test case. Signed-off-by: Alberto Garcia <berto@igalia.com> Message-id: fb53467cf48e95ff3330def1cf1003a5b862b7d9.1509718618.git.berto@igalia.com Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14qcow2: Prevent allocating L2 tables at offset 0Alberto Garcia
If the refcount data is corrupted then we can end up trying to allocate a new L2 table at offset 0 in the image, triggering an assertion in the qcow2 cache that would crash QEMU: qcow2_cache_entry_mark_dirty: Assertion `c->entries[i].offset != 0' failed This patch adds an explicit check for this scenario and a new test case. Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 92dac37191ae7844a2da22c122204eb493cc3133.1509718618.git.berto@igalia.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14qcow2: Prevent allocating refcount blocks at offset 0Alberto Garcia
Each entry in the qcow2 cache contains an offset field indicating the location of the data in the qcow2 image. If the offset is 0 then it means that the entry contains no data and is available to be used when needed. Because of that it is not possible to store in the cache the first cluster of the qcow2 image (offset = 0). This is not a problem because that cluster always contains the qcow2 header and we're not using this cache for that. However, if the qcow2 image is corrupted it can happen that we try to allocate a new refcount block at offset 0, triggering this assertion and crashing QEMU: qcow2_cache_entry_mark_dirty: Assertion `c->entries[i].offset != 0' failed This patch adds an explicit check for this scenario and a new test case. This problem was originally reported here: https://bugs.launchpad.net/qemu/+bug/1728615 Reported-by: R.Nageswara Sastry <nasastry@in.ibm.com> Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 92a2fadd10d58b423f269c1d1a309af161cdc73f.1509718618.git.berto@igalia.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into ↵Peter Maydell
staging Pull request The following disk I/O throttling fixes solve recent bugs. # gpg: Signature made Tue 14 Nov 2017 10:37:12 GMT # gpg: using RSA key 0x9CA4ABB381AB73C8 # gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" # gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" # Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8 * remotes/stefanha/tags/block-pull-request: qemu-iotests: Test I/O limits with removable media block: Leave valid throttle timers when removing a BDS from a backend block: Check for inserted BlockDriverState in blk_io_limits_disable() throttle-groups: drain before detaching ThrottleState block: all I/O should be completed before removing throttle timers. Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2017-11-13Merge remote-tracking branch 'remotes/famz/tags/docker-pull-request' into ↵Peter Maydell
staging # gpg: Signature made Fri 10 Nov 2017 13:41:01 GMT # gpg: using RSA key 0xCA35624C6A9171C6 # gpg: Good signature from "Fam Zheng <famz@redhat.com>" # Primary key fingerprint: 5003 7CB7 9706 0F76 F021 AD56 CA35 624C 6A91 71C6 * remotes/famz/tags/docker-pull-request: docker: correctly escape $BACKEND in the help output docker: Improved image checksum Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2017-11-13qemu-iotests: Test I/O limits with removable mediaAlberto Garcia
This test hotplugs a CD drive to a VM and checks that I/O limits can be set only when the drive has media inserted and that they are kept when the media is replaced. This also tests the removal of a device with valid I/O limits set but no media inserted. This involves deleting and disabling the limits of a BlockBackend without BlockDriverState, a scenario that has been crashing until the fixes from the last couple of patches. [Python PEP8 fixup: "Don't use spaces are the = sign when used to indicate a keyword argument or a default parameter value" --Stefan] Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 071eb397118ed207c5a7f01d58766e415ee18d6a.1510339534.git.berto@igalia.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2017-11-13Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2017-11-09' into ↵Peter Maydell
staging nbd patches for 2017-11-09 - Vladimir Sementsov-Ogievskiy: nbd/server: fix nbd_negotiate_handle_info - Eric Blake: 0/7 various NBD fixes for 2.11 # gpg: Signature made Thu 09 Nov 2017 16:56:58 GMT # gpg: using RSA key 0xA7A16B4A2527436A # gpg: Good signature from "Eric Blake <eblake@redhat.com>" # gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" # gpg: aka "[jpeg image of size 6874]" # Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A * remotes/ericb/tags/pull-nbd-2017-11-09: nbd/server: Fix structured read of length 0 nbd-client: Stricter enforcing of structured reply spec nbd-client: Short-circuit 0-length operations nbd: Fix struct name for structured reads nbd/client: Nicer trace of structured reply nbd-client: Refuse read-only client with BDRV_O_RDWR nbd-client: Fix error message typos nbd/server: fix nbd_negotiate_handle_info Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2017-11-10Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into ↵Peter Maydell
staging Pull request v2: * v1 emails 2/3 and 3/3 weren't sent due to an email failure * Included Sergio's updated wording in the commit description # gpg: Signature made Wed 08 Nov 2017 19:12:01 GMT # gpg: using RSA key 0x9CA4ABB381AB73C8 # gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" # gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" # Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8 * remotes/stefanha/tags/block-pull-request: util/async: use atomic_mb_set in qemu_bh_cancel tests-aio-multithread: fix /aio/multi/schedule race condition Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2017-11-09nbd-client: Refuse read-only client with BDRV_O_RDWREric Blake
The NBD spec says that clients should not try to write/trim to an export advertised as read-only by the server. But we failed to check that, and would allow the block layer to use NBD with BDRV_O_RDWR even when the server is read-only, which meant we were depending on the server sending a proper EPERM failure for various commands, and also exposes a leaky abstraction: using qemu-io in read-write mode would succeed on 'w -z 0 0' because of local short-circuiting logic, but 'w 0 0' would send a request over the wire (where it then depends on the server, and fails at least for qemu-nbd but might pass for other NBD implementations). With this patch, a client MUST request read-only mode to access a server that is doing a read-only export, or else it will get a message like: can't open device nbd://localhost:10809/foo: request for write access conflicts with read-only export It is no longer possible to even attempt writes over the wire (including the corner case of 0-length writes), because the block layer enforces the explicit read-only request; this matches the behavior of qcow2 when backed by a read-only POSIX file. Fix several iotests to comply with the new behavior (since qemu-nbd of an internal snapshot, as well as nbd-server-add over QMP, default to a read-only export, we must tell blockdev-add/qemu-io to set up a read-only client). CC: qemu-stable@nongnu.org Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20171108215703.9295-3-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2017-11-08tests: Run the luks tests in test-crypto-block only if encryption is availableThomas Huth
The test-crypto-block currently fails if encryption has not been compiled into QEMU: TEST: tests/test-crypto-block... (pid=22231) /crypto/block/qcow: OK /crypto/block/luks/default: Unexpected error in qcrypto_pbkdf2() at qemu/crypto/pbkdf-stub.c:41: FAIL GTester: last random seed: R02Sbbb5b6f299c6727f41bb50ba4aa6ef5c (pid=22237) /crypto/block/luks/aes-256-cbc-plain64: Unexpected error in qcrypto_pbkdf2() at qemu/crypto/pbkdf-stub.c:41: FAIL GTester: last random seed: R02S3e27992a5ab4cc95e141c4ed3c7f0d2e (pid=22239) /crypto/block/luks/aes-256-cbc-essiv: Unexpected error in qcrypto_pbkdf2() at qemu/crypto/pbkdf-stub.c:41: FAIL GTester: last random seed: R02S51b52bb02a66c42d8b331fd305384f53 (pid=22241) FAIL: tests/test-crypto-block So run the luks test only if the required encryption support is available. Signed-off-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-11-08tests-aio-multithread: fix /aio/multi/schedule race conditionStefan Hajnoczi
test_multi_co_schedule_entry() set to_schedule[id] in the final loop iteration before terminating the coroutine. There is a race condition where the main thread attempts to enter the terminating or terminated coroutine when signalling coroutines to stop: atomic_mb_set(&now_stopping, true); for (i = 0; i < NUM_CONTEXTS; i++) { ctx_run(i, finish_cb, NULL); <--- enters dead coroutine! to_schedule[i] = NULL; } Make sure only to set to_schedule[id] if this coroutine really needs to be scheduled! Reported-by: "R.Nageswara Sastry" <nasastry@in.ibm.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20171106190233.1175-1-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2017-11-08docker: correctly escape $BACKEND in the help outputPhilippe Mathieu-Daudé
In Makefiles the $ must be escaped as $$ in shell uses. Since 8a2390a4f47: $ make docker [...] NETWORK=1 Enable virtual network interface with default backend. NETWORK=ACKEND Enable virtual network interface with ACKEND. Once escaped: $ make docker [...] NETWORK=1 Enable virtual network interface with default backend. NETWORK=$BACKEND Enable virtual network interface with $BACKEND. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Fam Zheng <famz@redhat.com> Message-Id: <20171108024719.8389-1-f4bug@amsat.org> Signed-off-by: Fam Zheng <famz@redhat.com>
2017-11-08docker: Improved image checksumFam Zheng
When a base image locally defined by QEMU, such as in the debian images, is updated, the dockerfile checksum mechanism in docker.py still skips updating the derived image, because it only looks at the literal content of the dockerfile, without considering changes to the base image. For example we have a recent fix e58c1f9b35e81 that fixed debian-win64-cross by updating its base image, debian8-mxe, but due to above "feature" of docker.py the image in question is automatically NOT rebuilt unless you add NOCACHE=1. It is noticed on Shippable: https://app.shippable.com/github/qemu/qemu/runs/541/2/console because after the fix is merged, the error still occurs, and the log shows the container image is, as explained above, not updated. This is because at the time docker.py was written, there wasn't any dependencies between QEMU's docker images. Now improve this to preprocess any "FROM qemu:*" directives in the dockerfiles while doing checksum, and inline the base image's dockerfile content, recursively. This ensures any changes on the depended _QEMU_ images are taken into account. This means for external images that we expect to retrieve from docker registries, we still do it as before. It is not perfect, because registry images can get updated too. Technically we could substitute the image name with its hex ID as obtained with $(docker images $IMAGE --format="{{.Id}}"), but --format is not supported by RHEL 7, so leave it for now. Reported-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Fam Zheng <famz@redhat.com> Message-Id: <20171103131229.4737-1-famz@redhat.com> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Fam Zheng <famz@redhat.com>
2017-11-03Merge remote-tracking branch ↵Peter Maydell
'remotes/stsquad/tags/pull-ci-updates-for-softfreeze-021117-2' into staging One fix for win32/64 cross compiles. # gpg: Signature made Thu 02 Nov 2017 11:55:46 GMT # gpg: using RSA key 0xFBD0DB095A9E2A44 # gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" # Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44 * remotes/stsquad/tags/pull-ci-updates-for-softfreeze-021117-2: docker: add python stdlib dependency (required by keycodemapdb) Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2017-11-02docker: add python stdlib dependency (required by keycodemapdb)Philippe Mathieu-Daudé
Since 927128222b0a QEMU depends of keycodemapdb, which uses the python 'csv' module from stdlib to parse keymaps.csv. Without this package the build fails: GEN ui/input-keymap-linux-to-qcode.c Traceback (most recent call last): File "ui/keycodemapdb/tools/keymap-gen", line 15, in <module> import csv ImportError: No module named csv GEN ui/input-keymap-qcode-to-qnum.c Traceback (most recent call last): File "ui/keycodemapdb/tools/keymap-gen", line 15, in <module> import csv ImportError: No module named csv [...] CC ui/input-keymap.o ui/input-keymap.c:8:44: fatal error: ui/input-keymap-linux-to-qcode.c: No such file or directory make: *** [ui/input-keymap.o] Error 1 Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>