aboutsummaryrefslogtreecommitdiff
path: root/tests
AgeCommit message (Collapse)Author
2017-11-17iotests: Make 087 pass without AIO enabledMax Reitz
If AIO has not been enabled in the qemu build that is to be tested, we should skip the "aio=native without O_DIRECT" test instead of failing. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171115180732.31753-1-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17qcow2: Refuse to get unaligned offsets from cacheMax Reitz
Instead of using an assertion, it is better to emit a corruption event here. Checking all offsets for correct alignment can be tedious and it is easily possible to forget to do so. qcow2_cache_do_get() is a function every L2 and refblock access has to go through, so this is a good central point to add such a check. And for good measure, let us also add an assertion that the offset is non-zero. Making this a corruption event is not feasible, because a zero offset usually means something special (such as the cluster is unused), so all callers should be checking this anyway. If they do not, it is their fault, hence the assertion here. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171110203111.7666-6-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17qcow2: Add bounds check to get_refblock_offset()Max Reitz
Reported-by: R. Nageswara Sastry <nasastry@in.ibm.com> Buglink: https://bugs.launchpad.net/qemu/+bug/1728661 Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171110203111.7666-5-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17block: Guard against NULL bs->drvMax Reitz
We currently do not guard everywhere against a NULL bs->drv where we should be doing so. Most of the places fixed here just do not care about that case at all. Some care implicitly, e.g. through a prior function call to bdrv_getlength() which would always fail for an ejected BDS. Add an assert there to make it more obvious. Other places seem to care, but do so insufficiently: Freeing clusters in a qcow2 image is an error-free operation, but it may leave the image in an unusable state anyway. Giving qcow2_free_clusters() an error code is not really viable, it is much easier to note that bs->drv may be NULL even after a successful driver call. This concerns bdrv_co_flush(), and the way the check is added to bdrv_co_pdiscard() (in every iteration instead of only once). Finally, some places employ at least an assert(bs->drv); somewhere, that may be reasonable (such as in the reopen code), but in bdrv_has_zero_init(), it is definitely not. Returning 0 there in case of an ejected BDS saves us much headache instead. Reported-by: R. Nageswara Sastry <nasastry@in.ibm.com> Buglink: https://bugs.launchpad.net/qemu/+bug/1728660 Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171110203111.7666-4-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17qcow2: Unaligned zero cluster in handle_alloc()Max Reitz
We should check whether the cluster offset we are about to use is actually valid; that is, whether it is aligned to cluster boundaries. Reported-by: R. Nageswara Sastry <nasastry@in.ibm.com> Buglink: https://bugs.launchpad.net/qemu/+bug/1728643 Buglink: https://bugs.launchpad.net/qemu/+bug/1728657 Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171110203111.7666-3-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17qcow2: check_errors are fatalMax Reitz
When trying to repair a dirty image, qcow2_check() may apparently succeed (no really fatal error occurred that would prevent the check from continuing), but if check_errors in the result object is non-zero, we cannot trust the image to be usable. Reported-by: R. Nageswara Sastry <nasastry@in.ibm.com> Buglink: https://bugs.launchpad.net/qemu/+bug/1728639 Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171110203111.7666-2-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17iotests: Add test for failing qemu-img commitMax Reitz
Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20170616135847.17726-1-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17tests: Add check-qobject for equality testsMax Reitz
Add a new test file (check-qobject.c) for unit tests that concern QObjects as a whole. Its only purpose for now is to test the qobject_is_equal() function. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171114180128.17076-7-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17iotests: Add test for non-string option reopeningMax Reitz
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-id: 20171114180128.17076-6-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17qapi/qnull: Add own headerMax Reitz
Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-id: 20171114180128.17076-2-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-17qcow2: fix image corruption on commit with persistent bitmapEric Blake
If an image contains persistent bitmaps, we cannot use the fast path of bdrv_make_empty() to clear the image during qemu-img commit, because that will lose the clusters related to the bitmaps. Also leave a comment in qcow2_read_extensions to remind future feature additions to think about fast-path removal, since we just barely fixed the same bug for LUKS encryption. It's a pain that qemu-img has not yet been taught to manipulate, or even at a very minimum display, information about persistent bitmaps; instead, we have to use QMP commands. It's also a pain that only qeury-block and x-debug-block-dirty-bitmap-sha256 will allow bitmap introspection; but the former requires the node to be hooked to a block device, and the latter is experimental. Signed-off-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2017-11-17iotests: test clearing unknown autoclear_features by qcow2Vladimir Sementsov-Ogievskiy
Test clearing unknown autoclear_features by qcow2 on incoming migration. [ kwolf: Fixed wait for destination VM startup ] Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com>
2017-11-17qcow2: fix image corruption after committing qcow2 image into baseDaniel P. Berrange
After committing the qcow2 image contents into the base image, qemu-img will call bdrv_make_empty to drop the payload in the layered image. When this is done for qcow2 images, it blows away the LUKS encryption header, making the resulting image unusable. There are two codepaths for emptying a qcow2 image, and the second (slower) codepath leaves the LUKS header intact, so force use of that codepath. Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2017-11-17qemu-iotests: Use -nographic in 182Kevin Wolf
This avoids that random UI frontend error messages end up in the output. In particular, we were seeing this line in CI error logs: +Unable to init server: Could not connect: Connection refused Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Kashyap Chamarthy <kchamart@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com>
2017-11-16tests/bios-tables-test: Fix endianess problems when passing data to iaslThomas Huth
The bios-tables-test was writing out files that we pass to iasl in with the wrong endianness in the header when running on a big endian host. So instead of storing mixed endian information in our structures, let's keep everything in little endian and byte-swap it only when we need a value in the code. Reported-by: Daniel P. Berrange <berrange@redhat.com> Buglink: https://bugs.launchpad.net/qemu/+bug/1724570 Suggested-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com> Tested-by: "Daniel P. Berrange" <berrange@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2017-11-16tests/acpi-test-data: update _CRS in DSDTMichael S. Tsirkin
commit dadf988e81b15065ac1d6dbaf4b87b5b80c7b670 hw/pci-host: Fix x86 Host Bridges 64bit PCI hole Added a 64 bit hole to _CRS of PCI0. Update the expected files accordingly. Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2017-11-14qemu-iotests: update unsupported image formats in 194Jeff Cody
Test 194 checks for 'luks' to exclude as an unsupported format, However, most formats are unsupported, due to migration blockers. Rather than specifying a blacklist of unsupported formats, whitelist supported formats (specifically, qcow2, qed, raw, dmg). Tested-by: Alexey Kardashevskiy <aik@ozlabs.ru> Signed-off-by: Jeff Cody <jcody@redhat.com> Message-id: 23ca18c7f843c86a28b1529ca9ac6db4b35ca0e4.1510059970.git.jcody@redhat.com Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Denis V. Lunev <den@openvz.org> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14iotests: 077: Filter out 'resume' linesFam Zheng
In the "Overlapping multiple requests" cases, the 3rd reqs (the break point B) doesn't wait for the 2nd, and once resumed the I/O will just continue. This is because the 2nd is already waiting for the 1st, and in wait_serialising_requests() there is: /* If the request is already (indirectly) waiting for us, or * will wait for us as soon as it wakes up, then just go on * (instead of producing a deadlock in the former case). */ if (!req->waiting_for) { /* actually break */ ... } Consequently, the following "sleep 100; resume A" command races with the completion of that request, and sometimes results in an unexpected order of output: > @@ -56,9 +56,9 @@ > wrote XXX/XXX bytes at offset XXX > XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > blkdebug: Resuming request 'B' > +blkdebug: Resuming request 'A' > wrote XXX/XXX bytes at offset XXX > XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > -blkdebug: Resuming request 'A' > wrote XXX/XXX bytes at offset XXX > XXX bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) > wrote XXX/XXX bytes at offset XXX Filter out the "Resuming request" lines to make the output deterministic. Reported-by: Patchew <no-reply@patchew.org> Signed-off-by: Fam Zheng <famz@redhat.com> Message-id: 20171113150026.4743-1-famz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14qcow2: Check that corrupted images can be repaired in iotest 060Alberto Garcia
We just fixed a few bugs that caused QEMU to crash when trying to write to corrupted qcow2 images, and iotest 060 was expanded to test all those scenarios. In almost all cases the corrupted images can be repaired using qemu-img, so this patch verifies that. Signed-off-by: Alberto Garcia <berto@igalia.com> Message-id: 0b1b95340ecdfbc6927e36adf2fd42ae6198747a.1510143008.git.berto@igalia.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14iotests: Use new-style NBD connectionsEric Blake
Old-style NBD is deprecated upstream (it is documented, but no longer implemented in the reference implementation), and it is severely limited (it cannot support structured replies, which means it cannot support efficient handling of zeroes), when compared to new-style NBD. We are better off having our iotests favor new-style everywhere (although some explicit tests, particularly 83, still cover old-style for back-compat reasons); this is as simple as supplying the empty string as the default export name, as it does not change the URI needed to connect a client to the server. This also gives us more coverage of the just-added structured reply code, when not overriding $QEMU_NBD to intentionally point to an older server. Signed-off-by: Eric Blake <eblake@redhat.com> Message-id: 20171109221216.10248-1-eblake@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14iotests: Make 136 less flakyMax Reitz
136 executes some AIO requests without a final aio_flush; then it advances the virtual clock and thus expects the last access time of the device to be less than the current time when queried (i.e. idle_time_ns to be greater than 0). However, without the aio_flush, some requests may be settled after the clock_step invocation. In that case, idle_time_ns would be 0 and the test fails. Fix this by adding an aio_flush if any AIO request other than some other aio_flush has been executed. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20171109203025.27493-6-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14iotests: Make 083 less flakyMax Reitz
083 has (at least) two issues: 1. By launching the nbd-fault-injector in background, it may not be scheduled until the first grep on its output file is executed. However, until then, that file may not have been created yet -- so it either does not exist yet (thus making the grep emit an error), or it does exist but contains stale data (thus making the rest of the test case work connect to a wrong address). Fix this by explicitly overwriting the output file before executing nbd-fault-injector. 2. The nbd-fault-injector prints things other than "Listening on...". It also prints a "Closing connection" message from time to time. We currently invoke sed on the whole file in the hope of it only containing the "Listening on..." line yet. That hope is sometimes shattered by the brutal reality of race conditions, so make the sed script more robust. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171109203025.27493-5-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14iotests: Make 055 less flakyMax Reitz
First of all, test 055 does a valiant job of invoking pause_drive() sometimes, but that is worth nothing without blkdebug. So the first thing to do is to sprinkle a couple of "blkdebug::" in there -- with the exception of the transaction tests, because the blkdebug break points make the transaction QMP command hang (which is bad). In that case, we can get away with throttling the block job that it effectively is paused. Then, 055 usually does not pause the drive before starting a block job that should be cancelled. This means that the backup job might be completed already before block-job-cancel is invoked; thus making the test either fail (currently) or moot if cancel_and_wait() ignored this condition. Fix this by pausing the drive before starting the job. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20171109203025.27493-4-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14iotests: Add missing 'blkdebug::' in 040Max Reitz
040 tries to invoke pause_drive() on a drive that does not use blkdebug. Good idea, but let's use blkdebug to make it actually work. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20171109203025.27493-3-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14iotests: Make 030 less flakyMax Reitz
This patch fixes two race conditions in 030: 1. The first is in TestENOSPC.test_enospc(). After resuming the job, querying it to confirm it is no longer paused may fail because in the meantime it might have completed already. The same was fixed in TestEIO.test_ignore() already (in commit 2c3b44da07d341557a8203cc509ea07fe3605e11). 2. The second is in TestSetSpeed.test_set_speed_invalid(): Here, a stream job is started on a drive without any break points, with a block-job-set-speed invoked subsequently. However, without any break points, the job might have completed in the meantime (on tmpfs at least); or it might complete before cancel_and_wait() which expects the job to still exist. This can be fixed like everywhere else by pausing the drive (installing break points) before starting the job and letting cancel_and_wait() resume it. Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20171109203025.27493-2-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14qcow2: Add iotest for an empty refcount tableAlberto Garcia
This patch adds a simple iotest in which we try to write to an image with an empty refcount table (i.e. with all entries set to 0). This scenario was already handled by the existing consistency checks, but we add an explicit test case for completeness. Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 7e48b0e2ae1a0a18e0ee303b3045f130feec0474.1509718618.git.berto@igalia.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14qcow2: Add iotest for an image with header.refcount_table_offset == 0Alberto Garcia
This patch adds a simple iotest in which we try to write to an image with the refcount table offset set to 0. This scenario was already handled by the existing consistency checks, but we add an explicit test case for completeness. Signed-off-by: Alberto Garcia <berto@igalia.com> Message-id: feeceada92486bb8790b90f303fc9fe82a27391a.1509718618.git.berto@igalia.com Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14qcow2: Don't open images with header.refcount_table_clusters == 0Alberto Garcia
qcow2_do_open() is checking that header.refcount_table_clusters is not too large, but it doesn't check that it's greater than zero. Apart from the fact that an image like that is obviously corrupted, trying to use it crashes QEMU since we end up with a null s->refcount_table after qcow2_refcount_init(). These images can however be repaired, so allow opening them if the BDRV_O_CHECK flag is set. Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: f9750f50c80359babba11062e88f5075a47e8e16.1509718618.git.berto@igalia.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14qcow2: Prevent allocating compressed clusters at offset 0Alberto Garcia
If the refcount data is corrupted then we can end up trying to allocate a new compressed cluster at offset 0 in the image, triggering an assertion in qcow2_alloc_bytes() that would crash QEMU: qcow2_alloc_bytes: Assertion `offset' failed. This patch adds an explicit check for this scenario and a new test case. Signed-off-by: Alberto Garcia <berto@igalia.com> Message-id: fb53467cf48e95ff3330def1cf1003a5b862b7d9.1509718618.git.berto@igalia.com Reviewed-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14qcow2: Prevent allocating L2 tables at offset 0Alberto Garcia
If the refcount data is corrupted then we can end up trying to allocate a new L2 table at offset 0 in the image, triggering an assertion in the qcow2 cache that would crash QEMU: qcow2_cache_entry_mark_dirty: Assertion `c->entries[i].offset != 0' failed This patch adds an explicit check for this scenario and a new test case. Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 92dac37191ae7844a2da22c122204eb493cc3133.1509718618.git.berto@igalia.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14qcow2: Prevent allocating refcount blocks at offset 0Alberto Garcia
Each entry in the qcow2 cache contains an offset field indicating the location of the data in the qcow2 image. If the offset is 0 then it means that the entry contains no data and is available to be used when needed. Because of that it is not possible to store in the cache the first cluster of the qcow2 image (offset = 0). This is not a problem because that cluster always contains the qcow2 header and we're not using this cache for that. However, if the qcow2 image is corrupted it can happen that we try to allocate a new refcount block at offset 0, triggering this assertion and crashing QEMU: qcow2_cache_entry_mark_dirty: Assertion `c->entries[i].offset != 0' failed This patch adds an explicit check for this scenario and a new test case. This problem was originally reported here: https://bugs.launchpad.net/qemu/+bug/1728615 Reported-by: R.Nageswara Sastry <nasastry@in.ibm.com> Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 92a2fadd10d58b423f269c1d1a309af161cdc73f.1509718618.git.berto@igalia.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-11-14Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into ↵Peter Maydell
staging Pull request The following disk I/O throttling fixes solve recent bugs. # gpg: Signature made Tue 14 Nov 2017 10:37:12 GMT # gpg: using RSA key 0x9CA4ABB381AB73C8 # gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" # gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" # Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8 * remotes/stefanha/tags/block-pull-request: qemu-iotests: Test I/O limits with removable media block: Leave valid throttle timers when removing a BDS from a backend block: Check for inserted BlockDriverState in blk_io_limits_disable() throttle-groups: drain before detaching ThrottleState block: all I/O should be completed before removing throttle timers. Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2017-11-13Merge remote-tracking branch 'remotes/famz/tags/docker-pull-request' into ↵Peter Maydell
staging # gpg: Signature made Fri 10 Nov 2017 13:41:01 GMT # gpg: using RSA key 0xCA35624C6A9171C6 # gpg: Good signature from "Fam Zheng <famz@redhat.com>" # Primary key fingerprint: 5003 7CB7 9706 0F76 F021 AD56 CA35 624C 6A91 71C6 * remotes/famz/tags/docker-pull-request: docker: correctly escape $BACKEND in the help output docker: Improved image checksum Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2017-11-13qemu-iotests: Test I/O limits with removable mediaAlberto Garcia
This test hotplugs a CD drive to a VM and checks that I/O limits can be set only when the drive has media inserted and that they are kept when the media is replaced. This also tests the removal of a device with valid I/O limits set but no media inserted. This involves deleting and disabling the limits of a BlockBackend without BlockDriverState, a scenario that has been crashing until the fixes from the last couple of patches. [Python PEP8 fixup: "Don't use spaces are the = sign when used to indicate a keyword argument or a default parameter value" --Stefan] Signed-off-by: Alberto Garcia <berto@igalia.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 071eb397118ed207c5a7f01d58766e415ee18d6a.1510339534.git.berto@igalia.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2017-11-13Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2017-11-09' into ↵Peter Maydell
staging nbd patches for 2017-11-09 - Vladimir Sementsov-Ogievskiy: nbd/server: fix nbd_negotiate_handle_info - Eric Blake: 0/7 various NBD fixes for 2.11 # gpg: Signature made Thu 09 Nov 2017 16:56:58 GMT # gpg: using RSA key 0xA7A16B4A2527436A # gpg: Good signature from "Eric Blake <eblake@redhat.com>" # gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" # gpg: aka "[jpeg image of size 6874]" # Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A * remotes/ericb/tags/pull-nbd-2017-11-09: nbd/server: Fix structured read of length 0 nbd-client: Stricter enforcing of structured reply spec nbd-client: Short-circuit 0-length operations nbd: Fix struct name for structured reads nbd/client: Nicer trace of structured reply nbd-client: Refuse read-only client with BDRV_O_RDWR nbd-client: Fix error message typos nbd/server: fix nbd_negotiate_handle_info Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2017-11-10Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into ↵Peter Maydell
staging Pull request v2: * v1 emails 2/3 and 3/3 weren't sent due to an email failure * Included Sergio's updated wording in the commit description # gpg: Signature made Wed 08 Nov 2017 19:12:01 GMT # gpg: using RSA key 0x9CA4ABB381AB73C8 # gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" # gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" # Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8 * remotes/stefanha/tags/block-pull-request: util/async: use atomic_mb_set in qemu_bh_cancel tests-aio-multithread: fix /aio/multi/schedule race condition Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2017-11-09nbd-client: Refuse read-only client with BDRV_O_RDWREric Blake
The NBD spec says that clients should not try to write/trim to an export advertised as read-only by the server. But we failed to check that, and would allow the block layer to use NBD with BDRV_O_RDWR even when the server is read-only, which meant we were depending on the server sending a proper EPERM failure for various commands, and also exposes a leaky abstraction: using qemu-io in read-write mode would succeed on 'w -z 0 0' because of local short-circuiting logic, but 'w 0 0' would send a request over the wire (where it then depends on the server, and fails at least for qemu-nbd but might pass for other NBD implementations). With this patch, a client MUST request read-only mode to access a server that is doing a read-only export, or else it will get a message like: can't open device nbd://localhost:10809/foo: request for write access conflicts with read-only export It is no longer possible to even attempt writes over the wire (including the corner case of 0-length writes), because the block layer enforces the explicit read-only request; this matches the behavior of qcow2 when backed by a read-only POSIX file. Fix several iotests to comply with the new behavior (since qemu-nbd of an internal snapshot, as well as nbd-server-add over QMP, default to a read-only export, we must tell blockdev-add/qemu-io to set up a read-only client). CC: qemu-stable@nongnu.org Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20171108215703.9295-3-eblake@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2017-11-08tests: Run the luks tests in test-crypto-block only if encryption is availableThomas Huth
The test-crypto-block currently fails if encryption has not been compiled into QEMU: TEST: tests/test-crypto-block... (pid=22231) /crypto/block/qcow: OK /crypto/block/luks/default: Unexpected error in qcrypto_pbkdf2() at qemu/crypto/pbkdf-stub.c:41: FAIL GTester: last random seed: R02Sbbb5b6f299c6727f41bb50ba4aa6ef5c (pid=22237) /crypto/block/luks/aes-256-cbc-plain64: Unexpected error in qcrypto_pbkdf2() at qemu/crypto/pbkdf-stub.c:41: FAIL GTester: last random seed: R02S3e27992a5ab4cc95e141c4ed3c7f0d2e (pid=22239) /crypto/block/luks/aes-256-cbc-essiv: Unexpected error in qcrypto_pbkdf2() at qemu/crypto/pbkdf-stub.c:41: FAIL GTester: last random seed: R02S51b52bb02a66c42d8b331fd305384f53 (pid=22241) FAIL: tests/test-crypto-block So run the luks test only if the required encryption support is available. Signed-off-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
2017-11-08tests-aio-multithread: fix /aio/multi/schedule race conditionStefan Hajnoczi
test_multi_co_schedule_entry() set to_schedule[id] in the final loop iteration before terminating the coroutine. There is a race condition where the main thread attempts to enter the terminating or terminated coroutine when signalling coroutines to stop: atomic_mb_set(&now_stopping, true); for (i = 0; i < NUM_CONTEXTS; i++) { ctx_run(i, finish_cb, NULL); <--- enters dead coroutine! to_schedule[i] = NULL; } Make sure only to set to_schedule[id] if this coroutine really needs to be scheduled! Reported-by: "R.Nageswara Sastry" <nasastry@in.ibm.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20171106190233.1175-1-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2017-11-08docker: correctly escape $BACKEND in the help outputPhilippe Mathieu-Daudé
In Makefiles the $ must be escaped as $$ in shell uses. Since 8a2390a4f47: $ make docker [...] NETWORK=1 Enable virtual network interface with default backend. NETWORK=ACKEND Enable virtual network interface with ACKEND. Once escaped: $ make docker [...] NETWORK=1 Enable virtual network interface with default backend. NETWORK=$BACKEND Enable virtual network interface with $BACKEND. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Fam Zheng <famz@redhat.com> Message-Id: <20171108024719.8389-1-f4bug@amsat.org> Signed-off-by: Fam Zheng <famz@redhat.com>
2017-11-08docker: Improved image checksumFam Zheng
When a base image locally defined by QEMU, such as in the debian images, is updated, the dockerfile checksum mechanism in docker.py still skips updating the derived image, because it only looks at the literal content of the dockerfile, without considering changes to the base image. For example we have a recent fix e58c1f9b35e81 that fixed debian-win64-cross by updating its base image, debian8-mxe, but due to above "feature" of docker.py the image in question is automatically NOT rebuilt unless you add NOCACHE=1. It is noticed on Shippable: https://app.shippable.com/github/qemu/qemu/runs/541/2/console because after the fix is merged, the error still occurs, and the log shows the container image is, as explained above, not updated. This is because at the time docker.py was written, there wasn't any dependencies between QEMU's docker images. Now improve this to preprocess any "FROM qemu:*" directives in the dockerfiles while doing checksum, and inline the base image's dockerfile content, recursively. This ensures any changes on the depended _QEMU_ images are taken into account. This means for external images that we expect to retrieve from docker registries, we still do it as before. It is not perfect, because registry images can get updated too. Technically we could substitute the image name with its hex ID as obtained with $(docker images $IMAGE --format="{{.Id}}"), but --format is not supported by RHEL 7, so leave it for now. Reported-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Fam Zheng <famz@redhat.com> Message-Id: <20171103131229.4737-1-famz@redhat.com> Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Fam Zheng <famz@redhat.com>
2017-11-03Merge remote-tracking branch ↵Peter Maydell
'remotes/stsquad/tags/pull-ci-updates-for-softfreeze-021117-2' into staging One fix for win32/64 cross compiles. # gpg: Signature made Thu 02 Nov 2017 11:55:46 GMT # gpg: using RSA key 0xFBD0DB095A9E2A44 # gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" # Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44 * remotes/stsquad/tags/pull-ci-updates-for-softfreeze-021117-2: docker: add python stdlib dependency (required by keycodemapdb) Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2017-11-02docker: add python stdlib dependency (required by keycodemapdb)Philippe Mathieu-Daudé
Since 927128222b0a QEMU depends of keycodemapdb, which uses the python 'csv' module from stdlib to parse keymaps.csv. Without this package the build fails: GEN ui/input-keymap-linux-to-qcode.c Traceback (most recent call last): File "ui/keycodemapdb/tools/keymap-gen", line 15, in <module> import csv ImportError: No module named csv GEN ui/input-keymap-qcode-to-qnum.c Traceback (most recent call last): File "ui/keycodemapdb/tools/keymap-gen", line 15, in <module> import csv ImportError: No module named csv [...] CC ui/input-keymap.o ui/input-keymap.c:8:44: fatal error: ui/input-keymap-linux-to-qcode.c: No such file or directory make: *** [ui/input-keymap.o] Error 1 Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2017-10-30nbd: Minimal structured read for clientVladimir Sementsov-Ogievskiy
Minimal implementation: for structured error only error_report error message. Note that test 83 is now more verbose, because the implementation prints more warnings about unexpected communication errors; perhaps future patches should tone things down by using trace messages instead of traces, but the common case of successful communication is no noisier than before. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20171027104037.8319-13-eblake@redhat.com>
2017-10-29tests: check that migration parameters are really assignedJuan Quintela
Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2017-10-29tests: Don't abuse global_qtestJuan Quintela
As we have two guests running, just pass always who we want to send a message to. Once there, refactor return_or_event() into wait_command. Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2017-10-29tests: Factorize out migrate_test_start/endJuan Quintela
We fix global_test users left and right Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2017-10-29tests: Refactor setting of parameters/capabilitiesJuan Quintela
So we can use them in future tests Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2017-10-29tests: rename postcopy-test to migration-testJuan Quintela
Instead of repeating the code, we are going to bo more tests on this file Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2017-10-26iotests: Add cluster_size=64k to 125Max Reitz
Apparently it would be a good idea to test that, too. Signed-off-by: Max Reitz <mreitz@redhat.com> Message-id: 20171009215533.12530-4-mreitz@redhat.com Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Jeff Cody <jcody@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>