aboutsummaryrefslogtreecommitdiff
path: root/tests/qemu-iotests
AgeCommit message (Collapse)Author
2022-02-01iotests 60: more accurate set dirty bit in qcow2 headerVladimir Sementsov-Ogievskiy
Don't touch other incompatible bits, like compression-type. This makes the test pass with IMGOPTS='compression_type=zstd'. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20211223160144.1097696-18-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-02-01iotests: bash tests: filter compression typeVladimir Sementsov-Ogievskiy
We want iotests pass with both the default zlib compression and with IMGOPTS='compression_type=zstd'. Actually the only test that is interested in real compression type in test output is 287 (test for qcow2 compression type), so implement specific option for it. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-17-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-02-01iotest 39: use _qcow2_dump_headerVladimir Sementsov-Ogievskiy
_qcow2_dump_header has filter for compression type, so this change makes test pass with IMGOPTS='compression_type=zstd'. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20211223160144.1097696-16-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-02-01iotests: massive use _qcow2_dump_headerVladimir Sementsov-Ogievskiy
We are going to add filtering in _qcow2_dump_header and want all tests use it. The patch is generated by commands: cd tests/qemu-iotests sed -ie 's/$PYTHON qcow2.py "$TEST_IMG" dump-header\($\| \)/_qcow2_dump_header\1/' ??? tests/* (the difficulty is to avoid converting dump-header-exts) Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20211223160144.1097696-15-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-02-01iotests/common.rc: introduce _qcow2_dump_header helperVladimir Sementsov-Ogievskiy
We'll use it in tests instead of explicit qcow2.py. Then we are going to add some filtering in _qcow2_dump_header. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20211223160144.1097696-14-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-02-01iotest 302: use img_info_log() helperVladimir Sementsov-Ogievskiy
Instead of qemu_img_log("info", ..) use generic helper img_info_log(). img_info_log() has smarter logic. For example it use filter_img_info() to filter output, which in turns filter a compression type. So it will help us in future when we implement a possibility to use zstd compression by default (with help of some runtime config file or maybe build option). For now to test you should recompile qemu with a small addition into block/qcow2.c before "if (qcow2_opts->has_compression_type": if (!qcow2_opts->has_compression_type && version >= 3) { qcow2_opts->has_compression_type = true; qcow2_opts->compression_type = QCOW2_COMPRESSION_TYPE_ZSTD; } Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20211223160144.1097696-12-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-02-01iotests.py: filter compression type outVladimir Sementsov-Ogievskiy
We want iotests pass with both the default zlib compression and with IMGOPTS='compression_type=zstd'. Actually the only test that is interested in real compression type in test output is 287 (test for qcow2 compression type) and it's in bash. So for now we can safely filter out compression type in all qcow2 tests. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-11-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-02-01iotests.py: filter out successful output of qemu-img createVladimir Sementsov-Ogievskiy
The only "feature" of this "Formatting ..." line is that we have to update it every time we add new option. Let's drop it. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-10-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-02-01iotest 065: explicit compression typeVladimir Sementsov-Ogievskiy
The test checks different options. It of course fails if set IMGOPTS='compression_type=zstd'. So, let's be explicit in what compression type we want and independent of IMGOPTS. Test both existing compression types. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-9-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-02-01iotest 303: explicit compression typeVladimir Sementsov-Ogievskiy
The test prints qcow2 header fields which depends on chosen compression type. So, let's be explicit in what compression type we want and independent of IMGOPTS. Test both existing compression types. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20211223160144.1097696-8-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-02-01iotests.py: rewrite default luks support in qemu_imgVladimir Sementsov-Ogievskiy
Move the logic to more generic qemu_img_pipe_and_status(). Also behave better when we have several -o options. And reuse argument parser of course. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-7-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-02-01iotests: drop qemu_img_verbose() helperVladimir Sementsov-Ogievskiy
qemu_img_verbose() has a drawback of not going through generic qemu_img_pipe_and_status(). qemu_img_verbose() is not very popular, so update the only two users to qemu_img_log() and drop qemu_img_verbose() at all. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-6-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-02-01iotests.py: qemu_img*("create"): support IMGOPTS='compression_type=zstd'Vladimir Sementsov-Ogievskiy
Adding support of IMGOPTS (like in bash tests) allows user to pass a lot of different options. Still, some may require additional logic. Now we want compression_type option, so add some smart logic around it: ignore compression_type=zstd in IMGOPTS, if test want qcow2 in compatibility mode. As well, ignore compression_type for non-qcow2 formats. Note that we may instead add support only to qemu_img_create(), but that works bad: 1. We'll have to update a lot of tests to use qemu_img_create instead of qemu_img('create'). (still, we may want do it anyway, but no reason to create a dependancy between task of supporting IMGOPTS and updating a lot of tests) 2. Some tests use qemu_img_pipe('create', ..) - even more work on updating 3. Even if we update all tests to go through qemu_img_create, we'll need a way to avoid creating new tests using qemu_img*('create') - add assertions.. That doesn't seem good. So, let's add support of IMGOPTS to most generic qemu_img_pipe_and_status(). Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-5-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-02-01iotests: specify some unsupported_imgopts for python iotestsVladimir Sementsov-Ogievskiy
We are going to support IMGOPTS for python iotests. Still some iotests will not work with common IMGOPTS used with bash iotests like specifying refcount_bits and compat qcow2 options. So we should define corresponding unsupported_imgopts for now. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211223160144.1097696-4-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-02-01iotests.py: implement unsupported_imgoptsVladimir Sementsov-Ogievskiy
We are going to support some addition IMGOPTS in python iotests like in bash iotests. Similarly to bash iotests, we want a way to skip some tests which can't work with specific IMGOPTS. Globally for python iotests we will not support things like 'data_file=$TEST_IMG.ext_data_file' in IMGOPTS, so, forbid this globally in iotests.py. Suggested-by: Hanna Reitz <hreitz@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223160144.1097696-3-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-02-01iotests.py: img_info_log(): rename imgopts argumentVladimir Sementsov-Ogievskiy
We are going to support IMGOPTS environment variable like in bash tests. Corresponding global variable in iotests.py should be called imgopts. So to not interfere with function argument, rename it in advance. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20211223160144.1097696-2-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-02-01iotests/MRCE: Write data to sourceHanna Reitz
This test assumes that mirror flushes the source when entering the READY state, and that the format level will pass that flush on to the protocol level (where we intercept it with blkdebug). However, apparently that does not happen when using a VMDK image with zeroed_grain=on, which actually is the default set by testenv.py. Right now, Python tests ignore IMGOPTS, though, so this has no effect; but Vladimir has a series that will change this, so we need to fix this test before that series lands. We can fix it by writing data to the source before we start the mirror job; apparently that makes the (VMDK) format layer change its mind and pass on the pre-READY flush to the protocol level, so the test passes again. (I presume, without any data written, mirror just does a 64M zero write on the target, which VMDK with zeroed_grain=on basically just ignores.) Without this, we do not get a flush, and so blkdebug only sees a single flush at the end of the job instead of two, and therefore does not inject an error, which makes the block job complete instead of raising an error. Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211223165308.103793-1-hreitz@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2022-02-01tests/qemu-iotests: Fix 051 for binaries without 'lsi53c895a'Thomas Huth
The lsi53c895a SCSI adaptor might not be enabled in each and every x86 QEMU binary, e.g. it's disabled in the RHEL/CentOS build. Thus let's add a check to the 051 test so that it does not fail if this device is not available. Signed-off-by: Thomas Huth <thuth@redhat.com> Message-Id: <20211206143404.247032-1-thuth@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2022-01-28iotests/block-status-cache: New testHanna Reitz
Add a new test to verify that want_zero=false block-status calls do not pollute the block-status cache for want_zero=true calls. We check want_zero=true calls and their results using `qemu-img map` (over NBD), and want_zero=false calls also using `qemu-img map` over NBD, but using the qemu:allocation-depth context. (This test case cannot be integrated into nbd-qemu-allocation, because that is a qcow2 test, and this is a raw test.) Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20220118170000.49423-3-hreitz@redhat.com> Reviewed-by: Nir Soffer <nsoffer@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Tested-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2022-01-28check-block: replace -makecheck with TAP outputPaolo Bonzini
Let "meson test" take care of showing the results of the individual tests, consistently with other output from "make check V=1". Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-01-28build: make check-block a meson testPaolo Bonzini
"meson test" can be asked to run tests verbosely; this makes it usable also for qemu-iotests's own harness, and it lets "make check-block" reuse mtest2make.py's infrastructure to find and build test dependencies. Adjust check-block.sh to use the standard exit code that reports a test as skipped. Alternatively, in the future we could make it produce TAP output, which is consistent with all other "make check" tests. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-01-14iotests/testrunner.py: refactor test_field_widthVladimir Sementsov-Ogievskiy
A lot of Optional[] types doesn't make code beautiful. test_field_width defaults to 8, but that is never used in the code. More over, if we want some default behavior for single call of test_run(), it should just print the whole test name, not limiting or expanding its width, so 8 is bad default. So, just drop the default as unused for now. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211210201450.101576-1-vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2022-01-14block: drop BLK_PERM_GRAPH_MODVladimir Sementsov-Ogievskiy
First, this permission never protected a node from being changed, as generic child-replacing functions don't check it. Second, it's a strange thing: it presents a permission of parent node to change its child. But generally, children are replaced by different mechanisms, like jobs or qmp commands, not by nodes. Graph-mod permission is hard to understand. All other permissions describe operations which done by parent node on its child: read, write, resize. Graph modification operations are something completely different. The only place where BLK_PERM_GRAPH_MOD is used as "perm" (not shared perm) is mirror_start_job, for s->target. Still modern code should use bdrv_freeze_backing_chain() to protect from graph modification, if we don't do it somewhere it may be considered as a bug. So, it's a bit risky to drop GRAPH_MOD, and analyzing of possible loss of protection is hard. But one day we should do it, let's do it now. One more bit of information is that locking the corresponding byte in file-posix doesn't make sense at all. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20210902093754.2352-1-vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2022-01-14qemu-img: make is_allocated_sectors() more efficientVladimir Sementsov-Ogievskiy
Consider the case when the whole buffer is zero and end is unaligned. If i <= tail, we return 1 and do one unaligned WRITE, RMW happens. If i > tail, we do on aligned WRITE_ZERO (or skip if target is zeroed) and again one unaligned WRITE, RMW happens. Let's do better: don't fragment the whole-zero buffer and report it as ZERO: in case of zeroed target we just do nothing and avoid RMW. If target is not zeroes, one unaligned WRITE_ZERO should not be much worse than one unaligned WRITE. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211217164654.1184218-3-vsementsov@virtuozzo.com> Tested-by: Peter Lieven <pl@kamp.de> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2022-01-14iotests: Test qemu-img convert of zeroed data clusterKevin Wolf
This demonstrates what happens when the block status changes in sub-min_sparse granularity, but all of the parts are zeroed out. The alignment logic in is_allocated_sectors() prevents that the target image remains fully sparse as expected, but turns it into a data cluster of explicit zeros. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211217164654.1184218-2-vsementsov@virtuozzo.com> Tested-by: Peter Lieven <pl@kamp.de> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2022-01-14iotests/308: Fix for CAP_DAC_OVERRIDEHanna Reitz
With CAP_DAC_OVERRIDE (which e.g. root generally has), permission checks will be bypassed when opening files. 308 in one instance tries to open a read-only file (FUSE export) with qemu-io as read/write, and expects this to fail. However, when running it as root, opening will succeed (thanks to CAP_DAC_OVERRIDE) and only the actual write operation will fail. Note this as "Case not run", but have the test pass in either case. Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Fixes: 2c7dd057aa7bd7a875e9b1a53975c220d6380bc4 ("export/fuse: Pass default_permissions for mount") Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20220103120014.13061-1-hreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2022-01-14iotests/stream-error-on-reset: New testHanna Reitz
Test the following scenario: - Simple stream block in two-layer backing chain (base and top) - The job is drained via blk_drain(), then an error occurs while the job settles the ongoing request - And so the job completes while in blk_drain() This was reported as a segfault, but is fixed by "block-backend: prevent dangling BDS pointers across aio_poll()". Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2036178 Signed-off-by: Hanna Reitz <hreitz@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20220111153613.25453-3-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-12-28blockjob: drop BlockJob.blk fieldVladimir Sementsov-Ogievskiy
It's unused now (except for permission handling)[*]. The only reasonable user of it was block-stream job, recently updated to use own blk. And other block jobs prefer to use own source node related objects. So, the arguments of dropping the field are: - block jobs prefer not to use it - block jobs usually has more then one node to operate on, and better to operate symmetrically (for example has both source and target blk's in specific block-job state structure) *: BlockJob.blk is used to keep some permissions. We simply move permissions to block-job child created in block_job_create() together with blk. In mirror, we just should not care anymore about restoring state of blk. Most probably this code could be dropped long ago, after dropping bs->job pointer. Now it finally goes away together with BlockJob.blk itself. iotest 141 output is updated, as "bdrv_has_blk(bs)" check in qmp_blockdev_del() doesn't fail (we don't have blk now). Still, new error message looks even better. In iotest 283 we need to add a job id, otherwise "Invalid job ID" happens now earlier than permission check (as permissions moved from blk to block-job node). Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Nikita Lapshin <nikita.lapshin@virtuozzo.com>
2021-12-23Merge tag 'pull-nbd-2021-12-22-v2' of ↵Richard Henderson
https://src.openvz.org/scm/~vsementsov/qemu into staging nbd: reconnect-on-open feature v2: simple fix for mypy and pylint complains on patch 04 # gpg: Signature made Thu 23 Dec 2021 12:45:20 AM PST # gpg: using RSA key 8B9C26CDB2FD147C880E86A1561F24C1F19F79FB # gpg: Good signature from "Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>" [unknown] # gpg: WARNING: This key is not certified with a trusted signature! # gpg: There is no indication that the signature belongs to the owner. # Primary key fingerprint: 8B9C 26CD B2FD 147C 880E 86A1 561F 24C1 F19F 79FB * tag 'pull-nbd-2021-12-22-v2' of https://src.openvz.org/scm/~vsementsov/qemu: iotests: add nbd-reconnect-on-open test iotests.py: add qemu_io_popen() iotests.py: add and use qemu_io_wrap_args() iotests.py: add qemu_tool_popen() nbd/client-connection: improve error message of cancelled attempt nbd/client-connection: nbd_co_establish_connection(): return real error nbd: allow reconnect on open, with corresponding new options Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-12-23iotests: add nbd-reconnect-on-open testVladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Nikita Lapshin <nikita.lapshin@virtuozzo.com>
2021-12-23iotests.py: add qemu_io_popen()Vladimir Sementsov-Ogievskiy
Add qemu-io Popen constructor wrapper. To be used in the following new test commit. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Nikita Lapshin <nikita.lapshin@virtuozzo.com>
2021-12-23iotests.py: add and use qemu_io_wrap_args()Vladimir Sementsov-Ogievskiy
For qemu_io* functions support --image-opts argument, which conflicts with -f argument from qemu_io_args. For QemuIoInteractive use new wrapper as well, which allows relying on default format. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Nikita Lapshin <nikita.lapshin@virtuozzo.com>
2021-12-23iotests.py: add qemu_tool_popen()Vladimir Sementsov-Ogievskiy
Split qemu_tool_popen() from qemu_tool_pipe_and_status() to be used separately. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Nikita Lapshin <nikita.lapshin@virtuozzo.com>
2021-12-22iotests: check: multiprocessing supportVladimir Sementsov-Ogievskiy
Add -j <JOBS> parameter, to run tests in several jobs simultaneously. For realization - simply utilize multiprocessing.Pool class. Notes: 1. Of course, tests can't run simultaneously in same TEST_DIR. So, use subdirectories TEST_DIR/testname/ and SOCK_DIR/testname/ instead of simply TEST_DIR and SOCK_DIR 2. multiprocessing.Pool.starmap function doesn't support passing context managers, so we can't simply pass "self". Happily, we need self only for read-only access, and it just works if it is defined in global space. So, add a temporary link TestRunner.shared_self during run_tests(). Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211203122223.2780098-4-vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Tested-by: John Snow <jsnow@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-12-22iotests/testrunner.py: move updating last_elapsed to run_testsVladimir Sementsov-Ogievskiy
We are going to use do_run_test() in multiprocessing environment, where we'll not be able to change original runner object. Happily, the only thing we change is that last_elapsed and it's simple to do it in run_tests() instead. All other accesses to self in do_runt_test() and in run_test() are read-only. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211203122223.2780098-3-vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Tested-by: John Snow <jsnow@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-12-22iotests/testrunner.py: add doc string for run_test()Vladimir Sementsov-Ogievskiy
We are going to modify these methods and will add more documentation in further commit. As a preparation add basic documentation. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20211203122223.2780098-2-vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Tested-by: John Snow <jsnow@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-11-23iotests/149: Skip on unsupported ciphersHanna Reitz
Whenever qemu-img or qemu-io report that some cipher is unsupported, skip the whole test, because that is probably because qemu has been configured with the gnutls crypto backend. We could taylor the algorithm list to what gnutls supports, but this is a test that is run rather rarely anyway (because it requires password-less sudo), and so it seems better and easier to skip it. When this test is intentionally run to check LUKS compatibility, it seems better not to limit the algorithms but keep the list extensive. Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211117151707.52549-3-hreitz@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2021-11-23iotests: Use aes-128-cbcHanna Reitz
Our gnutls crypto backend (which is the default as of 8bd0931f6) supports neither twofish-128 nor the CTR mode. CBC and aes-128 are supported by all of our backends (as far as I can tell), so use aes-128-cbc in our iotests. (We could also use e.g. aes-256-cbc, but the different key sizes would lead to different key slot offsets and so change the reference output more, which is why I went with aes-128.) Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211117151707.52549-2-hreitz@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Tested-by: Thomas Huth <thuth@redhat.com>
2021-11-16file-posix: Fix alignment after reopen changing O_DIRECTKevin Wolf
At the end of a reopen, we already call bdrv_refresh_limits(), which should update bs->request_alignment according to the new file descriptor. However, raw_probe_alignment() relies on s->needs_alignment and just uses 1 if it isn't set. We neglected to update this field, so starting with cache=writeback and then reopening with cache=none means that we get an incorrect bs->request_alignment == 1 and unaligned requests fail instead of being automatically aligned. Fix this by recalculating s->needs_alignment in raw_refresh_limits() before calling raw_probe_alignment(). Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20211104113109.56336-1-kwolf@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20211115145409.176785-13-kwolf@redhat.com> [hreitz: Fix iotest 142 for block sizes greater than 512 by operating on a file with a size of 1 MB] Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211116101431.105252-1-hreitz@redhat.com>
2021-11-16iotests/030: Unthrottle parallel jobs in reverseHanna Reitz
See the comment for why this is necessary. Signed-off-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20211111120829.81329-11-hreitz@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20211115145409.176785-11-kwolf@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-11-03Merge remote-tracking branch 'remotes/kwolf/tags/for-upstream' into stagingRichard Henderson
Block layer patches - Fail gracefully when blockdev-snapshot creates loops - ide: Fix IDENTIFY DEVICE for disks > 128 GiB - file-posix: Fix return value translation for AIO discards - file-posix: add 'aio-max-batch' option - rbd: implement bdrv_co_block_status - Code cleanups and build fixes # gpg: Signature made Tue 02 Nov 2021 12:04:02 PM EDT # gpg: using RSA key DC3DEB159A9AF95D3D7456FE7F09B272C88F2FD6 # gpg: issuer "kwolf@redhat.com" # gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full] * remotes/kwolf/tags/for-upstream: block/nvme: Extract nvme_free_queue() from nvme_free_queue_pair() block/nvme: Display CQ/SQ pointer in nvme_free_queue_pair() block/nvme: Automatically free qemu_memalign() with QEMU_AUTO_VFREE block-backend: Silence clang -m32 compiler warning linux-aio: add `dev_max_batch` parameter to laio_io_unplug() linux-aio: add `dev_max_batch` parameter to laio_co_submit() file-posix: add `aio-max-batch` option block/export/fuse.c: fix musl build ide: Cap LBA28 capacity announcement to 2^28-1 block/rbd: implement bdrv_co_block_status block: Fail gracefully when blockdev-snapshot creates loops block/file-posix: Fix return value translation for AIO discards Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-11-02block: Fail gracefully when blockdev-snapshot creates loopsKevin Wolf
Using blockdev-snapshot to append a node as an overlay to itself, or to any of its parents, causes crashes. Catch the condition and return an error for these cases instead. Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1824363 Signed-off-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20211018134714.48438-1-kwolf@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-11-02pylint: fix errors and warnings generated by tests/qemu-iotests/297Emanuele Giuseppe Esposito
Test 297 in tests/qemu-iotests currently fails: pylint has learned new things to check, or we simply missed them. All fixes in this patch are related to additional spaces used or wrong indentation. No functional change intended. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Message-Id: <20211008062821.1010967-2-eesposit@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-11-01iotests/300: avoid abnormal shutdown race conditionJohn Snow
Wait for the destination VM to close itself instead of racing to shut it down first, which produces different error log messages from AQMP depending on precisely when we tried to shut it down. (For example: We may try to issue 'quit' immediately prior to the target VM closing its QMP socket, which will cause an ECONNRESET error to be logged. Waiting for the VM to exit itself avoids the race on shutdown behavior.) Reported-by: Hanna Reitz <hreitz@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211026175612.4127598-7-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01iotests: Conditionally silence certain AQMP errorsJohn Snow
AQMP likes to be very chatty about errors it encounters. In general, this is good because it allows us to get good diagnostic information for otherwise complex async failures. For example, during a failed QMP connection attempt, we might see: +ERROR:qemu.aqmp.qmp_client.qemub-2536319:Negotiation failed: EOFError +ERROR:qemu.aqmp.qmp_client.qemub-2536319:Failed to establish session: EOFError This might be nice in iotests output, because failure scenarios involving the new QMP library will be spelled out plainly in the output diffs. For tests that are intentionally causing this scenario though, filtering that log output could be a hassle. For now, add a context manager that simply lets us toggle this output off during a critical region. (Additionally, a forthcoming patch allows the use of either legacy or async QMP to be toggled with an environment variable. In this circumstance, we can't amend the iotest output to just always expect the error message, either. Just suppress it for now. More rigorous log filtering can be investigated later if/when it is deemed safe to permanently replace the legacy QMP library.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20211026175612.4127598-6-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01iotests: Accommodate async QMP Exception classesJohn Snow
(But continue to support the old ones for now, too.) There are very few cases of any user of QEMUMachine or a subclass thereof relying on a QMP Exception type. If you'd like to check for yourself, you want to grep for all of the derivatives of QMPError, excluding 'AQMPError' and its derivatives. That'd be these: - QMPError - QMPConnectError - QMPCapabilitiesError - QMPTimeoutError - QMPProtocolError - QMPResponseError - QMPBadPortError Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20211026175612.4127598-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01python/machine: remove has_quit argumentJohn Snow
If we spy on the QMP commands instead, we don't need callers to remember to pass it. Seems like a fair trade-off. The one slightly weird bit is overloading this instance variable for wait(), where we use it to mean "don't issue the qmp 'quit' command". This means that wait() will "fail" if the QEMU process does not terminate of its own accord. In most cases, we probably did already actually issue quit -- some iotests do this -- but in some others, we may be waiting for QEMU to terminate for some other reason, such as a test wherein we tell the guest (directly) to shut down. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20211026175612.4127598-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01iotests/linters: Add workaround for mypy bug #9852John Snow
This one is insidious: if you write an import as "from {namespace} import {subpackage}" as mirror-top-perms (now) does, mypy will fail on every-other invocation *if* the package being imported is a typed, installed, namespace-scoped package. Upsettingly, that's exactly what 'qemu.[aqmp|qmp|machine]' et al are in the context of Python CI tests. Now, I could just edit mirror-top-perms to avoid this invocation, but since I tripped on a landmine, I might as well head it off at the pass and make sure nobody else trips on that same landmine. It seems to have something to do with the order in which files are checked as well, meaning the random order in which set(os.listdir()) produces the list of files to test will cause problems intermittently and not just strictly "every other run". This will be fixed in mypy >= 0.920, which is not released yet. The workaround for now is to disable incremental checking, which avoids the issue. Note: This workaround is not applied when running iotest 297 directly, because the bug does not surface there! Given the nature of CI jobs not starting with any stale cache to begin with, this really only has a half-second impact on manual runs of the Python test suite when executed directly by a developer on their local machine. The workaround may be removed when the Python package requirements can stipulate mypy 0.920 or higher, which can happen as soon as it is released. (Barring any unforseen compatibility issues that 0.920 may bring with it.) See also: https://github.com/python/mypy/issues/11010 https://github.com/python/mypy/issues/9852 Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211019144918.3159078-14-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01iotests/linters: Add entry point for linting via Python CIJohn Snow
We need at least a tiny little shim here to join test file discovery with test invocation. This logic could conceivably be hosted somewhere in python/, but I felt it was strictly the least-rude thing to keep the test logic here in iotests/, even if this small function isn't itself an iotest. Note that we don't actually even need the executable bit here, we'll be relying on the ability to run this module as a script using Python CLI arguments. No chance it gets misunderstood as an actual iotest that way. (It's named, not in tests/, doesn't have the execute bit, and doesn't have an execution shebang.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211019144918.3159078-13-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01iotests: split linters.py out from 297John Snow
Now, 297 is just the iotests-specific incantations and linters.py is as minimal as I can think to make it. The only remaining element in here that ought to be configuration and not code is the list of skip files, but they're still numerous enough that repeating them for mypy and pylint configurations both would be ... a hassle. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-id: 20211019144918.3159078-12-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
generated by cgit v1.2.3 (git 2.26.2) at 2024-12-10 13:43:22 +0000