diff options
author | Stefan Hajnoczi <stefanha@redhat.com> | 2023-09-07 10:27:55 -0400 |
---|---|---|
committer | Stefan Hajnoczi <stefanha@redhat.com> | 2023-09-07 10:27:55 -0400 |
commit | c97d45d55798b27410253df8fc724d2a02189aa8 (patch) | |
tree | 8003a598b3e346635c2b77e207da550860c8fa6d | |
parent | 269e60635a72f8dec4cb210b5b99e9e7f8920f34 (diff) | |
parent | 06fca745d5ba3be8b693985e6075d6ba2782f145 (diff) |
Merge tag 'pull-parallels-2023-09-06' of https://src.openvz.org/scm/~den/qemu into staging
Parallels format driver changes:
* Fix comments formatting inside parallels driver
* Incorrect data end calculation in parallels_open()
* Check if data_end greater than the file size
* Add "explicit" argument to parallels_check_leak()
* Add data_start field to BDRVParallelsState
* Add checking and repairing duplicate offsets in BAT
* Image repairing in parallels_open()
* Use bdrv_co_getlength() in parallels_check_outside_image()
* Add data_off check
* Add data_off repairing to parallels_open()
* Fix record in MAINTAINERS
Parallels format driver tests:
* Add out-of-image check test for parallels format
* Add leak check test for parallels format
* Add test for BAT entries duplication check
* Refactor tests of parallels images checks (131)
* Fix cluster size in parallels images tests (131)
* Fix test 131 after repair was added to parallels_open()
* Add test for data_off check
# -----BEGIN PGP SIGNATURE-----
#
# iQHDBAABCgAtFiEE9vE2f3B8+RUZInytPzClrpN3nJ8FAmT4nUgPHGRlbkBvcGVu
# dnoub3JnAAoJED8wpa6Td5yf1F4L/j4RsGv+NRJRqZb9JNn2wUm4JdWGyv6ftuuh
# hT25F44B5S6J3tR3LalDFxHpr+kCXD1Xa3ZJNK14d1G9atw7Bsp5ntxpCmzEALBk
# 0PH+5fvNuhvt4ZnuYwQX70n3ZmalgzGpwf/jbs9mXUhdLinEr1RWi2f9yfCLmeZU
# x+0MSOhAdC6ZVsJOTJhGuRWWKL1q5KteuTwQlRCwDay8KF/Mc1OS/iPFqfmlWenM
# dc88PZBlg2Le15sWWNLc1AZHYguO+4xEPw6fk6RcswccILB2gCUPS6BJB0AuKNOO
# STPIgzUFMXfgIFhNUOvz58A7UnQGI4dMsRe/2UJIG+Y3qkM4DpjcZ7U/rHxhR6t0
# +GeeLS+a+aObz79TpB3gZi7leX2bpRUZ8nLkaAnL2umhtdFo5sdqD3xo4xcg4Ebk
# TbYSmgIM0eZ75d+48g7A+ddkyKYCmworGS9g9Cry6udclbs8yXhVB8KkUbYwtJlC
# HtNzgaWlw6J7n0MoSpz4OQVKq3bY0A==
# =grCk
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 06 Sep 2023 11:39:52 EDT
# gpg: using RSA key F6F1367F707CF91519227CAD3F30A5AE93779C9F
# gpg: issuer "den@openvz.org"
# gpg: Good signature from "Denis V. Lunev <den@openvz.org>" [unknown]
# gpg: WARNING: The key's User ID is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: F6F1 367F 707C F915 1922 7CAD 3F30 A5AE 9377 9C9F
* tag 'pull-parallels-2023-09-06' of https://src.openvz.org/scm/~den/qemu:
iotests: Add test for data_off check
iotests: Fix test 131 after repair was added to parallels_open()
iotests: Fix cluster size in parallels images tests (131)
iotests: Refactor tests of parallels images checks (131)
iotests: Add test for BAT entries duplication check
iotests: Add leak check test for parallels format
iotests: Add out-of-image check test for parallels format
parallels: Add data_off repairing to parallels_open()
parallels: Add data_off check
parallels: Use bdrv_co_getlength() in parallels_check_outside_image()
parallels: Image repairing in parallels_open()
parallels: Add checking and repairing duplicate offsets in BAT
parallels: Add data_start field to BDRVParallelsState
parallels: Add "explicit" argument to parallels_check_leak()
parallels: Check if data_end greater than the file size
parallels: Incorrect data end calculation in parallels_open()
parallels: Fix comments formatting inside parallels driver
MAINTAINERS: add tree to keep parallels format driver changes
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
-rw-r--r-- | MAINTAINERS | 1 | ||||
-rw-r--r-- | block/parallels.c | 346 | ||||
-rw-r--r-- | block/parallels.h | 1 | ||||
-rwxr-xr-x | tests/qemu-iotests/131 | 36 | ||||
-rw-r--r-- | tests/qemu-iotests/131.out | 59 | ||||
-rwxr-xr-x | tests/qemu-iotests/tests/parallels-checks | 145 | ||||
-rw-r--r-- | tests/qemu-iotests/tests/parallels-checks.out | 75 |
7 files changed, 564 insertions, 99 deletions
diff --git a/MAINTAINERS b/MAINTAINERS index 3b29568ed4..b471973e1e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -3693,6 +3693,7 @@ S: Supported F: block/parallels.c F: block/parallels-ext.c F: docs/interop/parallels.txt +T: git https://src.openvz.org/scm/~den/qemu.git parallels qed M: Stefan Hajnoczi <stefanha@redhat.com> diff --git a/block/parallels.c b/block/parallels.c index 18e34aef28..48c32d6821 100644 --- a/block/parallels.c +++ b/block/parallels.c @@ -136,6 +136,12 @@ static int cluster_remainder(BDRVParallelsState *s, int64_t sector_num, return MIN(nb_sectors, ret); } +static uint32_t host_cluster_index(BDRVParallelsState *s, int64_t off) +{ + off -= s->data_start << BDRV_SECTOR_BITS; + return off / s->cluster_size; +} + static int64_t block_status(BDRVParallelsState *s, int64_t sector_num, int nb_sectors, int *pnum) { @@ -188,7 +194,8 @@ allocate_clusters(BlockDriverState *bs, int64_t sector_num, idx = sector_num / s->tracks; to_allocate = DIV_ROUND_UP(sector_num + *pnum, s->tracks) - idx; - /* This function is called only by parallels_co_writev(), which will never + /* + * This function is called only by parallels_co_writev(), which will never * pass a sector_num at or beyond the end of the image (because the block * layer never passes such a sector_num to that function). Therefore, idx * is always below s->bat_size. @@ -196,7 +203,8 @@ allocate_clusters(BlockDriverState *bs, int64_t sector_num, * exceed the image end. Therefore, idx + to_allocate cannot exceed * s->bat_size. * Note that s->bat_size is an unsigned int, therefore idx + to_allocate - * will always fit into a uint32_t. */ + * will always fit into a uint32_t. + */ assert(idx < s->bat_size && idx + to_allocate <= s->bat_size); space = to_allocate * s->tracks; @@ -230,13 +238,15 @@ allocate_clusters(BlockDriverState *bs, int64_t sector_num, } } - /* Try to read from backing to fill empty clusters + /* + * Try to read from backing to fill empty clusters * FIXME: 1. previous write_zeroes may be redundant * 2. most of data we read from backing will be rewritten by * parallels_co_writev. On aligned-to-cluster write we do not need * this read at all. * 3. it would be good to combine write of data from backing and new - * data into one write call */ + * data into one write call. + */ if (bs->backing) { int64_t nb_cow_sectors = to_allocate * s->tracks; int64_t nb_cow_bytes = nb_cow_sectors << BDRV_SECTOR_BITS; @@ -440,6 +450,81 @@ static void parallels_check_unclean(BlockDriverState *bs, } } +/* + * Returns true if data_off is correct, otherwise false. In both cases + * correct_offset is set to the proper value. + */ +static bool parallels_test_data_off(BDRVParallelsState *s, + int64_t file_nb_sectors, + uint32_t *correct_offset) +{ + uint32_t data_off, min_off; + bool old_magic; + + /* + * There are two slightly different image formats: with "WithoutFreeSpace" + * or "WithouFreSpacExt" magic words. Call the first one as "old magic". + * In such images data_off field can be zero. In this case the offset is + * calculated as the end of BAT table plus some padding to ensure sector + * size alignment. + */ + old_magic = !memcmp(s->header->magic, HEADER_MAGIC, 16); + + min_off = DIV_ROUND_UP(bat_entry_off(s->bat_size), BDRV_SECTOR_SIZE); + if (!old_magic) { + min_off = ROUND_UP(min_off, s->cluster_size / BDRV_SECTOR_SIZE); + } + + if (correct_offset) { + *correct_offset = min_off; + } + + data_off = le32_to_cpu(s->header->data_off); + if (data_off == 0 && old_magic) { + return true; + } + + if (data_off < min_off || data_off > file_nb_sectors) { + return false; + } + + if (correct_offset) { + *correct_offset = data_off; + } + + return true; +} + +static int coroutine_fn GRAPH_RDLOCK +parallels_check_data_off(BlockDriverState *bs, BdrvCheckResult *res, + BdrvCheckMode fix) +{ + BDRVParallelsState *s = bs->opaque; + int64_t file_size; + uint32_t data_off; + + file_size = bdrv_co_nb_sectors(bs->file->bs); + if (file_size < 0) { + res->check_errors++; + return file_size; + } + + if (parallels_test_data_off(s, file_size, &data_off)) { + return 0; + } + + res->corruptions++; + if (fix & BDRV_FIX_ERRORS) { + s->header->data_off = cpu_to_le32(data_off); + res->corruptions_fixed++; + } + + fprintf(stderr, "%s data_off field has incorrect value\n", + fix & BDRV_FIX_ERRORS ? "Repairing" : "ERROR"); + + return 0; +} + static int coroutine_fn GRAPH_RDLOCK parallels_check_outside_image(BlockDriverState *bs, BdrvCheckResult *res, BdrvCheckMode fix) @@ -484,13 +569,13 @@ parallels_check_outside_image(BlockDriverState *bs, BdrvCheckResult *res, static int coroutine_fn GRAPH_RDLOCK parallels_check_leak(BlockDriverState *bs, BdrvCheckResult *res, - BdrvCheckMode fix) + BdrvCheckMode fix, bool explicit) { BDRVParallelsState *s = bs->opaque; int64_t size; int ret; - size = bdrv_getlength(bs->file->bs); + size = bdrv_co_getlength(bs->file->bs); if (size < 0) { res->check_errors++; return size; @@ -499,10 +584,13 @@ parallels_check_leak(BlockDriverState *bs, BdrvCheckResult *res, if (size > res->image_end_offset) { int64_t count; count = DIV_ROUND_UP(size - res->image_end_offset, s->cluster_size); - fprintf(stderr, "%s space leaked at the end of the image %" PRId64 "\n", - fix & BDRV_FIX_LEAKS ? "Repairing" : "ERROR", - size - res->image_end_offset); - res->leaks += count; + if (explicit) { + fprintf(stderr, + "%s space leaked at the end of the image %" PRId64 "\n", + fix & BDRV_FIX_LEAKS ? "Repairing" : "ERROR", + size - res->image_end_offset); + res->leaks += count; + } if (fix & BDRV_FIX_LEAKS) { Error *local_err = NULL; @@ -517,13 +605,148 @@ parallels_check_leak(BlockDriverState *bs, BdrvCheckResult *res, res->check_errors++; return ret; } - res->leaks_fixed += count; + if (explicit) { + res->leaks_fixed += count; + } } } return 0; } +static int coroutine_fn GRAPH_RDLOCK +parallels_check_duplicate(BlockDriverState *bs, BdrvCheckResult *res, + BdrvCheckMode fix) +{ + BDRVParallelsState *s = bs->opaque; + int64_t host_off, host_sector, guest_sector; + unsigned long *bitmap; + uint32_t i, bitmap_size, cluster_index, bat_entry; + int n, ret = 0; + uint64_t *buf = NULL; + bool fixed = false; + + /* + * Create a bitmap of used clusters. + * If a bit is set, there is a BAT entry pointing to this cluster. + * Loop through the BAT entries, check bits relevant to an entry offset. + * If bit is set, this entry is duplicated. Otherwise set the bit. + * + * We shouldn't worry about newly allocated clusters outside the image + * because they are created higher then any existing cluster pointed by + * a BAT entry. + */ + bitmap_size = host_cluster_index(s, res->image_end_offset); + if (bitmap_size == 0) { + return 0; + } + if (res->image_end_offset % s->cluster_size) { + /* A not aligned image end leads to a bitmap shorter by 1 */ + bitmap_size++; + } + + bitmap = bitmap_new(bitmap_size); + + buf = qemu_blockalign(bs, s->cluster_size); + + for (i = 0; i < s->bat_size; i++) { + host_off = bat2sect(s, i) << BDRV_SECTOR_BITS; + if (host_off == 0) { + continue; + } + + cluster_index = host_cluster_index(s, host_off); + assert(cluster_index < bitmap_size); + if (!test_bit(cluster_index, bitmap)) { + bitmap_set(bitmap, cluster_index, 1); + continue; + } + + /* this cluster duplicates another one */ + fprintf(stderr, "%s duplicate offset in BAT entry %u\n", + fix & BDRV_FIX_ERRORS ? "Repairing" : "ERROR", i); + + res->corruptions++; + + if (!(fix & BDRV_FIX_ERRORS)) { + continue; + } + + /* + * Reset the entry and allocate a new cluster + * for the relevant guest offset. In this way we let + * the lower layer to place the new cluster properly. + * Copy the original cluster to the allocated one. + * But before save the old offset value for repairing + * if we have an error. + */ + bat_entry = s->bat_bitmap[i]; + parallels_set_bat_entry(s, i, 0); + + ret = bdrv_co_pread(bs->file, host_off, s->cluster_size, buf, 0); + if (ret < 0) { + res->check_errors++; + goto out_repair_bat; + } + + guest_sector = (i * (int64_t)s->cluster_size) >> BDRV_SECTOR_BITS; + host_sector = allocate_clusters(bs, guest_sector, s->tracks, &n); + if (host_sector < 0) { + res->check_errors++; + goto out_repair_bat; + } + host_off = host_sector << BDRV_SECTOR_BITS; + + ret = bdrv_co_pwrite(bs->file, host_off, s->cluster_size, buf, 0); + if (ret < 0) { + res->check_errors++; + goto out_repair_bat; + } + + if (host_off + s->cluster_size > res->image_end_offset) { + res->image_end_offset = host_off + s->cluster_size; + } + + /* + * In the future allocate_cluster() will reuse holed offsets + * inside the image. Keep the used clusters bitmap content + * consistent for the new allocated clusters too. + * + * Note, clusters allocated outside the current image are not + * considered, and the bitmap size doesn't change. + */ + cluster_index = host_cluster_index(s, host_off); + if (cluster_index < bitmap_size) { + bitmap_set(bitmap, cluster_index, 1); + } + + fixed = true; + res->corruptions_fixed++; + + } + + if (fixed) { + /* + * When new clusters are allocated, the file size increases by + * 128 Mb. We need to truncate the file to the right size. Let + * the leak fix code make its job without res changing. + */ + ret = parallels_check_leak(bs, res, fix, false); + } + +out_free: + g_free(buf); + g_free(bitmap); + return ret; +/* + * We can get here only from places where index and old_offset have + * meaningful values. + */ +out_repair_bat: + s->bat_bitmap[i] = bat_entry; + goto out_free; +} + static void parallels_collect_statistics(BlockDriverState *bs, BdrvCheckResult *res, BdrvCheckMode fix) @@ -565,12 +788,22 @@ parallels_co_check(BlockDriverState *bs, BdrvCheckResult *res, WITH_QEMU_LOCK_GUARD(&s->lock) { parallels_check_unclean(bs, res, fix); + ret = parallels_check_data_off(bs, res, fix); + if (ret < 0) { + return ret; + } + ret = parallels_check_outside_image(bs, res, fix); if (ret < 0) { return ret; } - ret = parallels_check_leak(bs, res, fix); + ret = parallels_check_leak(bs, res, fix, true); + if (ret < 0) { + return ret; + } + + ret = parallels_check_duplicate(bs, res, fix); if (ret < 0) { return ret; } @@ -798,10 +1031,12 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, BDRVParallelsState *s = bs->opaque; ParallelsHeader ph; int ret, size, i; - int64_t file_nb_sectors; + int64_t file_nb_sectors, sector; + uint32_t data_start; QemuOpts *opts = NULL; Error *local_err = NULL; char *buf; + bool data_off_is_correct; ret = bdrv_open_file_child(NULL, options, "file", bs, errp); if (ret < 0) { @@ -859,15 +1094,6 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, ret = -ENOMEM; goto fail; } - s->data_end = le32_to_cpu(ph.data_off); - if (s->data_end == 0) { - s->data_end = ROUND_UP(bat_entry_off(s->bat_size), BDRV_SECTOR_SIZE); - } - if (s->data_end < s->header_size) { - /* there is not enough unused space to fit to block align between BAT - and actual data. We can't avoid read-modify-write... */ - s->header_size = size; - } ret = bdrv_pread(bs->file, 0, s->header_size, s->header, 0); if (ret < 0) { @@ -875,33 +1101,20 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, } s->bat_bitmap = (uint32_t *)(s->header + 1); - for (i = 0; i < s->bat_size; i++) { - int64_t off = bat2sect(s, i); - if (off >= file_nb_sectors) { - if (flags & BDRV_O_CHECK) { - continue; - } - error_setg(errp, "parallels: Offset %" PRIi64 " in BAT[%d] entry " - "is larger than file size (%" PRIi64 ")", - off << BDRV_SECTOR_BITS, i, - file_nb_sectors << BDRV_SECTOR_BITS); - ret = -EINVAL; - goto fail; - } - if (off >= s->data_end) { - s->data_end = off + s->tracks; - } - } - if (le32_to_cpu(ph.inuse) == HEADER_INUSE_MAGIC) { - /* Image was not closed correctly. The check is mandatory */ s->header_unclean = true; - if ((flags & BDRV_O_RDWR) && !(flags & BDRV_O_CHECK)) { - error_setg(errp, "parallels: Image was not closed correctly; " - "cannot be opened read/write"); - ret = -EACCES; - goto fail; - } + } + + data_off_is_correct = parallels_test_data_off(s, file_nb_sectors, + &data_start); + s->data_start = data_start; + s->data_end = s->data_start; + if (s->data_end < (s->header_size >> BDRV_SECTOR_BITS)) { + /* + * There is not enough unused space to fit to block align between BAT + * and actual data. We can't avoid read-modify-write... + */ + s->header_size = size; } opts = qemu_opts_create(¶llels_runtime_opts, NULL, 0, errp); @@ -962,10 +1175,41 @@ static int parallels_open(BlockDriverState *bs, QDict *options, int flags, bdrv_get_device_or_node_name(bs)); ret = migrate_add_blocker(s->migration_blocker, errp); if (ret < 0) { - error_free(s->migration_blocker); + error_setg(errp, "Migration blocker error"); goto fail; } qemu_co_mutex_init(&s->lock); + + for (i = 0; i < s->bat_size; i++) { + sector = bat2sect(s, i); + if (sector + s->tracks > s->data_end) { + s->data_end = sector + s->tracks; + } + } + + /* + * We don't repair the image here if it's opened for checks. Also we don't + * want to change inactive images and can't change readonly images. + */ + if ((flags & (BDRV_O_CHECK | BDRV_O_INACTIVE)) || !(flags & BDRV_O_RDWR)) { + return 0; + } + + /* + * Repair the image if it's dirty or + * out-of-image corruption was detected. + */ + if (s->data_end > file_nb_sectors || s->header_unclean + || !data_off_is_correct) { + BdrvCheckResult res; + ret = bdrv_check(bs, &res, BDRV_FIX_ERRORS | BDRV_FIX_LEAKS); + if (ret < 0) { + error_setg_errno(errp, -ret, "Could not repair corrupted image"); + migrate_del_blocker(s->migration_blocker); + goto fail; + } + } + return 0; fail_format: @@ -973,6 +1217,12 @@ fail_format: fail_options: ret = -EINVAL; fail: + /* + * "s" object was allocated by g_malloc0 so we can safely + * try to free its fields even they were not allocated. + */ + error_free(s->migration_blocker); + g_free(s->bat_dirty_bmap); qemu_vfree(s->header); return ret; } diff --git a/block/parallels.h b/block/parallels.h index f22f43f988..4e53e9572d 100644 --- a/block/parallels.h +++ b/block/parallels.h @@ -75,6 +75,7 @@ typedef struct BDRVParallelsState { uint32_t *bat_bitmap; unsigned int bat_size; + int64_t data_start; int64_t data_end; uint64_t prealloc_size; ParallelsPreallocMode prealloc_mode; diff --git a/tests/qemu-iotests/131 b/tests/qemu-iotests/131 index a847692b4c..304bbb3f61 100755 --- a/tests/qemu-iotests/131 +++ b/tests/qemu-iotests/131 @@ -44,31 +44,35 @@ _supported_os Linux inuse_offset=$((0x2c)) size=$((64 * 1024 * 1024)) -CLUSTER_SIZE=64k IMGFMT=parallels _make_test_img $size +# get cluster size in sectors from "tracks" header field +CLUSTER_SIZE_OFFSET=28 +CLUSTER_SIZE=$(peek_file_le $TEST_IMG $CLUSTER_SIZE_OFFSET 4) +CLUSTER_SIZE=$((CLUSTER_SIZE * 512)) +CLUSTER_HALF_SIZE=$((CLUSTER_SIZE / 2)) +CLUSTER_DBL_SIZE=$((CLUSTER_SIZE * 2)) + echo == read empty image == -{ $QEMU_IO -c "read -P 0 32k 64k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0 $CLUSTER_HALF_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir echo == write more than 1 block in a row == -{ $QEMU_IO -c "write -P 0x11 32k 128k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "write -P 0x11 $CLUSTER_HALF_SIZE $CLUSTER_DBL_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir echo == read less than block == -{ $QEMU_IO -c "read -P 0x11 32k 32k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0x11 $CLUSTER_HALF_SIZE $CLUSTER_HALF_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir echo == read exactly 1 block == -{ $QEMU_IO -c "read -P 0x11 64k 64k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir echo == read more than 1 block == -{ $QEMU_IO -c "read -P 0x11 32k 128k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0x11 $CLUSTER_HALF_SIZE $CLUSTER_DBL_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir echo == check that there is no trash after written == -{ $QEMU_IO -c "read -P 0 160k 32k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0 $((CLUSTER_HALF_SIZE + CLUSTER_DBL_SIZE)) $CLUSTER_HALF_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir echo == check that there is no trash before written == -{ $QEMU_IO -c "read -P 0 0 32k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +{ $QEMU_IO -c "read -P 0 0 $CLUSTER_HALF_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir -echo "== Corrupt image ==" +echo "== corrupt image ==" poke_file "$TEST_IMG" "$inuse_offset" "\x59\x6e\x6f\x74" -{ $QEMU_IO -c "read -P 0x11 64k 64k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir -_check_test_img -_check_test_img -r all -{ $QEMU_IO -c "read -P 0x11 64k 64k" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir +echo "== read corrupted image with repairing ==" +{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir echo "== allocate with backing ==" # Verify that allocating clusters works fine even when there is a backing image. @@ -83,7 +87,7 @@ TEST_IMG="$TEST_IMG.base" _make_test_img $size # Write some data to the base image (which would trigger an assertion failure if # interpreted as a QEMUIOVector) -$QEMU_IO -c 'write -P 42 0 64k' "$TEST_IMG.base" | _filter_qemu_io +$QEMU_IO -c "write -P 42 0 $CLUSTER_SIZE" "$TEST_IMG.base" | _filter_qemu_io # Parallels does not seem to support storing a backing filename in the image # itself, so we need to build our backing chain on the command line @@ -99,8 +103,8 @@ QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT \ QEMU_IO_OPTIONS=$QEMU_IO_OPTIONS_NO_FMT \ $QEMU_IO --image-opts "$imgopts" \ -c 'read -P 1 0 64' \ - -c "read -P 42 64 $((64 * 1024 - 64))" \ - -c "read -P 0 64k $((size - 64 * 1024))" \ + -c "read -P 42 64 $((CLUSTER_SIZE - 64))" \ + -c "read -P 0 $CLUSTER_SIZE $((size - CLUSTER_SIZE))" \ | _filter_qemu_io # success, all done diff --git a/tests/qemu-iotests/131.out b/tests/qemu-iotests/131.out index de5ef7a8f5..d2904578df 100644 --- a/tests/qemu-iotests/131.out +++ b/tests/qemu-iotests/131.out @@ -1,53 +1,42 @@ QA output created by 131 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 == read empty image == -read 65536/65536 bytes at offset 32768 -64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 524288 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == write more than 1 block in a row == -wrote 131072/131072 bytes at offset 32768 -128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 2097152/2097152 bytes at offset 524288 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == read less than block == -read 32768/32768 bytes at offset 32768 -32 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 524288/524288 bytes at offset 524288 +512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == read exactly 1 block == -read 65536/65536 bytes at offset 65536 -64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == read more than 1 block == -read 131072/131072 bytes at offset 32768 -128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 2097152/2097152 bytes at offset 524288 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == check that there is no trash after written == -read 32768/32768 bytes at offset 163840 -32 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 524288/524288 bytes at offset 2621440 +512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == check that there is no trash before written == -read 32768/32768 bytes at offset 0 -32 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -== Corrupt image == -qemu-io: can't open device TEST_DIR/t.parallels: parallels: Image was not closed correctly; cannot be opened read/write -ERROR image was not closed correctly - -1 errors were found on the image. -Data may be corrupted, or further writes to the image may corrupt it. +read 524288/524288 bytes at offset 0 +512 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +== corrupt image == +== read corrupted image with repairing == Repairing image was not closed correctly -The following inconsistencies were found and repaired: - - 0 leaked clusters - 1 corruptions - -Double checking the fixed image now... -No errors were found on the image. -read 65536/65536 bytes at offset 65536 -64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) == allocate with backing == Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864 -wrote 65536/65536 bytes at offset 0 -64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) wrote 64/64 bytes at offset 0 64 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) read 64/64 bytes at offset 0 64 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -read 65472/65472 bytes at offset 64 -63.938 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) -read 67043328/67043328 bytes at offset 65536 -63.938 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 1048512/1048512 bytes at offset 64 +1023.938 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +read 66060288/66060288 bytes at offset 1048576 +63 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) *** done diff --git a/tests/qemu-iotests/tests/parallels-checks b/tests/qemu-iotests/tests/parallels-checks new file mode 100755 index 0000000000..a7a1b357b5 --- /dev/null +++ b/tests/qemu-iotests/tests/parallels-checks @@ -0,0 +1,145 @@ +#!/usr/bin/env bash +# group: rw quick +# +# Test qemu-img check for parallels format +# +# Copyright (C) 2022 Virtuozzo International GmbH +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. +# + +# creator +owner=alexander.ivanov@virtuozzo.com + +seq=`basename $0` +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ../common.rc +. ../common.filter + +_supported_fmt parallels +_supported_proto file +_supported_os Linux + +SIZE=$((4 * 1024 * 1024)) +IMGFMT=parallels +CLUSTER_SIZE_OFFSET=28 +DATA_OFF_OFFSET=48 +BAT_OFFSET=64 + +_make_test_img $SIZE + +CLUSTER_SIZE=$(peek_file_le $TEST_IMG $CLUSTER_SIZE_OFFSET 4) +CLUSTER_SIZE=$((CLUSTER_SIZE * 512)) +LAST_CLUSTER_OFF=$((SIZE - CLUSTER_SIZE)) +LAST_CLUSTER=$((LAST_CLUSTER_OFF/CLUSTER_SIZE)) + +echo "== TEST OUT OF IMAGE CHECK ==" + +echo "== write pattern ==" +{ $QEMU_IO -c "write -P 0x11 0 $SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +echo "== corrupt image ==" +cluster=$(($LAST_CLUSTER + 2)) +poke_file "$TEST_IMG" "$BAT_OFFSET" "\x$cluster\x00\x00\x00" + +echo "== read corrupted image with repairing ==" +{ $QEMU_IO -c "read -P 0x00 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +# Clear image +_make_test_img $SIZE + +echo "== TEST LEAK CHECK ==" + +echo "== write pattern to last cluster ==" +echo "write -P 0x11 $LAST_CLUSTER_OFF $CLUSTER_SIZE" +{ $QEMU_IO -c "write -P 0x11 $LAST_CLUSTER_OFF $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +file_size=`stat --printf="%s" "$TEST_IMG"` +echo "file size: $file_size" + +echo "== extend image by 1 cluster ==" +fallocate -xl $((file_size + CLUSTER_SIZE)) "$TEST_IMG" + +file_size=`stat --printf="%s" "$TEST_IMG"` +echo "file size: $file_size" + +echo "== repair image ==" +_check_test_img -r all + +file_size=`stat --printf="%s" "$TEST_IMG"` +echo "file size: $file_size" + +echo "== check last cluster ==" +{ $QEMU_IO -c "read -P 0x11 $LAST_CLUSTER_OFF $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +# Clear image +_make_test_img $SIZE + +echo "== TEST DUPLICATION CHECK ==" + +echo "== write pattern to whole image ==" +{ $QEMU_IO -c "write -P 0x11 0 $SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +echo "== write another pattern to second cluster ==" +{ $QEMU_IO -c "write -P 0x55 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +echo "== check second cluster ==" +{ $QEMU_IO -c "read -P 0x55 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +echo "== corrupt image ==" +poke_file "$TEST_IMG" "$(($BAT_OFFSET + 4))" "\x01\x00\x00\x00" + +echo "== check second cluster ==" +{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +echo "== repair image ==" +_check_test_img -r all + +echo "== check second cluster ==" +{ $QEMU_IO -c "read -P 0x11 $CLUSTER_SIZE $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +echo "== check first cluster on host ==" +printf "content: 0x%02x\n" `peek_file_le $TEST_IMG $(($CLUSTER_SIZE)) 1` + +echo "== check second cluster on host ==" +printf "content: 0x%02x\n" `peek_file_le $TEST_IMG $(($CLUSTER_SIZE)) 1` + +# Clear image +_make_test_img $SIZE + +echo "== TEST DATA_OFF CHECK ==" + +echo "== write pattern to first cluster ==" +{ $QEMU_IO -c "write -P 0x55 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +echo "== spoil data_off field ==" +poke_file "$TEST_IMG" "$DATA_OFF_OFFSET" "\xff\xff\xff\xff" + +echo "== check first cluster ==" +{ $QEMU_IO -c "read -P 0x55 0 $CLUSTER_SIZE" "$TEST_IMG"; } 2>&1 | _filter_qemu_io | _filter_testdir + +# success, all done +echo "*** done" +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/tests/parallels-checks.out b/tests/qemu-iotests/tests/parallels-checks.out new file mode 100644 index 0000000000..98a3a7f55e --- /dev/null +++ b/tests/qemu-iotests/tests/parallels-checks.out @@ -0,0 +1,75 @@ +QA output created by parallels-checks +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304 +== TEST OUT OF IMAGE CHECK == +== write pattern == +wrote 4194304/4194304 bytes at offset 0 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +== corrupt image == +== read corrupted image with repairing == +Repairing cluster 0 is outside image +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304 +== TEST LEAK CHECK == +== write pattern to last cluster == +write -P 0x11 3145728 1048576 +wrote 1048576/1048576 bytes at offset 3145728 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +file size: 2097152 +== extend image by 1 cluster == +file size: 3145728 +== repair image == +Repairing space leaked at the end of the image 1048576 +The following inconsistencies were found and repaired: + + 1 leaked clusters + 0 corruptions + +Double checking the fixed image now... +No errors were found on the image. +file size: 2097152 +== check last cluster == +read 1048576/1048576 bytes at offset 3145728 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304 +== TEST DUPLICATION CHECK == +== write pattern to whole image == +wrote 4194304/4194304 bytes at offset 0 +4 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +== write another pattern to second cluster == +wrote 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +== check second cluster == +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +== corrupt image == +== check second cluster == +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +== repair image == +Repairing duplicate offset in BAT entry 1 +The following inconsistencies were found and repaired: + + 0 leaked clusters + 1 corruptions + +Double checking the fixed image now... +No errors were found on the image. +== check second cluster == +read 1048576/1048576 bytes at offset 1048576 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +== check first cluster on host == +content: 0x11 +== check second cluster on host == +content: 0x11 +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304 +== TEST DATA_OFF CHECK == +== write pattern to first cluster == +wrote 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +== spoil data_off field == +== check first cluster == +Repairing data_off field has incorrect value +read 1048576/1048576 bytes at offset 0 +1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +*** done |