aboutsummaryrefslogtreecommitdiff
path: root/hw/virtio/vhost-shadow-virtqueue.c
AgeCommit message (Collapse)Author
2024-10-02hw/virtio: freeing leaked memory from vhost_svq_get_buf in vhost_svq_pollMarc-André Lureau
vhost_svq_get_buf() may return a VirtQueueElement that should be freed. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Eugenio Pérez <eperezma@redhat.com>
2024-10-02hw/virtio: fix -Werror=maybe-uninitializedMarc-André Lureau
../hw/virtio/vhost-shadow-virtqueue.c:545:13: error: ‘r’ may be used uninitialized [-Werror=maybe-uninitialized] Set `r` to 0 at every loop, since we don't check vhost_svq_get_buf() return value. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
2023-10-18vhost: Expose vhost_svq_available_slots()Hawkins Jiawei
Next patches in this series will delay the polling and checking of buffers until either the SVQ is full or control commands shadow buffers are full, no longer perform an immediate poll and check of the device's used buffers for each CVQ state load command. To achieve this, this patch exposes vhost_svq_available_slots(), allowing QEMU to know whether the SVQ is full. Signed-off-by: Hawkins Jiawei <yin31149@gmail.com> Acked-by: Eugenio Pérez <eperezma@redhat.com> Message-Id: <25938079f0bd8185fd664c64e205e629f7a966be.1697165821.git.yin31149@gmail.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2023-10-04vhost: Add count argument to vhost_svq_poll()Hawkins Jiawei
Next patches in this series will no longer perform an immediate poll and check of the device's used buffers for each CVQ state load command. Instead, they will send CVQ state load commands in parallel by polling multiple pending buffers at once. To achieve this, this patch refactoring vhost_svq_poll() to accept a new argument `num`, which allows vhost_svq_poll() to wait for the device to use multiple elements, rather than polling for a single element. Signed-off-by: Hawkins Jiawei <yin31149@gmail.com> Acked-by: Eugenio Pérez <eperezma@redhat.com> Message-Id: <950b3bfcfc5d446168b9d6a249d554a013a691d4.1693287885.git.yin31149@gmail.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2023-07-10vhost: Fix false positive out-of-boundsHawkins Jiawei
QEMU uses vhost_svq_translate_addr() to translate addresses between the QEMU's virtual address and the SVQ IOVA. In order to validate this translation, QEMU checks whether the translated range falls within the mapped range. Yet the problem is that, the value of `needle_last`, which is calculated by `needle.translated_addr + iovec[i].iov_len`, should represent the exclusive boundary of the translated range, rather than the last inclusive addresses of the range. Consequently, QEMU fails the check when the translated range matches the size of the mapped range. This patch solves this problem by fixing the `needle_last` value to the last inclusive address of the translated range. Note that this bug cannot be triggered at the moment, because QEMU is unable to translate such a big range due to the truncation of the CVQ command in vhost_vdpa_net_handle_ctrl_avail(). Fixes: 34e3c94eda ("vdpa: Add custom IOTLB translations to SVQ") Signed-off-by: Hawkins Jiawei <yin31149@gmail.com> Message-Id: <ee31c5420ffc8e6a29705ddd30badb814ddbae1d.1688743107.git.yin31149@gmail.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2023-06-26vdpa: map shadow vrings with MAP_SHAREDEugenio Pérez
The vdpa devices that use va addresses neeeds these maps shared. Otherwise, vhost_vdpa checks will refuse to accept the maps. The mmap call will always return a page aligned address, so removing the qemu_memalign call. Keeping the ROUND_UP for the size as we still need to DMA-map them in full. Not applying fixes tag as it never worked with va devices. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Message-Id: <20230602143854.1879091-4-eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2023-05-19vhost: fix possible wrap in SVQ descriptor ringHawkins Jiawei
QEMU invokes vhost_svq_add() when adding a guest's element into SVQ. In vhost_svq_add(), it uses vhost_svq_available_slots() to check whether QEMU can add the element into SVQ. If there is enough space, then QEMU combines some out descriptors and some in descriptors into one descriptor chain, and adds it into `svq->vring.desc` by vhost_svq_vring_write_descs(). Yet the problem is that, `svq->shadow_avail_idx - svq->shadow_used_idx` in vhost_svq_available_slots() returns the number of occupied elements, or the number of descriptor chains, instead of the number of occupied descriptors, which may cause wrapping in SVQ descriptor ring. Here is an example. In vhost_handle_guest_kick(), QEMU forwards as many available buffers to device by virtqueue_pop() and vhost_svq_add_element(). virtqueue_pop() returns a guest's element, and then this element is added into SVQ by vhost_svq_add_element(), a wrapper to vhost_svq_add(). If QEMU invokes virtqueue_pop() and vhost_svq_add_element() `svq->vring.num` times, vhost_svq_available_slots() thinks QEMU just ran out of slots and everything should work fine. But in fact, virtqueue_pop() returns `svq->vring.num` elements or descriptor chains, more than `svq->vring.num` descriptors due to guest memory fragmentation, and this causes wrapping in SVQ descriptor ring. This bug is valid even before marking the descriptors used. If the guest memory is fragmented, SVQ must add chains so it can try to add more descriptors than possible. This patch solves it by adding `num_free` field in VhostShadowVirtqueue structure and updating this field in vhost_svq_add() and vhost_svq_get_buf(), to record the number of free descriptors. Fixes: 100890f7ca ("vhost: Shadow virtqueue buffers forwarding") Signed-off-by: Hawkins Jiawei <yin31149@gmail.com> Acked-by: Eugenio Pérez <eperezma@redhat.com> Message-Id: <20230509084817.3973-1-yin31149@gmail.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com> Tested-by: Lei Yang <leiyang@redhat.com>
2023-03-07vdpa: rewind at get_base, not set_baseEugenio Pérez
At this moment it is only possible to migrate to a vdpa device running with x-svq=on. As a protective measure, the rewind of the inflight descriptors was done at the destination. That way if the source sent a virtqueue with inuse descriptors they are always discarded. Since this series allows to migrate also to passthrough devices with no SVQ, the right thing to do is to rewind at the source so the base of vrings are correct. Support for inflight descriptors may be added in the future. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Jason Wang <jasowang@redhat.com> Message-Id: <20230303172445.1089785-5-eperezma@redhat.com> Tested-by: Lei Yang <leiyang@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2023-03-02vhost: avoid a potential use of an uninitialized variable in vhost_svq_poll()Carlos López
In vhost_svq_poll(), if vhost_svq_get_buf() fails due to a device providing invalid descriptors, len is left uninitialized and returned to the caller, potentally leaking stack data or causing undefined behavior. Fix this by initializing len to 0. Found with GCC 13 and -fanalyzer (abridged): ../hw/virtio/vhost-shadow-virtqueue.c: In function ‘vhost_svq_poll’: ../hw/virtio/vhost-shadow-virtqueue.c:538:12: warning: use of uninitialized value ‘len’ [CWE-457] [-Wanalyzer-use-of-uninitialized-value] 538 | return len; | ^~~ ‘vhost_svq_poll’: events 1-4 | | 522 | size_t vhost_svq_poll(VhostShadowVirtqueue *svq) | | ^~~~~~~~~~~~~~ | | | | | (1) entry to ‘vhost_svq_poll’ |...... | 525 | uint32_t len; | | ~~~ | | | | | (2) region created on stack here | | (3) capacity: 4 bytes |...... | 528 | if (vhost_svq_more_used(svq)) { | | ~ | | | | | (4) inlined call to ‘vhost_svq_more_used’ from ‘vhost_svq_poll’ (...) | 528 | if (vhost_svq_more_used(svq)) { | | ^~~~~~~~~~~~~~~~~~~~~~~~~ | | || | | |(8) ...to here | | (7) following ‘true’ branch... |...... | 537 | vhost_svq_get_buf(svq, &len); | | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | | | | | (9) calling ‘vhost_svq_get_buf’ from ‘vhost_svq_poll’ | +--> ‘vhost_svq_get_buf’: events 10-11 | | 416 | static VirtQueueElement *vhost_svq_get_buf(VhostShadowVirtqueue *svq, | | ^~~~~~~~~~~~~~~~~ | | | | | (10) entry to ‘vhost_svq_get_buf’ |...... | 423 | if (!vhost_svq_more_used(svq)) { | | ~ | | | | | (11) inlined call to ‘vhost_svq_more_used’ from ‘vhost_svq_get_buf’ | (...) | ‘vhost_svq_get_buf’: event 14 | | 423 | if (!vhost_svq_more_used(svq)) { | | ^ | | | | | (14) following ‘false’ branch... | ‘vhost_svq_get_buf’: event 15 | |cc1: | (15): ...to here | <------+ | ‘vhost_svq_poll’: events 16-17 | | 537 | vhost_svq_get_buf(svq, &len); | | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~ | | | | | (16) returning to ‘vhost_svq_poll’ from ‘vhost_svq_get_buf’ | 538 | return len; | | ~~~ | | | | | (17) use of uninitialized value ‘len’ here Note by Laurent Vivier <lvivier@redhat.com>: The return value is only used to detect an error: vhost_svq_poll vhost_vdpa_net_cvq_add vhost_vdpa_net_load_cmd vhost_vdpa_net_load_mac -> a negative return is only used to detect error vhost_vdpa_net_load_mq -> a negative return is only used to detect error vhost_vdpa_net_handle_ctrl_avail -> a negative return is only used to detect error Fixes: d368c0b052ad ("vhost: Do not depend on !NULL VirtQueueElement on vhost_svq_flush") Signed-off-by: Carlos López <clopez@suse.de> Message-Id: <20230213085747.19956-1-clopez@suse.de> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-12-21vhost: move iova_tree set to vhost_svq_startEugenio Pérez
Since we don't know if we will use SVQ at qemu initialization, let's allocate iova_tree only if needed. To do so, accept it at SVQ start, not at initialization. This will avoid to create it if the device does not support SVQ. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Jason Wang <jasowang@redhat.com> Message-Id: <20221215113144.322011-5-eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-12-21vhost: allocate SVQ device file descriptors at device startEugenio Pérez
The next patches will start control SVQ if possible. However, we don't know if that will be possible at qemu boot anymore. Delay device file descriptors until we know it at device start. This will avoid to create them if the device does not support SVQ. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Jason Wang <jasowang@redhat.com> Message-Id: <20221215113144.322011-4-eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-12-21vhost: set SVQ device call handler at SVQ startEugenio Pérez
By the end of this series CVQ is shadowed as long as the features support it. Since we don't know at the beginning of qemu running if this is supported, move the event notifier handler setting to the start of the SVQ, instead of the start of qemu run. This will avoid to create them if the device does not support SVQ. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Jason Wang <jasowang@redhat.com> Message-Id: <20221215113144.322011-3-eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-10-28vhost: Accept event idx flagEugenio Pérez
Enabling all the code path created before. Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-10-28vhost: use avail event idx on vhost_svq_kickEugenio Pérez
So SVQ code knows if an event is needed. The code is not reachable at the moment. Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-10-28vhost: toggle device callbacks using used event idxEugenio Pérez
Actually use the new field of the used ring and tell the device if SVQ wants to be notified. The code is not reachable at the moment. Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-10-28vhost: allocate event_idx fields on vringEugenio Pérez
There was not enough room to accomodate them. Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-09-29hw/virtio/vhost-shadow-virtqueue: Silence GCC error "maybe-uninitialized"Bernhard Beschow
GCC issues a false positive warning, resulting in build failure with -Werror: In file included from /usr/include/glib-2.0/glib.h:114, from src/include/glib-compat.h:32, from src/include/qemu/osdep.h:144, from ../src/hw/virtio/vhost-shadow-virtqueue.c:10: In function ‘g_autoptr_cleanup_generic_gfree’, inlined from ‘vhost_handle_guest_kick’ at ../src/hw/virtio/vhost-shadow-virtqueue.c:292:42: /usr/include/glib-2.0/glib/glib-autocleanups.h:28:3: error: ‘elem’ may be used uninitialized [-Werror=maybe-uninitialized] 28 | g_free (*pp); | ^~~~~~~~~~~~ ../src/hw/virtio/vhost-shadow-virtqueue.c: In function ‘vhost_handle_guest_kick’: ../src/hw/virtio/vhost-shadow-virtqueue.c:292:42: note: ‘elem’ was declared here 292 | g_autofree VirtQueueElement *elem; | ^~~~ cc1: all warnings being treated as errors There is actually no problem since "elem" is initialized in both branches. Silence the warning by initializig it with "NULL". $ gcc --version gcc (GCC) 12.2.0 Fixes: 9c2ab2f1ec333be8614cc12272d4b91960704dbe ("vhost: stop transfer elem ownership in vhost_handle_guest_kick") Signed-off-by: Bernhard Beschow <shentey@gmail.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20220910151117.6665-1-shentey@gmail.com> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2022-09-02vhost: Do not depend on !NULL VirtQueueElement on vhost_svq_flushEugenio Pérez
Since QEMU will be able to inject new elements on CVQ to restore the state, we need not to depend on a VirtQueueElement to know if a new element has been used by the device or not. Instead of check that, check if there are new elements only using used idx on vhost_svq_flush. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Jason Wang <jasowang@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-09-02vhost: Delete useless read memory barrierEugenio Pérez
As discussed in previous series [1], this memory barrier is useless with the atomic read of used idx at vhost_svq_more_used. Deleting it. [1] https://lists.nongnu.org/archive/html/qemu-devel/2022-07/msg02616.html Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Jason Wang <jasowang@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-09-02vhost: use SVQ element ndescs instead of opaque data for desc validationEugenio Pérez
Since we're going to allow SVQ to add elements without the guest's knowledge and without its own VirtQueueElement, it's easier to check if an element is a valid head checking a different thing than the VirtQueueElement. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Jason Wang <jasowang@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-09-02vhost: stop transfer elem ownership in vhost_handle_guest_kickEugenio Pérez
It was easier to allow vhost_svq_add to handle the memory. Now that we will allow qemu to add elements to a SVQ without the guest's knowledge, it's better to handle it in the caller. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Jason Wang <jasowang@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-09-02vhost: Always store new kick fd on vhost_svq_set_svq_kick_fdEugenio Pérez
We can unbind twice a file descriptor if we call twice vhost_svq_set_svq_kick_fd because of this. Since it comes from vhost and not from SVQ, that file descriptor could be a different thing that guest's vhost notifier. Likewise, it can happens the same if a guest start and stop the device multiple times. Reported-by: Lei Yang <leiyang@redhat.com> Fixes: dff4426fa6 ("vhost: Add Shadow VirtQueue kick forwarding capabilities") Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Jason Wang <jasowang@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-07-20vhost: Add svq avail_handler callbackEugenio Pérez
This allows external handlers to be aware of new buffers that the guest places in the virtqueue. When this callback is defined the ownership of the guest's virtqueue element is transferred to the callback. This means that if the user wants to forward the descriptor it needs to manually inject it. The callback is also free to process the command by itself and use the element with svq_push. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-07-20vhost: add vhost_svq_pollEugenio Pérez
It allows the Shadow Control VirtQueue to wait for the device to use the available buffers. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-07-20vhost: Expose vhost_svq_addEugenio Pérez
This allows external parts of SVQ to forward custom buffers to the device. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-07-20vhost: add vhost_svq_push_elemEugenio Pérez
This function allows external SVQ users to return guest's available buffers. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-07-20vhost: Track number of descs in SVQDescStateEugenio Pérez
A guest's buffer continuos on GPA may need multiple descriptors on qemu's VA, so SVQ should track its length sepparatedly. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-07-20vhost: Add SVQDescStateEugenio Pérez
This will allow SVQ to add context to the different queue elements. This patch only store the actual element, no functional change intended. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-07-20vhost: Decouple vhost_svq_add from VirtQueueElementEugenio Pérez
VirtQueueElement comes from the guest, but we're heading SVQ to be able to modify the element presented to the device without the guest's knowledge. To do so, make SVQ accept sg buffers directly, instead of using VirtQueueElement. Add vhost_svq_add_element to maintain element convenience. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Jason Wang <jasowang@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-07-20vhost: Check for queue full at vhost_svq_addEugenio Pérez
The series need to expose vhost_svq_add with full functionality, including checking for full queue. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-07-20vhost: Move vhost_svq_kick call to vhost_svq_addEugenio Pérez
The series needs to expose vhost_svq_add with full functionality, including kick Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-07-20vhost: Reorder vhost_svq_kickEugenio Pérez
Future code needs to call it from vhost_svq_add. No functional change intended. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-07-20vdpa: Avoid compiler to squash reads to used idxEugenio Pérez
In the next patch we will allow busypolling of this value. The compiler have a running path where shadow_used_idx, last_used_idx, and vring used idx are not modified within the same thread busypolling. This was not an issue before since we always cleared device event notifier before checking it, and that could act as memory barrier. However, the busypoll needs something similar to kernel READ_ONCE. Let's add it here, sepparated from the polling. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-07-20vhost: move descriptor translation to vhost_svq_vring_write_descsEugenio Pérez
It's done for both in and out descriptors so it's better placed here. Acked-by: Jason Wang <jasowang@redhat.com> Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-05-13vhost: Fix element in vhost_svq_add failureEugenio Pérez
Coverity rightly reports that is not free in that case. Fixes: Coverity CID 1487559 Fixes: 100890f7ca ("vhost: Shadow virtqueue buffers forwarding") Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Message-Id: <20220512175747.142058-7-eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-05-13vhost: Fix device's used descriptor dequeueEugenio Pérez
Only the first one of them were properly enqueued back. Fixes: 100890f7ca ("vhost: Shadow virtqueue buffers forwarding") Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Message-Id: <20220512175747.142058-3-eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-05-13vhost: Track descriptor chain in private at SVQEugenio Pérez
The device could have access to modify them, and it definitely have access when we implement packed vq. Harden SVQ maintaining a private copy of the descriptor chain. Other fields like buffer addresses are already maintained sepparatedly. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Message-Id: <20220512175747.142058-2-eperezma@redhat.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2022-04-06Replace qemu_real_host_page variables with inlined functionsMarc-André Lureau
Replace the global variables with inlined helper functions. getpagesize() is very likely annotated with a "const" function attribute (at least with glibc), and thus optimization should apply even better. This avoids the need for a constructor initialization too. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20220323155743.1585078-12-marcandre.lureau@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-03-15vdpa: Add custom IOTLB translations to SVQEugenio Pérez
Use translations added in VhostIOVATree in SVQ. Only introduce usage here, not allocation and deallocation. As with previous patches, we use the dead code paths of shadow_vqs_enabled to avoid commiting too many changes at once. These are impossible to take at the moment. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-03-15vhost: Shadow virtqueue buffers forwardingEugenio Pérez
Initial version of shadow virtqueue that actually forward buffers. There is no iommu support at the moment, and that will be addressed in future patches of this series. Since all vhost-vdpa devices use forced IOMMU, this means that SVQ is not usable at this point of the series on any device. For simplicity it only supports modern devices, that expects vring in little endian, with split ring and no event idx or indirect descriptors. Support for them will not be added in this series. It reuses the VirtQueue code for the device part. The driver part is based on Linux's virtio_ring driver, but with stripped functionality and optimizations so it's easier to review. However, forwarding buffers have some particular pieces: One of the most unexpected ones is that a guest's buffer can expand through more than one descriptor in SVQ. While this is handled gracefully by qemu's emulated virtio devices, it may cause unexpected SVQ queue full. This patch also solves it by checking for this condition at both guest's kicks and device's calls. The code may be more elegant in the future if SVQ code runs in its own iocontext. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-03-15virtio: Add vhost_svq_get_vring_addrEugenio Pérez
It reports the shadow virtqueue address from qemu virtual address space. Since this will be different from the guest's vaddr, but the device can access it, SVQ takes special care about its alignment & lack of garbage data. It assumes that IOMMU will work in host_page_size ranges for that. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-03-15vhost: Add vhost_svq_valid_features to shadow vqEugenio Pérez
This allows SVQ to negotiate features with the guest and the device. For the device, SVQ is a driver. While this function bypasses all non-transport features, it needs to disable the features that SVQ does not support when forwarding buffers. This includes packed vq layout, indirect descriptors or event idx. Future changes can add support to offer more features to the guest, since the use of VirtQueue gives this for free. This is left out at the moment for simplicity. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-03-15vhost: Add Shadow VirtQueue call forwarding capabilitiesEugenio Pérez
This will make qemu aware of the device used buffers, allowing it to write the guest memory with its contents if needed. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-03-15vhost: Add Shadow VirtQueue kick forwarding capabilitiesEugenio Pérez
At this mode no buffer forwarding will be performed in SVQ mode: Qemu will just forward the guest's kicks to the device. Host memory notifiers regions are left out for simplicity, and they will not be addressed in this series. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
2022-03-15vhost: Add VhostShadowVirtqueueEugenio Pérez
Vhost shadow virtqueue (SVQ) is an intermediate jump for virtqueue notifications and buffers, allowing qemu to track them. While qemu is forwarding the buffers and virtqueue changes, it is able to commit the memory it's being dirtied, the same way regular qemu's VirtIO devices do. This commit only exposes basic SVQ allocation and free. Next patches of the series add functionality like notifications and buffers forwarding. Signed-off-by: Eugenio Pérez <eperezma@redhat.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Jason Wang <jasowang@redhat.com>
generated by cgit v1.2.3 (git 2.26.2) at 2025-03-20 21:24:59 +0000