diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/about/deprecated.rst | 18 | ||||
| -rw-r--r-- | docs/about/removed-features.rst | 13 | ||||
| -rw-r--r-- | docs/conf.py | 4 | ||||
| -rw-r--r-- | docs/devel/fuzzing.rst | 22 | ||||
| -rw-r--r-- | docs/devel/vfio-migration.rst | 72 | ||||
| -rw-r--r-- | docs/meson.build | 1 | ||||
| -rw-r--r-- | docs/system/arm/nuvoton.rst | 2 | ||||
| -rw-r--r-- | docs/tools/index.rst | 1 | ||||
| -rw-r--r-- | docs/tools/virtiofsd.rst | 403 |
9 files changed, 50 insertions, 486 deletions
diff --git a/docs/about/deprecated.rst b/docs/about/deprecated.rst index 2827b0c0be..ee95bcb1a6 100644 --- a/docs/about/deprecated.rst +++ b/docs/about/deprecated.rst @@ -330,24 +330,6 @@ versions, aliases will point to newer CPU model versions depending on the machine type, so management software must resolve CPU model aliases before starting a virtual machine. -Tools ------ - -virtiofsd -''''''''' - -There is a new Rust implementation of ``virtiofsd`` at -``https://gitlab.com/virtio-fs/virtiofsd``; -since this is now marked stable, new development should be done on that -rather than the existing C version in the QEMU tree. -The C version will still accept fixes and patches that -are already in development for the moment, but will eventually -be deleted from this tree. -New deployments should use the Rust version, and existing systems -should consider moving to it. The command line and feature set -is very close and moving should be simple. - - QEMU guest agent ---------------- diff --git a/docs/about/removed-features.rst b/docs/about/removed-features.rst index e901637ce5..5b258b446b 100644 --- a/docs/about/removed-features.rst +++ b/docs/about/removed-features.rst @@ -889,3 +889,16 @@ The VXHS code did not compile since v2.12.0. It was removed in 5.1. The corresponding upstream server project is no longer maintained. Users are recommended to switch to an alternative distributed block device driver such as RBD. + +Tools +----- + +virtiofsd (removed in 8.0) +'''''''''''''''''''''''''' + +There is a newer Rust implementation of ``virtiofsd`` at +``https://gitlab.com/virtio-fs/virtiofsd``; this has been +stable for some time and is now widely used. +The command line and feature set is very close to the removed +C implementation. + diff --git a/docs/conf.py b/docs/conf.py index 73a287a4f2..00767b0e24 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -290,10 +290,6 @@ man_pages = [ ('tools/virtfs-proxy-helper', 'virtfs-proxy-helper', 'QEMU 9p virtfs proxy filesystem helper', ['M. Mohan Kumar'], 1), - ('tools/virtiofsd', 'virtiofsd', - 'QEMU virtio-fs shared file system daemon', - ['Stefan Hajnoczi <stefanha@redhat.com>', - 'Masayoshi Mizuma <m.mizuma@jp.fujitsu.com>'], 1), ] man_make_section_directory = False diff --git a/docs/devel/fuzzing.rst b/docs/devel/fuzzing.rst index 715330c856..3bfcb33fc4 100644 --- a/docs/devel/fuzzing.rst +++ b/docs/devel/fuzzing.rst @@ -19,11 +19,6 @@ responsibility to ensure that state is reset between fuzzing-runs. Building the fuzzers -------------------- -*NOTE*: If possible, build a 32-bit binary. When forking, the 32-bit fuzzer is -much faster, since the page-map has a smaller size. This is due to the fact that -AddressSanitizer maps ~20TB of memory, as part of its detection. This results -in a large page-map, and a much slower ``fork()``. - To build the fuzzers, install a recent version of clang: Configure with (substitute the clang binaries with the version you installed). Here, enable-sanitizers, is optional but it allows us to reliably detect bugs @@ -296,10 +291,9 @@ input. It is also responsible for manually calling ``main_loop_wait`` to ensure that bottom halves are executed and any cleanup required before the next input. Since the same process is reused for many fuzzing runs, QEMU state needs to -be reset at the end of each run. There are currently two implemented -options for resetting state: +be reset at the end of each run. For example, this can be done by rebooting the +VM, after each run. -- Reboot the guest between runs. - *Pros*: Straightforward and fast for simple fuzz targets. - *Cons*: Depending on the device, does not reset all device state. If the @@ -308,15 +302,3 @@ options for resetting state: reboot. - *Example target*: ``i440fx-qtest-reboot-fuzz`` - -- Run each test case in a separate forked process and copy the coverage - information back to the parent. This is fairly similar to AFL's "deferred" - fork-server mode [3] - - - *Pros*: Relatively fast. Devices only need to be initialized once. No need to - do slow reboots or vmloads. - - - *Cons*: Not officially supported by libfuzzer. Does not work well for - devices that rely on dedicated threads. - - - *Example target*: ``virtio-net-fork-fuzz`` diff --git a/docs/devel/vfio-migration.rst b/docs/devel/vfio-migration.rst index 673057c90d..c214c73e28 100644 --- a/docs/devel/vfio-migration.rst +++ b/docs/devel/vfio-migration.rst @@ -7,46 +7,43 @@ the guest is running on source host and restoring this saved state on the destination host. This document details how saving and restoring of VFIO devices is done in QEMU. -Migration of VFIO devices consists of two phases: the optional pre-copy phase, -and the stop-and-copy phase. The pre-copy phase is iterative and allows to -accommodate VFIO devices that have a large amount of data that needs to be -transferred. The iterative pre-copy phase of migration allows for the guest to -continue whilst the VFIO device state is transferred to the destination, this -helps to reduce the total downtime of the VM. VFIO devices can choose to skip -the pre-copy phase of migration by returning pending_bytes as zero during the -pre-copy phase. +Migration of VFIO devices currently consists of a single stop-and-copy phase. +During the stop-and-copy phase the guest is stopped and the entire VFIO device +data is transferred to the destination. + +The pre-copy phase of migration is currently not supported for VFIO devices. +Support for VFIO pre-copy will be added later on. + +Note that currently VFIO migration is supported only for a single device. This +is due to VFIO migration's lack of P2P support. However, P2P support is planned +to be added later on. A detailed description of the UAPI for VFIO device migration can be found in -the comment for the ``vfio_device_migration_info`` structure in the header -file linux-headers/linux/vfio.h. +the comment for the ``vfio_device_mig_state`` structure in the header file +linux-headers/linux/vfio.h. VFIO implements the device hooks for the iterative approach as follows: -* A ``save_setup`` function that sets up the migration region and sets _SAVING - flag in the VFIO device state. +* A ``save_setup`` function that sets up migration on the source. -* A ``load_setup`` function that sets up the migration region on the - destination and sets _RESUMING flag in the VFIO device state. +* A ``load_setup`` function that sets the VFIO device on the destination in + _RESUMING state. * A ``state_pending_exact`` function that reads pending_bytes from the vendor driver, which indicates the amount of data that the vendor driver has yet to save for the VFIO device. -* A ``save_live_iterate`` function that reads the VFIO device's data from the - vendor driver through the migration region during iterative phase. - * A ``save_state`` function to save the device config space if it is present. -* A ``save_live_complete_precopy`` function that resets _RUNNING flag from the - VFIO device state and iteratively copies the remaining data for the VFIO - device until the vendor driver indicates that no data remains (pending bytes - is zero). +* A ``save_live_complete_precopy`` function that sets the VFIO device in + _STOP_COPY state and iteratively copies the data for the VFIO device until + the vendor driver indicates that no data remains. * A ``load_state`` function that loads the config section and the data - sections that are generated by the save functions above + sections that are generated by the save functions above. * ``cleanup`` functions for both save and load that perform any migration - related cleanup, including unmapping the migration region + related cleanup. The VFIO migration code uses a VM state change handler to change the VFIO @@ -71,13 +68,13 @@ tracking can identify dirtied pages, but any page pinned by the vendor driver can also be written by the device. There is currently no device or IOMMU support for dirty page tracking in hardware. -By default, dirty pages are tracked when the device is in pre-copy as well as -stop-and-copy phase. So, a page pinned by the vendor driver will be copied to -the destination in both phases. Copying dirty pages in pre-copy phase helps -QEMU to predict if it can achieve its downtime tolerances. If QEMU during -pre-copy phase keeps finding dirty pages continuously, then it understands -that even in stop-and-copy phase, it is likely to find dirty pages and can -predict the downtime accordingly. +By default, dirty pages are tracked during pre-copy as well as stop-and-copy +phase. So, a page pinned by the vendor driver will be copied to the destination +in both phases. Copying dirty pages in pre-copy phase helps QEMU to predict if +it can achieve its downtime tolerances. If QEMU during pre-copy phase keeps +finding dirty pages continuously, then it understands that even in stop-and-copy +phase, it is likely to find dirty pages and can predict the downtime +accordingly. QEMU also provides a per device opt-out option ``pre-copy-dirty-page-tracking`` which disables querying the dirty bitmap during pre-copy phase. If it is set to @@ -111,23 +108,22 @@ Live migration save path | migrate_init spawns migration_thread Migration thread then calls each device's .save_setup() - (RUNNING, _SETUP, _RUNNING|_SAVING) + (RUNNING, _SETUP, _RUNNING) | - (RUNNING, _ACTIVE, _RUNNING|_SAVING) + (RUNNING, _ACTIVE, _RUNNING) If device is active, get pending_bytes by .state_pending_exact() If total pending_bytes >= threshold_size, call .save_live_iterate() - Data of VFIO device for pre-copy phase is copied Iterate till total pending bytes converge and are less than threshold | On migration completion, vCPU stops and calls .save_live_complete_precopy for - each active device. The VFIO device is then transitioned into _SAVING state - (FINISH_MIGRATE, _DEVICE, _SAVING) + each active device. The VFIO device is then transitioned into _STOP_COPY state + (FINISH_MIGRATE, _DEVICE, _STOP_COPY) | For the VFIO device, iterate in .save_live_complete_precopy until pending data is 0 - (FINISH_MIGRATE, _DEVICE, _STOPPED) + (FINISH_MIGRATE, _DEVICE, _STOP) | - (FINISH_MIGRATE, _COMPLETED, _STOPPED) + (FINISH_MIGRATE, _COMPLETED, _STOP) Migraton thread schedules cleanup bottom half and exits Live migration resume path @@ -136,7 +132,7 @@ Live migration resume path :: Incoming migration calls .load_setup for each device - (RESTORE_VM, _ACTIVE, _STOPPED) + (RESTORE_VM, _ACTIVE, _STOP) | For each device, .load_state is called for that device section data (RESTORE_VM, _ACTIVE, _RESUMING) diff --git a/docs/meson.build b/docs/meson.build index 9136fed3b7..bbcdccce68 100644 --- a/docs/meson.build +++ b/docs/meson.build @@ -48,7 +48,6 @@ if build_docs 'qemu-storage-daemon.1': (have_tools ? 'man1' : ''), 'qemu-trace-stap.1': (stap.found() ? 'man1' : ''), 'virtfs-proxy-helper.1': (have_virtfs_proxy_helper ? 'man1' : ''), - 'virtiofsd.1': (have_virtiofsd ? 'man1' : ''), 'qemu.1': 'man1', 'qemu-block-drivers.7': 'man7', 'qemu-cpu-models.7': 'man7' diff --git a/docs/system/arm/nuvoton.rst b/docs/system/arm/nuvoton.rst index c38df32bde..0424cae4b0 100644 --- a/docs/system/arm/nuvoton.rst +++ b/docs/system/arm/nuvoton.rst @@ -49,6 +49,7 @@ Supported devices * SMBus controller (SMBF) * Ethernet controller (EMC) * Tachometer + * Peripheral SPI controller (PSPI) Missing devices --------------- @@ -64,7 +65,6 @@ Missing devices * Ethernet controller (GMAC) * USB device (USBD) - * Peripheral SPI controller (PSPI) * SD/MMC host * PECI interface * PCI and PCIe root complex and bridges diff --git a/docs/tools/index.rst b/docs/tools/index.rst index 2151adcf78..8e65ce0dfc 100644 --- a/docs/tools/index.rst +++ b/docs/tools/index.rst @@ -16,4 +16,3 @@ command line utilities and other standalone programs. qemu-pr-helper qemu-trace-stap virtfs-proxy-helper - virtiofsd diff --git a/docs/tools/virtiofsd.rst b/docs/tools/virtiofsd.rst deleted file mode 100644 index 995a754a7b..0000000000 --- a/docs/tools/virtiofsd.rst +++ /dev/null @@ -1,403 +0,0 @@ -QEMU virtio-fs shared file system daemon -======================================== - -Synopsis --------- - -**virtiofsd** [*OPTIONS*] - -Description ------------ - -Share a host directory tree with a guest through a virtio-fs device. This -program is a vhost-user backend that implements the virtio-fs device. Each -virtio-fs device instance requires its own virtiofsd process. - -This program is designed to work with QEMU's ``--device vhost-user-fs-pci`` -but should work with any virtual machine monitor (VMM) that supports -vhost-user. See the Examples section below. - -This program must be run as the root user. The program drops privileges where -possible during startup although it must be able to create and access files -with any uid/gid: - -* The ability to invoke syscalls is limited using seccomp(2). -* Linux capabilities(7) are dropped. - -In "namespace" sandbox mode the program switches into a new file system -namespace and invokes pivot_root(2) to make the shared directory tree its root. -A new pid and net namespace is also created to isolate the process. - -In "chroot" sandbox mode the program invokes chroot(2) to make the shared -directory tree its root. This mode is intended for container environments where -the container runtime has already set up the namespaces and the program does -not have permission to create namespaces itself. - -Both sandbox modes prevent "file system escapes" due to symlinks and other file -system objects that might lead to files outside the shared directory. - -Options -------- - -.. program:: virtiofsd - -.. option:: -h, --help - - Print help. - -.. option:: -V, --version - - Print version. - -.. option:: -d - - Enable debug output. - -.. option:: --syslog - - Print log messages to syslog instead of stderr. - -.. option:: -o OPTION - - * debug - - Enable debug output. - - * flock|no_flock - - Enable/disable flock. The default is ``no_flock``. - - * modcaps=CAPLIST - Modify the list of capabilities allowed; CAPLIST is a colon separated - list of capabilities, each preceded by either + or -, e.g. - ''+sys_admin:-chown''. - - * log_level=LEVEL - - Print only log messages matching LEVEL or more severe. LEVEL is one of - ``err``, ``warn``, ``info``, or ``debug``. The default is ``info``. - - * posix_lock|no_posix_lock - - Enable/disable remote POSIX locks. The default is ``no_posix_lock``. - - * readdirplus|no_readdirplus - - Enable/disable readdirplus. The default is ``readdirplus``. - - * sandbox=namespace|chroot - - Sandbox mode: - - namespace: Create mount, pid, and net namespaces and pivot_root(2) into - the shared directory. - - chroot: chroot(2) into shared directory (use in containers). - The default is "namespace". - - * source=PATH - - Share host directory tree located at PATH. This option is required. - - * timeout=TIMEOUT - - I/O timeout in seconds. The default depends on cache= option. - - * writeback|no_writeback - - Enable/disable writeback cache. The cache allows the FUSE client to buffer - and merge write requests. The default is ``no_writeback``. - - * xattr|no_xattr - - Enable/disable extended attributes (xattr) on files and directories. The - default is ``no_xattr``. - - * posix_acl|no_posix_acl - - Enable/disable posix acl support. Posix ACLs are disabled by default. - - * security_label|no_security_label - - Enable/disable security label support. Security labels are disabled by - default. This will allow client to send a MAC label of file during - file creation. Typically this is expected to be SELinux security - label. Server will try to set that label on newly created file - atomically wherever possible. - - * killpriv_v2|no_killpriv_v2 - - Enable/disable ``FUSE_HANDLE_KILLPRIV_V2`` support. KILLPRIV_V2 is enabled - by default as long as the client supports it. Enabling this option helps - with performance in write path. - -.. option:: --socket-path=PATH - - Listen on vhost-user UNIX domain socket at PATH. - -.. option:: --socket-group=GROUP - - Set the vhost-user UNIX domain socket gid to GROUP. - -.. option:: --fd=FDNUM - - Accept connections from vhost-user UNIX domain socket file descriptor FDNUM. - The file descriptor must already be listening for connections. - -.. option:: --thread-pool-size=NUM - - Restrict the number of worker threads per request queue to NUM. The default - is 0. - -.. option:: --cache=none|auto|always - - Select the desired trade-off between coherency and performance. ``none`` - forbids the FUSE client from caching to achieve best coherency at the cost of - performance. ``auto`` acts similar to NFS with a 1 second metadata cache - timeout. ``always`` sets a long cache lifetime at the expense of coherency. - The default is ``auto``. - -Extended attribute (xattr) mapping ----------------------------------- - -By default the name of xattr's used by the client are passed through to the server -file system. This can be a problem where either those xattr names are used -by something on the server (e.g. selinux client/server confusion) or if the -``virtiofsd`` is running in a container with restricted privileges where it -cannot access some attributes. - -Mapping syntax -~~~~~~~~~~~~~~ - -A mapping of xattr names can be made using -o xattrmap=mapping where the ``mapping`` -string consists of a series of rules. - -The first matching rule terminates the mapping. -The set of rules must include a terminating rule to match any remaining attributes -at the end. - -Each rule consists of a number of fields separated with a separator that is the -first non-white space character in the rule. This separator must then be used -for the whole rule. -White space may be added before and after each rule. - -Using ':' as the separator a rule is of the form: - -``:type:scope:key:prepend:`` - -**scope** is: - -- 'client' - match 'key' against a xattr name from the client for - setxattr/getxattr/removexattr -- 'server' - match 'prepend' against a xattr name from the server - for listxattr -- 'all' - can be used to make a single rule where both the server - and client matches are triggered. - -**type** is one of: - -- 'prefix' - is designed to prepend and strip a prefix; the modified - attributes then being passed on to the client/server. - -- 'ok' - Causes the rule set to be terminated when a match is found - while allowing matching xattr's through unchanged. - It is intended both as a way of explicitly terminating - the list of rules, and to allow some xattr's to skip following rules. - -- 'bad' - If a client tries to use a name matching 'key' it's - denied using EPERM; when the server passes an attribute - name matching 'prepend' it's hidden. In many ways it's use is very like - 'ok' as either an explicit terminator or for special handling of certain - patterns. - -- 'unsupported' - If a client tries to use a name matching 'key' it's - denied using ENOTSUP; when the server passes an attribute - name matching 'prepend' it's hidden. In many ways it's use is very like - 'ok' as either an explicit terminator or for special handling of certain - patterns. - -**key** is a string tested as a prefix on an attribute name originating -on the client. It maybe empty in which case a 'client' rule -will always match on client names. - -**prepend** is a string tested as a prefix on an attribute name originating -on the server, and used as a new prefix. It may be empty -in which case a 'server' rule will always match on all names from -the server. - -e.g.: - - ``:prefix:client:trusted.:user.virtiofs.:`` - - will match 'trusted.' attributes in client calls and prefix them before - passing them to the server. - - ``:prefix:server::user.virtiofs.:`` - - will strip 'user.virtiofs.' from all server replies. - - ``:prefix:all:trusted.:user.virtiofs.:`` - - combines the previous two cases into a single rule. - - ``:ok:client:user.::`` - - will allow get/set xattr for 'user.' xattr's and ignore - following rules. - - ``:ok:server::security.:`` - - will pass 'security.' xattr's in listxattr from the server - and ignore following rules. - - ``:ok:all:::`` - - will terminate the rule search passing any remaining attributes - in both directions. - - ``:bad:server::security.:`` - - would hide 'security.' xattr's in listxattr from the server. - -A simpler 'map' type provides a shorter syntax for the common case: - -``:map:key:prepend:`` - -The 'map' type adds a number of separate rules to add **prepend** as a prefix -to the matched **key** (or all attributes if **key** is empty). -There may be at most one 'map' rule and it must be the last rule in the set. - -Note: When the 'security.capability' xattr is remapped, the daemon has to do -extra work to remove it during many operations, which the host kernel normally -does itself. - -Security considerations -~~~~~~~~~~~~~~~~~~~~~~~ - -Operating systems typically partition the xattr namespace using -well defined name prefixes. Each partition may have different -access controls applied. For example, on Linux there are multiple -partitions - - * ``system.*`` - access varies depending on attribute & filesystem - * ``security.*`` - only processes with CAP_SYS_ADMIN - * ``trusted.*`` - only processes with CAP_SYS_ADMIN - * ``user.*`` - any process granted by file permissions / ownership - -While other OS such as FreeBSD have different name prefixes -and access control rules. - -When remapping attributes on the host, it is important to -ensure that the remapping does not allow a guest user to -evade the guest access control rules. - -Consider if ``trusted.*`` from the guest was remapped to -``user.virtiofs.trusted*`` in the host. An unprivileged -user in a Linux guest has the ability to write to xattrs -under ``user.*``. Thus the user can evade the access -control restriction on ``trusted.*`` by instead writing -to ``user.virtiofs.trusted.*``. - -As noted above, the partitions used and access controls -applied, will vary across guest OS, so it is not wise to -try to predict what the guest OS will use. - -The simplest way to avoid an insecure configuration is -to remap all xattrs at once, to a given fixed prefix. -This is shown in example (1) below. - -If selectively mapping only a subset of xattr prefixes, -then rules must be added to explicitly block direct -access to the target of the remapping. This is shown -in example (2) below. - -Mapping examples -~~~~~~~~~~~~~~~~ - -1) Prefix all attributes with 'user.virtiofs.' - -:: - - -o xattrmap=":prefix:all::user.virtiofs.::bad:all:::" - - -This uses two rules, using : as the field separator; -the first rule prefixes and strips 'user.virtiofs.', -the second rule hides any non-prefixed attributes that -the host set. - -This is equivalent to the 'map' rule: - -:: - - -o xattrmap=":map::user.virtiofs.:" - -2) Prefix 'trusted.' attributes, allow others through - -:: - - "/prefix/all/trusted./user.virtiofs./ - /bad/server//trusted./ - /bad/client/user.virtiofs.// - /ok/all///" - - -Here there are four rules, using / as the field -separator, and also demonstrating that new lines can -be included between rules. -The first rule is the prefixing of 'trusted.' and -stripping of 'user.virtiofs.'. -The second rule hides unprefixed 'trusted.' attributes -on the host. -The third rule stops a guest from explicitly setting -the 'user.virtiofs.' path directly to prevent access -control bypass on the target of the earlier prefix -remapping. -Finally, the fourth rule lets all remaining attributes -through. - -This is equivalent to the 'map' rule: - -:: - - -o xattrmap="/map/trusted./user.virtiofs./" - -3) Hide 'security.' attributes, and allow everything else - -:: - - "/bad/all/security./security./ - /ok/all///' - -The first rule combines what could be separate client and server -rules into a single 'all' rule, matching 'security.' in either -client arguments or lists returned from the host. This stops -the client seeing any 'security.' attributes on the server and -stops it setting any. - -SELinux support ---------------- -One can enable support for SELinux by running virtiofsd with option -"-o security_label". But this will try to save guest's security context -in xattr security.selinux on host and it might fail if host's SELinux -policy does not permit virtiofsd to do this operation. - -Hence, it is preferred to remap guest's "security.selinux" xattr to say -"trusted.virtiofs.security.selinux" on host. - -"-o xattrmap=:map:security.selinux:trusted.virtiofs.:" - -This will make sure that guest and host's SELinux xattrs on same file -remain separate and not interfere with each other. And will allow both -host and guest to implement their own separate SELinux policies. - -Setting trusted xattr on host requires CAP_SYS_ADMIN. So one will need -add this capability to daemon. - -"-o modcaps=+sys_admin" - -Giving CAP_SYS_ADMIN increases the risk on system. Now virtiofsd is more -powerful and if gets compromised, it can do lot of damage to host system. -So keep this trade-off in my mind while making a decision. - -Examples --------- - -Export ``/var/lib/fs/vm001/`` on vhost-user UNIX domain socket -``/var/run/vm001-vhost-fs.sock``: - -.. parsed-literal:: - - host# virtiofsd --socket-path=/var/run/vm001-vhost-fs.sock -o source=/var/lib/fs/vm001 - host# |qemu_system| \\ - -chardev socket,id=char0,path=/var/run/vm001-vhost-fs.sock \\ - -device vhost-user-fs-pci,chardev=char0,tag=myfs \\ - -object memory-backend-memfd,id=mem,size=4G,share=on \\ - -numa node,memdev=mem \\ - ... - guest# mount -t virtiofs myfs /mnt |