aboutsummaryrefslogtreecommitdiff
path: root/util
AgeCommit message (Collapse)Author
2020-02-25mem-prealloc: optimize large guest startupbauerchen
[desc]: Large memory VM starts slowly when using -mem-prealloc, and there are some areas to optimize in current method; 1、mmap will be used to alloc threads stack during create page clearing threads, and it will attempt mm->mmap_sem for write lock, but clearing threads have hold read lock, this competition will cause threads createion very slow; 2、methods of calcuating pages for per threads is not well;if we use 64 threads to split 160 hugepage,63 threads clear 2page,1 thread clear 34 page,so the entire speed is very slow; to solve the first problem,we add a mutex in thread function,and start all threads when all threads finished createion; and the second problem, we spread remainder to other threads,in situation that 160 hugepage and 64 threads, there are 32 threads clear 3 pages,and 32 threads clear 2 pages. [test]: 320G 84c VM start time can be reduced to 10s 680G 84c VM start time can be reduced to 18s Signed-off-by: bauerchen <bauerchen@tencent.com> Reviewed-by: Pan Rui <ruippan@tencent.com> Reviewed-by: Ivan Ren <ivanren@tencent.com> [Simplify computation of the number of pages per thread. - Paolo] Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-22module: check module wasn't already initializedAlexander Bulekov
The virtual-device fuzzer must initialize QOM, prior to running vl:qemu_init, so that it can use the qos_graph to identify the arguments required to initialize a guest for libqos-assisted fuzzing. This change prevents errors when vl:qemu_init tries to (re)initialize the previously initialized QOM module. Signed-off-by: Alexander Bulekov <alxndr@bu.edu> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Darren Kenny <darren.kenny@oracle.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20200220041118.23264-4-alxndr@bu.edu Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22aio-posix: make AioHandler dispatch O(1) with epollStefan Hajnoczi
File descriptor monitoring is O(1) with epoll(7), but aio_dispatch_handlers() still scans all AioHandlers instead of dispatching just those that are ready. This makes aio_poll() O(n) with respect to the total number of registered handlers. Add a local ready_list to aio_poll() so that each nested aio_poll() builds a list of handlers ready to be dispatched. Since file descriptor polling is level-triggered, nested aio_poll() calls also see fds that were ready in the parent but not yet dispatched. This guarantees that nested aio_poll() invocations will dispatch all fds, even those that became ready before the nested invocation. Since only handlers ready to be dispatched are placed onto the ready_list, the new aio_dispatch_ready_handlers() function provides O(1) dispatch. Note that AioContext polling is still O(n) and currently cannot be fully disabled. This still needs to be fixed before aio_poll() is fully O(1). Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Sergio Lopez <slp@redhat.com> Message-id: 20200214171712.541358-6-stefanha@redhat.com [Fix compilation error on macOS where there is no epoll(87). The aio_epoll() prototype was out of date and aio_add_ready_list() needed to be moved outside the ifdef. --Stefan] Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22aio-posix: make AioHandler deletion O(1)Stefan Hajnoczi
It is not necessary to scan all AioHandlers for deletion. Keep a list of deleted handlers instead of scanning the full list of all handlers. The AioHandler->deleted field can be dropped. Let's check if the handler has been inserted into the deleted list instead. Add a new QLIST_IS_INSERTED() API for this check. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Sergio Lopez <slp@redhat.com> Message-id: 20200214171712.541358-5-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22aio-posix: don't pass ns timeout to epoll_wait()Stefan Hajnoczi
Don't pass the nanosecond timeout into epoll_wait(), which expects milliseconds. The epoll_wait() timeout value does not matter if qemu_poll_ns() determined that the poll fd is ready, but passing a value in the wrong units is still ugly. Pass a 0 timeout to epoll_wait() instead. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Sergio Lopez <slp@redhat.com> Message-id: 20200214171712.541358-3-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22aio-posix: fix use after leaving scope in aio_poll()Stefan Hajnoczi
epoll_handler is a stack variable and must not be accessed after it goes out of scope: if (aio_epoll_check_poll(ctx, pollfds, npfd, timeout)) { AioHandler epoll_handler; ... add_pollfd(&epoll_handler); ret = aio_epoll(ctx, pollfds, npfd, timeout); } ... ... /* if we have any readable fds, dispatch event */ if (ret > 0) { for (i = 0; i < npfd; i++) { nodes[i]->pfd.revents = pollfds[i].revents; } } nodes[0] is &epoll_handler, which has already gone out of scope. There is no need to use pollfds[] for epoll. We don't need an AioHandler for the epoll fd. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Sergio Lopez <slp@redhat.com> Message-id: 20200214171712.541358-2-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22util/async: make bh_aio_poll() O(1)Stefan Hajnoczi
The ctx->first_bh list contains all created BHs, including those that are not scheduled. The list is iterated by the event loop and therefore has O(n) time complexity with respected to the number of created BHs. Rewrite BHs so that only scheduled or deleted BHs are enqueued. Only BHs that actually require action will be iterated. One semantic change is required: qemu_bh_delete() enqueues the BH and therefore invokes aio_notify(). The tests/test-aio.c:test_source_bh_delete_from_cb() test case assumed that g_main_context_iteration(NULL, false) returns false after qemu_bh_delete() but it now returns true for one iteration. Fix up the test case. This patch makes aio_compute_timeout() and aio_bh_poll() drop from a CPU profile reported by perf-top(1). Previously they combined to 9% CPU utilization when AioContext polling is commented out and the guest has 2 virtio-blk,num-queues=1 and 99 virtio-blk,num-queues=32 devices. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200221093951.1414693-1-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22aio-posix: avoid reacquiring rcu_read_lock() when pollingStefan Hajnoczi
The first rcu_read_lock/unlock() is expensive. Nested calls are cheap. This optimization increases IOPS from 73k to 162k with a Linux guest that has 2 virtio-blk,num-queues=1 and 99 virtio-blk,num-queues=32 devices. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Message-id: 20200218182708.914552-1-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-21Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-5.0-20200221' ↵Peter Maydell
into staging ppc patch queue 2020-02-21 Here's the next patch of ppc target patches. Highlights are: * Some fixes for CAS / unplug interactions * Remove some leaks of device trees * Some fixes for the PHB3 and PHB4 devices * Support for NVDIMMs on the pseries machine type * Assorted other fixes and cleanups # gpg: Signature made Fri 21 Feb 2020 03:35:40 GMT # gpg: using RSA key 75F46586AE61A66CC44E87DC6C38CACA20D9B392 # gpg: Good signature from "David Gibson <david@gibson.dropbear.id.au>" [full] # gpg: aka "David Gibson (Red Hat) <dgibson@redhat.com>" [full] # gpg: aka "David Gibson (ozlabs.org) <dgibson@ozlabs.org>" [full] # gpg: aka "David Gibson (kernel.org) <dwg@kernel.org>" [unknown] # Primary key fingerprint: 75F4 6586 AE61 A66C C44E 87DC 6C38 CACA 20D9 B392 * remotes/dgibson/tags/ppc-for-5.0-20200221: hw/ppc/virtex_ml507:fix leak of fdevice tree blob spapr: Fix handling of unplugged devices during CAS and migration spapr: Don't use spapr_drc_needed() in CAS code ppc: free 'fdt' after reset the machine target/ppc/cpu.h: Clean up comments in the struct CPUPPCState definition target/ppc/cpu.h: Move fpu related members closer in cpu env target/ppc: Fix typo in comments spapr: Allow changing offset for -kernel image pnv/phb3: Add missing break statement pnv/phb4: Fix error path in pnv_pec_realize() pnv/phb3: Convert 1u to 1ull target/ppc/cpu.h: Remove duplicate includes spapr: Add Hcalls to support PAPR NVDIMM device spapr: Add NVDIMM device support nvdimm: add uuid property to nvdimm mem: move nvdimm_device_list to utilities ppc: function to setup latest class options ppc/pnv: Fix PCI_EXPRESS dependency qtest: Fix rtas dependencies spapr/rtas: Print message from "ibm,os-term" Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-21mem: move nvdimm_device_list to utilitiesShivaprasad G Bhat
nvdimm_device_list is required for parsing the list for devices in subsequent patches. Move it to common utility area. Signed-off-by: Shivaprasad G Bhat <sbhat@linux.ibm.com> Reviewed-by: Igor Mammedov <imammedo@redhat.com> Reviewed-by: David Gibson <david@gibson.dropbear.id.au> Message-Id: <158131055857.2897.15658377276504711773.stgit@lep8c.aus.stglabs.ibm.com> Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2020-02-20Merge remote-tracking branch ↵Peter Maydell
'remotes/vivier2/tags/linux-user-for-5.0-pull-request' into staging Implement membarrier, SO_RCVTIMEO and SO_SNDTIMEO Disable by default build of fdt, slirp and tools with linux-user Improve strace and use qemu_log to send trace to a file Add partial ALSA ioctl supports # gpg: Signature made Thu 20 Feb 2020 09:20:20 GMT # gpg: using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C # gpg: issuer "laurent@vivier.eu" # gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full] # gpg: aka "Laurent Vivier <laurent@vivier.eu>" [full] # gpg: aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full] # Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F 5173 F30C 38BD 3F2F BE3C * remotes/vivier2/tags/linux-user-for-5.0-pull-request: linux-user: Add support for selected alsa timer instructions using ioctls linux-user: Add support for getting/setting selected alsa timer parameters using ioctls linux-user: Add support for selecting alsa timer using ioctl linux-user: Add support for getting/setting specified alsa timer parameters using ioctls linux-user: Add support for getting alsa timer version and id linux-user: remove gemu_log from the linux-user tree linux-user: Use `qemu_log' for strace linux-user: Use `qemu_log' for non-strace logging configure: Avoid compiling system tools on user build by default linux-user/strace: Improve output of various syscalls configure: linux-user doesn't need neither fdt nor slirp linux-user: implement getsockopt SO_RCVTIMEO and SO_SNDTIMEO linux-user: Implement membarrier syscall Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-19linux-user: Use `qemu_log' for straceJosh Kunz
This change switches linux-user strace logging to use the newer `qemu_log` logging subsystem rather than the older `gemu_log` (notice the "g") logger. `qemu_log` has several advantages, namely that it allows logging to a file, and provides a more unified interface for configuration of logging (via the QEMU_LOG environment variable or options). This change introduces a new log mask: `LOG_STRACE` which is used for logging of user-mode strace messages. Reviewed-by: Laurent Vivier <laurent@vivier.eu> Signed-off-by: Josh Kunz <jkz@google.com> Message-Id: <20200204025416.111409-3-jkz@google.com> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2020-02-18Report stringified errno in VFIO related errorsMichal Privoznik
In a few places we report errno formatted as a negative integer. This is not as user friendly as it can be. Use strerror() and/or error_setg_errno() instead. Signed-off-by: Michal Privoznik <mprivozn@redhat.com> Reviewed-by: Ján Tomko <jtomko@redhat.com> Reviewed-by: Cornelia Huck <cohuck@redhat.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Alex Williamson <alex.williamson@redhat.com> Message-Id: <4949c3ecf1a32189b8a4b5eb4b0fd04c1122501d.1581674006.git.mprivozn@redhat.com> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2020-02-12Remove support for CLOCK_MONOTONIC not being definedPeter Maydell
Some older parts of QEMU's codebase assume that CLOCK_MONOTONIC might not be defined by the host OS, and have workarounds to deal with this. However, more recently (notably in commit 50290c002c045280f8d for qemu-img in mid-2019, but also much earlier in 2011 in commit 22795174a37e0 for ui/spice-display.c) we've written code that assumes CLOCK_MONOTONIC is always defined. The only host OS anybody's ever noticed this on is OSX 10.11 and earlier, which we don't support. So we can assume that all our host OSes have the #define, and we can remove some now-unnecessary ifdefs. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Message-Id: <20200201172252.6605-1-peter.maydell@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-01-31Merge remote-tracking branch 'remotes/stefanha/tags/tracing-pull-request' ↵Peter Maydell
into staging Pull request # gpg: Signature made Thu 30 Jan 2020 21:38:06 GMT # gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8 # gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full] # gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full] # Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8 * remotes/stefanha/tags/tracing-pull-request: qemu_set_log_filename: filename argument may be NULL hw/display/qxl.c: Use trace_event_get_state_backends() memory.c: Use trace_event_get_state_backends() docs/devel/tracing.txt: Recommend only trace_event_get_state_backends() Makefile: Keep trace-events-subdirs ordered Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-01-30qemu_set_log_filename: filename argument may be NULLSalvador Fandino
NULL is a valid log filename used to indicate we want to use stderr but qemu_set_log_filename (which is called by bsd-user/main.c) was not handling it correctly. That also made redundant a couple of NULL checks in calling code which have been removed. Signed-off-by: Salvador Fandino <salvador@qindel.com> Message-Id: <20200123193626.19956-1-salvador@qindel.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-01-30util/async: add aio interfaces for io_uringAarushi Mehta
Signed-off-by: Aarushi Mehta <mehta.aaru20@gmail.com> Acked-by: Stefano Garzarella <sgarzare@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20200120141858.587874-7-stefanha@redhat.com Message-Id: <20200120141858.587874-7-stefanha@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-01-21util/cacheinfo: fix crash when compiling with uClibcCarlos Santos
uClibc defines _SC_LEVEL1_ICACHE_LINESIZE and _SC_LEVEL1_DCACHE_LINESIZE but the corresponding sysconf calls returns -1, which is a valid result, meaning that the limit is indeterminate. Handle this situation using the fallback values instead of crashing due to an assertion failure. Signed-off-by: Carlos Santos <casantos@redhat.com> Message-Id: <20191017123713.30192-1-casantos@redhat.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2020-01-13Merge remote-tracking branch ↵Peter Maydell
'remotes/vivier2/tags/trivial-branch-pull-request' into staging Fix some uninitialized variable warnings, some memory leak warnings and update MAINTAINERS file. # gpg: Signature made Wed 08 Jan 2020 16:02:11 GMT # gpg: using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C # gpg: issuer "laurent@vivier.eu" # gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full] # gpg: aka "Laurent Vivier <laurent@vivier.eu>" [full] # gpg: aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full] # Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F 5173 F30C 38BD 3F2F BE3C * remotes/vivier2/tags/trivial-branch-pull-request: vl: fix memory leak in configure_accelerators arm/translate-a64: fix uninitialized variable warning nbd: fix uninitialized variable warning util/module: fix a memory leak MAINTAINERS: Update Yuval Shaia's email address Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-01-10Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into stagingPeter Maydell
* Compat machines fix (Denis) * Command line parsing fixes (Michal, Peter, Xiaoyao) * Cooperlake CPU model fixes (Xiaoyao) * i386 gdb fix (mkdolata) * IOEventHandler cleanup (Philippe) * icount fix (Pavel) * RR support for random number sources (Pavel) * Kconfig fixes (Philippe) # gpg: Signature made Wed 08 Jan 2020 10:41:00 GMT # gpg: using RSA key BFFBD25F78C7AE83 # gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full] # gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" [full] # Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1 # Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83 * remotes/bonzini/tags/for-upstream: (38 commits) chardev: Use QEMUChrEvent enum in IOEventHandler typedef chardev: use QEMUChrEvent instead of int chardev/char: Explicit we ignore some QEMUChrEvent in IOEventHandler monitor/hmp: Explicit we ignore a QEMUChrEvent in IOEventHandler monitor/qmp: Explicit we ignore few QEMUChrEvent in IOEventHandler virtio-console: Explicit we ignore some QEMUChrEvent in IOEventHandler vhost-user-blk: Explicit we ignore few QEMUChrEvent in IOEventHandler vhost-user-net: Explicit we ignore few QEMUChrEvent in IOEventHandler vhost-user-crypto: Explicit we ignore some QEMUChrEvent in IOEventHandler ccid-card-passthru: Explicit we ignore QEMUChrEvent in IOEventHandler hw/usb/redirect: Explicit we ignore few QEMUChrEvent in IOEventHandler hw/usb/dev-serial: Explicit we ignore few QEMUChrEvent in IOEventHandler hw/char/terminal3270: Explicit ignored QEMUChrEvent in IOEventHandler hw/ipmi: Explicit we ignore some QEMUChrEvent in IOEventHandler hw/ipmi: Remove unnecessary declarations target/i386: Add missed features to Cooperlake CPU model target/i386: Add new bit definitions of MSR_IA32_ARCH_CAPABILITIES target/i386: Fix handling of k_gs_base register in 32-bit mode in gdbstub hw/rtc/mc146818: Add missing dependency on ISA Bus hw/nvram/Kconfig: Restrict CHRP NVRAM to machines using OpenBIOS or SLOF ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-01-08util/module: fix a memory leakPan Nengyuan
spotted by ASAN Fixes: 81d8ccb1bea4fb9eaaf4c8e30bd4021180a9a39f Reported-by: Euler Robot <euler.robot@huawei.com> Signed-off-by: Pan Nengyuan <pannengyuan@huawei.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <1576805650-16380-1-git-send-email-pannengyuan@huawei.com> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2020-01-07chardev: generate an internal id when none givenMarc-André Lureau
Internally, qemu may create chardev without ID. Those will not be looked up with qemu_chr_find(), which prevents using qdev_prop_set_chr(). Use id_generate(), to generate an internal name (prefixed with #), so no conflict exist with user-named chardev. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: xiaoqiang zhao <zxq_yx_007@163.com>
2020-01-07replay: record and replay random number sourcesPavel Dovgalyuk
Record/replay feature of icount allows deterministic running of execution scenarios. Some CPUs and peripheral devices read random numbers from external sources making deterministic execution impossible. This patch adds recording and replaying of random read operations into guest-random module, which is used by the virtual hardware. Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru> Message-Id: <157675984852.14505.15709141760677102489.stgit@pasha-Precision-3630-Tower> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-01-06Merge remote-tracking branch ↵Peter Maydell
'remotes/elmarco/tags/dbus-vmstate7-pull-request' into staging Add dbus-vmstate Hi, With external processes or helpers participating to the VM support, it becomes necessary to handle their migration. Various options exist to transfer their state: 1) as the VM memory, RAM or devices (we could say that's how vhost-user devices can be handled today, they are expected to restore from ring state) 2) other "vmstate" (as with TPM emulator state blobs) 3) left to be handled by management layer 1) is not practical, since an external processes may legitimatelly need arbitrary state date to back a device or a service, or may not even have an associated device. 2) needs ad-hoc code for each helper, but is simple and working 3) is complicated for management layer, QEMU has the migration timing The proposed "dbus-vmstate" object will connect to a given D-Bus address, and save/load from org.qemu.VMState1 owners on migration. Thus helpers can easily have their state migrated with QEMU, without implementing ad-hoc support (such as done for TPM emulation) D-Bus is ubiquitous on Linux (it is systemd IPC), and can be made to work on various other OSes. There are several implementations and good bindings for various languages. (the tests/dbus-vmstate-test.c is a good example of how simple the implementation of services can be, even in C) dbus-vmstate is put into use by the libvirt series "[PATCH 00/23] Use a slirp helper process". v2: - fix build with broken mingw-glib # gpg: Signature made Mon 06 Jan 2020 14:43:35 GMT # gpg: using RSA key 87A9BD933F87C606D276F62DDAE8E10975969CE5 # gpg: issuer "marcandre.lureau@redhat.com" # gpg: Good signature from "Marc-André Lureau <marcandre.lureau@redhat.com>" [full] # gpg: aka "Marc-André Lureau <marcandre.lureau@gmail.com>" [full] # Primary key fingerprint: 87A9 BD93 3F87 C606 D276 F62D DAE8 E109 7596 9CE5 * remotes/elmarco/tags/dbus-vmstate7-pull-request: tests: add dbus-vmstate-test tests: add migration-helpers unit dockerfiles: add dbus-daemon to some of latest distributions configure: add GDBUS_CODEGEN Add dbus-vmstate object util: add dbus helper unit docs: start a document to describe D-Bus usage vmstate: replace DeviceState with VMStateIf vmstate: add qom interface to get id Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-01-06util: add dbus helper unitMarc-André Lureau
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-02osdep: add qemu_unlink()Marc-André Lureau
Add a helper function to match qemu_open() which may return files under the /dev/fdset prefix. Those shouldn't be removed, since it's only a qemu namespace. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-12-20Merge remote-tracking branch ↵Peter Maydell
'remotes/stsquad/tags/pull-tesing-and-misc-191219-1' into staging Various testing and logging updates - test tci with Travis - enable multiarch testing in Travis - default to out-of-tree builds - make changing logfile safe via RCU - remove redundant tests - remove gtester test from docker - convert DEBUG_MMAP to tracepoints - remove hand rolled glob function - trigger tcg re-configure when needed # gpg: Signature made Thu 19 Dec 2019 08:24:08 GMT # gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44 # gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full] # Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44 * remotes/stsquad/tags/pull-tesing-and-misc-191219-1: (25 commits) tests/tcg: ensure we re-configure if configure.sh is updated trace: replace hand-crafted pattern_glob with g_pattern_match_simple linux-user: convert target_munmap debug to a tracepoint linux-user: log page table changes under -d page linux-user: add target_mmap_complete tracepoint linux-user: convert target_mmap debug to tracepoint linux-user: convert target_mprotect debug to tracepoint travis.yml: Remove the redundant clang-with-MAIN_SOFTMMU_TARGETS entry docker: gtester is no longer used Added tests for close and change of logfile. Add use of RCU for qemu_logfile. qemu_log_lock/unlock now preserves the qemu_logfile handle. Add a mutex to guarantee single writer to qemu_logfile handle. Cleaned up flow of code in qemu_set_log(), to simplify and clarify. Fix double free issue in qemu_set_log_filename(). ci: build out-of-tree travis.yml: Enable builds on arm64, ppc64le and s390x tests/test-util-filemonitor: Skip test on non-x86 Travis containers tests/hd-geo-test: Skip test when images can not be created iotests: Skip test 079 if it is not possible to create large files ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-12-20Merge remote-tracking branch ↵Peter Maydell
'remotes/vivier2/tags/trivial-branch-pull-request' into staging Trivial fixes (20191218) # gpg: Signature made Wed 18 Dec 2019 13:00:34 GMT # gpg: using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C # gpg: issuer "laurent@vivier.eu" # gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full] # gpg: aka "Laurent Vivier <laurent@vivier.eu>" [full] # gpg: aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full] # Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F 5173 F30C 38BD 3F2F BE3C * remotes/vivier2/tags/trivial-branch-pull-request: qemu-doc: Remove the unused "Guest Agent" node Revert "qemu-options.hx: Update for reboot-timeout parameter" target/sparc: Remove old TODO file test-keyval: Tighten test of trailing crap after size util/cutils: Turn FIXME comment into QEMU_BUILD_BUG_ON() monitor: Remove unused define MAINTAINERS: Add hw/sd/ssi-sd.c in the SD section Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-12-20Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into stagingPeter Maydell
* More uses of RCU_READ_LOCK_GUARD (Dave, myself) * QOM doc improvments (Greg) * Cleanups from the Meson conversion (Marc-André) * Support for multiple -accel options (myself) * Many x86 machine cleanup (Philippe, myself) * tests/migration-test cleanup (Juan) * PC machine removal and next round of deprecation (Thomas) * kernel-doc integration (Peter, myself) # gpg: Signature made Wed 18 Dec 2019 01:35:02 GMT # gpg: using RSA key BFFBD25F78C7AE83 # gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full] # gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" [full] # Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1 # Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83 * remotes/bonzini/tags/for-upstream: (87 commits) vga: cleanup mapping of VRAM for non-PCI VGA hw/display: Remove "rombar" hack from vga-pci and vmware_vga hw/pci: Remove the "command_serr_enable" property hw/audio: Remove the "use_broken_id" hack from the AC97 device hw/i386: Remove the deprecated machines 0.12 up to 0.15 hw/pci-host: Add Kconfig entry to select the IGD Passthrough Host Bridge hw/pci-host/i440fx: Extract the IGD passthrough host bridge device hw/pci-host/i440fx: Use definitions instead of magic values hw/pci-host/i440fx: Use size_t to iterate over ARRAY_SIZE() hw/pci-host/i440fx: Extract PCII440FXState to "hw/pci-host/i440fx.h" hw/pci-host/i440fx: Correct the header description Fix some comment spelling errors. target/i386: remove unused pci-assign codes WHPX: refactor load library migration: check length directly to make sure the range is aligned memory: include MemoryListener documentation and some missing function parameters docs: add memory API reference memory.h: Silence kernel-doc complaints docs: Create bitops.rst as example of kernel-docs bitops.h: Silence kernel-doc complaints ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-12-18Add use of RCU for qemu_logfile.Robert Foley
This now allows changing the logfile while logging is active, and also solves the issue of a seg fault while changing the logfile. Any read access to the qemu_logfile handle will use the rcu_read_lock()/unlock() around the use of the handle. To fetch the handle we will use atomic_rcu_read(). We also in many cases do a check for validity of the logfile handle before using it to deal with the case where the file is closed and set to NULL. The cases where we write to the qemu_logfile will use atomic_rcu_set(). Writers will also use call_rcu() with a newly added qemu_logfile_free function for freeing/closing when readers have finished. Signed-off-by: Robert Foley <robert.foley@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20191118211528.3221-6-robert.foley@linaro.org>
2019-12-18Add a mutex to guarantee single writer to qemu_logfile handle.Robert Foley
Also added qemu_logfile_init() for initializing the logfile mutex. Note that inside qemu_set_log() we needed to add a pair of qemu_mutex_unlock() calls in order to avoid a double lock in qemu_log_close(). This unavoidable temporary ugliness will be cleaned up in a later patch in this series. Signed-off-by: Robert Foley <robert.foley@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20191118211528.3221-4-robert.foley@linaro.org>
2019-12-18Cleaned up flow of code in qemu_set_log(), to simplify and clarify.Robert Foley
Also added some explanation of the reasoning behind the branches. Signed-off-by: Robert Foley <robert.foley@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20191118211528.3221-3-robert.foley@linaro.org>
2019-12-18Fix double free issue in qemu_set_log_filename().Robert Foley
After freeing the logfilename, we set logfilename to NULL, in case of an error which returns without setting logfilename. Signed-off-by: Robert Foley <robert.foley@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20191118211528.3221-2-robert.foley@linaro.org>
2019-12-18util/cutils: Turn FIXME comment into QEMU_BUILD_BUG_ON()Markus Armbruster
qemu_strtoi64() assumes int64_t is long long. This is marked FIXME. Replace by a QEMU_BUILD_BUG_ON() to avoid surprises. Same for qemu_strtou64(). Fix a typo in qemu_strtoul()'s contract while there. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20191125133846.27790-2-armbru@redhat.com> [lv: removed trailing whitespace] Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-12-18error: make Error **errp const where it is appropriateVladimir Sementsov-Ogievskiy
Mostly, Error ** is for returning error from the function, so the callee sets it. However these three functions get already filled errp parameter. They don't change the pointer itself, only change the internal state of referenced Error object. So we can make it Error *const * errp, to stress the behavior. It will also help coccinelle script (in future) to distinguish such cases from common errp usage. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20191205174635.18758-4-vsementsov@virtuozzo.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Commit message typo fixed] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2019-12-18error: Fix -msg timestamp defaultMarkus Armbruster
-msg parameter "timestamp" defaults to "off" if you don't specify msg, and to "on" if you do. Messed up right in commit 5e2ac51917 "add timestamp to error_report()". Mostly harmless, because "timestamp" is the only parameter, so "if you do" is "-msg ''", which nobody does. Change the default to "off" no matter what. While there, rename enable_timestamp_msg to error_with_timestamp, and polish documentation. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20191010081508.8978-1-armbru@redhat.com>
2019-12-17qsp: Use WITH_RCU_READ_LOCK_GUARDDr. David Alan Gilbert
The automatic rcu read lock maintenance works quite nicely in this case where it previously relied on a comment to delimit the lifetime and now has a block. Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-12-17stubs: replace stubs with lnot if applicablePaolo Bonzini
The stubs mechanism relies on static libraries and compilation order, which is a bit brittle and should be avoided unless necessary. Replace it with Boolean operations on CONFIG_* symbols. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-12-16Memory: Enable writeback for given memory regionBeata Michalska
Add an option to trigger memory writeback to sync given memory region with the corresponding backing store, case one is available. This extends the support for persistent memory, allowing syncing on-demand. Signed-off-by: Beata Michalska <beata.michalska@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-id: 20191121000843.24844-3-beata.michalska@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-11-25util/cutils: Fix incorrect integer->float conversion caught by clangFangrui Song
Clang does not like do_strtosz()'s code to guard against overflow: qemu/util/cutils.c:245:23: error: implicit conversion from 'unsigned long' to 'double' changes value from 18446744073709550592 to 18446744073709551616 [-Werror,-Wimplicit-int-float-conversion] The warning will be enabled by default in clang 10. It is not available for clang <= 9. val * mul >= 0xfffffffffffffc00 is indeed wrong. 0xfffffffffffffc00 is not representable exactly as double. It's half-way between the representable values 0xfffffffffffff800 and 0x10000000000000000. Which one we get is implementation-defined. Bad. We want val * mul > (the largest uint64_t exactly representable as double). That's 0xfffffffffffff800. Write it as nextafter(0x1p64, 0) with a suitable comment. Signed-off-by: Fangrui Song <i@maskray.me> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Patch split, commit message improved] Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20191122080039.12771-3-armbru@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Juan Quintela <quintela@redhat.com>
2019-11-18qemu-coroutine-sleep: Silence Coverity warningEric Blake
Coverity warns that we store the address of a stack variable through a pointer passed in by the caller, which would let the caller trivially trigger use-after-free if that stored value is still present when we finish execution. However, the way coroutines work is that after our call to qemu_coroutine_yield(), control is temporarily continued in the caller prior to our function concluding, and in order to resume our coroutine, the caller must poll until the variable has been set to NULL. Thus, we can add an assert that we do not leak stack storage to the caller on function exit. Fixes: Coverity CID 1406474 CC: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20191111203524.21912-1-eblake@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
2019-10-30Merge remote-tracking branch ↵Peter Maydell
'remotes/stsquad/tags/pull-tcg-plugins-281019-4' into staging TCG Plugins initial implementation - use --enable-plugins @ configure - low impact introspection (-plugin empty.so to measure overhead) - plugins cannot alter guest state - example plugins included in source tree (tests/plugins) - -d plugin to enable plugin output in logs - check-tcg runs extra tests when plugins enabled - documentation in docs/devel/plugins.rst # gpg: Signature made Mon 28 Oct 2019 15:13:23 GMT # gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44 # gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full] # Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44 * remotes/stsquad/tags/pull-tcg-plugins-281019-4: (57 commits) travis.yml: enable linux-gcc-debug-tcg cache MAINTAINERS: add me for the TCG plugins code scripts/checkpatch.pl: don't complain about (foo, /* empty */) .travis.yml: add --enable-plugins tests include/exec: wrap cpu_ldst.h in CONFIG_TCG accel/stubs: reduce headers from tcg-stub tests/plugin: add hotpages to analyse memory access patterns tests/plugin: add instruction execution breakdown tests/plugin: add a hotblocks plugin tests/tcg: enable plugin testing tests/tcg: drop test-i386-fprem from TESTS when not SLOW tests/tcg: move "virtual" tests to EXTRA_TESTS tests/tcg: set QEMU_OPTS for all cris runs tests/tcg/Makefile.target: fix path to config-host.mak tests/plugin: add sample plugins linux-user: support -plugin option vl: support -plugin option plugin: add qemu_plugin_outs helper plugin: add qemu_plugin_insn_disas helper plugin: expand the plugin_init function to include an info block ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-10-28plugin: add qemu_plugin_outs helperAlex Bennée
Having the plugins grab stdout and spew stuff there is a bit ugly and certainly makes the tests look ugly. Provide a hook back into QEMU which can be redirected as needed. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Aaron Lindsay <aaron@os.amperecomputing.com>
2019-10-28Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-10-28' ↵Peter Maydell
into staging Block patches for softfreeze: - iotest patches - Improve performance of the mirror block job in write-blocking mode - Limit memory usage for the backup block job - Add discard and write-zeroes support to the NVMe host block driver - Fix a bug in the mirror job - Prevent the qcow2 driver from creating technically non-compliant qcow2 v3 images (where there is not enough extra data for snapshot table entries) - Allow callers of bdrv_truncate() (etc.) to determine whether the file must be resized to the exact given size or whether it is OK for block devices not to shrink # gpg: Signature made Mon 28 Oct 2019 12:13:53 GMT # gpg: using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40 # gpg: issuer "mreitz@redhat.com" # gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full] # Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40 * remotes/maxreitz/tags/pull-block-2019-10-28: (69 commits) qemu-iotests: restrict 264 to qcow2 only Revert "qemu-img: Check post-truncation size" block: Pass truncate exact=true where reasonable block: Let format drivers pass @exact block: Evaluate @exact in protocol drivers block: Add @exact parameter to bdrv_co_truncate() block: Do not truncate file node when formatting block/cor: Drop cor_co_truncate() block: Handle filter truncation like native impl. iotests: Test qcow2's snapshot table handling iotests: Add peek_file* functions qcow2: Fix v3 snapshot table entry compliancy qcow2: Repair snapshot table with too many entries qcow2: Fix overly long snapshot tables qcow2: Keep track of the snapshot table length qcow2: Fix broken snapshot table entries qcow2: Add qcow2_check_fix_snapshot_table() qcow2: Separate qcow2_check_read_snapshot_table() qcow2: Write v3-compliant snapshot list on upgrade qcow2: Put qcow2_upgrade() into its own function ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-10-28util: introduce SharedResourceVladimir Sementsov-Ogievskiy
Introduce an API for some shared splittable resource, like memory. It's going to be used by backup. Backup uses both read/write io and copy_range. copy_range may consume memory implictly, so the new API is abstract: it doesn't allocate any real memory but only hands out tickets. The idea is that we have some total amount of something and callers should wait in coroutine queue if there is not enough of the resource at the moment. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20191022111805.3432-5-vsementsov@virtuozzo.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-28hbitmap: handle set/reset with zero lengthVladimir Sementsov-Ogievskiy
Passing zero length to these functions leads to unpredicted results. Zero-length set/reset may occur in active-mirror, on zero-length write (which is unlikely, but not guaranteed to never happen). Let's just do nothing on zero-length request. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20191011090711.19940-2-vsementsov@virtuozzo.com Reviewed-by: Max Reitz <mreitz@redhat.com> Cc: qemu-stable@nongnu.org Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-26core: replace getpagesize() with qemu_real_host_page_sizeWei Yang
There are three page size in qemu: real host page size host page size target page size All of them have dedicate variable to represent. For the last two, we use the same form in the whole qemu project, while for the first one we use two forms: qemu_real_host_page_size and getpagesize(). qemu_real_host_page_size is defined to be a replacement of getpagesize(), so let it serve the role. [Note] Not fully tested for some arch or device. Signed-off-by: Wei Yang <richardw.yang@linux.intel.com> Message-Id: <20191013021145.16011-3-richardw.yang@linux.intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-10-26Merge commit 'df84f17' into HEADPaolo Bonzini
This merge fixes a semantic conflict with the trivial tree. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-10-25Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into ↵Peter Maydell
staging Pull request # gpg: Signature made Fri 25 Oct 2019 20:18:23 BST # gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8 # gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full] # gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full] # Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8 * remotes/stefanha/tags/block-pull-request: yield_until_fd_readable: make it work with any AioContect virtio-blk: Add blk_drain() to virtio_blk_device_unrealize() Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-10-25yield_until_fd_readable: make it work with any AioContectDietmar Maurer
Simply use qemu_get_current_aio_context(). Signed-off-by: Dietmar Maurer <dietmar@proxmox.com> Message-Id: <20191024045610.9071-1-dietmar@proxmox.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>