aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2018-03-19tests/test-blockjob: test cancellationsJohn Snow
Whatever the state a blockjob is in, it should be able to be canceled by the block layer. Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19iotests: test manual job dismissalJohn Snow
Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: Expose manual propertyJohn Snow
Expose the "manual" property via QAPI for the backup-related jobs. As of this commit, this allows the management API to request the "concluded" and "dismiss" semantics for backup jobs. Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: add block-job-finalizeJohn Snow
Instead of automatically transitioning from PENDING to CONCLUDED, gate the .prepare() and .commit() phases behind an explicit acknowledgement provided by the QMP monitor if auto_finalize = false has been requested. This allows us to perform graph changes in prepare and/or commit so that graph changes do not occur autonomously without knowledge of the controlling management layer. Transactions that have reached the "PENDING" state together can all be moved to invoke their finalization methods by issuing block_job_finalize to any one job in the transaction. Jobs in a transaction with mixed job->auto_finalize settings will all remain stuck in the "PENDING" state, as if the entire transaction was specified with auto_finalize = false. Jobs that specified auto_finalize = true, however, will still not emit the PENDING event. Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: add PENDING status and eventJohn Snow
For jobs utilizing the new manual workflow, we intend to prohibit them from modifying the block graph until the management layer provides an explicit ACK via block-job-finalize to move the process forward. To distinguish this runstate from "ready" or "waiting," we add a new "pending" event and status. For now, the transition from PENDING to CONCLUDED/ABORTING is automatic, but a future commit will add the explicit block-job-finalize step. Transitions: Waiting -> Pending: Normal transition. Pending -> Concluded: Normal transition. Pending -> Aborting: Late transactional failures and cancellations. Removed Transitions: Waiting -> Concluded: Jobs must go to PENDING first. Verbs: Cancel: Can be applied to a pending job. +---------+ |UNDEFINED| +--+------+ | +--v----+ +---------+CREATED+-----------------+ | +--+----+ | | | | | +--+----+ +------+ | +---------+RUNNING<----->PAUSED| | | +--+-+--+ +------+ | | | | | | | +------------------+ | | | | | | +--v--+ +-------+ | | +---------+READY<------->STANDBY| | | | +--+--+ +-------+ | | | | | | | +--v----+ | | +---------+WAITING<---------------+ | | +--+----+ | | | | | +--v----+ | +---------+PENDING| | | +--+----+ | | | | +--v-----+ +--v------+ | |ABORTING+--->CONCLUDED| | +--------+ +--+------+ | | | +--v-+ | |NULL<--------------------+ +----+ Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: add waiting statusJohn Snow
For jobs that are stuck waiting on others in a transaction, it would be nice to know that they are no longer "running" in that sense, but instead are waiting on other jobs in the transaction. Jobs that are "waiting" in this sense cannot be meaningfully altered any longer as they have left their running loop. The only meaningful user verb for jobs in this state is "cancel," which will cancel the whole transaction, too. Transitions: Running -> Waiting: Normal transition. Ready -> Waiting: Normal transition. Waiting -> Aborting: Transactional cancellation. Waiting -> Concluded: Normal transition. Removed Transitions: Running -> Concluded: Jobs must go to WAITING first. Ready -> Concluded: Jobs must go to WAITING first. Verbs: Cancel: Can be applied to WAITING jobs. +---------+ |UNDEFINED| +--+------+ | +--v----+ +---------+CREATED+-----------------+ | +--+----+ | | | | | +--v----+ +------+ | +---------+RUNNING<----->PAUSED| | | +--+-+--+ +------+ | | | | | | | +------------------+ | | | | | | +--v--+ +-------+ | | +---------+READY<------->STANDBY| | | | +--+--+ +-------+ | | | | | | | +--v----+ | | +---------+WAITING<---------------+ | | +--+----+ | | | | +--v-----+ +--v------+ | |ABORTING+--->CONCLUDED| | +--------+ +--+------+ | | | +--v-+ | |NULL<--------------------+ +----+ Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: add prepare callbackJohn Snow
Some jobs upon finalization may need to perform some work that can still fail. If these jobs are part of a transaction, it's important that these callbacks fail the entire transaction. We allow for a new callback in addition to commit/abort/clean that allows us the opportunity to have fairly late-breaking failures in the transactional process. The expected flow is: - All jobs in a transaction converge to the PENDING state, added in a forthcoming commit. - Upon being finalized, either automatically or explicitly by the user, jobs prepare to complete. - If any job fails preparation, all jobs call .abort. - Otherwise, they succeed and call .commit. Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: add block_job_txn_apply functionJohn Snow
Simply apply a function transaction-wide. A few more uses of this in forthcoming patches. Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: add commit, abort, clean helpersJohn Snow
The completed_single function is getting a little mucked up with checking to see which callbacks exist, so let's factor them out. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: ensure abort is called for cancelled jobsJohn Snow
Presently, even if a job is canceled post-completion as a result of a failing peer in a transaction, it will still call .commit because nothing has updated or changed its return code. The reason why this does not cause problems currently is because backup's implementation of .commit checks for cancellation itself. I'd like to simplify this contract: (1) Abort is called if the job/transaction fails (2) Commit is called if the job/transaction succeeds To this end: A job's return code, if 0, will be forcibly set as -ECANCELED if that job has already concluded. Remove the now redundant check in the backup job implementation. We need to check for cancellation in both block_job_completed AND block_job_completed_single, because jobs may be cancelled between those two calls; for instance in transactions. This also necessitates an ABORTING -> ABORTING transition to be allowed. The check in block_job_completed could be removed, but there's no point in starting to attempt to succeed a transaction that we know in advance will fail. This does NOT affect mirror jobs that are "canceled" during their synchronous phase. The mirror job itself forcibly sets the canceled property to false prior to ceding control, so such cases will invoke the "commit" callback. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: add block_job_dismissJohn Snow
For jobs that have reached their CONCLUDED state, prior to having their last reference put down (meaning jobs that have completed successfully, unsuccessfully, or have been canceled), allow the user to dismiss the job's lingering status report via block-job-dismiss. This gives management APIs the chance to conclusively determine if a job failed or succeeded, even if the event broadcast was missed. Note: block_job_do_dismiss and block_job_decommission happen to do exactly the same thing, but they're called from different semantic contexts, so both aliases are kept to improve readability. Note 2: Don't worry about the 0x04 flag definition for AUTO_DISMISS, she has a friend coming in a future patch to fill the hole where 0x02 is. Verbs: Dismiss: operates on CONCLUDED jobs only. Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: add NULL stateJohn Snow
Add a new state that specifically demarcates when we begin to permanently demolish a job after it has performed all work. This makes the transition explicit in the STM table and highlights conditions under which a job may be demolished. Alongside this state, add a new helper command "block_job_decommission", which transitions to the NULL state and puts down our implicit reference. This separates instances in the code for "block_job_unref" which merely undo a matching "block_job_ref" with instances intended to initiate the full destruction of the object. This decommission action also sets a number of fields to make sure that block internals or external users that are holding a reference to a job to see when it "finishes" are convinced that the job object is "done." This is necessary, for instance, to do a block_job_cancel_sync on a created object which will not make any progress. Now, all jobs must go through block_job_decommission prior to being freed, giving us start-to-finish state machine coverage for jobs. Transitions: Created -> Null: Early failure event before the job is started Concluded -> Null: Standard transition. Verbs: None. This should not ever be visible to the monitor. +---------+ |UNDEFINED| +--+------+ | +--v----+ +---------+CREATED+------------------+ | +--+----+ | | | | | +--v----+ +------+ | +---------+RUNNING<----->PAUSED| | | +--+-+--+ +------+ | | | | | | | +------------------+ | | | | | | +--v--+ +-------+ | | +---------+READY<------->STANDBY| | | | +--+--+ +-------+ | | | | | | +--v-----+ +--v------+ | | |ABORTING+--->CONCLUDED<-------------+ | +--------+ +--+------+ | | | +--v-+ | |NULL<---------------------+ +----+ Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: add CONCLUDED stateJohn Snow
add a new state "CONCLUDED" that identifies a job that has ceased all operations. The wording was chosen to avoid any phrasing that might imply success, error, or cancellation. The task has simply ceased all operation and can never again perform any work. ("finished", "done", and "completed" might all imply success.) Transitions: Running -> Concluded: normal completion Ready -> Concluded: normal completion Aborting -> Concluded: error and cancellations Verbs: None as of this commit. (a future commit adds 'dismiss') +---------+ |UNDEFINED| +--+------+ | +--v----+ +---------+CREATED| | +--+----+ | | | +--v----+ +------+ +---------+RUNNING<----->PAUSED| | +--+-+--+ +------+ | | | | | +------------------+ | | | | +--v--+ +-------+ | +---------+READY<------->STANDBY| | | +--+--+ +-------+ | | | | +--v-----+ +--v------+ | |ABORTING+--->CONCLUDED<-------------+ +--------+ +---------+ Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: add ABORTING stateJohn Snow
Add a new state ABORTING. This makes transitions from normative states to error states explicit in the STM, and serves as a disambiguation for which states may complete normally when normal end-states (CONCLUDED) are added in future commits. Notably, Paused/Standby jobs do not transition directly to aborting, as they must wake up first and cooperate in their cancellation. Transitions: Created -> Aborting: can be cancelled (by the system) Running -> Aborting: can be cancelled or encounter an error Ready -> Aborting: can be cancelled or encounter an error Verbs: None. The job must finish cleaning itself up and report its final status. +---------+ |UNDEFINED| +--+------+ | +--v----+ +---------+CREATED| | +--+----+ | | | +--v----+ +------+ +---------+RUNNING<----->PAUSED| | +--+----+ +------+ | | | +--v--+ +-------+ +---------+READY<------->STANDBY| | +-----+ +-------+ | +--v-----+ |ABORTING| +--------+ Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: add block_job_verb permission tableJohn Snow
Which commands ("verbs") are appropriate for jobs in which state is also somewhat burdensome to keep track of. As of this commit, it looks rather useless, but begins to look more interesting the more states we add to the STM table. A recurring theme is that no verb will apply to an 'undefined' job. Further, it's not presently possible to restrict the "pause" or "resume" verbs any more than they are in this commit because of the asynchronous nature of how jobs enter the PAUSED state; justifications for some seemingly erroneous applications are given below. ===== Verbs ===== Cancel: Any state except undefined. Pause: Any state except undefined; 'created': Requests that the job pauses as it starts. 'running': Normal usage. (PAUSED) 'paused': The job may be paused for internal reasons, but the user may wish to force an indefinite user-pause, so this is allowed. 'ready': Normal usage. (STANDBY) 'standby': Same logic as above. Resume: Any state except undefined; 'created': Will lift a user's pause-on-start request. 'running': Will lift a pause request before it takes effect. 'paused': Normal usage. 'ready': Will lift a pause request before it takes effect. 'standby': Normal usage. Set-speed: Any state except undefined, though ready may not be meaningful. Complete: Only a 'ready' job may accept a complete request. ======= Changes ======= (1) To facilitate "nice" error checking, all five major block-job verb interfaces in blockjob.c now support an errp parameter: - block_job_user_cancel is added as a new interface. - block_job_user_pause gains an errp paramter - block_job_user_resume gains an errp parameter - block_job_set_speed already had an errp parameter. - block_job_complete already had an errp parameter. (2) block-job-pause and block-job-resume will no longer no-op when trying to pause an already paused job, or trying to resume a job that isn't paused. These functions will now report that they did not perform the action requested because it was not possible. iotests have been adjusted to address this new behavior. (3) block-job-complete doesn't worry about checking !block_job_started, because the permission table guards against this. (4) test-bdrv-drain's job implementation needs to announce that it is 'ready' now, in order to be completed. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19iotests: add pause_waitJohn Snow
Split out the pause command into the actual pause and the wait. Not every usage presently needs to resubmit a pause request. The intent with the next commit will be to explicitly disallow redundant or meaningless pause/resume requests, so the tests need to become more judicious to reflect that. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: add state transition tableJohn Snow
The state transition table has mostly been implied. We're about to make it a bit more complex, so let's make the STM explicit instead. Perform state transitions with a function that for now just asserts the transition is appropriate. Transitions: Undefined -> Created: During job initialization. Created -> Running: Once the job is started. Jobs cannot transition from "Created" to "Paused" directly, but will instead synchronously transition to running to paused immediately. Running -> Paused: Normal workflow for pauses. Running -> Ready: Normal workflow for jobs reaching their sync point. (e.g. mirror) Ready -> Standby: Normal workflow for pausing ready jobs. Paused -> Running: Normal resume. Standby -> Ready: Resume of a Standby job. +---------+ |UNDEFINED| +--+------+ | +--v----+ |CREATED| +--+----+ | +--v----+ +------+ |RUNNING<----->PAUSED| +--+----+ +------+ | +--v--+ +-------+ |READY<------->STANDBY| +-----+ +-------+ Notably, there is no state presently defined as of this commit that deals with a job after the "running" or "ready" states, so this table will be adjusted alongside the commits that introduce those states. Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: add status enumJohn Snow
We're about to add several new states, and booleans are becoming unwieldly and difficult to reason about. It would help to have a more explicit bookkeeping of the state of blockjobs. To this end, add a new "status" field and add our existing states in a redundant manner alongside the bools they are replacing: UNDEFINED: Placeholder, default state. Not currently visible to QMP unless changes occur in the future to allow creating jobs without starting them via QMP. CREATED: replaces !!job->co && paused && !busy RUNNING: replaces effectively (!paused && busy) PAUSED: Nearly redundant with info->paused, which shows pause_count. This reports the actual status of the job, which almost always matches the paused request status. It differs in that it is strictly only true when the job has actually gone dormant. READY: replaces job->ready. STANDBY: Paused, but job->ready is true. New state additions in coming commits will not be quite so redundant: WAITING: Waiting on transaction. This job has finished all the work it can until the transaction converges, fails, or is canceled. PENDING: Pending authorization from user. This job has finished all the work it can until the job or transaction is finalized via block_job_finalize. This implies the transaction has converged and left the WAITING phase. ABORTING: Job has encountered an error condition and is in the process of aborting. CONCLUDED: Job has ceased all operations and has a return code available for query and may be dismissed via block_job_dismiss. NULL: Job has been dismissed and (should) be destroyed. Should never be visible to QMP. Some of these states appear somewhat superfluous, but it helps define the expected flow of a job; so some of the states wind up being synchronous empty transitions. Importantly, jobs can be in only one of these states at any given time, which helps code and external users alike reason about the current condition of a job unambiguously. Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19Blockjobs: documentation touchupJohn Snow
Trivial; Document what the job creation flags do, and some general tidying. Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: model single jobs as transactionsJohn Snow
model all independent jobs as single job transactions. It's one less case we have to worry about when we add more states to the transition machine. This way, we can just treat all job lifetimes exactly the same. This helps tighten assertions of the STM graph and removes some conditionals that would have been needed in the coming commits adding a more explicit job lifetime management API. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19blockjobs: fix set-speed kickJohn Snow
If speed is '0' it's not actually "less than" the previous speed. Kick the job in this case too. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-17Merge remote-tracking branch 'remotes/xtensa/tags/20180316-xtensa' into stagingPeter Maydell
target/xtensa linux-user support. - small cleanup for xtensa registers dumping (-d cpu); - add support for debugging linux-user process with xtensa-linux-gdb (as opposed to xtensa-elf-gdb), which can only access unprivileged registers; - enable MTTCG for target/xtensa; - cleanup in linux-user/mmap area making sure that it works correctly with limited 30-bit-wide user address space; - import xtensa-specific definitions from the linux kernel, conditionalize user-only/softmmu-only code and add handlers for signals, exceptions, process/thread creation and core registers dumping. # gpg: Signature made Fri 16 Mar 2018 16:46:19 GMT # gpg: using RSA key 51F9CC91F83FA044 # gpg: Good signature from "Max Filippov <filippov@cadence.com>" # gpg: aka "Max Filippov <max.filippov@cogentembedded.com>" # gpg: aka "Max Filippov <jcmvbkbc@gmail.com>" # Primary key fingerprint: 2B67 854B 98E5 327D CDEB 17D8 51F9 CC91 F83F A044 * remotes/xtensa/tags/20180316-xtensa: MAINTAINERS: fix W: address for xtensa qemu-binfmt-conf.sh: add qemu-xtensa target/xtensa: add linux-user support linux-user: drop unused target_msync function linux-user: fix target_mprotect/target_munmap error return values linux-user: fix assertion in shmdt linux-user: fix mmap/munmap/mprotect/mremap/shmat target/xtensa: support MTTCG target/xtensa: use correct number of registers in gdbstub target/xtensa: mark register windows in the dump target/xtensa: dump correct physical registers Signed-off-by: Peter Maydell <peter.maydell@linaro.org> # Conflicts: # linux-user/syscall.c
2018-03-17Merge remote-tracking branch ↵Peter Maydell
'remotes/ehabkost/tags/machine-next-pull-request' into staging PC compat bug fix for 2.12 # gpg: Signature made Fri 16 Mar 2018 19:30:25 GMT # gpg: using RSA key 2807936F984DC5A6 # gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" # Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF D1AA 2807 936F 984D C5A6 * remotes/ehabkost/tags/machine-next-pull-request: pc: correct misspelled CPU model-id for pc 2.2 Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-16pc: correct misspelled CPU model-id for pc 2.2Wang Xin
Signed-off-by: Wang Xin <wangxinxin.wang@huawei.com> Message-Id: <1517367668-25048-1-git-send-email-wangxinxin.wang@huawei.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-03-16Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20180316' into stagingPeter Maydell
Queued TCG patches # gpg: Signature made Thu 15 Mar 2018 17:17:31 GMT # gpg: using RSA key 64DF38E8AF7E215F # gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" # Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A 05C0 64DF 38E8 AF7E 215F * remotes/rth/tags/pull-tcg-20180316: tcg: Add choose_vector_size tcg/i386: Support INDEX_op_dup2_vec for -m32 tcg: Improve tcg_gen_muli_i32/i64 Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-16MAINTAINERS: fix W: address for xtensaMax Filippov
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
2018-03-16qemu-binfmt-conf.sh: add qemu-xtensaMax Filippov
Register qemu-xtensa and qemu-xtensaeb for transparent linux userspace emulation. Cc: Riku Voipio <riku.voipio@iki.fi> Cc: Laurent Vivier <laurent@vivier.eu> Reviewed-by: Laurent Vivier <laurent@vivier.eu> Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
2018-03-16target/xtensa: add linux-user supportMax Filippov
Import list of syscalls from the kernel source. Conditionalize code/data that is only used with softmmu. Implement exception handlers. Implement signal hander (only the core registers for now, no coprocessors or TIE). Cc: Riku Voipio <riku.voipio@iki.fi> Cc: Laurent Vivier <laurent@vivier.eu> Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
2018-03-16Merge remote-tracking branch 'remotes/jnsnow/tags/bitmaps-pull-request' into ↵Peter Maydell
staging # gpg: Signature made Tue 13 Mar 2018 21:11:43 GMT # gpg: using RSA key 7DEF8106AAFC390E # gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" # Primary key fingerprint: FAEB 9711 A12C F475 812F 18F2 88A9 064D 1835 61EB # Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76 CBD0 7DEF 8106 AAFC 390E * remotes/jnsnow/tags/bitmaps-pull-request: iotests: add dirty bitmap postcopy test iotests: add dirty bitmap migration test migration: add postcopy migration of dirty bitmaps migration: allow qmp command migrate-start-postcopy for any postcopy migration: add is_active_iterate handler migration/qemu-file: add qemu_put_counted_string() migration: include migrate_dirty_bitmaps in migrate_postcopy qapi: add dirty-bitmaps migration capability migration: introduce postcopy-only pending dirty-bitmap: add locked state block/dirty-bitmap: add _locked version of bdrv_reclaim_dirty_bitmap block/dirty-bitmap: fix locking in bdrv_reclaim_dirty_bitmap block/dirty-bitmap: add bdrv_dirty_bitmap_enable_successor() Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-16Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2018-03-13-v2' ↵Peter Maydell
into staging nbd patches for 2018-03-13 - Eric Blake: iotests: Fix stuck NBD process on 33 - Vladimir Sementsov-Ogievskiy: 0/5 nbd server fixing and refactoring before BLOCK_STATUS - Eric Blake: nbd/server: Honor FUA request on NBD_CMD_TRIM - Stefan Hajnoczi: 0/2 block: fix nbd-server-stop crash after blockdev-snapshot-sync - Vladimir Sementsov-Ogievskiy: nbd block status base:allocation # gpg: Signature made Tue 13 Mar 2018 20:48:37 GMT # gpg: using RSA key A7A16B4A2527436A # gpg: Good signature from "Eric Blake <eblake@redhat.com>" # gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" # gpg: aka "[jpeg image of size 6874]" # Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A * remotes/ericb/tags/pull-nbd-2018-03-13-v2: iotests: new test 209 for NBD BLOCK_STATUS iotests: add file_path helper iotests.py: tiny refactor: move system imports up nbd: BLOCK_STATUS for standard get_block_status function: client part block/nbd-client: save first fatal error in nbd_iter_error nbd: BLOCK_STATUS for standard get_block_status function: server part nbd/server: add nbd_read_opt_name helper nbd/server: add nbd_opt_invalid helper iotests: add 208 nbd-server + blockdev-snapshot-sync test case block: let blk_add/remove_aio_context_notifier() tolerate BDS changes nbd/server: Honor FUA request on NBD_CMD_TRIM nbd/server: refactor nbd_trip: split out nbd_handle_request nbd/server: refactor nbd_trip: cmd_read and generic reply nbd/server: fix: check client->closing before sending reply nbd/server: fix sparse read nbd/server: move nbd_co_send_structured_error up iotests: Fix stuck NBD process on 33 Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-16Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into stagingPeter Maydell
* Record-replay lockstep execution, log dumper and fixes (Alex, Pavel) * SCSI fix to pass maximum transfer size (Daniel Barboza) * chardev fixes and improved iothread support (Daniel Berrangé, Peter) * checkpatch tweak (Eric) * make help tweak (Marc-André) * make more PCI NICs available with -net or -nic (myself) * change default q35 NIC to e1000e (myself) * SCSI support for NDOB bit (myself) * membarrier system call support (myself) * SuperIO refactoring (Philippe) * miscellaneous cleanups and fixes (Thomas) # gpg: Signature made Mon 12 Mar 2018 16:10:52 GMT # gpg: using RSA key BFFBD25F78C7AE83 # gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" # gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" # Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1 # Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83 * remotes/bonzini/tags/for-upstream: (69 commits) tcg: fix cpu_io_recompile replay: update documentation replay: save vmstate of the asynchronous events replay: don't process async events when warping the clock scripts/replay-dump.py: replay log dumper replay: avoid recursive call of checkpoints replay: check return values of fwrite replay: push replay_mutex_lock up the call tree replay: don't destroy mutex at exit replay: make locking visible outside replay code replay/replay-internal.c: track holding of replay_lock replay/replay.c: bump REPLAY_VERSION again replay: save prior value of the host clock replay: added replay log format description replay: fix save/load vm for non-empty queue replay: fixed replay_enable_events replay: fix processing async events cpu-exec: fix exception_index handling hw/i386/pc: Factor out the superio code hw/alpha/dp264: Use the TYPE_SMC37C669_SUPERIO ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org> # Conflicts: # default-configs/i386-softmmu.mak # default-configs/x86_64-softmmu.mak
2018-03-16Merge remote-tracking branch 'remotes/awilliam/tags/vfio-update-20180313.0' ↵Peter Maydell
into staging VFIO updates 2018-03-13 - Display support for vGPUs (Gerd Hoffmann) - Enable new kernel support for mmaps overlapping MSI-X vector table, disable MSI-X emulation on POWER (Alexey Kardashevskiy) # gpg: Signature made Tue 13 Mar 2018 19:48:49 GMT # gpg: using RSA key 239B9B6E3BB08B22 # gpg: Good signature from "Alex Williamson <alex.williamson@redhat.com>" # gpg: aka "Alex Williamson <alex@shazbot.org>" # gpg: aka "Alex Williamson <alwillia@redhat.com>" # gpg: aka "Alex Williamson <alex.l.williamson@gmail.com>" # Primary key fingerprint: 42F6 C04E 540B D1A9 9E7B 8A90 239B 9B6E 3BB0 8B22 * remotes/awilliam/tags/vfio-update-20180313.0: ppc/spapr, vfio: Turn off MSIX emulation for VFIO devices vfio-pci: Allow mmap of MSIX BAR vfio/pci: Relax DMA map errors for MMIO regions vfio/display: adding dmabuf support vfio/display: adding region support vfio/display: core & wireup vfio/common: cleanup in vfio_region_finalize secondary-vga: properly close QemuConsole on unplug console: minimal hotplug suport ui/pixman: add qemu_drm_format_to_pixman() standard-headers: add drm/drm_fourcc.h Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-15Merge remote-tracking branch ↵Peter Maydell
'remotes/berrange/tags/socket-next-pull-request' into staging # gpg: Signature made Tue 13 Mar 2018 18:12:14 GMT # gpg: using RSA key BE86EBB415104FDF # gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>" # gpg: aka "Daniel P. Berrange <berrange@redhat.com>" # Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF * remotes/berrange/tags/socket-next-pull-request: char: allow passing pre-opened socket file descriptor at startup char: refactor parsing of socket address information sockets: allow SocketAddress 'fd' to reference numeric file descriptors sockets: check that the named file descriptor is a socket sockets: move fd_is_socket() into common sockets code sockets: strengthen test suite IP protocol availability checks sockets: pull code for testing IP availability out of specific test cutils: add qemu_strtoi & qemu_strtoui parsers for int/unsigned int types char: don't silently skip tn3270 protocol init when TLS is enabled Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-15Merge remote-tracking branch ↵Peter Maydell
'remotes/vivier2/tags/linux-user-for-2.12-pull-request' into staging # gpg: Signature made Tue 13 Mar 2018 17:33:03 GMT # gpg: using RSA key F30C38BD3F2FBE3C # gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" # gpg: aka "Laurent Vivier <laurent@vivier.eu>" # gpg: aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" # Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F 5173 F30C 38BD 3F2F BE3C * remotes/vivier2/tags/linux-user-for-2.12-pull-request: linux-user: init_guest_space: Add a comment about search strategy linux-user: init_guest_space: Don't try to align if we'll reject it linux-user: init_guest_space: Clean up control flow a bit linux-user: init_guest_commpage: Add a comment about size check linux-user: init_guest_space: Clarify page alignment logic linux-user: init_guest_space: Correctly handle guest_start in commpage initialization linux-user: init_guest_space: Clean up if we can't initialize the commpage linux-user: Rename validate_guest_space => init_guest_commpage linux-user: Use #if to only call validate_guest_space for 32-bit ARM target qemu-binfmt-conf.sh: add qemu-xtensa linux-user: drop unused target_msync function linux-user: fix target_mprotect/target_munmap error return values linux-user: fix assertion in shmdt linux-user: fix mmap/munmap/mprotect/mremap/shmat linux-user: Support f_flags in statfs when available. linux-user: allows to use "--systemd ALL" with qemu-binfmt-conf.sh linux-user: Remove the unused "not implemented" signal handling stubs linux-user: Drop unicore32 code Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-16tcg: Add choose_vector_sizeRichard Henderson
This unifies 5 copies of checks for supported vector size, and in the process fixes a missing check in tcg_gen_gvec_2s. This lead to an assertion failure for 64-bit vector multiply, which is not available in the AVX instruction set. Suggested-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2018-03-16tcg/i386: Support INDEX_op_dup2_vec for -m32Richard Henderson
Unknown why -m32 was passing with gcc but not clang; it should have failed for both. This would be used for tcg_gen_dup_i64_vec, and visible with the right TB and an aarch64 guest. Reported-by: Max Reitz <mreitz@redhat.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2018-03-16tcg: Improve tcg_gen_muli_i32/i64Richard Henderson
Convert multiplication by power of two to left shift. Reviewed-by: Emilio G. Cota <cota@braap.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2018-03-15Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream-sev' into ↵Peter Maydell
staging * Migrate MSR_SMI_COUNT (Liran) * Update kernel headers (Gerd, myself) * SEV support (Brijesh) I have not tested non-x86 compilation, but I reordered the SEV patches so that all non-x86-specific changes go first to catch any possible issues (which weren't there anyway :)). # gpg: Signature made Tue 13 Mar 2018 16:37:06 GMT # gpg: using RSA key BFFBD25F78C7AE83 # gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" # gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" # Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1 # Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83 * remotes/bonzini/tags/for-upstream-sev: (22 commits) sev/i386: add sev_get_capabilities() sev/i386: qmp: add query-sev-capabilities command sev/i386: qmp: add query-sev-launch-measure command sev/i386: hmp: add 'info sev' command cpu/i386: populate CPUID 0x8000_001F when SEV is active sev/i386: add migration blocker sev/i386: finalize the SEV guest launch flow sev/i386: add support to LAUNCH_MEASURE command target/i386: encrypt bios rom sev/i386: add command to encrypt guest memory region sev/i386: add command to create launch memory encryption context sev/i386: register the guest memory range which may contain encrypted data sev/i386: add command to initialize the memory encryption context include: add psp-sev.h header file sev/i386: qmp: add query-sev command target/i386: add Secure Encrypted Virtualization (SEV) object kvm: introduce memory encryption APIs kvm: add memory encryption context docs: add AMD Secure Encrypted Virtualization (SEV) machine: add memory-encryption option ... Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-15Merge remote-tracking branch ↵Peter Maydell
'remotes/stsquad/tags/pull-travis-speedup-130318-1' into staging Some updates to reduce timeouts in Travis # gpg: Signature made Tue 13 Mar 2018 16:44:46 GMT # gpg: using RSA key FBD0DB095A9E2A44 # gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" # Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44 * remotes/stsquad/tags/pull-travis-speedup-130318-1: .travis.yml: add --disable-user with the rest of the disables .travis.yml: split default config into system and user .travis.yml: drop setting default log output Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-15Merge remote-tracking branch ↵Peter Maydell
'remotes/vivier/tags/m68k-for-2.12-pull-request' into staging # gpg: Signature made Tue 13 Mar 2018 15:58:42 GMT # gpg: using RSA key F30C38BD3F2FBE3C # gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" # gpg: aka "Laurent Vivier <laurent@vivier.eu>" # gpg: aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" # Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F 5173 F30C 38BD 3F2F BE3C * remotes/vivier/tags/m68k-for-2.12-pull-request: target/m68k: implement fcosh target/m68k: implement fsinh target/m68k: implement ftanh target/m68k: implement fatanh target/m68k: implement facos target/m68k: implement fasin target/m68k: implement fatan target/m68k: implement fsincos target/m68k: implement fcos target/m68k: implement fsin target/m68k: implement ftan Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-13iotests: add dirty bitmap postcopy testVladimir Sementsov-Ogievskiy
Test - start two vms (vm_a, vm_b) - in a - do writes from set A - do writes from set B - fix bitmap sha256 - clear bitmap - do writes from set A - start migration - than, in b - wait vm start (postcopy should start) - do writes from set B - check bitmap sha256 The test should verify postcopy migration and then merging with delta (changes in target, during postcopy process). Reduce supported cache modes to only 'none', because with cache on time from source.STOP to target.RESUME is unpredictable and we can fail with timout while waiting for target.RESUME. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20180313180320.339796-14-vsementsov@virtuozzo.com
2018-03-13iotests: add dirty bitmap migration testVladimir Sementsov-Ogievskiy
The test starts two vms (vm_a, vm_b), create dirty bitmap in the first one, do several writes to corresponding device and then migrate vm_a to vm_b. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-id: 20180313180320.339796-13-vsementsov@virtuozzo.com
2018-03-13migration: add postcopy migration of dirty bitmapsVladimir Sementsov-Ogievskiy
Postcopy migration of dirty bitmaps. Only named dirty bitmaps are migrated. If destination qemu is already containing a dirty bitmap with the same name as a migrated bitmap (for the same node), then, if their granularities are the same the migration will be done, otherwise the error will be generated. If destination qemu doesn't contain such bitmap it will be created. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-id: 20180313180320.339796-12-vsementsov@virtuozzo.com [Changed '+' to '*' as per list discussion. --js] Signed-off-by: John Snow <jsnow@redhat.com>
2018-03-13migration: allow qmp command migrate-start-postcopy for any postcopyVladimir Sementsov-Ogievskiy
Allow migrate-start-postcopy for any postcopy type Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-id: 20180313180320.339796-11-vsementsov@virtuozzo.com
2018-03-13migration: add is_active_iterate handlerVladimir Sementsov-Ogievskiy
Only-postcopy savevm states (dirty-bitmap) don't need live iteration, so to disable them and stop transporting empty sections there is a new savevm handler. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Message-id: 20180313180320.339796-10-vsementsov@virtuozzo.com
2018-03-13migration/qemu-file: add qemu_put_counted_string()Vladimir Sementsov-Ogievskiy
Add function opposite to qemu_get_counted_string. qemu_put_counted_string puts one-byte length of the string (string should not be longer than 255 characters), and then it puts the string, without last zero byte. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Message-id: 20180313180320.339796-9-vsementsov@virtuozzo.com
2018-03-13migration: include migrate_dirty_bitmaps in migrate_postcopyVladimir Sementsov-Ogievskiy
Enable postcopy if dirty bitmap migration is enabled. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Message-id: 20180313180320.339796-8-vsementsov@virtuozzo.com
2018-03-13qapi: add dirty-bitmaps migration capabilityVladimir Sementsov-Ogievskiy
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Message-id: 20180313180320.339796-7-vsementsov@virtuozzo.com
2018-03-13migration: introduce postcopy-only pendingVladimir Sementsov-Ogievskiy
There would be savevm states (dirty-bitmap) which can migrate only in postcopy stage. The corresponding pending is introduced here. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com> Message-id: 20180313180320.339796-6-vsementsov@virtuozzo.com
2018-03-13dirty-bitmap: add locked stateVladimir Sementsov-Ogievskiy
Add special state, when qmp operations on the bitmap are disabled. It is needed during bitmap migration. "Frozen" state is not appropriate here, because it looks like bitmap is unchanged. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-id: 20180207155837.92351-5-vsementsov@virtuozzo.com Signed-off-by: John Snow <jsnow@redhat.com>