aboutsummaryrefslogtreecommitdiff
path: root/python
AgeCommit message (Collapse)Author
2021-06-01python: add pylint import exceptionsJohn Snow
Pylint 2.5.x - 2.7.x have regressions that make import checking inconsistent, see: https://github.com/PyCQA/pylint/issues/3609 https://github.com/PyCQA/pylint/issues/3624 https://github.com/PyCQA/pylint/issues/3651 Pinning to 2.4.4 is worse, because it mandates versions of shared dependencies that are too old for features we want in isort and mypy. Oh well. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-16-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: Add pipenv supportJohn Snow
pipenv is a tool used for managing virtual environments with pinned, explicit dependencies. It is used for precisely recreating python virtual environments. pipenv uses two files to do this: (1) Pipfile, which is similar in purpose and scope to what setup.cfg lists. It specifies the requisite minimum to get a functional environment for using this package. (2) Pipfile.lock, which is similar in purpose to `pip freeze > requirements.txt`. It specifies a canonical virtual environment used for deployment or testing. This ensures that all users have repeatable results. The primary benefit of using this tool is to ensure *rock solid* repeatable CI results with a known set of packages. Although I endeavor to support as many versions as I can, the fluid nature of the Python toolchain often means tailoring code for fairly specific versions. Note that pipenv is *not* required to install or use this module; this is purely for the sake of repeatable testing by CI or developers. Here, a "blank" pipfile is added with no dependencies, but specifies Python 3.6 for the virtual environment. Pipfile will specify our version minimums, while Pipfile.lock specifies an exact loadout of packages that were known to operate correctly. This latter file provides the real value for easy setup of container images and CI environments. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-15-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add MANIFEST.inJohn Snow
When creating a source or binary distribution via 'python3 setup.py <sdist|bdist>', the VERSION and PACKAGE.rst files aren't bundled by default. Create a MANIFEST.in file that instructs the build tools to include these so that installation from these files won't fail. This is required by 'tox', as well as by the tooling needed to upload packages to PyPI. Exclude the 'README.rst' file -- that's intended as a guidebook to our source tree, not a file that needs to be distributed. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-14-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add directory structure README.rst filesJohn Snow
Add short readmes to python/, python/qemu/, python/qemu/machine, python/qemu/qmp, and python/qemu/utils that explain the directory hierarchy. These readmes are visible when browsing the source on e.g. gitlab/github and are designed to help new developers/users quickly make sense of the source tree. They are not designed for inclusion in a published manual. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-13-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add VERSION fileJohn Snow
Python infrastructure as it exists today is not capable reliably of single-sourcing a package version from a parent directory. The authors of pip are working to correct this, but as of today this is not possible. The problem is that when using pip to build and install a python package, it copies files over to a temporary directory and performs its build there. This loses access to any information in the parent directory, including git itself. Further, Python versions have a standard (PEP 440) that may or may not follow QEMU's versioning. In general, it does; but naturally QEMU does not follow PEP 440. To avoid any automatically-generated conflict, a manual version file is preferred. I am proposing: - Python tooling follows the QEMU version, indirectly, but with a major version of 0 to indicate that the API is not expected to be stable. This would mean version 0.5.2.0, 0.5.1.1, 0.5.3.0, etc. - In the event that a Python package needs to be updated independently of the QEMU version, a pre-release alpha version should be preferred, but *only* after inclusion to the qemu development or stable branches. e.g. 0.5.2.0a1, 0.5.2.0a2, and so on should be preferred prior to 5.2.0's release. - The Python core tooling makes absolutely no version compatibility checks or constraints. It *may* work with releases of QEMU from the past or future, but it is not required to. i.e., "qemu.machine" will, for now, remain in lock-step with QEMU. - We reserve the right to split the qemu package into independently versioned subpackages at a later date. This might allow for us to begin versioning QMP independently from QEMU at a later date, if we so choose. Implement this versioning scheme by adding a VERSION file and setting it to 0.6.0.0a1. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-12-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add qemu package installerJohn Snow
Add setup.cfg and setup.py, necessary for installing a package via pip. Add a ReST document (PACKAGE.rst) explaining the basics of what this package is for and who to contact for more information. This document will be used as the landing page for the package on PyPI. List the subpackages we intend to package by name instead of using find_namespace because find_namespace will naively also packages tests, things it finds in the dist/ folder, etc. I could not figure out how to modify this behavior; adding allow/deny lists to setuptools kept changing the packaged hierarchy. This works, roll with it. I am not yet using a pyproject.toml style package manifest, because "editable" installs are not defined (yet?) by PEP-517/518. I consider editable installs crucial for development, though they have (apparently) always been somewhat poorly defined. Pip now (19.2 and later) now supports editable installs for projects using pyproject.toml manifests, but might require the use of the --no-use-pep517 flag, which somewhat defeats the point. Full support for setup.py-less editable installs was not introduced until pip 21.1.1: https://github.com/pypa/pip/pull/9547/commits/7a95720e796a5e56481c1cc20b6ce6249c50f357 For now, while the dust settles, stick with the de-facto setup.py/setup.cfg combination supported by setuptools. It will be worth re-evaluating this point again in the future when our supported build platforms all ship a fairly modern pip. Additional reading on this matter: https://github.com/pypa/packaging-problems/issues/256 https://github.com/pypa/pip/issues/6334 https://github.com/pypa/pip/issues/6375 https://github.com/pypa/pip/issues/6434 https://github.com/pypa/pip/issues/6438 Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-11-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: create qemu packagesJohn Snow
move python/qemu/*.py to python/qemu/[machine, qmp, utils]/*.py and update import directives across the tree. This is done to create a PEP420 namespace package, in which we may create subpackages. To do this, the namespace directory ("qemu") should not have any modules in it. Those files will go into new 'machine', 'qmp' and 'utils' subpackages instead. Implement machine/__init__.py making the top-level classes and functions from its various modules available directly inside the package. Change qmp.py to qmp/__init__.py similarly, such that all of the useful QMP library classes are available directly from "qemu.qmp" instead of "qemu.qmp.qmp". Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-10-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/machine: Trim line length to below 80 charsJohn Snow
One more little delinting fix that snuck in. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-8-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/machine: disable warning for Popen in _launch()John Snow
We handle this resource rather meticulously in shutdown/kill/wait/__exit__ et al, through the laborious mechanisms in _do_shutdown(). Quiet this pylint warning here. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-7-jsnow@redhat.com Message-id: 20210517184808.3562549-7-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/machine: Disable pylint warning for open() in _pre_launchJohn Snow
Shift the open() call later so that the pylint pragma applies *only* to that one open() call. Add a note that suggests why this is safe: the resource is unconditionally cleaned up in _post_shutdown(). _post_shutdown is called after failed launches (see launch()), and unconditionally after every call to shutdown(), and therefore also on __exit__. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-6-jsnow@redhat.com Message-id: 20210517184808.3562549-6-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/console_socket: Add a pylint ignoreJohn Snow
We manage cleaning up this resource ourselves. Pylint should shush. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-5-jsnow@redhat.com Message-id: 20210517184808.3562549-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/machine: use subprocess.run instead of subprocess.PopenJohn Snow
use run() instead of Popen() -- to assert to pylint that we are not forgetting to close a long-running program. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-4-jsnow@redhat.com Message-id: 20210517184808.3562549-4-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/machine: use subprocess.DEVNULL instead of open(os.path.devnull)John Snow
One less file resource to manage, and it helps quiet some pylint >= 2.8.0 warnings about not using a with-context manager for the open call. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-3-jsnow@redhat.com Message-id: 20210517184808.3562549-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/console_socket: avoid one-letter variableJohn Snow
Fixes pylint warnings. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20210527211715.394144-2-jsnow@redhat.com Message-id: 20210517184808.3562549-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01Python: add utility function for retrieving port redirectionCleber Rosa
Slightly different versions for the same utility code are currently present on different locations. This unifies them all, giving preference to the version from virtiofs_submounts.py, because of the last tweaks added to it. While at it, this adds a "qemu.utils" module to host the utility function and a test. Signed-off-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Eric Auger <eric.auger@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20210412044644.55083-4-crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com> [Squashed in below fix. --js] Signed-off-by: John Snow <jsnow@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210601154546.130870-2-crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01Python: expose QEMUMachine's temporary directoryCleber Rosa
Each instance of qemu.machine.QEMUMachine currently has a "test directory", which may not have any relation to a "test", and it's really a temporary directory. Users instantiating the QEMUMachine class will be able to set the location of the directory that will *contain* the QEMUMachine unique temporary directory, so that parameter name has been changed from test_dir to base_temp_dir. A property has been added to allow users to access it without using private attributes, and with that, the directory is created on first use of the property. Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210211220146.2525771-3-crosa@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-02-15Python: close the log file kept by QEMUMachine before reading itCleber Rosa
Closing a file that is open for writing, and then reading from it sounds like a better idea than the opposite, given that the content will be flushed. Reference: https://docs.python.org/3/library/io.html#io.IOBase.close Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210211220146.2525771-2-crosa@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-01-02python: add __repr__ to ConsoleSocket to aid debuggingAlex Bennée
While attempting to debug some console weirdness I thought it would be worth making it easier to see what it had inside. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-Id: <20201210190417.31673-6-alex.bennee@linaro.org>
2020-12-10treewide: do not use short-form boolean optionsPaolo Bonzini
They are going to be deprecated, avoid warnings on stdout while the tests run. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-11-23tests: add prefixes to the bare mkdtemp callsAlex Bennée
The first step to debug a thing is to know what created the thing in the first place. Add some prefixes so random tmpdir's have something grep in the code. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-Id: <20201117173635.29101-3-alex.bennee@linaro.org>
2020-10-20python/qemu/qmp.py: Fix settimeout operationJohn Snow
We enabled callers to interface directly with settimeout, but this reacts poorly with blocking/nonblocking operation; as they are using the same internal mechanism. 1. Whenever we change the blocking mechanism temporarily, always set it back to what it was afterwards. 2. Disallow callers from setting a timeout of "0", which means Non-blocking mode. This is going to create more weird problems than anybody wants, so just forbid it. I opt not to coerce '0' to 'None' to maintain the principal of least surprise in mirroring the semantics of Python's interface. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20201009175123.249009-4-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu/qmp.py: re-raise OSError when encounteredJohn Snow
Nested if conditions don't change when the exception block fires; we need to explicitly re-raise the error if we didn't intend to capture and suppress it. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20201009175123.249009-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python: add mypy configJohn Snow
Formalize the options used for checking the python library. You can run mypy from the directory that mypy.ini is in by typing `mypy qemu/`. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20201009175123.249009-2-jsnow@redhat.com [Edit: Added newline; thanks Bin Meng --js] Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu/qmp.py: Preserve error context on re-raiseJohn Snow
Use the "from ..." phrasing when re-raising errors to preserve their initial context, to help aid debugging when things go wrong. This also silences a pylint 2.6.0+ error. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-18-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu/console_socket.py: avoid encoding to/from stringJohn Snow
We can work directly in bytes instead of translating back and forth to string, which removes the question of which encodings to use. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-17-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu/console_socket.py: Add type hint annotationsJohn Snow
Finish the typing of console_socket.py with annotations and no code changes. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-16-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu/console_socket.py: Clarify type of drain_threadJohn Snow
Mypy needs just a little help to guess the type here. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-15-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu/console_socket.py: fix typing of settimeoutJohn Snow
The types and names of the parameters must match the socket.socket interface. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-14-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu/console_socket.py: Correct type of recv()John Snow
The type and parameter names of recv() should match socket.socket(). OK, easy enough, but in the cases we don't pass straight through to the real socket implementation, we probably can't accept such flags. OK, for now, assert that we don't receive flags in such cases. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-13-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu: Add mypy type annotationsJohn Snow
These should all be purely annotations with no changes in behavior at all. You need to be in the python folder, but you should be able to confirm that these annotations are correct (or at least self-consistent) by running `mypy --strict qemu`. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-12-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu: make 'args' style arguments immutableJohn Snow
These arguments don't need to be mutable and aren't really used as such. Clarify their types as immutable and adjust code to match where necessary. In general, It's probably best not to accept a user-defined mutable object and store it as internal object state unless there's a strong justification for doing so. Instead, try to use generic types as input with empty tuples as the default, and coerce to list where necessary. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20201006235817.3280413-10-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/machine.py: fix _popen accessJohn Snow
As always, Optional[T] causes problems with unchecked access. Add a helper that asserts the pipe is present before we attempt to talk with it. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-9-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/machine.py: Add _qmp access shimJohn Snow
Like many other Optional[] types, it's not always a given that this object will be set. Wrap it in a type-shim that raises a meaningful error and will always return a concrete type. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-8-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/machine.py: use qmp.commandJohn Snow
machine.py and qmp.py both do the same thing here; refactor machine.py to use qmp.py's functionality more directly. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20201006235817.3280413-7-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/machine.py: Handle None events in events_waitJohn Snow
If the timeout is 0, we can get None back. Handle this explicitly. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-6-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/machine.py: Don't modify state in _base_args()John Snow
Don't append to the _remove_files list during _base_args; instead do so during _launch. Rework _base_args as a @property to help facilitate this impression. This has the additional benefit of making the type of _console_address easier to analyze statically. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/machine.py: reorder __init__John Snow
Put the init arg handling all at the top, and mostly in order (deviating when one is dependent on another), and put what is effectively runtime state declaration at the bottom. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-4-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/machine.py: Fix monitor address typingJohn Snow
Prior to this, it's difficult for mypy to intuit what the concrete type of the monitor address is; it has difficulty inferring the type across two variables. Create _monitor_address as a property that always returns a valid address to simplify static type analysis. To preserve our ability to clean up, use a simple boolean to indicate whether or not we should try to clean up the sock file after execution. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-10-20python/qemu: use isort to lay out importsJohn Snow
Borrowed from the QAPI cleanup series, use the same configuration to standardize the way we write and sort imports. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-id: 20201006235817.3280413-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2020-07-27python/qemu: Change ConsoleSocket to optionally drain socket.Robert Foley
The primary purpose of this change is to clean up machine.py's console_socket property to return a single type, a ConsoleSocket. ConsoleSocket now derives from a socket, which means that in the default case (of not draining), machine.py will see the same behavior as it did prior to ConsoleSocket. Signed-off-by: Robert Foley <robert.foley@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200717203041.9867-3-robert.foley@linaro.org> Message-Id: <20200724064509.331-16-alex.bennee@linaro.org>
2020-07-27python/qemu: Cleanup changes to ConsoleSocketRobert Foley
The changes to console_socket.py and machine.py are to cleanup for pylint and flake8. Signed-off-by: Robert Foley <robert.foley@linaro.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20200717203041.9867-2-robert.foley@linaro.org> Message-Id: <20200724064509.331-15-alex.bennee@linaro.org>
2020-07-25python/machine: Change default timeout to 30 secondsJohn Snow
3 seconds is too short for some tests running inside busy VMs. Build it out to a rather generous 30 seconds to find out conclusively if there are more severe problems in the merge/CI tests. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Message-id: 20200720160252.104139-2-jsnow@redhat.com Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-07-14python/qmp.py: add QMPProtocolErrorJohn Snow
In the case that we receive a reply but are unable to understand it, use this exception name to indicate that case. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20200710052220.3306-7-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-07-14python/qmp.py: add casts to JSON deserializationJohn Snow
mypy and python type hints are not powerful enough to properly describe JSON messages in Python 3.6. The best we can do, generally, is describe them as Dict[str, Any]. Add casts to coerce this type for static analysis; but do NOT enforce this type at runtime in any way. Note: Python 3.8 adds a TypedDict construct which allows for the description of more arbitrary Dictionary shapes. There is a third-party module, "Pydantic", which is compatible with 3.6 that can be used instead of the JSON library that parses JSON messages to fully-typed Python objects, and may be preferable in some cases. (That is well beyond the scope of this commit or series.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20200710052220.3306-6-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-07-14python/qmp.py: Do not return None from cmd_objJohn Snow
This makes typing the qmp library difficult, as it necessitates wrapping Optional[] around the type for every return type up the stack. At some point, it becomes difficult to discern or remember why it's None instead of the expected object. Use the python exception system to tell us exactly why we didn't get an object. Remove this special-cased return. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20200710052220.3306-5-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-07-14python/qmp.py: re-absorb MonitorResponseErrorJohn Snow
When I initially split this out, I considered this more of a machine error than a QMP protocol error, but I think that's misguided. Move this back to qmp.py and name it QMPResponseError. Convert qmp.command() to use this exception type. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20200710052220.3306-4-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-07-14python/qmp.py: Define common typesJohn Snow
Define some common types that we'll need to annotate a lot of other functions going forward. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-Id: <20200710052220.3306-2-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-07-14python/machine.py: change default wait timeout to 3 secondsJohn Snow
Machine.wait() does not appear to be used except in the acceptance tests, and an infinite timeout by default in a test suite is not the most helpful. Change it to 3 seconds, like the default shutdown timeout. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20200710050649.32434-13-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-07-14python/machine.py: re-add sigkill warning suppressionJohn Snow
If the user kills QEMU on purpose, we don't need to warn them about that having happened: they know already. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20200710050649.32434-12-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-07-14python/machine.py: split shutdown into hard and soft flavorsJohn Snow
This is done primarily to avoid the 'bare except' pattern, which suppresses all exceptions during shutdown and can obscure errors. Replace this with a pattern that isolates the different kind of shutdown paradigms (_hard_shutdown and _soft_shutdown), and a new fallback shutdown handler (_do_shutdown) that gracefully attempts one before the other. This split now also ensures that no matter what happens, _post_shutdown() is always invoked. shutdown() changes in behavior such that if it attempts to do a graceful shutdown and is unable to, it will now always raise an exception to indicate this. This can be avoided by the test writer in three ways: 1. If the VM is expected to have already exited or is in the process of exiting, wait() can be used instead of shutdown() to clean up resources instead. This helps avoid race conditions in shutdown. 2. If a test writer is expecting graceful shutdown to fail, shutdown should be called in a try...except block. 3. If the test writer has no interest in performing a graceful shutdown at all, kill() can be used instead. Handling shutdown in this way makes it much more explicit which type of shutdown we want and allows the library to report problems with this process. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20200710050649.32434-11-jsnow@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-22 05:36:10 +0000