aboutsummaryrefslogtreecommitdiff
path: root/python
AgeCommit message (Collapse)Author
2021-09-01python:QEMUMachine: template typing for self returning methodsVladimir Sementsov-Ogievskiy
mypy thinks that return value of these methods in subclusses is QEMUMachine, which is wrong. So, make typing smarter. Suggested-by: John Snow <jsnow@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Message-Id: <20210824083856.17408-26-vsementsov@virtuozzo.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-01python/qemu/machine: QEMUMachine: improve qmp() methodVladimir Sementsov-Ogievskiy
We often call qmp() with unpacking dict, like qmp('foo', **{...}). mypy don't really like it, it thinks that passed unpacked dict is a positional argument and complains that it type should be bool (because second argument of qmp() is conv_keys: bool). Allow passing dict directly, simplifying interface, and giving a way to satisfy mypy. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-Id: <20210824083856.17408-25-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-01python/qemu/machine.py: refactor _qemu_args()Vladimir Sementsov-Ogievskiy
- use shorter construction - don't create new dict if not needed - drop extra unpacking key-val arguments - drop extra default values Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Message-Id: <20210824083856.17408-24-vsementsov@virtuozzo.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-01qemu-iotests: add option to show qemu binary logs on stdoutEmanuele Giuseppe Esposito
Using the flag -p, allow the qemu binary to print to stdout. Also create the common function _close_qemu_log_file() to avoid accessing machine.py private fields directly and have duplicate code. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210809090114.64834-16-eesposit@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-01python: qemu: pass the wrapper field from QEMUQtestmachine to QEMUMachineEmanuele Giuseppe Esposito
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Acked-by: John Snow <jsnow@redhat.com> Message-Id: <20210809090114.64834-4-eesposit@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-01python: Reduce strictness of pylint's duplicate-code checkJohn Snow
Pylint prior to 2.8.3 (We pin at >= 2.8.0) includes function and method signatures as part of its duplicate checking algorithm. This check does not listen to pragmas, so the only way to disable it is to turn it off completely or increase the minimum duplicate lines so that it doesn't trigger for functions with long, multi-line signatures. When we decide to upgrade to pylint 2.8.3 or greater, we will be able to use 'ignore-signatures = true' to the config instead. I'd prefer not to keep us on the very bleeding edge of pylint if I can help it -- 2.8.3 came out only three days ago at time of writing. See: https://github.com/PyCQA/pylint/pull/4474 Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Acked-by: John Snow <jsnow@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210809090114.64834-3-eesposit@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-01python: qemu: add timer parameter for qmp.accept socketEmanuele Giuseppe Esposito
Also add a new _qmp_timer field to the QEMUMachine class. Let's change the default socket timeout to None, so that if a subclass needs to add a timer, it can be done by modifying this private field. At the same time, restore the timer to be 15 seconds in iotests.py, to give an upper bound to the QMP monitor test command execution. Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: John Snow <jsnow@redhat.com> Acked-by: John Snow <jsnow@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20210809090114.64834-2-eesposit@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-07-13python: Configure tox to skip missing interpretersWainer dos Santos Moschetta
Currently tox tests against the installed interpreters, however if any supported interpreter is absent then it will return fail. It seems not reasonable to expect developers to have all supported interpreters installed on their systems. Luckily tox can be configured to skip missing interpreters. This changed the tox setup so that missing interpreters are skipped by default. On the CI, however, we still want to enforce it tests against all supported. This way on CI the --skip-missing-interpreters=false option is passed to tox. Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20210630184546.456582-1-wainersm@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-07-13python/qemu: Add args property to the QEMUMachine classWainer dos Santos Moschetta
This added the args property to QEMUMachine so that users of the class can access and handle the list of arguments to be given to the QEMU binary. Reviewed-by: Cleber Rosa <crosa@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-Id: <20210430133414.39905-6-wainersm@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-07-13Acceptance Tests: distinguish between temp and logs dirCleber Rosa
Logs can be very important to debug issues, and currently QEMUMachine instances will remove logs that are created under the temporary directories. With this change, the stdout and stderr generated by the QEMU process started by QEMUMachine will always be kept along the test results directory. Signed-off-by: Cleber Rosa <crosa@redhat.com> Message-Id: <20210211220146.2525771-6-crosa@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-06-30python: Fix broken ReST docstringsJohn Snow
This patch *doesn't* update all of the docstring standards across the QEMU package directory to make our docstring usage consistent. It *doesn't* fix the formatting to make it look pretty or reasonable in generated output. It *does* fix a few small instances where Sphinx would emit a build warning because of malformed ReST -- If we built our Python docs with Sphinx. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-id: 20210629214323.1329806-16-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-30python: remove auto-generated pyproject.toml fileJohn Snow
For reasons that at-present escape me, pipenv insists on creating a stub pyproject.toml file. This file is a nuisance, because its mere presence changes the behavior of various tools. For instance, this stub file will cause "pip install --user -e ." to fail in spectacular fashion with misleading errors. "pip install -e ." works okay, but for some reason pip does not support editable installs to the user directory when using PEP517. References: https://github.com/pypa/pip/pull/9990 https://github.com/pypa/pip/issues/7953 As outlined in ea1213b7ccc, it is still too early for us to consider moving to a PEP-517 exclusive package. We must support older distributions, so squash the annoyance for now. (Python 3.6 shipped Dec 2016, PEP517 support showed up in pip sometime in 2019 or so.) Add 'pyproject.toml' to the 'make clean' target, and also delete it after every pipenv invocation issued by the Makefile. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-id: 20210629214323.1329806-15-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-30python: Update help text on 'make clean', 'make distclean'John Snow
Update for visual parity with all the remaining targets. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-id: 20210629214323.1329806-14-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-30python: Update help text on 'make check', 'make develop'John Snow
Update for visual parity with the other targets. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-id: 20210629214323.1329806-13-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-30python: add 'make check-dev' invocationJohn Snow
This is a *third* way to run the Python tests. Unlike the first two (check-pipenv, check-tox), this version does not require any specific interpreter version -- making it a lot easier to tell people to run it as a quick smoketest prior to submission to GitLab CI. Summary: Checked via GitLab CI: - check-pipenv: tests our oldest python & dependencies - check-tox: tests newest dependencies on all non-EOL python versions Executed only incidentally: - check-dev: tests newest dependencies on whichever python version ('make check' does not set up any environment at all, it just runs the tests in your current environment. All four invocations perform the exact same tests, just in different execution environments.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-id: 20210629214323.1329806-12-jsnow@redhat.com [Maintainer edit: added .dev-venv/ to .gitignore. --js] Acked-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Acked-by: Willian Rampazzo <willianr@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-30python: only check qemu/ subdir with flake8John Snow
flake8 is a little eager to check everything it can. Limit it to checking inside the qemu namespace directory only. Update setup.cfg now that the exclude patterns are no longer necessary. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-id: 20210629214323.1329806-11-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-30python: Fix .PHONY Make specifiersJohn Snow
I missed the 'check-tox' target. Add that, but split the large .PHONY specifier at the top into its component pieces and move them near the targets they describe so that they're much harder to forget to update. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-id: 20210629214323.1329806-10-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-30python: update help text for check-toxJohn Snow
Move it up near the check-pipenv help text, and update it to suggest parity. (At the time I first added it, I wasn't sure if I would be keeping it, but I've come to appreciate it as it has actually helped uncover bugs I would not have noticed without it. It should stay.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-id: 20210629214323.1329806-9-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-30python: rename 'venv-check' target to 'check-pipenv'John Snow
Well, Cleber was right, this is a better name. In preparation for adding a different kind of virtual environment check (One that simply uses whichever version of Python you happen to have), rename this test 'check-pipenv' so that it matches the CI job 'check-python-pipenv'. Remove the "If you don't know which test to run" hint, because it's not actually likely you have Python 3.6 installed to be able to run the test. It's still the test I'd most prefer you to run, but it's not the test you are most likely to be able to run. Rename the 'venv' target to 'pipenv' as well, and move the more pertinent help text under the 'check-pipenv' target. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-id: 20210629214323.1329806-8-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-30python: Add no-install usage instructionsJohn Snow
It's not encouraged, but it's legitimate to want to know how to do. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-id: 20210629214323.1329806-7-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-30python: README.rst touchupsJohn Snow
Clarifying a few points; removing the reference to 'setuptools' because it isn't referenced anywhere else in this document and doesn't really provide any useful information to a Python newcomer. Adjusting the language elsewhere to be less ambiguous and have fewer run-on sentences. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-id: 20210629214323.1329806-6-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-30python: Re-lock pipenv at *oldest* supported versionsJohn Snow
tox is already testing the most recent versions. Let's use pipenv to test the oldest versions we claim to support. This matches the stylistic choice to have pipenv always test our oldest supported Python version, 3.6. The effect of this is that the python-check-pipenv CI job on gitlab will now test against much older versions of these linters, which will help highlight incompatible changes that might otherwise go unnoticed. Update instructions for adding and bumping versions in setup.cfg. The reason for deleting the line that gets added to Pipfile is largely just to avoid having the version minimums specified in multiple places in config checked into the tree. (This patch was written by deleting Pipfile and Pipfile.lock, then explicitly installing each dependency manually at a specific version. Then, I restored the prior Pipfile and re-ran `pipenv lock --dev --keep-outdated` to re-add the qemu dependency back to the pipenv environment while keeping the "old" packages. It's annoying, yes, but I think the improvement to test coverage is worthwhile.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-id: 20210629214323.1329806-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-30python: Remove global pylint suppressionsJohn Snow
These suppressions only apply to a small handful of places. Instead of disabling them globally, disable them just in the cases where we need. The design of the machine class grew quite organically with tons of constructor and class instance variables -- there's little chance of meaningfully refactoring it in the near term, so just suppress the warnings for that class. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-id: 20210629214323.1329806-4-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-30python: expose typing information via PEP 561John Snow
https://www.python.org/dev/peps/pep-0561/#specification Create 'py.typed' files in each subpackage that indicate to mypy that this is a typed module, so that users of any of these packages can use mypy to check their code as well. Note: Theoretically it's possible to ditch MANIFEST.in in favor of using package_data in setup.cfg, but I genuinely could not figure out how to get it to include things from the *source root* into the *package root*; only how to include things from each subpackage. I tried! Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Message-id: 20210629214323.1329806-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-30python/qom: Do not use 'err' name at module scopeJohn Snow
Pylint updated to 2.9.0 upstream, adding new warnings for things that re-use the 'err' variable. Luckily, this only breaks the python-check-tox job, which is allowed to fail as a warning. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com> Reviewed-by: Willian Rampazzo <willianr@redhat.com> Message-id: 20210629214323.1329806-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18python: add qmp-shell entry pointJohn Snow
now 'qmp-shell' should be available from the command line when installing the python package. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20210607200649.1840382-42-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18scripts/qmp-shell: move to python/qemu/qmp/qmp_shell.pyJohn Snow
The script will be unavailable for a commit or two, which will help preserve development history attached to the new file. A forwarder will be added shortly afterwards. With qmp_shell in the python qemu.qmp package, now it is fully type checked, linted, etc. via the Python CI. It will be quite a bit harder to accidentally break it again in the future. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20210607200649.1840382-41-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18python/qmp: return generic type from context managerJohn Snow
__enter__ can be invoked from a subclass, so it needs a more flexible type. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20210607200649.1840382-31-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18python/qmp: add QMPObject type aliasJohn Snow
This is meant to represent any generic object seen in a QMPMessage, not just the root object itself. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20210607200649.1840382-27-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18python/qemu-ga-client: add entry pointJohn Snow
Remove the shebang, and add a package-defined entry point instead. Now, it can be accessed using 'qemu-ga-client' from the command line after installing the package. The next commit adds a forwarder shim that allows the running of this script without needing to install the package again. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20210604155532.1499282-11-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18scripts/qemu-ga-client: move to python/qemu/qmp/qemu_ga_client.pyJohn Snow
The script itself will be unavailable for a few commits before being restored, with no way to run it right after this commit. This helps move git history into the new file. To prevent linter regressions, though, we do need to immediately touch up the filename to remove dashes (to make the module importable), and remove the executable bit. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20210604155532.1499282-10-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18python/qmp: Correct type of QMPReturnValueJohn Snow
It's only a Dict[str, Any] most of the time. It's not actually guaranteed to be anything in particular. Fix this type to be more accurate to the reality we live in. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20210604155532.1499282-8-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18python/qmp: add fuse command to 'qom' toolsJohn Snow
The 'fuse' command will be unavailable if 'fusepy' is not installed. It will simply not load and subsequently be unavailable as a subcommand. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20210603003719.1321369-20-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18scripts/qom-fuse: move to python/qemu/qmp/qom_fuse.pyJohn Snow
Move qom-fuse over to the python package now that it passes the linter. Update the import paradigms so that it continues to pass in the context of the Python package. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20210603003719.1321369-18-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18python: add optional FUSE dependenciesJohn Snow
In preparation for moving qom-fuse over to the python package, we need some new dependencies to support it. Add an optional 'fusepy' dependency that users of the package can opt into with e.g. "pip install qemu[fuse]" which installs the requirements necessary to obtain the additional functionality. Add the same fusepy dependency to the 'devel' extras group -- unfortunately I do not see a way for optional groups to imply other optional groups at present, so the dependency is repeated. The development group needs to include the full set of dependencies for the purpose of static analysis of all features offered by this library. Lastly, add the [fuse] extras group to tox's configuration as a workaround so that if a stale tox environment is found when running `make check-tox`, tox will know to rebuild its environments. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20210603003719.1321369-17-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18scripts/qom-fuse: add static type hintsJohn Snow
Because fusepy does not have type hints, add some targeted warning suppressions. Namely, we need to allow subclassing something of an unknown type (in qom_fuse.py), and we need to allow missing imports (recorded against fuse itself) because mypy will be unable to import fusepy (even when installed) as it has no types nor type stubs available. Note: Until now, it was possible to run invocations like 'mypy qemu/' from ./python and have that work. However, these targeted suppressions require that you run 'mypy -p qemu/' instead. The correct, canonical invocation is recorded in ./python/tests/mypy.sh and all of the various CI invocations always use this correct form. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20210603003719.1321369-16-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18python: Add 'fh' to known-good variable namesJohn Snow
fd and fh are fine: we often use these for "file descriptor" or "file handle" accordingly. It is rarely the case that you need to enforce a more semantically meaningful name beyond "This is the file we are using right now." While we're here: add comments for all of the non-standard pylint names. (And the underscore.) Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20210603003719.1321369-10-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18python/qmp: add qom script entry pointsJohn Snow
Add the 'qom', 'qom-set', 'qom-get', 'qom-list', and 'qom-tree' scripts to the qemu.qmp package. When you install this package, these scripts will become available on your command line. (e.g. when inside of a venv, `cd python && pip install .` will add 'qom', 'qom-set', etc to your $PATH.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20210603003719.1321369-6-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18python/qmp: Add qom script rewritesJohn Snow
Inspired by qom-set, qom-get, qom-tree and qom-list; combine all four of those scripts into a single script. A later addition of qom-fuse as an 'extension' necessitates that some common features are split out and shared between them. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20210603003719.1321369-5-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18python/qmp: add parse_address classmethodJohn Snow
This takes the place of qmp-shell's __get_address function. It also allows other utilities to share the same parser and syntax for specifying QMP locations. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20210603003719.1321369-4-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18python/qmp: Fix type of SocketAddrTJohn Snow
In porting the qom tools, qmp-shell, etc; it becomes evident that this type is wrong. This is an integer, not a string. We didn't catch this before because none of QEMUMonitorProtocol's *users* happen to be checked, and the internal logic of this class is otherwise self-consistent. Additionally, mypy was not introspecting into the socket() interface to realize we were passing a bad type for AF_INET. Fixed now. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20210603003719.1321369-3-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18python/pipenv: Update Pipfile.lockJohn Snow
In a previous commit, I added tox to the development requirements of the Python library. I never bothered to add them to the Pipfile, because they aren't needed there. Here, I sync it anyway in its own commit so that when we add new packages later that the diffstats will not confusingly appear to pull in lots of extra packages. Ideally I could tell Pipenv simply not to install these, but it doesn't seem to support that, exactly. The alternative is removing Tox from the development requires, which I'd rather not do. The other alternative is re-specifying all of the dependencies of setup.cfg in the Pipfile, which I'd also rather not do. Picking what feels least-worst here. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20210603003719.1321369-2-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add tox supportJohn Snow
This is intended to be a manually run, non-CI script. Use tox to test the linters against all python versions from 3.6 to 3.10. This will only work if you actually have those versions installed locally, but Fedora makes this easy: > sudo dnf install python3.6 python3.7 python3.8 python3.9 python3.10 Unlike the pipenv tests (make venv-check), this pulls "whichever" versions of the python packages, so they are unpinned and may break as time goes on. In the case that breakages are found, setup.cfg should be amended accordingly to avoid the bad dependant versions, or the code should be amended to work around the issue. With confidence that the tests pass on 3.6 through 3.10 inclusive, add the appropriate classifiers to setup.cfg to indicate which versions we claim to support. Tox 3.18.0 or above is required to use the 'allowlist_externals' option. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-31-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add .gitignoreJohn Snow
Ignore *Python* build and package output (build, dist, qemu.egg-info); these files are not created as part of a QEMU build. They are created by running the commands 'python3 setup.py <sdist|bdist>' when preparing tarballs to upload to e.g. PyPI. Ignore miscellaneous cached python confetti (mypy, pylint, et al) Ignore .idea (pycharm) .vscode, and .venv (pipenv et al). Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-30-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add Makefile for some common tasksJohn Snow
Add "make venv" to create the pipenv-managed virtual environment that contains our explicitly pinned dependencies. Add "make check" to run the python linters [in the host execution environment]. Add "make venv-check" which combines the above two: create/update the venv, then run the linters in that explicitly managed environment. Add "make develop" which canonizes the runes needed to get both the linting pre-requisites (the "[devel]" part), and the editable live-install (the "-e" part) of these python libraries. make clean: delete miscellaneous python packaging output possibly created by pipenv, pip, or other python packaging utilities make distclean: delete the above, the .venv, and the editable "qemu" package forwarder (qemu.egg-info) if there is one. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-29-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add avocado-framework and testsJohn Snow
Try using avocado to manage our various tests; even though right now they're only invoking shell scripts and not really running any python-native code. Create tests/, and add shell scripts which call out to mypy, flake8, pylint and isort to enforce the standards in this directory. Add avocado-framework to the setup.cfg development dependencies, and add avocado.cfg to store some preferences for how we'd like the test output to look. Finally, add avocado-framework to the Pipfile environment and lock the new dependencies. We are using avocado >= 87.0 here to take advantage of some features that Cleber has helpfully added to make the test output here *very* friendly and easy to read for developers that might chance upon the output in Gitlab CI. [Note: ALL of the dependencies get updated to the most modern versions that exist at the time of this writing. No way around it that I have seen. Not ideal, but so it goes.] Provided you have the right development dependencies (mypy, flake8, isort, pylint, and now avocado-framework) You should be able to run "avocado --config avocado.cfg run tests/" from the python folder to run all of these linters with the correct arguments. (A forthcoming commit adds the much easier 'make check'.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-28-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: add devel package requirements to setuptoolsJohn Snow
setuptools doesn't have a formal understanding of development requires, but it has an optional feataures section. Fine; add a "devel" feature and add the requirements to it. To avoid duplication, we can modify pipenv to install qemu[devel] instead. This enables us to run invocations like "pip install -e .[devel]" and test the package on bleeding-edge packages beyond those specified in Pipfile.lock. Importantly, this also allows us to install the qemu development packages in a non-networked mode: `pip3 install --no-index -e .[devel]` will now fail if the proper development dependencies are not already met. This can be useful for automated build scripts where fetching network packages may be undesirable. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-27-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/qemu: add qemu package itself to pipenvJohn Snow
This adds the python qemu packages themselves to the pipenv manifest. 'pipenv sync' will create a virtual environment sufficient to use the SDK. 'pipenv sync --dev' will create a virtual environment sufficient to use and test the SDK (with pylint, mypy, isort, flake8, etc.) The qemu packages are installed in 'editable' mode; all changes made to the python package inside the git tree will be reflected in the installed package without reinstallation. This includes changes made via git pull and so on. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Tested-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-26-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python/qemu: add isort to pipenvJohn Snow
isort 5.0.0 through 5.0.4 has a bug that causes it to misinterpret certain "from ..." clauses that are not related to imports. isort < 5.1.1 has a bug where it does not handle comments near import statements correctly. Require 5.1.2 or greater. isort can be run (in "check" mode) with 'isort -c qemu' from the python root. isort can also be used to fix/rewrite import order automatically by using 'isort qemu'. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-25-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01python: move .isort.cfg into setup.cfgJohn Snow
Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Cleber Rosa <crosa@redhat.com> Message-id: 20210527211715.394144-24-jsnow@redhat.com Signed-off-by: John Snow <jsnow@redhat.com>
generated by cgit v1.2.3 (git 2.26.2) at 2024-12-28 23:11:14 +0000