aboutsummaryrefslogtreecommitdiff
path: root/scripts/qapi
AgeCommit message (Collapse)Author
2018-08-28qapi: Add comments to aid debugging generated introspectionEric Blake
We consciously chose in commit 1a9a507b to hide QAPI type names from the introspection output on the wire, but added a command line option -u to unmask the type name when doing a debug build. The unmask option still remains useful to some other forms of automated analysis, so it will not be removed; however, when it is not in use, the generated .c file can be hard to read. At the time when we first introduced masking, the generated file consisted only of a monolithic C string, so there was no clean way to inject any comments. Later, in commit 7d0f982b, we switched the generation to output a QLit object, in part to make it easier for future addition of conditional compilation. In fact, commit d626b6c1 took advantage of this by passing a tuple instead of a bare object for encoding the output of conditionals. By extending that tuple, we can now interject strategic comments. For now, type name debug aid comments are only output once per meta-type, rather than at all uses of the number used to encode the type within the introspection data. But this is still a lot more convenient than having to regenerate the file with the unmask operation temporarily turned on - merely search the generated file for '"NNN" =' to learn the corresponding source name and associated definition of type NNN. The generated qapi-introspect.c changes only with the addition of comments, such as: | @@ -14755,6 +15240,7 @@ | { "name", QLIT_QSTR("[485]"), }, | {} | })), | + /* "485" = QCryptoBlockInfoLUKSSlot */ | QLIT_QDICT(((QLitDictEntry[]) { | { "members", QLIT_QLIST(((QLitObject[]) { | QLIT_QDICT(((QLitDictEntry[]) { Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20180827213943.33524-3-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Rebased, update to qapi-code-gen.txt corrected] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-08-28qapi: Minor introspect.py cleanupsEric Blake
Commit 7d0f982b changed generated introspection output to no longer produce long lines in the generated .c file, but failed to adjust comments to match. Add some clarity that the shorter length that matters most is the overall QMP response on the wire. Commit 25b1ef31 triggers a pep8 formatting nit. Signed-off-by: Eric Blake <eblake@redhat.com> Message-Id: <20180827213943.33524-2-eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-08-28qapi: Emit a blank line before dummy declarationMarkus Armbruster
We emit a dummy variable in each .c file "to shut up OSX toolchain warnings about empty .o files" (commit 252dc3105fc). Separate it from the code preceding it (if any) with a blank line. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180828120736.32323-2-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2018-08-28qapi: Drop qapi_event_send_FOO()'s Error ** argumentPeter Xu
The generated qapi_event_send_FOO() take an Error ** argument. They can't actually fail, because all they do with the argument is passing it to functions that can't fail: the QObject output visitor, and the @qmp_emit callback, which is either monitor_qapi_event_queue() or event_test_emit(). Drop the argument, and pass &error_abort to the QObject output visitor and @qmp_emit instead. Suggested-by: Eric Blake <eblake@redhat.com> Suggested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Peter Xu <peterx@redhat.com> Message-Id: <20180815133747.25032-4-peterx@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Commit message rewritten, update to qapi-code-gen.txt corrected] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-08-28qapi: Fix build_params() for empty parameter listMarkus Armbruster
build_params() returns '' instead of 'void' when there are no parameters. Can't happen now, but the next commit will change that. Signed-off-by: Markus Armbruster <armbru@redhat.com> [peterx: compose the patch from email replies] Signed-off-by: Peter Xu <peterx@redhat.com> Message-Id: <20180815133747.25032-3-peterx@redhat.com>
2018-08-15qapi: Fix some pycodestyle-3 complaintsMarkus Armbruster
Fix the following issues: common.py:873:13: E129 visually indented line with same indent as next logical line common.py:1766:5: E741 ambiguous variable name 'l' common.py:1784:1: E305 expected 2 blank lines after class or function definition, found 1 common.py:1833:1: E305 expected 2 blank lines after class or function definition, found 1 common.py:1843:1: E305 expected 2 blank lines after class or function definition, found 1 visit.py:181:18: E127 continuation line over-indented for visual indent Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Message-Id: <20180621083551.775-1-armbru@redhat.com> [Fixup squashed in:] Message-ID: <871sd0nzw9.fsf@dusky.pond.sub.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-07-23qapi: Make 'allow-oob' optional in SchemaInfoCommandMarkus Armbruster
Making 'allow-oob' optional in SchemaInfoCommand permits omitting it in the common case. Shrinks query-qmp-schema's output from 122.1KiB to 118.6KiB for me. Note that out-of-band execution is still experimental (you have to configure the monitor with x-oob=on to use it). Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180718090557.17248-1-armbru@redhat.com> Reviewed-by: Peter Xu <peterx@redhat.com>
2018-07-16qapi: Do not expose "allow-preconfig" in query-qmp-schemaMarkus Armbruster
According to commit 047f7038f58, option --preconfig [...] allows pausing QEMU in the new RUN_STATE_PRECONFIG state, allowing the configuration of QEMU from QMP before the machine jumps into board initialization code of machine_run_board_init() The intent is to allow management to query machine state and additionally configure it using previous query results within one QEMU instance (i.e. eliminate the need to start QEMU twice, 1st to query board specific parameters and 2nd for actual VM start using query results for additional parameters). The implementation is a bit of a hack: it splices in an additional main loop before machine creation, in special runstate preconfig. New command exit-preconfig exits that main loop. QEMU continues initializing, creates the machine, and runs the good old main loop. The replacement of the main loop is transparent to monitors. Sadly, some commands expect initialization to be complete. Running them in --preconfig's main loop violates their preconditions. Since we don't really know which commands are safe, we use a whitelist. This drags the concept of run state into the QMP core. The whitelist is done as a command flag in the QAPI schema (commit d6fe3d02e9a). Drags the concept of run state further into the QAPI language. The command flag is exposed in query-qmp-schema (also commit d6fe3d02e9a). This makes it ABI. I consider the whole thing an offensively ugly hack, but sometimes an ugly hack is the best we can do to solve a problem people have. The need described by the commit message quote above is genuine. The proper solution would be a main loop that permits complete configuration via QMP. This is out of reach, thus the hack. However, even though the need is genuine, it isn't urgent: libvirt is not going to use this anytime soon. Baking a hack into ABI before it has any users is a bad idea. This commit reverts the parts of commit d6fe3d02e9a that affect ABI via query-qmp-schema. The commit did the following: (1) Add command flag 'allow-preconfig' to the QAPI schema language (2) Pass it to code generators (3) Have the commands.py code generator pass it to the command registry (so commit 047f7038f58 can use it as whitelist) (4) Add 'allow-preconfig' to SchemaInfoCommand (neglecting to update qapi-code-gen.txt section "Client JSON Protocol introspection") (5) Set 'allow-preconfig': true for commands qmp_capabilities, query-commands, query-command-line-options, query-status Revert exactly (4), plus a bit of documentation added to qemu-tech.info in commit 047f7038f58. Shrinks query-qmp-schema's output from 126.5KiB to 121.8KiB for me. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180705091402.26244-2-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Acked-by: Eduardo Habkost <ehabkost@redhat.com> Acked-by: Igor Mammedov <imammedo@redhat.com> [Straightforward conflict with commit d626b6c1ae7 resolved]
2018-07-03qapi: add 'If:' section to generated documentationMarc-André Lureau
The documentation is generated only once, and doesn't know C pre-conditions. Add 'If:' sections for top-level entities. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180703155648.11933-13-marcandre.lureau@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03qapi-types: add #if conditions to types & visitorsMarkus Armbruster
Types & visitors are coupled and must be handled together to avoid temporary build regression. Wrap generated types/visitor code with #if/#endif using the context helpers. Derived from a patch by Marc-André. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180703155648.11933-12-marcandre.lureau@redhat.com>
2018-07-03qapi/events: add #if conditions to eventsMarc-André Lureau
Wrap generated code with #if/#endif using an 'ifcontext' on QAPIGenCSnippet objects. This makes a conditional event's qapi_event_send_FOO() compile-time conditional, but its enum QAPIEvent member remains unconditional for now. A follow up patch "qapi-event: add 'if' condition to implicit event enum" will improve this. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180703155648.11933-11-marcandre.lureau@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03qapi/commands: add #if conditions to commandsMarc-André Lureau
Wrap generated code with #if/#endif using an 'ifcontext' on QAPIGenCSnippet objects. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180703155648.11933-10-marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Line breaks tweaked] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03qapi-introspect: add preprocessor conditions to generated QLitMarc-André Lureau
This commit adds 'ifcond' conditions to top-level QLit objects. Future work will add them to object and enum type members, i.e. within QLit objects. Extend the QLit generator to_qlit() to accept (@obj, @cond) tuples in addition to just @obj. The tuple causes the QLit generated for objects for @obj with #if/#endif conditions for @cond. See generated tests/test-qmp-introspect.c. Example diff after this patch: --- before 2018-01-08 11:55:24.757083654 +0100 +++ tests/test-qmp-introspect.c 2018-01-08 13:08:44.477641629 +0100 @@ -51,6 +51,8 @@ { "name", QLIT_QSTR("EVENT_F"), }, {} })), +#if defined(TEST_IF_CMD) +#if defined(TEST_IF_STRUCT) QLIT_QDICT(((QLitDictEntry[]) { { "arg-type", QLIT_QSTR("5"), }, { "meta-type", QLIT_QSTR("command"), }, @@ -58,12 +60,16 @@ { "ret-type", QLIT_QSTR("0"), }, {} })), +#endif /* defined(TEST_IF_STRUCT) */ +#endif /* defined(TEST_IF_CMD) */ Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180703155648.11933-9-marcandre.lureau@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03qapi-introspect: modify to_qlit() to append ',' on level > 0Marc-André Lureau
The following patch is going to break list entries with #if/#endif, so they should have the trailing ',' as suffix. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180703155648.11933-8-marcandre.lureau@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03qapi: add #if/#endif helpersMarc-André Lureau
Add helpers to wrap generated code with #if/#endif lines. A later patch wants to use QAPIGen for generating C snippets rather than full C files with copyright headers etc. Splice in class QAPIGenCCode between QAPIGen and QAPIGenC. Add a 'with' statement context manager that will be used to wrap generator visitor methods. The manager will check if code was generated before adding #if/#endif lines on QAPIGenCSnippet objects. Used in the following patches. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180703155648.11933-7-marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03qapi: mcgen() shouldn't indent # linesMarc-André Lureau
Skip preprocessor lines when adding indentation, since that would likely result in invalid code. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180703155648.11933-6-marcandre.lureau@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03qapi: add 'ifcond' to visitor methodsMarc-André Lureau
Modify the test visitor to check correct passing of values. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180703155648.11933-5-marcandre.lureau@redhat.com> [Accidental change to roms/seabios dropped] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03qapi: leave the ifcond attribute undefined until check()Marc-André Lureau
We commonly initialize attributes to None in .init(), then set their real value in .check(). Accessing the attribute before .check() yields None. If we're lucky, the code that accesses the attribute prematurely chokes on None. It won't for .ifcond, because None is a legitimate value. Leave the ifcond attribute undefined until check(). Suggested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180703155648.11933-4-marcandre.lureau@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03qapi: pass 'if' condition into QAPISchemaEntity objectsMarc-André Lureau
Built-in objects remain unconditional. Explicitly defined objects use the condition specified in the schema. Implicitly defined objects inherit their condition from their users. For most of them, there is exactly one user, so the condition to use is obvious. The exception is wrapped types generated for simple union variants, which can be shared by any number of simple unions. The tight condition would be the disjunction of the conditions of these simple unions. For now, use the wrapped type's condition instead. Much simpler and good enough for now. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180703155648.11933-3-marcandre.lureau@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-07-03qapi: add 'if' to top-level expressionsMarc-André Lureau
Accept 'if' key in top-level elements, accepted as string or list of string type. The following patches will modify the test visitor to check the value is correctly saved, and generate #if/#endif code (as a single #if/endif line or a series for a list). Example of 'if' key: { 'struct': 'TestIfStruct', 'data': { 'foo': 'int' }, 'if': 'defined(TEST_IF_STRUCT)' } The generated code is for now *unconditional*. Later patches generate the conditionals. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180703155648.11933-2-marcandre.lureau@redhat.com> [Commit message and Documentation improved] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-06-22qapi/introspect: Eliminate pointless variable in .visit_end()Markus Armbruster
Commit 1a9a507b2e3 "qapi-introspect: Hide type names" added local variable @jsons to improve sorting, but also removed the sorting. It was part of a big series that went to v8, and it made sense until v2 or so... Commit 7d0f982bfbb replaced @jsons by @qlits, preserving the uselessness. Get rid of it. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180620124742.16979-1-armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2018-06-22qapi: Open files with encoding='utf-8'Markus Armbruster
Python 2 happily reads UTF-8 files in text mode, but Python 3 requires either UTF-8 locale or an explicit encoding passed to open(). Commit d4e5ec877ca fixed this by setting the en_US.UTF-8 locale. Falls apart when the locale isn't be available. Matthias Maier and Arfrever Frehtes Taifersar Arahesis proposed to use binary mode instead, with manual conversion from bytes to str. Works, but opening with an explicit encoding is simpler, so do that. Since Python 2's open() doesn't support the encoding parameter, we need to suppress it with a version check. Reported-by: Arfrever Frehtes Taifersar Arahesis <arfrever.fta@gmail.com> Reported-by: Matthias Maier <tamiko@43-1.org> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180618175958.29073-2-armbru@redhat.com> Reviewed-by: Eduardo Habkost <ehabkost@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>
2018-06-22qapi: allow empty branches in flat unionsAnton Nefedov
It often happens that just a few discriminator values imply extra data in a flat union. Existing checks did not make possible to leave other values uncovered. Such cases had to be worked around by either stating a dummy (empty) type or introducing another (subset) discriminator enumeration. Both options create redundant entities in qapi files for little profit. With this patch it is not necessary anymore to add designated union fields for every possible value of a discriminator enumeration. Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com> Message-Id: <1529311206-76847-2-git-send-email-anton.nefedov@virtuozzo.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-06-22qapi/events: generate event enum in main moduleMarc-André Lureau
The event generator produces an enum, and put it in the last visited module. It fits better in the main module, since it's the set of all visited events, from all modules. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180321115211.17937-3-marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-06-22qapi/visit: remove useless prefix argumentMarc-André Lureau
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180321115211.17937-2-marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-06-08python: Remove scripts/ordereddict.pyEduardo Habkost
Python 2.7 (the minimum Python version we require) provides collections.OrderedDict on the standard library, so we don't need to carry our own implementation. Signed-off-by: Eduardo Habkost <ehabkost@redhat.com> Message-Id: <20180608175252.25110-3-ehabkost@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-05-30qapi: introduce new cmd option "allow-preconfig"Igor Mammedov
New option will be used to allow commands, which are prepared/need to run, during preconfig state. Other commands that should be able to run in preconfig state, should be amended to not expect machine in initialized state or deal with it. For compatibility reasons, commands that don't use new flag 'allow-preconfig' explicitly are not permitted to run in preconfig state but allowed in all other states like they used to be. Within this patch allow following commands in preconfig state: qmp_capabilities query-qmp-schema query-commands query-command-line-options query-status exit-preconfig to allow qmp connection, basic introspection and moving to the next state. PS: set-numa-node and query-hotpluggable-cpus will be enabled later in a separate patches. Signed-off-by: Igor Mammedov <imammedo@redhat.com> Message-Id: <1526057503-39287-1-git-send-email-imammedo@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [ehabkost: Changed "since 2.13" to "since 3.0"] Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-05-04qapi: add SysEmuTarget to "common.json"Laszlo Ersek
We'll soon need an enumeration type that lists all the softmmu targets that QEMU (the project) supports. Introduce @SysEmuTarget to "common.json". The enum constant @x86_64 doesn't match the QAPI convention of preferring hyphen ("-") over underscore ("_"). This is intentional; the @SysEmuTarget constants are supposed to produce QEMU executable names when stringified and appended to the "qemu-system-" prefix. Put differently, the replacement text of the TARGET_NAME preprocessor macro must be possible to look up in the list of (stringified) enum constants. Like other enum types, @SysEmuTarget too can be used for discriminator fields in unions. For the @i386 constant, a C-language union member called "i386" would be generated. On mingw build hosts, "i386" is a macro however. Add "i386" to "polluted_words" at once. Cc: "Daniel P. Berrange" <berrange@redhat.com> Cc: Eric Blake <eblake@redhat.com> Cc: Markus Armbruster <armbru@redhat.com> Signed-off-by: Laszlo Ersek <lersek@redhat.com> Message-Id: <20180427192852.15013-3-lersek@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-05-04qobject: Replace qobject_incref/QINCREF qobject_decref/QDECREFMarc-André Lureau
Now that we can safely call QOBJECT() on QObject * as well as its subtypes, we can have macros qobject_ref() / qobject_unref() that work everywhere instead of having to use QINCREF() / QDECREF() for QObject and qobject_incref() / qobject_decref() for its subtypes. The replacement is mechanical, except I broke a long line, and added a cast in monitor_qmp_cleanup_req_queue_locked(). Unlike qobject_decref(), qobject_unref() doesn't accept void *. Note that the new macros evaluate their argument exactly once, thus no need to shout them. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180419150145.24795-4-marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Rebased, semantic conflict resolved, commit message improved] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-03-27qapi: restrict allow-oob value to be "true"Peter Xu
It was missed in the first version of OOB series. We should check this to make sure we throw the right error when fault value is passed in. Signed-off-by: Peter Xu <peterx@redhat.com> Message-Id: <20180326063901.27425-5-peterx@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-19qapi: introduce new cmd option "allow-oob"Peter Xu
Here "oob" stands for "Out-Of-Band". When "allow-oob" is set, it means the command allows out-of-band execution. The "oob" idea is proposed by Markus Armbruster in following thread: https://lists.gnu.org/archive/html/qemu-devel/2017-09/msg02057.html This new "allow-oob" boolean will be exposed by "query-qmp-schema" as well for command entries, so that QMP clients can know which commands can be used in out-of-band calls. For example the command "migrate" originally looks like: {"name": "migrate", "ret-type": "17", "meta-type": "command", "arg-type": "86"} And it'll be changed into: {"name": "migrate", "ret-type": "17", "allow-oob": false, "meta-type": "command", "arg-type": "86"} This patch only provides the QMP interface level changes. It does not contain the real out-of-band execution implementation yet. Suggested-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Fam Zheng <famz@redhat.com> Signed-off-by: Peter Xu <peterx@redhat.com> Message-Id: <20180309090006.10018-18-peterx@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [eblake: rebase on introspection done by qlit] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-19qapi: generate a literal qobject for introspectionMarc-André Lureau
Replace the generated json string with a literal qobject. The later is easier to deal with, at run time as well as compile time: adding #if conditionals will be easier than in a json string. The output of query-qmp-schema is not changed. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180305172951.2150-5-marcandre.lureau@redhat.com> [eblake: fix python 3 failure] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-19qapi2texi: minor python code simplificationMarc-André Lureau
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180305172951.2150-2-marcandre.lureau@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Move qapi-schema.json to qapi/, rename generated filesMarkus Armbruster
Move qapi-schema.json to qapi/, so it's next to its modules, and all files get generated to qapi/, not just the ones generated for modules. Consistently name the generated files qapi-MODULE.EXT: qmp-commands.[ch] become qapi-commands.[ch], qapi-event.[ch] become qapi-events.[ch], and qmp-introspect.[ch] become qapi-introspect.[ch]. This gets rid of the temporary hacks in scripts/qapi/commands.py, scripts/qapi/events.py, and scripts/qapi/common.py. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180211093607.27351-28-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> [eblake: Fix trailing dot in tpm.c, undo temporary hack for OSX toolchain] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02Include less of the generated modular QAPI headersMarkus Armbruster
In my "build everything" tree, a change to the types in qapi-schema.json triggers a recompile of about 4800 out of 5100 objects. The previous commit split up qmp-commands.h, qmp-event.h, qmp-visit.h, qapi-types.h. Each of these headers still includes all its shards. Reduce compile time by including just the shards we actually need. To illustrate the benefits: adding a type to qapi/migration.json now recompiles some 2300 instead of 4800 objects. The next commit will improve it further. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180211093607.27351-24-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> [eblake: rebase to master] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Generate separate .h, .c for each moduleMarkus Armbruster
Our qapi-schema.json is composed of modules connected by include directives, but the generated code is monolithic all the same: one qapi-types.h with all the types, one qapi-visit.h with all the visitors, and so forth. These monolithic headers get included all over the place. In my "build everything" tree, adding a QAPI type recompiles about 4800 out of 5100 objects. We wouldn't write such monolithic headers by hand. It stands to reason that we shouldn't generate them, either. Split up generated qapi-types.h to mirror the schema's modular structure: one header per module. Name the main module's header qapi-types.h, and sub-module D/B.json's header D/qapi-types-B.h. Mirror the schema's includes in the headers, so that qapi-types.h gets you everything exactly as before. If you need less, you can include one or more of the sub-module headers. To be exploited shortly. Split up qapi-types.c, qapi-visit.h, qapi-visit.c, qmp-commands.h, qmp-commands.c, qapi-event.h, qapi-event.c the same way. qmp-introspect.h, qmp-introspect.c and qapi.texi remain monolithic. The split of qmp-commands.c duplicates static helper function qmp_marshal_output_str() in qapi-commands-char.c and qapi-commands-misc.c. This happens when commands returning the same type occur in multiple modules. Not worth avoiding. Since I'm going to rename qapi-event.[ch] to qapi-events.[ch], and qmp-commands.[ch] to qapi-commands.[ch], name the shards that way already, to reduce churn. This requires temporary hacks in commands.py and events.py. Similarly, c_name() must temporarily be taught to munge '/' in common.py. They'll go away with the rename. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180211093607.27351-23-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [eblake: declare a dummy variable in each .c file, to shut up OSX toolchain warnings about empty .o files, including hacking c_name()] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi/common: Fix guardname() for funny filenamesMarkus Armbruster
guardname() fails to return a valid C identifier for arguments containing anything but [A-Za-z0-9_.-']. Fix that. Don't bother protecting ticklish identifiers; header guards are all-caps, and no ticklish identifiers are. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180211093607.27351-22-armbru@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi/types qapi/visit: Generate built-in stuff into separate filesMarkus Armbruster
Linking code from multiple separate QAPI schemata into the same program is possible, but involves some weirdness around built-in types: * We generate code for built-in types into .c only with option --builtins. The user is responsible for generating code for exactly one QAPI schema per program with --builtins. * We generate code for built-in types into .h regardless of --builtins, but guarded by #ifndef QAPI_VISIT_BUILTIN. Because all copies of this code are exactly the same, including any combination of these headers works. Replace this contraption by something more conventional: generate code for built-in types into their very own files: qapi-builtin-types.c, qapi-builtin-visit.c, qapi-builtin-types.h, qapi-builtin-visit.h, but only with --builtins. Obey --output-dir, but ignore --prefix for them. Make qapi-types.h include qapi-builtin-types.h. With multiple schemata you now have multiple qapi-types.[ch], but only one qapi-builtin-types.[ch]. Same for qapi-visit.[ch] and qapi-builtin-visit.[ch]. Bonus: if all you need is built-in stuff, you can include a much smaller header. To be exploited shortly. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180211093607.27351-21-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> [eblake: fix octal constant for python 3] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Make code-generating visitors use QAPIGen moreMarkus Armbruster
The use of QAPIGen is rather shallow so far: most of the output accumulation is not converted. Take the next step: convert output accumulation in the code-generating visitor classes. Helper functions outside these classes are not converted. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180211093607.27351-20-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> [eblake: rebase to earlier guardstart cleanup] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Rename generated qmp-marshal.c to qmp-commands.cMarkus Armbruster
All generated .c are named like their .h, except for qmp-marshal.c and qmp-commands.h. To add to the confusion, tests-qmp-commands.c falsely matches generated test-qmp-commands.h. Get rid of this unnecessary complication. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180211093607.27351-19-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Record 'include' directives in intermediate representationMarkus Armbruster
The include directive permits modular QAPI schemata, but the generated code is monolithic all the same. To permit generating modular code, the front end needs to pass more information on inclusions to the back ends. The commit before last added the necessary information to the parse tree. This commit adds it to the intermediate representation and its QAPISchemaVisitor. A later commit will use this to to generate modular code. New entity QAPISchemaInclude represents inclusions. Call new visitor method visit_include() for it, so visitors can see the sub-modules a module includes. Note that unlike other entities, QAPISchemaInclude has no name, and is therefore not added to entity_dict. New QAPISchemaEntity attribute @module names the entity's source file. Call new visitor method visit_module() when it changes during a visit, so visitors can keep track of the module being visited. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-18-armbru@redhat.com> [eblake: avoid accidental deletion of self._predefining] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Generate in source orderMarkus Armbruster
The generators' conversion to visitors (merge commit 9e72681d16) changed the processing order of entities from source order to alphabetical order. The next commit needs source order, so change it back. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-17-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Record 'include' directives in parse treeMarkus Armbruster
The parse tree is a list of expressions. Except include expressions currently get replaced by the included file's parse tree. Instead of throwing away the include expression, keep it with the file name expanded so you don't have to track the including file's directory to make sense of it. A future commit will put this include expression to use. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-16-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [eblake: fix check of expr after assignment] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Concentrate QAPISchemaParser.exprs updates in .__init__()Markus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-15-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Lift error reporting from QAPISchema.__init__() to callersMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-14-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi/common: Eliminate QAPISchema.exprsMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-13-armbru@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Improve include file name reporting in error messagesMarkus Armbruster
Error messages print absolute file names of included files even if the user gave a relative one on the command line: $ PYTHONPATH=scripts python -B tests/qapi-schema/test-qapi.py tests/qapi-schema/include-cycle.json In file included from tests/qapi-schema/include-cycle.json:1: In file included from /work/armbru/qemu/tests/qapi-schema/include-cycle-b.json:1: /work/armbru/qemu/tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json Improve this to In file included from tests/qapi-schema/include-cycle.json:1: In file included from tests/qapi-schema/include-cycle-b.json:1: tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json The error message when an include file can't be opened prints the include directive's file name, which is relative to the including file. Change this to print the file name relative to the working directory. Visible in tests/qapi-schema/include-no-file.err. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-12-armbru@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Touch generated files only when they changeMarkus Armbruster
A massive number of objects depends on QAPI-generated headers. In my "build everything" tree, it's roughly 4800 out of 5100. This is particularly annoying when only some of the generated files change, say for a doc fix. Improve qapi-gen.py to touch its output files only if they actually change. Rebuild time for a QAPI doc fix drops from many minutes to a few seconds. Rebuilds get faster for certain code changes, too. For instance, adding a simple QMP event now recompiles less than 200 instead of 4800 objects. But adding a QAPI type is as bad as ever; we've clearly got more work to do. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180211093607.27351-11-armbru@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> [eblake: fix octal constant for python3] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi-gen: Convert from getopt to argparseMarkus Armbruster
argparse is nicer to use than getopt, and gives us --help almost for free. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20180211093607.27351-10-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> [eblake: Fix --output-dir editing accident] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi-gen: New common driver for code and doc generatorsMarkus Armbruster
Whenever qapi-schema.json changes, we run six programs eleven times to update eleven files. Similar for qga/qapi-schema.json. This is silly. Replace the six programs by a single program that spits out all eleven files. The programs become modules in new Python package qapi, along with the helper library. This requires moving them to scripts/qapi/. While moving them, consistently drop executable mode bits. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-9-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> [eblake: move change to one-line 'blurb' earlier in series, mention mode bit change as intentional, update qapi-code-gen.txt to match actual generated events.c file] Signed-off-by: Eric Blake <eblake@redhat.com>
generated by cgit v1.2.3 (git 2.26.2) at 2024-12-03 04:56:57 +0000