aboutsummaryrefslogtreecommitdiff
path: root/scripts/qapi2texi.py
AgeCommit message (Collapse)Author
2018-03-02qapi-gen: New common driver for code and doc generatorsMarkus Armbruster
Whenever qapi-schema.json changes, we run six programs eleven times to update eleven files. Similar for qga/qapi-schema.json. This is silly. Replace the six programs by a single program that spits out all eleven files. The programs become modules in new Python package qapi, along with the helper library. This requires moving them to scripts/qapi/. While moving them, consistently drop executable mode bits. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-9-armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com> [eblake: move change to one-line 'blurb' earlier in series, mention mode bit change as intentional, update qapi-code-gen.txt to match actual generated events.c file] Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-02qapi: Streamline boilerplate comment generationMarkus Armbruster
Every generator has separate boilerplate for .h and .c, and their differences are boring. All of them repeat the license note. Reduce the repetition as follows. Move common text like the license note to common open_output(), next to the existing common text there. For each generator, replace the two separate descriptions by a single one. While there, emit an "automatically generated" note into generated documentation, too. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20180211093607.27351-3-armbru@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>
2018-02-05qapi: use items()/values() intead of iteritems()/itervalues()Daniel P. Berrange
The iteritems()/itervalues() methods are gone in py3, but the items()/values() methods are still around. The latter are less efficient than the former in py2, but this has unmeasurably small impact on QEMU build time, so taking portability over efficiency is a net win. Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <20180116134217.8725-3-berrange@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-02-05qapi: convert to use python print function instead of statementDaniel P. Berrange
Python 3 no longer supports the bare "print" statement, it must be called as a normal function with round brackets. It is possible to opt-in to this new syntax with Python 2.6 onwards by importing the "print_function" from the "__future__" module, making it easy to support Python 2 and 3 in parallel. Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <20180116134217.8725-2-berrange@redhat.com> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2017-12-20qapi2texi: De-duplicate code to add blank line before symbolMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20171002141341.24616-12-armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2017-12-20qapi2texi: Simplify representation of section textMarkus Armbruster
Use a string instead of a list of strings. While there, generate fewer superfluous blank lines. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20171002141341.24616-10-armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2017-12-20qapi: Simplify representation of QAPIDoc section textMarkus Armbruster
Use a string instead of a list of strings. This makes qapi2texi.py generate additional blank lines. They're harmless, and the next commit will get rid of them again. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20171002141341.24616-9-armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2017-12-20qapi: Unify representation of doc section without nameMarkus Armbruster
We have two representations of sections without a name: the main section uses name=None, the others name=''. Standardize on name=None. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20171002141341.24616-8-armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2017-12-20qapi2texi: Clean up texi_sections()Markus Armbruster
Repurposing the function parameter doc for stepping through doc.sections.__str__() is not nice. Use new variable @text instead. While there, eliminate variables name and func. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20171002141341.24616-7-armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2017-07-31docs: fix broken paths to docs/devel/qapi-code-gen.txtPhilippe Mathieu-Daudé
With the move of some docs to docs/interop on ac06724a71, a couple of references were not updated. Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
2017-03-21qapi2texi: Fix translation of *strong* and _emphasized_Markus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <1490015515-25851-7-git-send-email-armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2017-03-21qapi2texi: Fix to actually fail when 'doc-required' is falseMarkus Armbruster
Messed up in commit bc52d03. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <1490015515-25851-3-git-send-email-armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2017-03-16qapi2texi: Use category "Object" for all object typesMarkus Armbruster
At the protocol level, the distinction between struct, flat union and simple union is meaningless, they are all JSON objects. Document them that way. Example change (qemu-qmp-ref.txt): - -- Simple Union: InputEvent + -- Object: InputEvent Input event union. This also fixes the completely broken headings for flat and simple unions in qemu-qmp-ref.7 and qemu-ga-ref.7, by sidestepping a bug in texi2pod.pl. For instance, it mistranslates "@deftp {Simple Union} InputEvent" to "B<Union> (Simple)", but translates "@deftp Object InputEvent" to "B<SocketAddress> (Object)". Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-30-git-send-email-armbru@redhat.com>
2017-03-16qapi2texi: Generate descriptions for simple union tagsMarkus Armbruster
Simple union tags carry no type information, because their type is implicit. Their description should make up for it, but many have none. Generate one automatically then. Example change (qemu-qmp-ref.txt): -- Simple Union: ImageInfoSpecific A discriminated record of image format specific information structures. Members: 'type' - Not documented + One of "qcow2", "vmdk", "luks" 'data: ImageInfoSpecificQCow2' when 'type' is "qcow2" 'data: ImageInfoSpecificVmdk' when 'type' is "vmdk" 'data: QCryptoBlockInfoLUKS' when 'type' is "luks" Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-29-git-send-email-armbru@redhat.com>
2017-03-16qapi2texi: Generate documentation for variant membersMarkus Armbruster
A flat union's branch brings in the members of another type. Generate a suitable reference to that type. Example change (qemu-qmp-ref.txt): -- Flat Union: QCryptoBlockOpenOptions The options that are available for all encryption formats when opening an existing volume Members: The members of 'QCryptoBlockOptionsBase' + The members of 'QCryptoBlockOptionsQCow' when 'format' is "qcow" + The members of 'QCryptoBlockOptionsLUKS' when 'format' is "luks" Since: 2.6 A simple union's branch adds a member 'data' of some other type. Generate documentation for that member. Example change (qemu-qmp-ref.txt): -- Simple Union: SocketAddress Captures the address of a socket, which could also be a named file descriptor Members: 'type' Not documented + 'data: InetSocketAddress' when 'type' is "inet" + 'data: UnixSocketAddress' when 'type' is "unix" + 'data: VsockSocketAddress' when 'type' is "vsock" + 'data: String' when 'type' is "fd" Since: 1.3 Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-28-git-send-email-armbru@redhat.com>
2017-03-16qapi2texi: Generate reference to base type membersMarkus Armbruster
The generated documentation doesn't mention object type members inherited from a base type. Fix that. Example change (qemu-qmp-ref.txt): -- Struct: VncServerInfo The network connection information for server Members: 'auth' (optional) authentication method used for the plain (non-websocket) VNC server + The members of 'VncBasicInfo' Since: 2.1 Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-27-git-send-email-armbru@redhat.com>
2017-03-16qapi2texi: Include member type in generated documentationMarkus Armbruster
The recent merge of docs/qmp-commands.txt and docs/qmp-events.txt into the schema lost type information. Fix this documentation regression. Example change (qemu-qmp-ref.txt): -- Struct: InputKeyEvent Keyboard input event. Members: - 'button' + 'button: InputButton' Which button this event is for. - 'down' + 'down: boolean' True for key-down and false for key-up events. Since: 2.0 Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-26-git-send-email-armbru@redhat.com>
2017-03-16qapi2texi: Implement boxed argument documentationMarkus Armbruster
This replaces manual references like "For the arguments, see the documentation of ..." by a generated reference "Arguments: the members of ...". Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-25-git-send-email-armbru@redhat.com>
2017-03-16qapi2texi: Don't hide undocumented members and argumentsMarkus Armbruster
Show undocumented object, alternate type members and command, event arguments exactly like undocumented enumeration type values. Example change (qemu-qmp-ref.txt): -- Command: query-rocker Return rocker switch information. + Arguments: + 'name' + Not documented + Returns: 'Rocker' information Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-24-git-send-email-armbru@redhat.com>
2017-03-16qapi2texi: Explain enum value undocumentedness more clearlyMarkus Armbruster
Instead of not saying anything when we have no documentation, say "Not documented". Example change (qemu-qmp-ref.txt): -- Enum: GuestPanicAction An enumeration of the actions taken when guest OS panic is detected Values: 'pause' system pauses 'poweroff' + Not documented Since: 2.1 (poweroff since 2.8) Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-23-git-send-email-armbru@redhat.com>
2017-03-16qapi2texi: Present the table of members more clearlyMarkus Armbruster
The table of members follows the main descriptive text immediately. Makes it hard to see what it is about. Start a new paragraph, and lead with a line "Members:" for object and alternate types, "Values:" for enumeration types, and "Arguments:" for commands and events. Example change (qemu-qmp-ref.txt): -- Command: set_link Sets the link status of a virtual network adapter. + + Arguments: 'name' the device name of the virtual network adapter 'up' true to set the link status to be up Returns: Nothing on success If 'name' is not a valid network device, DeviceNotFound Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-22-git-send-email-armbru@redhat.com>
2017-03-16qapi2texi: Plainer enum value and member name formattingMarkus Armbruster
Use @code{%s} instead of @code{'%s'}. Impact, using @id as example: * Texinfo -@item @code{'id'} +@item @code{id} * HTML -<dt><code>'id'</code></dt> +<dt><code>id</code></dt> * POD (for manual pages): -=item C<'id'> +=item C<id> * Formatted manual pages: -'id' +"id" * Plain text: - ''id'' + 'id' Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-21-git-send-email-armbru@redhat.com>
2017-03-16qapi: Prefer single-quoted strings more consistentlyMarkus Armbruster
PEP 8 advises: In Python, single-quoted strings and double-quoted strings are the same. This PEP does not make a recommendation for this. Pick a rule and stick to it. When a string contains single or double quote characters, however, use the other one to avoid backslashes in the string. It improves readability. The QAPI generators succeed at picking a rule, but fail at sticking to it. Convert a bunch of double-quoted strings to single-quoted ones. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-20-git-send-email-armbru@redhat.com>
2017-03-16qapi: The #optional tag is redundant, dropMarkus Armbruster
We traditionally mark optional members #optional in the doc comment. Before commit 3313b61, this was entirely manual. Commit 3313b61 added some automation because its qapi2texi.py relied on #optional to determine whether a member is optional. This is no longer the case since the previous commit: the only thing qapi2texi.py still does with #optional is stripping it out. We still reject bogus qapi-schema.json and six places for qga/qapi-schema.json. Thus, you can't actually rely on #optional to see whether something is optional. Yet we still make people add it manually. That's just busy-work. Drop the code to check, fix up and strip out #optional, along with all instances of #optional. To keep it out, add code to reject it, to be dropped again once the dust settles. No change to generated documentation. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-18-git-send-email-armbru@redhat.com>
2017-03-16qapi2texi: Convert to QAPISchemaVisitorMarkus Armbruster
qapi2texi works with schema expression trees. Such a tight coupling to schema language syntax is not a good idea. Convert it to the visitor interface the other generators use. No change to generated documentation. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-17-git-send-email-armbru@redhat.com>
2017-03-16qapi: Conjure up QAPIDoc.ArgSection for undocumented membersMarkus Armbruster
qapi2texi.py already conjures up ArgSections for undocumented enumeration values, in texi_enum. Drop that, and conjure them up for all kinds of "arguments" (enumeration values, object and alternate type members) in qapi.py instead. Take care to keep generated documentation exactly the same for now. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <1489582656-31133-16-git-send-email-armbru@redhat.com>
2017-03-16qapi: Avoid unwanted blank lines in QAPIDocMarkus Armbruster
We silently fix missing #optional tags for QAPIDoc by appending a line "#optional" to the section's .content. However, this interferes with .__repr__ stripping trailing blank lines from .content. Use new ArgSection instance variable .optional instead, and leave .content alone. To permit testing .optional in texi_body(), clean up texi_enum()'s hack to add empty documentation for undocumented enum values: add an ArgSection instead of ''. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <1489582656-31133-12-git-send-email-armbru@redhat.com>
2017-03-16qapi2texi: Fix up output around #optionalMarkus Armbruster
We use tag #optional to mark optional members, like this: # @name: #optional The name of the guest texi_body() strips #optional, but not whitespace around it. For the above, we get in qemu-qmp-qapi.texi @item @code{'name'} (optional) The name of the guest @end table The extra space can lead to artifacts in output, e.g in qemu-qmp-ref.7.pod =item C<'name'> (optional) The name of the guest and then in qemu-qmp-ref.7 .IX Item "name (optional)" .Vb 1 \& The name of the guest .Ve instead of intended plain .IX Item "name (optional)" The name of the guest Get rid of these artifacts by removing whitespace around #optional along with it. This turns three minus signs in qapi-schema.json into markup, because they're now at the beginning of the line. Drop them, they're unwanted there. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <1489582656-31133-11-git-send-email-armbru@redhat.com>
2017-03-16qapi: Make doc comments optional where we don't need themMarkus Armbruster
Since we added the documentation generator in commit 3313b61, doc comments are mandatory. That's a very good idea for a schema that needs to be documented, but has proven to be annoying for testing. Make doc comments optional again, but add a new directive { 'pragma': { 'doc-required': true } } to let a QAPI schema require them. Add test cases for the new pragma directive. While there, plug a minor hole in includ directive test coverage. Require documentation in the schemas we actually want documented: qapi-schema.json and qga/qapi-schema.json. We could probably make qapi2texi.py cope with incomplete documentation, but for now, simply make it refuse to run unless the schema has 'doc-required': true. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <1489582656-31133-3-git-send-email-armbru@redhat.com> [qapi-code-gen.txt wording tweaked] Reviewed-by: Eric Blake <eblake@redhat.com>
2017-03-16qapi2texi: change texi formattersMarc-André Lureau
STRUCT_FMT is generic enough, rename it to TYPE_FMT, use it for unions. Rename COMMAND_FMT to MSG_FMT, since it applies to both commands and events. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20170125130308.16104-2-marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-02-20qapi2texi: replace quotation by bold section nameMarc-André Lureau
When we build qemu-qmp-ref.txt this causes texinfo to complain several times: "Negative repeat count does nothing at /usr/share/texinfo/Texinfo/Convert/Line.pm line 124." It also doesn't display correctly, because the "Notes" text disappears entirely in the HTML version because it thinks there's no actual quotation text. The text file output formatting is also not good. To solve those problems, remove usage of @quotation, and simply use bold face for the section name. Reported-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20170217093416.27688-1-marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-01-16qapi: add qapi2texi scriptMarc-André Lureau
As the name suggests, the qapi2texi script converts JSON QAPI description into a texi file suitable for different target formats (info/man/txt/pdf/html...). It parses the following kind of blocks: Free-form: ## # = Section # == Subsection # # Some text foo with *emphasis* # 1. with a list # 2. like that # # And some code: # | $ echo foo # | -> do this # | <- get that # ## Symbol description: ## # @symbol: # # Symbol body ditto ergo sum. Foo bar # baz ding. # # @param1: the frob to frobnicate # @param2: #optional how hard to frobnicate # # Returns: the frobnicated frob. # If frob isn't frobnicatable, GenericError. # # Since: version # Notes: notes, comments can have # - itemized list # - like this # # Example: # # -> { "execute": "quit" } # <- { "return": {} } # ## That's roughly following the following EBNF grammar: api_comment = "##\n" comment "##\n" comment = freeform_comment | symbol_comment freeform_comment = { "# " text "\n" | "#\n" } symbol_comment = "# @" name ":\n" { member | tag_section | freeform_comment } member = "# @" name ':' [ text ] "\n" freeform_comment tag_section = "# " ( "Returns:", "Since:", "Note:", "Notes:", "Example:", "Examples:" ) [ text ] "\n" freeform_comment text = free text with markup Note that the grammar is ambiguous: a line "# @foo:\n" can be parsed both as freeform_comment and as symbol_comment. The actual parser recognizes symbol_comment. See docs/qapi-code-gen.txt for more details. Deficiencies and limitations: - the generated QMP documentation includes internal types - union type support is lacking - type information is lacking in generated documentation - doc comment error message positions are imprecise, they point to the beginning of the comment. - a few minor issues, all marked TODO/FIXME in the code Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-Id: <20170113144135.5150-16-marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [test-qapi.py tweaked to avoid trailing empty lines in .out] Signed-off-by: Markus Armbruster <armbru@redhat.com>