aboutsummaryrefslogtreecommitdiff
path: root/scripts/qapi
AgeCommit message (Collapse)Author
2024-09-10qapi: Smarter camel_to_upper() to reduce need for 'prefix'Markus Armbruster
camel_to_upper() converts its argument from camel case to upper case with '_' between words. Used for generated enumeration constant prefixes. When some of the words are spelled all caps, where exactly to insert '_' is guesswork. camel_to_upper()'s guesses are bad enough in places to make people override them with a 'prefix' in the schema. Rewrite it to guess better: 1. Insert '_' after a non-upper case character followed by an upper case character: OneTwo -> ONE_TWO One2Three -> ONE2_THREE 2. Insert '_' before the last upper case character followed by a non-upper case character: ACRONYMWord -> ACRONYM_Word Except at the beginning (as in OneTwo above), or when there is already one: AbCd -> AB_CD This changes the default enumeration constant prefix for a number of enums. Generated enumeration constants change only where the default is not overridden with 'prefix'. The following enumerations without a 'prefix' change: enum old camel_to_upper() new camel_to_upper() ------------------------------------------------------------------ DisplayGLMode DISPLAYGL_MODE DISPLAY_GL_MODE EbpfProgramID EBPF_PROGRAMID EBPF_PROGRAM_ID HmatLBDataType HMATLB_DATA_TYPE HMAT_LB_DATA_TYPE HmatLBMemoryHierarchy HMATLB_MEMORY_HIERARCHY HMAT_LB_MEMORY_HIERARCHY MultiFDCompression MULTIFD_COMPRESSION MULTI_FD_COMPRESSION OffAutoPCIBAR OFF_AUTOPCIBAR OFF_AUTO_PCIBAR QCryptoBlockFormat Q_CRYPTO_BLOCK_FORMAT QCRYPTO_BLOCK_FORMAT QCryptoBlockLUKSKeyslotState Q_CRYPTO_BLOCKLUKS_KEYSLOT_STATE QCRYPTO_BLOCK_LUKS_KEYSLOT_STATE QKeyCode Q_KEY_CODE QKEY_CODE XDbgBlockGraphNodeType X_DBG_BLOCK_GRAPH_NODE_TYPE XDBG_BLOCK_GRAPH_NODE_TYPE TestUnionEnumA TEST_UNION_ENUMA TEST_UNION_ENUM_A Add a 'prefix' so generated code doesn't change now. Subsequent commits will remove most of them again. Two will remain: MULTIFD_COMPRESSION, because migration code generally spells "multifd" that way, and Q_KEY_CODE, because that one is baked into subprojects/keycodemapdb/tools/keymap-gen. The following enumerations with a 'prefix' change so that the prefix is now superfluous: enum old camel_to_upper() new camel_to_upper() [equal to prefix] ------------------------------------------------------------------ BlkdebugIOType BLKDEBUGIO_TYPE BLKDEBUG_IO_TYPE QCryptoTLSCredsEndpoint Q_CRYPTOTLS_CREDS_ENDPOINT QCRYPTO_TLS_CREDS_ENDPOINT QCryptoSecretFormat Q_CRYPTO_SECRET_FORMAT QCRYPTO_SECRET_FORMAT QCryptoCipherMode Q_CRYPTO_CIPHER_MODE QCRYPTO_CIPHER_MODE QCryptodevBackendType Q_CRYPTODEV_BACKEND_TYPE QCRYPTODEV_BACKEND_TYPE QType [builtin] Q_TYPE QTYPE Drop these prefixes. The following enumerations with a 'prefix' change without making the 'prefix' superfluous: enum old camel_to_upper() new camel_to_upper() [equal to prefix] prefix ------------------------------------------------------------------ CpuS390Entitlement CPUS390_ENTITLEMENT CPU_S390_ENTITLEMENT S390_CPU_ENTITLEMENT CpuS390Polarization CPUS390_POLARIZATION CPU_S390_POLARIZATION S390_CPU_POLARIZATION CpuS390State CPUS390_STATE CPU_S390_STATE S390_CPU_STATE QAuthZListFormat Q_AUTHZ_LIST_FORMAT QAUTH_Z_LIST_FORMAT QAUTHZ_LIST_FORMAT QAuthZListPolicy Q_AUTHZ_LIST_POLICY QAUTH_Z_LIST_POLICY QAUTHZ_LIST_POLICY QCryptoAkCipherAlgorithm Q_CRYPTO_AK_CIPHER_ALGORITHM QCRYPTO_AK_CIPHER_ALGORITHM QCRYPTO_AKCIPHER_ALG QCryptoAkCipherKeyType Q_CRYPTO_AK_CIPHER_KEY_TYPE QCRYPTO_AK_CIPHER_KEY_TYPE QCRYPTO_AKCIPHER_KEY_TYPE QCryptoCipherAlgorithm Q_CRYPTO_CIPHER_ALGORITHM QCRYPTO_CIPHER_ALGORITHM QCRYPTO_CIPHER_ALG QCryptoHashAlgorithm Q_CRYPTO_HASH_ALGORITHM QCRYPTO_HASH_ALGORITHM QCRYPTO_HASH_ALG QCryptoIVGenAlgorithm Q_CRYPTOIV_GEN_ALGORITHM QCRYPTO_IV_GEN_ALGORITHM QCRYPTO_IVGEN_ALG QCryptoRSAPaddingAlgorithm Q_CRYPTORSA_PADDING_ALGORITHM QCRYPTO_RSA_PADDING_ALGORITHM QCRYPTO_RSA_PADDING_ALG QCryptodevBackendAlgType Q_CRYPTODEV_BACKEND_ALG_TYPE QCRYPTODEV_BACKEND_ALG_TYPE QCRYPTODEV_BACKEND_ALG QCryptodevBackendServiceType Q_CRYPTODEV_BACKEND_SERVICE_TYPE QCRYPTODEV_BACKEND_SERVICE_TYPE QCRYPTODEV_BACKEND_SERVICE Subsequent commits will tweak things to remove most of these prefixes. Only QAUTHZ_LIST_FORMAT and QAUTHZ_LIST_POLICY will remain. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20240904111836.3273842-2-armbru@redhat.com>
2024-07-17qapi: remove "Example" doc sectionJohn Snow
Fully eliminate the "Example" sections in QAPI doc blocks now that they have all been converted to arbitrary rST syntax using the ".. qmp-example::" directive. Update tests to match. Migrating to the new syntax --------------------------- The old "Example:" or "Examples:" section syntax is now caught as an error, but "Example::" is stil permitted as explicit rST syntax for an un-lexed, generic preformatted text block. ('Example' is not special in this case, any sentence that ends with "::" will start an indented code block in rST.) Arbitrary rST for Examples is now possible, but it's strongly recommended that documentation authors use the ".. qmp-example::" directive for consistent visual formatting in rendered HTML docs. The ":title:" directive option may be used to add extra information into the title bar for the example. The ":annotated:" option can be used to write arbitrary rST instead, with nested "::" blocks applying QMP formatting where desired. Other choices available are ".. code-block:: QMP" which will not create an "Example:" box, or the short-form "::" code-block syntax which will not apply QMP highlighting when used outside of the qmp-example directive. Why? ---- This patch has several benefits: 1. Example sections can now be written more arbitrarily, mixing explanatory paragraphs and code blocks however desired. 2. Example sections can now use fully arbitrary rST. 3. All code blocks are now lexed and validated as QMP; increasing usability of the docs and ensuring validity of example snippets. (To some extent - This patch only gaurantees it lexes correctly, not that it's valid under the JSON or QMP grammars. It will catch most small mistakes, however.) 4. Each qmp-example can be titled or annotated independently without bypassing the QMP lexer/validator. (i.e. code blocks are now for *code* only, so we don't have to sacrifice exposition for having lexically valid examples.) NOTE: As with the "Notes" conversion (d461c279737), this patch (and the three preceding) may change the rendering order for Examples in the current generator. The forthcoming qapidoc rewrite will fix this by always generating documentation in source order. Signed-off-by: John Snow <jsnow@redhat.com> Message-ID: <20240717021312.606116-10-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2024-07-06qapi/parser: don't parse rST markup as section headersJohn Snow
The double-colon synax is rST formatting that precedes a literal code block. We do not want to capture these as QAPI-specific sections. Coerce blocks that start with e.g. "Example::" to be parsed as untagged paragraphs instead of special tagged sections. Signed-off-by: John Snow <jsnow@redhat.com> Message-ID: <20240626222128.406106-14-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Indentation tweaked for consistency] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2024-07-06qapi: convert "Note" sections to plain rSTJohn Snow
We do not need a dedicated section for notes. By eliminating a specially parsed section, these notes can be treated as normal rST paragraphs in the new QMP reference manual, and can be placed and styled much more flexibly. Convert all existing "Note" and "Notes" sections to pure rST. As part of the conversion, capitalize the first letter of each sentence and add trailing punctuation where appropriate to ensure notes look sensible and consistent in rendered HTML documentation. Markup is also re-aligned to the de-facto standard of 3 spaces for directives. Update docs/devel/qapi-code-gen.rst to reflect the new paradigm, and update the QAPI parser to prohibit "Note" sections while suggesting a new syntax. The exact formatting to use is a matter of taste, but a good candidate is simply: .. note:: lorem ipsum ... ... dolor sit amet ... ... consectetur adipiscing elit ... ... but there are other choices, too. The Sphinx readthedocs theme offers theming for the following forms (capitalization unimportant); all are adorned with a (!) symbol () in the title bar for rendered HTML docs. See https://sphinx-rtd-theme.readthedocs.io/en/stable/demo/demo.html#admonitions for examples of each directive/admonition in use. These are rendered in orange: .. Attention:: ... .. Caution:: ... .. WARNING:: ... These are rendered in red: .. DANGER:: ... .. Error:: ... These are rendered in green: .. Hint:: ... .. Important:: ... .. Tip:: ... These are rendered in blue: .. Note:: ... .. admonition:: custom title admonition body text This patch uses ".. note::" almost everywhere, with just two "caution" directives. Several instances of "Notes:" have been converted to merely ".. note::", or multiple ".. note::" where appropriate. ".. admonition:: notes" is used in a few places where we had an ordered list of multiple notes that would not make sense as standalone/separate admonitions. Two "Note:" following "Example:" have been turned into ordinary paragraphs within the example. NOTE: Because qapidoc.py does not attempt to preserve source ordering of sections, the conversion of Notes from a "tagged section" to an "untagged section" means that rendering order for some notes *may change* as a result of this patch. The forthcoming qapidoc.py rewrite strictly preserves source ordering in the rendered documentation, so this issue will be rectified in the new generator. Signed-off-by: John Snow <jsnow@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> [for block*.json] Message-ID: <20240626222128.406106-11-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Commit message clarified slightly, period added to one more note] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2024-07-06qapi/parser: fix comment parsing immediately following a doc blockJohn Snow
If a comment immediately follows a doc block, the parser doesn't ignore that token appropriately. Fix that. e.g. > ## > # = Hello World! > ## > > # I'm a comment! will break the parser, because it does not properly ignore the comment token if it immediately follows a doc block. Fixes: 3d035cd2cca6 (qapi: Rewrite doc comment parser) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240626222128.406106-7-jsnow@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2024-07-06qapi/parser: preserve indentation in QAPIDoc sectionsJohn Snow
Change get_doc_indented() to preserve indentation on all subsequent text lines, and create a compatibility dedent() function for qapidoc.py that removes indentation the same way get_doc_indented() did. This is being done for the benefit of a new qapidoc generator which requires that indentation in argument and features sections are preserved. Prior to this patch, a section like this: ``` @name: lorem ipsum dolor sit amet consectetur adipiscing elit ``` would have its body text be parsed into: ``` lorem ipsum dolor sit amet consectetur adipiscing elit ``` We want to preserve the indentation for even the first body line so that the entire block can be parsed directly as rST. This patch would now parse that segment into: ``` lorem ipsum dolor sit amet consectetur adipiscing elit ``` This is helpful for formatting arguments and features as field lists in rST, where the new generator will format this information as: ``` :arg type name: lorem ipsum dolor sit amet consectetur apidiscing elit ``` ...and can be formed by the simple concatenation of the field list construct and the body text. The indents help preserve the continuation of a block-level element, and further allow the use of additional rST block-level constructs such as code blocks, lists, and other such markup. This understandably breaks the existing qapidoc.py; so a new function is added there to dedent the text for compatibility. Once the new generator is merged, this function will not be needed any longer and can be dropped. I verified this patch changes absolutely nothing by comparing the md5sums of the QMP ref html pages both before and after the change, so it's certified inert. QAPI test output has been updated to reflect the new strategy of preserving indents for rST. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240626222128.406106-6-jsnow@redhat.com> [Lost commit message paragraph restored] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2024-07-06qapi: linter fixupsJohn Snow
Fix minor irritants to pylint/flake8 et al. (Yes, these need to be guarded by the Python tests. That's a work in progress, a series that's quite likely to follow once I finish this Sphinx project. Please pardon the temporary irritation.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240626222128.406106-3-jsnow@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>
2024-05-06qapi: Simplify QAPISchemaVariants @tag_memberMarkus Armbruster
For union types, the tag member is known only after .check(). We used to code this in a simple way: QAPISchemaVariants attribute .tag_member was None for union types until .check(). Since this complicated typing, recent commit "qapi/schema: fix typing for QAPISchemaVariants.tag_member" hid it behind a property. The previous commit lets us treat .tag_member just like the other attributes that become known only in .check(): declare, but don't initialize it in .__init__(). Signed-off-by: Markus Armbruster <armbru@redhat.com>
2024-05-06qapi: Move conditional code from QAPISchemaVariants to its subtypesMarkus Armbruster
QAPISchemaVariants.check()'s code is almost entirely conditional on union vs. alternate type. Move the conditional code to QAPISchemaBranches.check() and QAPISchemaAlternatives.check(), where the conditions are always satisfied. Attribute QAPISchemaVariants.tag_name is now only used by QAPISchemaBranches. Move it there. Refactor the three types' .__init__() to make them a bit simpler. Signed-off-by: Markus Armbruster <armbru@redhat.com>
2024-05-06qapi: Rename QAPISchemaAlternateType.variants to .alternativesMarkus Armbruster
A previous commit narrowed the type of QAPISchemaAlternateType.variants from QAPISchemaVariants to QAPISchemaAlternatives. Rename it to .alternatives. Same for .__init__() parameter @variants. Signed-off-by: Markus Armbruster <armbru@redhat.com>
2024-05-06qapi: Rename QAPISchemaObjectType.variants to .branchesMarkus Armbruster
A previous commit narrowed the type of QAPISchemaObjectType.variants from QAPISchemaVariants to QAPISchemaBranches. Rename it to .branches. Same for .__init__() parameter @variants. Signed-off-by: Markus Armbruster <armbru@redhat.com>
2024-05-06qapi: Rename visitor parameter @variants to @alternativesMarkus Armbruster
A previous commit narrowed the type of .visit_alternate_type() parameter @variants from QAPISchemaVariants to QAPISchemaAlternatives. Rename it to @alternatives. One of them passes @alternatives to helper function gen_visit_alternate(). Rename its @variants parameter to @alternatives as well. Signed-off-by: Markus Armbruster <armbru@redhat.com>
2024-05-06qapi: Rename visitor parameter @variants to @branchesMarkus Armbruster
The previous commit narrowed the type of .visit_object_type() parameter @variants from QAPISchemaVariants to QAPISchemaBranches. Rename it to @branches. Same for .visit_object_type_flat(). A few of these pass @branches to helper functions: QAPISchemaGenRSTVisitor.visit_object_type() to ._nodes_for_members() and ._nodes_for_variant_when(), and QAPISchemaGenVisitVisitor.visit_object_type() to gen_visit_object_members(). Rename the helpers' @variants parameters to @branches as well. Signed-off-by: Markus Armbruster <armbru@redhat.com>
2024-05-06qapi: New QAPISchemaBranches, QAPISchemaAlternativesMarkus Armbruster
QAPISchemaVariants represents either a union type's branches, or an alternate type's alternatives. Much of its code is conditional on which one it actually is. Create QAPISchemaBranches for branches, and QAPISchemaAlternatives for alternatives, both subtypes of QAPISchemaVariants. Replace QAPISchemaVariants by one of them where possible. Keep it only where we actually deal with either of them. QAPISchemaVariants.__init__() takes @tag_name and @tag_member, where exactly one must be None: @tag_name for alternatives, @tag_member for branches. Let QAPISchemaBranches.__init__() take just @tag_name, and QAPISchemaAlternatives.__init__() take just @tag_member. A later patch will move the conditional code to the subtypes. Signed-off-by: Markus Armbruster <armbru@redhat.com>
2024-04-24Merge tag 'pull-qapi-2024-04-24' of https://repo.or.cz/qemu/armbru into stagingRichard Henderson
QAPI patches patches for 2024-04-24 # -----BEGIN PGP SIGNATURE----- # # iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmYovX4SHGFybWJydUBy # ZWRoYXQuY29tAAoJEDhwtADrkYZT0RQP/1R1oOSdfWmiSR5gdW+IDE88VrjNY+Md # lZJ7dTtbIDbwABP6s7/wGIxlc4bf6lwIwhWNnfOa+y71jJ/u9aX2Q6D8wHY5lQnu # b8jhzoP3UyB+LQV5TTX1nTqTaO72Bj0pxw/0/DeKCzg7kgjys06a1Ln9P0oyWs6x # li/Cs133wPtZ5rISqif5yOmssber0h2D584k5MN2VK7eaGidLioQQIRmSDikPE6Z # TpnOEqySInIFhPJmm77il19ZDpCrgdCoD7lXoqX1C6XScYz7dU+m/TaToFk1lECw # VstR2SJT39TzbOLdis1O5/vsLP0QfciMRQbUktSrf4jQHumrkSa/OE6xKwJC3x82 # axJWc+BygcosylKc5CYVzwSlHugHw6Lf39qui//yzi5CXakzO6owKX7Q8AcH3PPy # 6thncy0dvw1ggq/BZGYhjyG+6MXBCWipPGVXFp9Gf2cAayTALhwyNtPvDNX57fzT # UZA/fa+/Wc9xZv1YAnxLaKyo4o65YWXVnq1eHXV17ny08BzZNYOC2ZuXim+rMThM # lxzcrTkrMLgsGXetp59uhZw9JRnVvaxLqNXfC2bwpoRzXzuifvnPithl6nkSm1YB # TvHGZZdO3B498kOW6947KrMVFh3t4aNWkdbOyetAMECy71H3Q4CTHYGFa5TzXT9O # Rjz31NYdVNBE # =uyfg # -----END PGP SIGNATURE----- # gpg: Signature made Wed 24 Apr 2024 01:06:22 AM PDT # gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653 # gpg: issuer "armbru@redhat.com" # gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [undefined] # gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [undefined] # gpg: WARNING: This key is not certified with a trusted signature! # gpg: There is no indication that the signature belongs to the owner. # Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653 * tag 'pull-qapi-2024-04-24' of https://repo.or.cz/qemu/armbru: (25 commits) qapi: Dumb down QAPISchema.lookup_entity() qapi: Tighten check whether implicit object type already exists qapi/schema: remove unnecessary asserts qapi/schema: turn on mypy strictness qapi/schema: add type hints qapi/parser.py: assert member.info is present in connect_member qapi/parser: demote QAPIExpression to Dict[str, Any] qapi/schema: assert inner type of QAPISchemaVariants in check_clash() qapi/schema: fix typing for QAPISchemaVariants.tag_member qapi/schema: Don't initialize "members" with `None` qapi/schema: add _check_complete flag qapi/schema: assert info is present when necessary qapi/schema: fix QAPISchemaArrayType.check's call to resolve_type qapi: Assert built-in types exist qapi/schema: assert resolve_type has 'info' and 'what' args on error qapi/schema: add type narrowing to lookup_type() qapi/schema: adjust type narrowing for mypy's benefit qapi/schema: make c_type() and json_type() abstract methods qapi/schema: declare type for QAPISchemaArrayType.element_type qapi/schema: declare type for QAPISchemaObjectTypeMember.type ... Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2024-04-24qapi: Dumb down QAPISchema.lookup_entity()Markus Armbruster
QAPISchema.lookup_entity() takes an optional type argument, a subtype of QAPISchemaDefinition, and returns that type or None. Callers can use this to save themselves an isinstance() test. The only remaining user of this convenience feature is .lookup_type(). But we don't actually save anything anymore there: we still need the isinstance() to help mypy over the hump. Drop the .lookup_entity() argument, and adjust .lookup_type(). Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-26-armbru@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> [Commit message typo fixed]
2024-04-24qapi: Tighten check whether implicit object type already existsMarkus Armbruster
Entities with names starting with q_obj_ are implicit object types. Therefore, QAPISchema._make_implicit_object_type()'s .lookup_entity() can only return a QAPISchemaObjectType. Assert that. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-25-armbru@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: John Snow <jsnow@redhat.com>
2024-04-24qapi/schema: remove unnecessary assertsJohn Snow
With strict typing enabled, these runtime statements aren't necessary anymore; we can prove them statically. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-24-armbru@redhat.com>
2024-04-24qapi/schema: turn on mypy strictnessJohn Snow
This patch can be rolled in with the previous one once the series is ready for merge, but for work-in-progress' sake, it's separate here. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-23-armbru@redhat.com>
2024-04-24qapi/schema: add type hintsJohn Snow
This patch only adds type hints, which aren't utilized at runtime and don't change the behavior of this module in any way. In a scant few locations, type hints are removed where no longer necessary due to inference power from typing all of the rest of creation; and any type hints that no longer need string quotes are changed. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-22-armbru@redhat.com>
2024-04-24qapi/parser.py: assert member.info is present in connect_memberJohn Snow
Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-21-armbru@redhat.com>
2024-04-24qapi/parser: demote QAPIExpression to Dict[str, Any]John Snow
Dict[str, object] is a stricter type, but with the way that code is currently arranged, it is infeasible to enforce this strictness. In particular, although expr.py's entire raison d'être is normalization and type-checking of QAPI Expressions, that type information is not "remembered" in any meaningful way by mypy because each individual expression is not downcast to a specific expression type that holds all the details of each expression's unique form. As a result, all of the code in schema.py that deals with actually creating type-safe specialized structures has no guarantee (myopically) that the data it is being passed is correct. There are two ways to solve this: (1) Re-assert that the incoming data is in the shape we expect it to be, or (2) Disable type checking for this data. (1) is appealing to my sense of strictness, but I gotta concede that it is asinine to re-check the shape of a QAPIExpression in schema.py when expr.py has just completed that work at length. The duplication of code and the nightmare thought of needing to update both locations if and when we change the shape of these structures makes me extremely reluctant to go down this route. (2) allows us the chance to miss updating types in the case that types are updated in expr.py, but it *is* an awful lot simpler and, importantly, gets us closer to type checking schema.py *at all*. Something is better than nothing, I'd argue. So, do the simpler dumber thing and worry about future strictness improvements later. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-20-armbru@redhat.com>
2024-04-24qapi/schema: assert inner type of QAPISchemaVariants in check_clash()John Snow
QAPISchemaVariant's "variants" field is typed as List[QAPISchemaVariant], where the typing for QAPISchemaVariant allows its type field to be any QAPISchemaType. However, QAPISchemaVariant expects that all of its variants contain the narrower QAPISchemaObjectType. This relationship is enforced at runtime in QAPISchemaVariants.check(). This relationship is not embedded in the type system though, so QAPISchemaVariants.check_clash() needs to re-assert this property in order to call QAPISchemaVariant.type.check_clash(). Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-19-armbru@redhat.com>
2024-04-24qapi/schema: fix typing for QAPISchemaVariants.tag_memberJohn Snow
There are two related changes here: (1) We need to perform type narrowing for resolving the type of tag_member during check(), and (2) tag_member is a delayed initialization field, but we can hide it behind a property that raises an Exception if it's called too early. This simplifies the typing in quite a few places and avoids needing to assert that the "tag_member is not None" at a dozen callsites, which can be confusing and suggest the wrong thing to a drive-by contributor. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-18-armbru@redhat.com>
2024-04-24qapi/schema: Don't initialize "members" with `None`John Snow
Declare, but don't initialize the "members" field with type List[QAPISchemaObjectTypeMember]. This simplifies the typing from what would otherwise be Optional[List[T]] to merely List[T]. This removes the need to add assertions to several callsites that this value is not None - which it never will be after the delayed initialization in check() anyway. The type declaration without initialization trick will cause accidental uses of this field prior to full initialization to raise an AttributeError. (Note that it is valid to have an empty members list, see the internal q_empty object as an example. For this reason, we cannot use the empty list as a replacement test for full initialization and instead rely on the _checked/_check_complete fields.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-17-armbru@redhat.com>
2024-04-24qapi/schema: add _check_complete flagJohn Snow
Instead of using the None value for the members field, use a dedicated flag to detect recursive misconfigurations. This is intended to assist with subsequent patches that seek to remove the "None" value from the members field (which can never hold that value after the final call to check()) in order to simplify the static typing of that field; avoiding the need of assertions littered at many callsites to eliminate the possibility of the None value. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-16-armbru@redhat.com>
2024-04-24qapi/schema: assert info is present when necessaryJohn Snow
QAPISchemaInfo arguments can often be None because built-in definitions don't have such information. The type hint can only be Optional[QAPISchemaInfo] then. But, mypy gets upset about all the places where we exploit that it can't actually be None there. Add assertions that will help mypy over the hump, to enable adding type hints in a forthcoming commit. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-15-armbru@redhat.com>
2024-04-24qapi/schema: fix QAPISchemaArrayType.check's call to resolve_typeJohn Snow
Adjust the expression at the callsite to work around mypy's weak type introspection that believes this expression can resolve to QAPISourceInfo; it cannot. (Fundamentally: self.info only resolves to false in a boolean expression when it is None; therefore this expression may only ever produce Optional[str]. mypy does not know that 'info', when it is a QAPISourceInfo object, cannot ever be false.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-14-armbru@redhat.com>
2024-04-24qapi: Assert built-in types existMarkus Armbruster
QAPISchema.lookup_type('FOO') returns a QAPISchemaType when type 'FOO' exists, else None. It won't return None for built-in types like 'int'. Since mypy can't see that, it'll complain that we assign the Optional[QAPISchemaType] returned by .lookup_type() to QAPISchemaType variables. Add assertions to help it over the hump. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-13-armbru@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com>
2024-04-24qapi/schema: assert resolve_type has 'info' and 'what' args on errorJohn Snow
resolve_type() is generally used to resolve configuration-provided type names into type objects, and generally requires valid 'info' and 'what' parameters. In some cases, such as with QAPISchemaArrayType.check(), resolve_type may be used to resolve built-in types and as such will not have an 'info' argument, but also must not fail in this scenario. Use an assertion to sate mypy that we will indeed have 'info' and 'what' parameters for the error pathway in resolve_type. Note: there are only three callsites to resolve_type at present where "info" is perceived by mypy to be possibly None: 1) QAPISchemaArrayType.check() 2) QAPISchemaObjectTypeMember.check() 3) QAPISchemaEvent.check() Of those three, only the first actually ever passes None; the other two are limited by their base class initializers which accept info=None, but neither subclass actually use a None value in practice, currently. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-12-armbru@redhat.com>
2024-04-24qapi/schema: add type narrowing to lookup_type()John Snow
This function is a bit hard to type as-is; mypy needs some assertions to assist with the type narrowing. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-11-armbru@redhat.com>
2024-04-24qapi/schema: adjust type narrowing for mypy's benefitJohn Snow
We already take care to perform some type narrowing for arg_type and ret_type, but not in a way where mypy can utilize the result once we add type hints, e.g.: qapi/schema.py:833: error: Incompatible types in assignment (expression has type "QAPISchemaType", variable has type "Optional[QAPISchemaObjectType]") [assignment] qapi/schema.py:893: error: Incompatible types in assignment (expression has type "QAPISchemaType", variable has type "Optional[QAPISchemaObjectType]") [assignment] A simple change to use a temporary variable helps the medicine go down. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-10-armbru@redhat.com>
2024-04-24qapi/schema: make c_type() and json_type() abstract methodsJohn Snow
These methods should always return a str, it's only the default abstract implementation that doesn't. They can be marked "abstract", which requires subclasses to override the method with the proper return type. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-9-armbru@redhat.com>
2024-04-24qapi/schema: declare type for QAPISchemaArrayType.element_typeJohn Snow
A QAPISchemaArrayType's element type gets resolved only during .check(). We have QAPISchemaArrayType.__init__() initialize self.element_type = None, and .check() assign the actual type. Using .element_type before .check() is wrong, and hopefully crashes due to the value being None. Works. However, it makes for awkward typing. With .element_type: Optional[QAPISchemaType], mypy is of course unable to see that it's None before .check(), and a QAPISchemaType after. To help it over the hump, we'd have to assert self.element_type is not None before all the (valid) uses. The assertion catches invalid uses, but only at run time; mypy can't flag them. Instead, declare .element_type in .__init__() as QAPISchemaType *without* initializing it. Using .element_type before .check() now certainly crashes, which is an improvement. Mypy still can't flag invalid uses, but that's okay. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-8-armbru@redhat.com>
2024-04-24qapi/schema: declare type for QAPISchemaObjectTypeMember.typeJohn Snow
A QAPISchemaObjectTypeMember's type gets resolved only during .check(). We have QAPISchemaObjectTypeMember.__init__() initialize self.type = None, and .check() assign the actual type. Using .type before .check() is wrong, and hopefully crashes due to the value being None. Works. However, it makes for awkward typing. With .type: Optional[QAPISchemaType], mypy is of course unable to see that it's None before .check(), and a QAPISchemaType after. To help it over the hump, we'd have to assert self.type is not None before all the (valid) uses. The assertion catches invalid uses, but only at run time; mypy can't flag them. Instead, declare .type in .__init__() as QAPISchemaType *without* initializing it. Using .type before .check() now certainly crashes, which is an improvement. Mypy still can't flag invalid uses, but that's okay. Addresses typing errors such as these: qapi/schema.py:657: error: "None" has no attribute "alternate_qtype" [attr-defined] qapi/schema.py:662: error: "None" has no attribute "describe" [attr-defined] Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-7-armbru@redhat.com>
2024-04-24qapi: create QAPISchemaDefinitionJohn Snow
Include entities don't have names, but we generally expect "entities" to have names. Reclassify all entities with names as *definitions*, leaving the nameless include entities as QAPISchemaEntity instances. This is primarily to help simplify typing around expectations of what callers expect for properties of an "entity". Suggested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-6-armbru@redhat.com>
2024-04-24qapi/schema: add pylint suppressionsJohn Snow
With this patch, pylint is happy with the file, so enable it in the configuration. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-5-armbru@redhat.com>
2024-04-24qapi: sort pylint suppressionsJohn Snow
Suggested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-4-armbru@redhat.com>
2024-04-24qapi/parser: shush up pylintJohn Snow
Shhh! Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-3-armbru@redhat.com>
2024-04-24qapi/parser: fix typo - self.returns.info => self.errors.infoJohn Snow
Small copy-pasto. The correct info field to use in this conditional block is self.errors.info. Fixes: 3a025d3d1ffa Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-2-armbru@redhat.com>
2024-04-24qapi: Inline and remove QERR_INVALID_PARAMETER_TYPE definitionPhilippe Mathieu-Daudé
Address the comment added in commit 4629ed1e98 ("qerror: Finally unused, clean up"), from 2015: /* * These macros will go away, please don't use * in new code, and do not add new ones! */ Manual changes (escaping the format in qapi/visit.py). Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240312141343.3168265-8-armbru@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
2024-03-04qapi: Reject "Returns" section when command doesn't return anythingMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-14-armbru@redhat.com>
2024-03-04qapi: New documentation section tag "Errors"Markus Armbruster
We use section "Returns" for documenting both success and error response of commands. I intend to generate better command success response documentation. Easier when "Returns" documents just he success response. Create new section tag "Errors". The next two commits will move error response documentation from "Returns" sections to "Errors" sections. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-4-armbru@redhat.com>
2024-03-04qapi: Slightly clearer error message for invalid "Returns" sectionMarkus Armbruster
Change "'Returns:' is only valid for commands" to "'Returns' section is only valid for commands". Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-3-armbru@redhat.com>
2024-03-04qapi: Memorize since & returns sectionsMarkus Armbruster
This is chiefly to make code that looks up these sections easier to read. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240227113921.236097-2-armbru@redhat.com>
2024-02-26qapi: Divorce QAPIDoc from QAPIParseErrorMarkus Armbruster
QAPIDoc stores a reference to QAPIParser just to pass it to QAPIParseError. The resulting error position depends on the state of the parser. It happens to be the current comment line. Servicable, but action at a distance. The commit before previous moved most uses of QAPIParseError from QAPIDoc to QAPIParser. There are just three left. Convert them to QAPISemError. This involves passing info to a few methods. Then drop the reference to QAPIParser. The three errors lose the column number. Not really interesting here: it's the comment line's indentation. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240216145841.2099240-17-armbru@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2024-02-26qapi: Reject multiple and empty feature descriptionsMarkus Armbruster
The parser recognizes only the first "Features:" line. Any subsequent ones are treated as ordinary text, as visible in test case doc-duplicate-features. Recognize "Features:" lines anywhere. A second one is an error. A 'Features:' line without any features is useless, but not an error. Make it an error. This makes detecting a second "Features:" line easier. qapi/run-state.json actually has an instance of this since commit fe17522d854 (qapi: Remove deprecated 'singlestep' member of StatusInfo). Clean it up. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240216145841.2099240-16-armbru@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2024-02-26qapi: Rewrite doc comment parserMarkus Armbruster
QAPISchemaParser is a conventional recursive descent parser. Except QAPISchemaParser.get_doc() delegates most of the doc comment parsing work to a state machine in QAPIDoc. The state machine doesn't get tokens like a recursive descent parser, it is fed tokens. I find this state machine rather opaque and hard to maintain. Replace it by a conventional parser, all in QAPISchemaParser. Less code, and (at least in my opinion) easier to understand. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240216145841.2099240-15-armbru@redhat.com> Tested-by: Daniel P. Berrangé <berrange@redhat.com>
2024-02-26qapi: Merge adjacent untagged sectionsMarkus Armbruster
The parser mostly doesn't create adjacent untagged sections, and merging the ones it does create is hardly worth the bother. I'm doing it to avoid behavioral change in the next commit. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240216145841.2099240-14-armbru@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2024-02-26qapi: Call QAPIDoc.check() alwaysMarkus Armbruster
We currently call QAPIDoc.check() only for definition documentation. Calling it for free-form documentation as well is simpler. No change, because it doesn't actually do anything there. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240216145841.2099240-13-armbru@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-21 21:34:24 +0000