aboutsummaryrefslogtreecommitdiff
path: root/tests
AgeCommit message (Collapse)Author
2018-09-24Merge remote-tracking branch 'remotes/xanclic/tags/pull-block-2018-08-31-v2' ↵Peter Maydell
into staging Block patches: - (Block) job exit refactoring, part 1 (removing job_defer_to_main_loop()) - test-bdrv-drain leak fix # gpg: Signature made Fri 31 Aug 2018 15:30:33 BST # gpg: using RSA key F407DB0061D5CF40 # gpg: Good signature from "Max Reitz <mreitz@redhat.com>" # Primary key fingerprint: 91BE B60A 30DB 3E88 57D1 1829 F407 DB00 61D5 CF40 * remotes/xanclic/tags/pull-block-2018-08-31-v2: jobs: remove job_defer_to_main_loop jobs: remove ret argument to job_completed; privatize it block/backup: make function variables consistently named jobs: utilize job_exit shim block/mirror: utilize job_exit shim block/commit: utilize job_exit shim jobs: add exit shim jobs: canonize Error object jobs: change start callback to run callback tests: fix bdrv-drain leak Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-09-24Revert "tests: add qmp/qom-set-without-value test"Peter Maydell
Markus spotted some issues with this new test case which unfortunately I didn't notice had been flagged until after I'd applied the pull request. Revert the relevant commit. This reverts commit 2b70ea92766f5a1a735a44e28c92cdfba3c4054f. Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-09-24Merge remote-tracking branch ↵Peter Maydell
'remotes/huth-gitlab/tags/pull-request-2018-08-31' into staging Removal of deprecated options and improvements for the qtests # gpg: Signature made Fri 31 Aug 2018 09:10:23 BST # gpg: using RSA key 2ED9D774FE702DB5 # gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" # gpg: aka "Thomas Huth <thuth@redhat.com>" # gpg: aka "Thomas Huth <huth@tuxfamily.org>" # gpg: aka "Thomas Huth <th.huth@posteo.de>" # Primary key fingerprint: 27B8 8847 EEE0 2501 18F3 EAB9 2ED9 D774 FE70 2DB5 * remotes/huth-gitlab/tags/pull-request-2018-08-31: tests: add a qmp success-response test tests: add qmp/qom-set-without-value test tests: add qmp/object-add-without-props test tests: add qmp_assert_error_class() tests/libqos: Utilize newer glib spawn check net: Remove the deprecated -tftp, -bootp, -redir and -smb options Remove the deprecated options -startdate, -localtime and -rtc-td-hack Remove the deprecated -nodefconfig option Remove the deprecated -balloon option Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-09-24Merge remote-tracking branch 'remotes/cohuck/tags/s390x-20180829' into stagingPeter Maydell
- various fixes and improvements in the tcg code - split off the individual virtio-ccw devices into separate files # gpg: Signature made Wed 29 Aug 2018 10:38:03 BST # gpg: using RSA key DECF6B93C6F02FAF # gpg: Good signature from "Cornelia Huck <conny@cornelia-huck.de>" # gpg: aka "Cornelia Huck <huckc@linux.vnet.ibm.com>" # gpg: aka "Cornelia Huck <cornelia.huck@de.ibm.com>" # gpg: aka "Cornelia Huck <cohuck@kernel.org>" # gpg: aka "Cornelia Huck <cohuck@redhat.com>" # Primary key fingerprint: C3D0 D66D C362 4FF6 A8C0 18CE DECF 6B93 C6F0 2FAF * remotes/cohuck/tags/s390x-20180829: target/s390x: use regular spaces in translate.c hw/s390x: Move virtio-ccw-blk code to a separate file hw/s390x: Move virtio-ccw-net code to a separate file hw/s390x: Move virtio-ccw-input code to a separate file hw/s390x: Move virtio-ccw-gpu code to a separate file hw/s390x: Move vhost-vsock-ccw code to a separate file hw/s390x: Move virtio-ccw-crypto code to a separate file hw/s390x: Move virtio-ccw-9p code to a separate file hw/s390x: Move virtio-ccw-rng code to a separate file hw/s390x: Move virtio-ccw-scsi code to a separate file hw/s390x: Move virtio-ccw-balloon code to a separate file hw/s390x: Move virtio-ccw-serial code to a separate file hw/s390x/virtio-ccw: Consolidate calls to virtio_ccw_unrealize() target/s390x: fix PACK reading 1 byte less and writing 1 byte more target/s390x: add EX support for TRT and TRTR target/s390x: fix IPM polluting irrelevant bits target/s390x: fix CSST decoding and runtime alignment check target/s390x: add BAL and BALR instructions tests/tcg: add a simple s390x test Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-08-31jobs: utilize job_exit shimJohn Snow
Utilize the job_exit shim by not calling job_defer_to_main_loop, and where applicable, converting the deferred callback into the job_exit callback. This converts backup, stream, create, and the unit tests all at once. Most of these jobs do not see any changes to the order in which they clean up their resources, except the test-blockjob-txn test, which now puts down its bs before job_completed is called. This is safe for the same reason the reordering in the mirror job is safe, because job_completed no longer runs under two locks, making the unref safe even if it causes a flush. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20180830015734.19765-7-jsnow@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-08-31jobs: canonize Error objectJohn Snow
Jobs presently use both an Error object in the case of the create job, and char strings in the case of generic errors elsewhere. Unify the two paths as just j->err, and remove the extra argument from job_completed. The integer error code for job_completed is kept for now, to be removed shortly in a separate patch. Signed-off-by: John Snow <jsnow@redhat.com> Message-id: 20180830015734.19765-3-jsnow@redhat.com [mreitz: Dropped a superfluous g_strdup()] Reviewed-by: Eric Blake <eblake@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-08-31jobs: change start callback to run callbackJohn Snow
Presently we codify the entry point for a job as the "start" callback, but a more apt name would be "run" to clarify the idea that when this function returns we consider the job to have "finished," except for any cleanup which occurs in separate callbacks later. As part of this clarification, change the signature to include an error object and a return code. The error ptr is not yet used, and the return code while captured, will be overwritten by actions in the job_completed function. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-id: 20180830015734.19765-2-jsnow@redhat.com Reviewed-by: Jeff Cody <jcody@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-08-31tests: fix bdrv-drain leakMarc-André Lureau
Spotted by ASAN: ================================================================= ==5378==ERROR: LeakSanitizer: detected memory leaks Direct leak of 65536 byte(s) in 1 object(s) allocated from: #0 0x7f788f83bc48 in malloc (/lib64/libasan.so.5+0xeec48) #1 0x7f788c9923c5 in g_malloc (/lib64/libglib-2.0.so.0+0x523c5) #2 0x5622a1fe37bc in coroutine_trampoline /home/elmarco/src/qq/util/coroutine-ucontext.c:116 #3 0x7f788a15d75f in __correctly_grouped_prefixwc (/lib64/libc.so.6+0x4c75f) (Broken in commit 4c8158e359d.) Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Message-id: 20180809114417.28718-3-marcandre.lureau@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-08-31tests: add a qmp success-response testMarc-André Lureau
Verify the usage of this schema feature and the API behaviour. This should be the only case where qmp_dispatch() returns NULL. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2018-08-31tests: add qmp/qom-set-without-value testMarc-André Lureau
test_qom_set_without_value() is about a bug in infrastructure used by the QMP core, fixed in commit c489780203. We covered the bug in infrastructure unit tests (commit bce3035a44). I wrote that test earlier, to cover QMP level as well, the test could go into qmp-test. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2018-08-31tests: add qmp/object-add-without-props testMarc-André Lureau
test_object_add_without_props() tests a bug in qmp_object_add() we fixed in commit e64c75a975. Sadly, we don't have systematic object-add tests. This lone test can go into qmp-cmd-test for want of a better home. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2018-08-31tests: add qmp_assert_error_class()Marc-André Lureau
This helper will simplify a bunch of code checking for QMP errors and can be shared by various tests. Note that test-qga does check for error description as well, so don't replace the code there for now. Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2018-08-31tests/libqos: Utilize newer glib spawn checkEric Blake
During development, I got a 'make check' failure that claimed: qemu-img returned status code 32512 ** ERROR:tests/libqos/libqos.c:202:mkimg: assertion failed: (!rc) But 32512 is too big for a normal exit status value, which means we failed to use WEXITSTATUS() to shift the bits to the desired value for printing. However, instead of worrying about how to portably parse g_spawn()'s rc in the proper platform-dependent manner, it's better to just rely on the fact that we now require glib 2.40 (since commit e7b3af815) and can therefore use glib's portable checker instead, where the message under my same condition improves to: Child process exited with code 127 ** ERROR:tests/libqos/libqos.c:192:mkimg: assertion failed: (ret && !err) Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>
2018-08-28qapi: Drop qapi_event_send_FOO()'s Error ** argumentPeter Xu
The generated qapi_event_send_FOO() take an Error ** argument. They can't actually fail, because all they do with the argument is passing it to functions that can't fail: the QObject output visitor, and the @qmp_emit callback, which is either monitor_qapi_event_queue() or event_test_emit(). Drop the argument, and pass &error_abort to the QObject output visitor and @qmp_emit instead. Suggested-by: Eric Blake <eblake@redhat.com> Suggested-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Peter Xu <peterx@redhat.com> Message-Id: <20180815133747.25032-4-peterx@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Commit message rewritten, update to qapi-code-gen.txt corrected] Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-08-28target/s390x: fix PACK reading 1 byte less and writing 1 byte morePavel Zbitskiy
PACK fails on the test from the Principles of Operation: F1F2F3F4 becomes 0000234C instead of 0001234C due to an off-by-one error. Furthermore, it overwrites one extra byte to the left of F1. If len_dest is 0, then we only want to flip the 1st byte and never loop over the rest. Therefore, the loop condition should be > and not >=. If len_src is 1, then we should flip the 1st byte and pack the 2nd. Since len_src is already decremented before the loop, the first condition should be >=, and not >. Likewise for len_src == 2 and the second condition. Signed-off-by: Pavel Zbitskiy <pavel.zbitskiy@gmail.com> Message-Id: <20180821025104.19604-7-pavel.zbitskiy@gmail.com> Reviewed-by: David Hildenbrand <david@redhat.com> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2018-08-28target/s390x: add EX support for TRT and TRTRPavel Zbitskiy
Improves "b213c9f5: target/s390x: Implement TRTR" by introducing the intermediate functions, which are compatible with dx_helper type. Signed-off-by: Pavel Zbitskiy <pavel.zbitskiy@gmail.com> Message-Id: <20180821025104.19604-6-pavel.zbitskiy@gmail.com> Reviewed-by: David Hildenbrand <david@redhat.com> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2018-08-28target/s390x: fix IPM polluting irrelevant bitsPavel Zbitskiy
Suppose psw.mask=0x0000000080000000, cc=2, r1=0 and we do "ipm 1". This command must touch only bits 32-39, so the expected output is r1=0x20000000. However, currently qemu yields r1=0x20008000, because irrelevant parts of PSW leak into r1 during program mask transfer. Signed-off-by: Pavel Zbitskiy <pavel.zbitskiy@gmail.com> Message-Id: <20180821025104.19604-5-pavel.zbitskiy@gmail.com> Reviewed-by: David Hildenbrand <david@redhat.com> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2018-08-28target/s390x: fix CSST decoding and runtime alignment checkPavel Zbitskiy
CSST is defined as: C(0xc802, CSST, SSF, CASS, la1, a2, 0, 0, csst, 0) It means that the first parameter is handled by in1_la1(). in1_la1() fills addr1 field, and not in1. Furthermore, when extract32() is used for the alignment check, the third parameter should specify the number of trailing bits that must be 0. For FC these numbers are: FC=0 (word, 4 bytes): 2 FC=1 (double word, 8 bytes): 3 FC=2 (quad word, 16 bytes): 4 For SC these numbers correspond to the size: SC=0: 0 SC=1: 1 SC=2: 2 SC=3: 3 SC=4: 4 Signed-off-by: Pavel Zbitskiy <pavel.zbitskiy@gmail.com> Message-Id: <20180821025104.19604-4-pavel.zbitskiy@gmail.com> Reviewed-by: David Hildenbrand <david@redhat.com> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2018-08-28tests/tcg: add a simple s390x testPavel Zbitskiy
Copied from alpha. Signed-off-by: Pavel Zbitskiy <pavel.zbitskiy@gmail.com> Message-Id: <20180821025104.19604-2-pavel.zbitskiy@gmail.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: David Hildenbrand <david@redhat.com> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2018-08-27check: Move wdt_ib700 test to commonJuan Quintela
It is protected by CONFIG_WDT_IB700. Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com>
2018-08-27check: Move endianess test to commonJuan Quintela
It is already protected by CONFIG_ISA_TESTDEV in all architectures. Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-08-27check: Move VMXNET3 test to commonJuan Quintela
We protect it with CONFIG_VMXNET3_PCI now, so no need to also put it on i386. Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-08-27check: Only test boot-serial when sga is compiled inJuan Quintela
This is only for x86* architecture. Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-08-27check: Only test ivshm when it is compiled inJuan Quintela
Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-08-27check: Only test usb-xhci-nec when it is compiled inJuan Quintela
Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-08-27check: Only test isa-testdev when it is compiled inJuan Quintela
Once there, untangle endianness-test and boot-serial-test. Signed-off-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> --- boot-serial-test don't depend on isa-testdev. Thanks Thomas.
2018-08-24json: Support %% in JSON strings when interpolatingMarkus Armbruster
The previous commit makes JSON strings containing '%' awkward to express in templates: you'd have to mask the '%' with an Unicode escape \u0025. No template currently contains such JSON strings. Support the printf conversion specification %% in JSON strings as a convenience anyway, because it's trivially easy to do. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-58-armbru@redhat.com>
2018-08-24json: Improve safety of qobject_from_jsonf_nofail() & friendsMarkus Armbruster
The JSON parser optionally supports interpolation. This is used to build QObjects by parsing string templates. The templates are C literals, so parse errors (such as invalid interpolation specifications) are actually programming errors. Consequently, the functions providing parsing with interpolation (qobject_from_jsonf_nofail(), qobject_from_vjsonf_nofail(), qdict_from_jsonf_nofail(), qdict_from_vjsonf_nofail()) pass &error_abort to the parser. However, there's another, more dangerous kind of programming error: since we use va_arg() to get the value to interpolate, behavior is undefined when the variable argument isn't consistent with the interpolation specification. The same problem exists with printf()-like functions, and the solution is to have the compiler check consistency. This is what GCC_FMT_ATTR() is about. To enable this type checking for interpolation as well, we carefully chose our interpolation specifications to match printf conversion specifications, and decorate functions parsing templates with GCC_FMT_ATTR(). Note that this only protects against undefined behavior due to type errors. It can't protect against use of invalid interpolation specifications that happen to be valid printf conversion specifications. However, there's still a gaping hole in the type checking: GCC recognizes '%' as start of printf conversion specification anywhere in the template, but the parser recognizes it only outside JSON strings. For instance, if someone were to pass a "{ '%s': %d }" template, GCC would require a char * and an int argument, but the parser would va_arg() only an int argument, resulting in undefined behavior. Avoid undefined behavior by catching the programming error at run time: have the parser recognize and reject '%' in JSON strings. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-57-armbru@redhat.com>
2018-08-24tests/drive_del-test: Fix harmless JSON interpolation bugMarkus Armbruster
test_after_failed_device_add() does this: response = qmp("{'execute': 'device_add'," " 'arguments': {" " 'driver': 'virtio-blk-%s'," " 'drive': 'drive0'" "}}", qvirtio_get_dev_type()); Wrong. An interpolation specification must be a JSON token, it doesn't work within JSON string tokens. The code above doesn't use the value of qvirtio_get_dev_type(), and sends arguments {"driver": "virtio-blk-%s", "drive": "drive0"}} The command fails because there is no driver named "virtio-blk-%". Harmless, since the test wants the command to fail. Screwed up in commit 2f84a92ec63. Fix the obvious way. The command now fails because the drive is empty, like it did before commit 2f84a92ec63. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-55-armbru@redhat.com>
2018-08-24json: Clean up headersMarkus Armbruster
The JSON parser has three public headers, json-lexer.h, json-parser.h, json-streamer.h. They all contain stuff that is of no interest outside qobject/json-*.c. Collect the public interface in include/qapi/qmp/json-parser.h, and everything else in qobject/json-parser-int.h. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-54-armbru@redhat.com>
2018-08-24qjson: Have qobject_from_json() & friends reject empty and blankMarkus Armbruster
The last case where qobject_from_json() & friends return null without setting an error is empty or blank input. Callers: * block.c's parse_json_protocol() reports "Could not parse the JSON options". It's marked as a work-around, because it also covered actual bugs, but they got fixed in the previous few commits. * qobject_input_visitor_new_str() reports "JSON parse error". Also marked as work-around. The recent fixes have made this unreachable, because it currently gets called only for input starting with '{'. * check-qjson.c's empty_input() and blank_input() demonstrate the behavior. * The other callers are not affected since they only pass input with exactly one JSON value or, in the case of negative tests, one error. Fail with "Expecting a JSON value" instead of returning null, and simplify callers. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-48-armbru@redhat.com>
2018-08-24json: Fix streamer not to ignore trailing unterminated structuresMarkus Armbruster
json_message_process_token() accumulates tokens until it got the sequence of tokens that comprise a single JSON value (it counts curly braces and square brackets to decide). It feeds those token sequences to json_parser_parse(). If a non-empty sequence of tokens remains at the end of the parse, it's silently ignored. check-qjson.c cases unterminated_array(), unterminated_array_comma(), unterminated_dict(), unterminated_dict_comma() demonstrate this bug. Fix as follows. Introduce a JSON_END_OF_INPUT token. When the streamer receives it, it feeds the accumulated tokens to json_parser_parse(). Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-46-armbru@redhat.com>
2018-08-24qjson: Fix qobject_from_json() & friends for multiple valuesMarkus Armbruster
qobject_from_json() & friends use the consume_json() callback to receive either a value or an error from the parser. When they are fed a string that contains more than either one JSON value or one JSON syntax error, consume_json() gets called multiple times. When the last call receives a value, qobject_from_json() returns that value. Any other values are leaked. When any call receives an error, qobject_from_json() sets the first error received. Any other errors are thrown away. When values follow errors, qobject_from_json() returns both a value and sets an error. That's bad. Impact: * block.c's parse_json_protocol() ignores and leaks the value. It's used to to parse pseudo-filenames starting with "json:". The pseudo-filenames can come from the user or from image meta-data such as a QCOW2 image's backing file name. * vl.c's parse_display_qapi() ignores and leaks the error. It's used to parse the argument of command line option -display. * vl.c's main() case QEMU_OPTION_blockdev ignores the error and leaves it in @err. main() will then pass a pointer to a non-null Error * to net_init_clients(), which is forbidden. It can lead to assertion failure or other misbehavior. * check-qjson.c's multiple_values() demonstrates the badness. * The other callers are not affected since they only pass strings with exactly one JSON value or, in the case of negative tests, one error. The impact on the _nofail() functions is relatively harmless. They abort when any call receives an error. Else they return the last value, and leak the others, if any. Fix consume_json() as follows. On the first call, save value and error as before. On subsequent calls, if any, don't save them. If the first call saved a value, the next call, if any, replaces the value by an "Expecting at most one JSON value" error. Take care not to leak values or errors that aren't saved. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-44-armbru@redhat.com>
2018-08-24json: Replace %I64d, %I64u by %PRId64, %PRIu64Markus Armbruster
Support for %I64d got added in commit 2c0d4b36e7f "json: fix PRId64 on Win32". We had to hard-code I64d because we used the lexer's finite state machine to check interpolations. No more, so clean this up. Additional conversion specifications would be easy enough to implement when needed. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-42-armbru@redhat.com>
2018-08-24json: Leave rejecting invalid interpolation to parserMarkus Armbruster
Both lexer and parser reject invalid interpolation specifications. The parser's check is useless. The lexer ends the token right after the first bad character. This tends to lead to suboptimal error reporting. For instance, input [ %04d ] produces the tokens JSON_LSQUARE [ JSON_ERROR %0 JSON_INTEGER 4 JSON_KEYWORD d JSON_RSQUARE ] The parser then yields an error, an object and two more errors: error: Invalid JSON syntax object: 4 error: JSON parse error, invalid keyword error: JSON parse error, expecting value Dumb down the lexer to accept [A-Za-z0-9]*. The parser's check is now used. Emit a proper error there. The lexer now produces JSON_LSQUARE [ JSON_INTERP %04d JSON_RSQUARE ] and the parser reports just JSON parse error, invalid interpolation '%04d' Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-41-armbru@redhat.com>
2018-08-24json: Pass lexical errors and limit violations to callbackMarkus Armbruster
The callback to consume JSON values takes QObject *json, Error *err. If both are null, the callback is supposed to make up an error by itself. This sucks. qjson.c's consume_json() neglects to do so, which makes qobject_from_json() null instead of failing. I consider that a bug. The culprit is json_message_process_token(): it passes two null pointers when it runs into a lexical error or a limit violation. Fix it to pass a proper Error object then. Update the callbacks: * monitor.c's handle_qmp_command(): the code to make up an error is now dead, drop it. * qga/main.c's process_event(): lumps the "both null" case together with the "not a JSON object" case. The former is now gone. The error message "Invalid JSON syntax" is misleading for the latter. Improve it to "Input must be a JSON object". * qobject/qjson.c's consume_json(): no update; check-qjson demonstrates qobject_from_json() now sets an error on lexical errors, but still doesn't on some other errors. * tests/libqtest.c's qmp_response(): the Error object is now reliable, so use it to improve the error message. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-40-armbru@redhat.com>
2018-08-24json: Treat unwanted interpolation as lexical errorMarkus Armbruster
The JSON parser optionally supports interpolation. The lexer recognizes interpolation tokens unconditionally. The parser rejects them when interpolation is disabled, in parse_interpolation(). However, it neglects to set an error then, which can make json_parser_parse() fail without setting an error. Move the check for unwanted interpolation from the parser's parse_interpolation() into the lexer's finite state machine. When interpolation is disabled, '%' is now handled like any other unexpected character. The next commit will improve how such lexical errors are handled. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-39-armbru@redhat.com>
2018-08-24json: Redesign the callback to consume JSON valuesMarkus Armbruster
The classical way to structure parser and lexer is to have the client call the parser to get an abstract syntax tree, the parser call the lexer to get the next token, and the lexer call some function to get input characters. Another way to structure them would be to have the client feed characters to the lexer, the lexer feed tokens to the parser, and the parser feed abstract syntax trees to some callback provided by the client. This way is more easily integrated into an event loop that dispatches input characters as they arrive. Our JSON parser is kind of between the two. The lexer feeds tokens to a "streamer" instead of a real parser. The streamer accumulates tokens until it got the sequence of tokens that comprise a single JSON value (it counts curly braces and square brackets to decide). It feeds those token sequences to a callback provided by the client. The callback passes each token sequence to the parser, and gets back an abstract syntax tree. I figure it was done that way to make a straightforward recursive descent parser possible. "Get next token" becomes "pop the first token off the token sequence". Drawback: we need to store a complete token sequence. Each token eats 13 + input characters + malloc overhead bytes. Observations: 1. This is not the only way to use recursive descent. If we replaced "get next token" by a coroutine yield, we could do without a streamer. 2. The lexer reports errors by passing a JSON_ERROR token to the streamer. This communicates the offending input characters and their location, but no more. 3. The streamer reports errors by passing a null token sequence to the callback. The (already poor) lexical error information is thrown away. 4. Having the callback receive a token sequence duplicates the code to convert token sequence to abstract syntax tree in every callback. 5. Known bug: the streamer silently drops incomplete token sequences. This commit rectifies 4. by lifting the call of the parser from the callbacks into the streamer. Later commits will address 3. and 5. The lifting removes a bug from qjson.c's parse_json(): it passed a pointer to a non-null Error * in certain cases, as demonstrated by check-qjson.c. json_parser_parse() is now unused. It's a stupid wrapper around json_parser_parse_err(). Drop it, and rename json_parser_parse_err() to json_parser_parse(). Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-35-armbru@redhat.com>
2018-08-24check-qjson: Fix and enable utf8_string()'s disabled partMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-31-armbru@redhat.com>
2018-08-24json: Fix \uXXXX for surrogate pairsMarkus Armbruster
The JSON parser treats each half of a surrogate pair as unpaired surrogate. Fix it to recognize surrogate pairs. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-30-armbru@redhat.com>
2018-08-24json: Reject invalid \uXXXX, fix \u0000Markus Armbruster
The JSON parser translates invalid \uXXXX to garbage instead of rejecting it, and swallows \u0000. Fix by using mod_utf8_encode() instead of flawed wchar_to_utf8(). Valid surrogate pairs are now differently broken: they're rejected instead of translated to garbage. The next commit will fix them. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-29-armbru@redhat.com>
2018-08-24json: Accept overlong \xC0\x80 as U+0000 ("modified UTF-8")Markus Armbruster
Since the JSON grammer doesn't accept U+0000 anywhere, this merely exchanges one kind of parse error for another. It's purely for consistency with qobject_to_json(), which accepts \xC0\x80 (see commit e2ec3f97680). Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-26-armbru@redhat.com>
2018-08-24json: Reject invalid UTF-8 sequencesMarkus Armbruster
We reject bytes that can't occur in valid UTF-8 (\xC0..\xC1, \xF5..\xFF in the lexer. That's insufficient; there's plenty of invalid UTF-8 not containing these bytes, as demonstrated by check-qjson: * Malformed sequences - Unexpected continuation bytes - Missing continuation bytes after start bytes other than \xC0..\xC1, \xF5..\xFD. * Overlong sequences with start bytes other than \xC0..\xC1, \xF5..\xFD. * Invalid code points Fixing this in the lexer would be bothersome. Fixing it in the parser is straightforward, so do that. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-23-armbru@redhat.com>
2018-08-24check-qjson: Document we expect invalid UTF-8 to be rejectedMarkus Armbruster
The JSON parser rejects some invalid sequences, but accepts others without correcting the problem. We should either reject all invalid sequences, or minimize overlong sequences and replace all other invalid sequences by a suitable replacement character. A common choice for replacement is U+FFFD. I'm going to implement the former. Update the comments in utf8_string() to expect this. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-22-armbru@redhat.com>
2018-08-24json: Reject unescaped control charactersMarkus Armbruster
Fix the lexer to reject unescaped control characters in JSON strings, in accordance with RFC 8259 "The JavaScript Object Notation (JSON) Data Interchange Format". Bonus: we now recover more nicely from unclosed strings. E.g. {"one: 1}\n{"two": 2} now recovers cleanly after the newline, where before the lexer remained confused until the next unpaired double quote or lexical error. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-19-armbru@redhat.com>
2018-08-24check-qjson: Cover interpolation more thoroughlyMarkus Armbruster
Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-17-armbru@redhat.com>
2018-08-24check-qjson qmp-test: Cover control characters more thoroughlyMarkus Armbruster
RFC 8259 "The JavaScript Object Notation (JSON) Data Interchange Format" requires control characters in strings to be escaped. Demonstrate the JSON parser accepts U+0001 .. U+001F unescaped. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-16-armbru@redhat.com>
2018-08-24check-qjson: Fix utf8_string() to test all invalid sequencesMarkus Armbruster
Some of utf8_string()'s test_cases[] contain multiple invalid sequences. Testing that qobject_from_json() fails only tests we reject at least one invalid sequence. That's incomplete. Additionally test each non-space sequence in isolation. This demonstrates that the JSON parser accepts invalid sequences starting with \xC2..\xF4. Add a FIXME comment. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-15-armbru@redhat.com>
2018-08-24check-qjson: Simplify utf8_string()Markus Armbruster
The previous commit made utf8_string()'s test_cases[].utf8_in superfluous: we can use .json_in instead. Except for the case testing U+0000. \x00 doesn't work in C strings, so it tests \\u0000 instead. But testing \\uXXXX is escaped_string()'s job. It's covered there. Test U+0001 here, and drop .utf8_in. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-14-armbru@redhat.com>
2018-08-24check-qjson: Cover UTF-8 in single quoted stringsMarkus Armbruster
utf8_string() tests only double quoted strings. Cover single quoted strings, too: store the strings to test without quotes, then wrap them in either kind of quote. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20180823164025.12553-13-armbru@redhat.com>
generated by cgit v1.2.3 (git 2.26.2) at 2024-12-25 03:54:21 +0000