aboutsummaryrefslogtreecommitdiff
path: root/test/functional/test_framework
AgeCommit message (Collapse)Author
2017-10-05[tests] do not allow assert_raises_message to be called with JSONRPCExceptionJohn Newbery
2017-10-05[tests] remove direct testing on JSONRPCException from individual test casesJohn Newbery
2017-09-29Merge #11319: [qa] Fix error introduced into p2p-segwit.py, and prevent ↵MarcoFalke
future similar errors f97ab35fa qa: Fix bug introduced in p2p-segwit.py (Suhas Daftuar) a7820422e qa: Treat mininode p2p exceptions as fatal (Suhas Daftuar) Pull request description: #11121 inadvertently broke the constructor for the `TestNode()` object in `p2p-segwit.py`, silently breaking at least one of the tests. Although the python code was raising exceptions due to a `TestNode()` object not existing (or having the right type), mininode was masking these from anyone running the test through the test_runner (like travis), because it catches all exceptions during message delivery and just prints a log message and continues. Such "graceful" handling of errors is almost certainly something we don't want in our test suite, so the first commit here attempts to prevent that type of failure from ever being masked. The second commit fixes the particular bug in `p2p-segwit.py`. Tree-SHA512: b6646e3cb1e05c35c28e8899c44104bf2e2d0384643ca87042ab7f6ec0960d89f5bf25a7b95bab6e32d401c20a6018226160500f6ddceb923e81ffb04adb4f2f
2017-09-29Merge #10552: [Test] Tests for zmqpubrawtx and zmqpubrawblockMarcoFalke
d3677ab75 Tests for zmqpubrawtx and zmqpubrawblock (Andrew Chow) Pull request description: Tree-SHA512: 9e367fd8936514bfb567ef3f3d83770d374287354b59c9187e844056dd086e8aa2de32ce55d35486cecd706e7c93cd1c1e2709ee82d3dddb805827be8d2bcb14
2017-09-29Merge #11167: Full BIP173 (Bech32) supportWladimir J. van der Laan
8213838 [Qt] tolerate BIP173/bech32 addresses during input validation (Jonas Schnelli) 06eaca6 [RPC] Wallet: test importing of native witness scripts (NicolasDorier) fd0041a Use BIP173 addresses in segwit.py test (Pieter Wuille) e278f12 Support BIP173 in addwitnessaddress (Pieter Wuille) c091b99 Implement BIP173 addresses and tests (Pieter Wuille) bd355b8 Add regtest testing to base58_tests (Pieter Wuille) 6565c55 Convert base58_tests from type/payload to scriptPubKey comparison (Pieter Wuille) 8fd2267 Import Bech32 C++ reference code & tests (Pieter Wuille) 1e46ebd Implement {Encode,Decode}Destination without CBitcoinAddress (Pieter Wuille) Pull request description: Builds on top of #11117. This adds support for: * Creating BIP173 addresses for testing (through `addwitnessaddress`, though by default it still produces P2SH versions) * Sending to BIP173 addresses (including non-v0 ones) * Analysing BIP173 addresses (through `validateaddress`) It includes a reformatted version of the [C++ Bech32 reference code](https://github.com/sipa/bech32/tree/master/ref/c%2B%2B) and an independent implementation of the address encoding/decoding logic (integrated with CTxDestination). All BIP173 test vectors are included. Not included (and intended for other PRs): * Full wallet support for SegWit (which would include automatically adding witness scripts to the wallet during automatic keypool topup, SegWit change outputs, ...) [see #11403] * Splitting base58.cpp and tests/base58_tests.cpp up into base58-specific code, and "address encoding"-code [see #11372] * Error locating in UI for BIP173 addresses. Tree-SHA512: 238031185fd07f3ac873c586043970cc2db91bf7735c3c168cb33a3db39a7bda81d4891b649685bb17ef90dc63af0328e7705d8cd3e8dafd6c4d3c08fb230341
2017-09-28Use BIP173 addresses in segwit.py testPieter Wuille
2017-09-20[test] Add restart_node to BitcoinTestFrameworkJoão Barbosa
2017-09-18Tests for zmqpubrawtx and zmqpubrawblockAndrew Chow
2017-09-18Merge #11323: mininode: add an optimistic write and disable nagleMarcoFalke
1817398b3 mininode: add an optimistic write and disable nagle (Cory Fields) Pull request description: Disclaimer: I'm not familiar with asyncore, so I'm unclear how safe this is. It works for me (tm). Because the poll/select loop may pause for 100msec before actually doing a send, and we have no way to force the loop awake, try sending from the calling thread if the queue is empty. Also, disable nagle as all sends should be either full messages or unfinished sends. This shaves an average of ~1 minute or so off of my accumulated runtime, and 10-15 seconds off of actual runtime. Tree-SHA512: 6b61b8058e621dacf0b4dd353c10e3666fbda0691440eb6ebc432491ebada80a781dcd09291bf03e70112a41d3c2a0c91775ed08824b79bf8d0ebed11595c28b
2017-09-13mininode: add an optimistic write and disable nagleCory Fields
Because the poll/select loop may pause for 100msec before actually doing a send, and we have no way to force the loop awake, try sending from the calling thread if the queue is empty. Also, disable nagle as all sends should be either full messages or unfinished sends. This shaves an average of ~1 minute or so off of my accumulated runtime, and 10-15 seconds off of actual runtime.
2017-09-13Merge #11078: [tests] Make p2p-leaktests.py more robustMarcoFalke
0063d2c3d [tests] Make p2p-leaktests.py more robust (John Newbery) Pull request description: There has been an example of p2p-leaktests.py failing on travis in the new service bits test (introduced in #11001 . It appeared to me that the previous p2p connections had not been fully disconnected before attempting to add new p2p connections. I've added a sleep and restarted the NetworkThread, but I don't know whether this will fix the problem, since I'm unable to reproduce the failure locally. @MarcoFalke - not sure what you want to do here? I don't think this change could make things any worse. Tree-SHA512: f5427c26267185a903c9b75bb3925bf153b8afce70c8e493bf8f585f57d809d20643b4ee69081300b211d22e960242aecc3d719f4ddd230aa08fdc5484b55055
2017-09-13qa: Treat mininode p2p exceptions as fatalSuhas Daftuar
2017-09-07[test] Replace check_output with low level versionJoão Barbosa
2017-09-07[test] Add assert_raises_process_error to assert process errorsJoão Barbosa
2017-09-07[test] Add support for custom arguments to TestNodeCLIJoão Barbosa
2017-09-07[test] Improve assert_raises_jsonrpc docstringJoão Barbosa
2017-09-06[qa] TestNode: Add wait_until_stopped helper methodMarcoFalke
2017-09-01Merge #11121: TestNode tidyupsMarcoFalke
7148b74dc [tests] Functional tests must explicitly set num_nodes (John Newbery) 5448a1471 [tests] don't override __init__() in individual tests (John Newbery) 6cf094a02 [tests] Avoid passing around member variables in test_framework (John Newbery) 36b626867 [tests] TestNode: separate add_node from start_node (John Newbery) be2a2ab6a [tests] fix - use rpc_timeout as rpc timeout (John Newbery) Pull request description: Some additional tidyups after the introduction of TestNode: - commit 1 makes TestNode use the correct rpc timeout. This should have been included in #11077 - commit 2 separates `add_node()` from `start_node()` as originally discussed here: https://github.com/bitcoin/bitcoin/pull/10556#discussion_r121161453 with @kallewoof . The test writer no longer needs to assign to `self.nodes` when starting/stopping nodes. - commit 3 adds a `set_test_params()` method, so individual tests don't need to override `__init__()` and call `super().__init__()` Tree-SHA512: 0adb030623b96675b5c29e2890ce99ccd837ed05f721d0c91b35378c5ac01b6658174aac12f1f77402e1d38b61f39b3c43b4df85c96952565dde1cda05b0db84
2017-09-01[tests] Functional tests must explicitly set num_nodesJohn Newbery
2017-09-01[tests] don't override __init__() in individual testsJohn Newbery
Almost all test scripts currently need to override the __init__() method. When they do that they need to call into super().__init__() as the base class does some generic initialization. This commit makes the base class __init__() call into set_test_params() method. Individual test cases can override set_test_params() to setup their test parameters.
2017-09-01[tests] Avoid passing around member variables in test_frameworkJohn Newbery
2017-09-01[tests] TestNode: separate add_node from start_nodeJohn Newbery
Separates the act of creating a TestNode object from starting the node. The test_framework now keeps track of its list of TestNodes, and test writers can call start_node() and stop_node() without having to update the self.nodes list.
2017-09-01[tests] fix - use rpc_timeout as rpc timeoutJohn Newbery
2017-08-28Use the variable name _ for unused return valuespracticalswift
2017-08-28Remove unused variables and/or function callspracticalswift
2017-08-24[tests] Make p2p-leaktests.py more robustJohn Newbery
2017-08-24Merge #10798: [tests] [utils] test bitcoin-cliMarcoFalke
c6ec4358a [tests] Add bitcoin_cli.py test script (John Newbery) b23549f6e [tests] add TestNodeCLI class for calling bitcoin-cli for a node (John Newbery) Pull request description: We don't test bitcoin-cli at all. That means that we can miss inconsistencies between the bitcoin-cli client and the RPC interface, such as #10698 and #10747. It also means that the various bitcoin-cli options and features are untested and regressions could be silently introduced. Let's fix that. This PR adds bitcoin-cli testing in the python functional test_framework: 1. Add a bitcoin_cli.py test script that tests bitcoin-cli. At the moment it only tests that the result of `getinfo` is the same if you run it as an RPC or through bitcoin-cli, but can easily be extended to test additional bitcoin-cli features **EDIT: `--usecli` option is moved to a separate PR. This PR now only covers the bitcoin_cli.py test.** 2. ~Add a `--usecli` option to the test framework. This changes the test to use bitcoin-cli for all RPC calls instead of using direct HTTP requests. This is somewhat experimental. It works for most tests, but there are some cases where it can't work transparently because:~ - ~the testcase is asserting on a specific error code, and bitcoin-cli returns a different error code from the direct RPC~ - ~we're sending a very large RPC request (eg `submitblock`) and it can't be serialized into a shell bitcoin-cli call.~ ~I think that even though `--usecli` doesn't work on all tests, it's still a useful experimental feature. Future potential enhancements:~ - ~enhance the framework to automatically skip tests that are known to fail with bitcoin-cli if the `--usecli` option is used.~ - ~run a subset of tests in Travis with `-usecli`~ This builds on and requires the `TestNode` PR #10711 . As an aside, this is a good demonstration of how tidy it is to add additional features/interfaces now that test node logic/state is encapsulated in a TestNode class. Addresses #10791 Tree-SHA512: a1e6be12e8e007f6f67b3d3bbcd142d835787300831eb38e6027a1ad25ca9d79c4bc99a41b19e31ee95205cba1b3b2d21a688b5909316aad70bfc2b4eb6d8a52
2017-08-23Merge #11068: qa: Move wait_until to utilMarcoFalke
08ce33f8e qa: Move wait_until to util (MarcoFalke) Pull request description: This moves `wait_until` to `util.py` to make it generally available to python tests. Also, `wait_until` now takes an optional lock that is acquired while testing the predicate. Previously the lock was always acquired, even when it was not necessary, cf. `disconnect_ban.py`. Tree-SHA512: 18e452a017a6566fa8ad09bde058e1b841e167039dc63299e70cfa7a6dcbc779581e60ca3e8eb2f1b610767d5208b9376c203eb11015b250fd0542b5eb4215a8
2017-08-23[tests] fix timeout issues from TestNodeJohn Newbery
Fixes a couple of bugs from the introduction of TestNode: - test scripts were no longer able to specify a custom timeout for starting a node. Therefore tests with nodes that take a long time to start up (eg pruning.py) would fail. - the test for whether a node has failed on start up was broken by changing 'assert x is None' to 'assert not x'. Since subprocess.poll() can return None (indicating the node is still running) or 0 (indicating the node exited with return code 0), this was a regression.
2017-08-20test: Increase initial RPC timeout to 60 secondsWladimir J. van der Laan
When running the tests locally with a parallelism of 4 on an otherwise busy system, RPC can take quite a wait to come up. Change the timeout to 60 seconds just to be safe.
2017-08-17qa: Move wait_until to utilMarcoFalke
2017-08-15[tests] add TestNodeCLI class for calling bitcoin-cli for a nodeJohn Newbery
2017-08-15Merge #10711: [tests] Introduce TestNodeMarcoFalke
789733891 [tests] Introduce TestNode (John Newbery) Pull request description: Continues #10082 TestNode is a class responsible for all state related to a bitcoind node under test. It stores local state, is responsible for tracking the bitcoind process and delegates unrecognised messages to the RPC connection. This commit changes start_nodes and stop_nodes to start and stop the bitcoind nodes in parallel, making test setup and teardown much faster. On my vm, this changeset reduces total test_runner runtime for the base set of tests (including building the cache) from 263s to 195s (a 25% speedup). Note that the time reported by test_runner does not include time spent building the cache: *with TestNode*: ``` → date +"%T" ; ./test_runner.py -q ; date +"%T" 12:48:04 .................................................................................................................................................................................................................................................................................................................................. TEST | STATUS | DURATION abandonconflict.py | ✓ Passed | 12 s bip68-112-113-p2p.py | ✓ Passed | 19 s blockchain.py | ✓ Passed | 8 s bumpfee.py | ✓ Passed | 13 s decodescript.py | ✓ Passed | 3 s disablewallet.py | ✓ Passed | 3 s disconnect_ban.py | ✓ Passed | 6 s fundrawtransaction.py | ✓ Passed | 37 s getchaintips.py | ✓ Passed | 4 s httpbasics.py | ✓ Passed | 3 s import-rescan.py | ✓ Passed | 4 s importmulti.py | ✓ Passed | 6 s importprunedfunds.py | ✓ Passed | 3 s invalidblockrequest.py | ✓ Passed | 4 s invalidtxrequest.py | ✓ Passed | 4 s keypool.py | ✓ Passed | 7 s listsinceblock.py | ✓ Passed | 4 s listtransactions.py | ✓ Passed | 33 s mempool_limit.py | ✓ Passed | 4 s mempool_persist.py | ✓ Passed | 15 s mempool_reorg.py | ✓ Passed | 4 s mempool_resurrect_test.py | ✓ Passed | 3 s mempool_spendcoinbase.py | ✓ Passed | 3 s merkle_blocks.py | ✓ Passed | 3 s multi_rpc.py | ✓ Passed | 4 s net.py | ✓ Passed | 3 s nulldummy.py | ✓ Passed | 3 s p2p-compactblocks.py | ✓ Passed | 28 s p2p-fullblocktest.py | ✓ Passed | 126 s p2p-leaktests.py | ✓ Passed | 8 s p2p-mempool.py | ✓ Passed | 3 s p2p-segwit.py | ✓ Passed | 59 s p2p-versionbits-warning.py | ✓ Passed | 8 s preciousblock.py | ✓ Passed | 3 s prioritise_transaction.py | ✓ Passed | 5 s proxy_test.py | ✓ Passed | 3 s rawtransactions.py | ✓ Passed | 9 s receivedby.py | ✓ Passed | 19 s reindex.py | ✓ Passed | 12 s rest.py | ✓ Passed | 9 s rpcnamedargs.py | ✓ Passed | 3 s segwit.py | ✓ Passed | 7 s sendheaders.py | ✓ Passed | 24 s signmessages.py | ✓ Passed | 3 s signrawtransactions.py | ✓ Passed | 3 s txn_clone.py | ✓ Passed | 4 s txn_doublespend.py --mineblock | ✓ Passed | 4 s uptime.py | ✓ Passed | 3 s wallet-accounts.py | ✓ Passed | 3 s wallet-dump.py | ✓ Passed | 7 s wallet-encryption.py | ✓ Passed | 8 s wallet-hd.py | ✓ Passed | 15 s wallet.py | ✓ Passed | 31 s walletbackup.py | ✓ Passed | 104 s zapwallettxes.py | ✓ Passed | 9 s zmq_test.py | ○ Skipped | 0 s ALL | ✓ Passed | 735 s (accumulated) Runtime: 189 s 12:51:19 ``` *master*: ``` → date +"%T" ; ./test_runner.py -q ; date +"%T" 12:40:13 .......................................................................................................................................................................................................................................................................................................................................................................................................................................... TEST | STATUS | DURATION abandonconflict.py | ✓ Passed | 15 s bip68-112-113-p2p.py | ✓ Passed | 19 s blockchain.py | ✓ Passed | 8 s bumpfee.py | ✓ Passed | 20 s decodescript.py | ✓ Passed | 3 s disablewallet.py | ✓ Passed | 3 s disconnect_ban.py | ✓ Passed | 8 s fundrawtransaction.py | ✓ Passed | 36 s getchaintips.py | ✓ Passed | 11 s httpbasics.py | ✓ Passed | 7 s import-rescan.py | ✓ Passed | 16 s importmulti.py | ✓ Passed | 10 s importprunedfunds.py | ✓ Passed | 5 s invalidblockrequest.py | ✓ Passed | 4 s invalidtxrequest.py | ✓ Passed | 3 s keypool.py | ✓ Passed | 7 s listsinceblock.py | ✓ Passed | 11 s listtransactions.py | ✓ Passed | 37 s mempool_limit.py | ✓ Passed | 4 s mempool_persist.py | ✓ Passed | 23 s mempool_reorg.py | ✓ Passed | 7 s mempool_resurrect_test.py | ✓ Passed | 3 s mempool_spendcoinbase.py | ✓ Passed | 3 s merkle_blocks.py | ✓ Passed | 10 s multi_rpc.py | ✓ Passed | 6 s net.py | ✓ Passed | 6 s nulldummy.py | ✓ Passed | 3 s p2p-compactblocks.py | ✓ Passed | 30 s p2p-fullblocktest.py | ✓ Passed | 126 s p2p-leaktests.py | ✓ Passed | 8 s p2p-mempool.py | ✓ Passed | 3 s p2p-segwit.py | ✓ Passed | 62 s p2p-versionbits-warning.py | ✓ Passed | 8 s preciousblock.py | ✓ Passed | 8 s prioritise_transaction.py | ✓ Passed | 7 s proxy_test.py | ✓ Passed | 10 s rawtransactions.py | ✓ Passed | 15 s receivedby.py | ✓ Passed | 28 s reindex.py | ✓ Passed | 12 s rest.py | ✓ Passed | 12 s rpcnamedargs.py | ✓ Passed | 3 s segwit.py | ✓ Passed | 12 s sendheaders.py | ✓ Passed | 26 s signmessages.py | ✓ Passed | 3 s signrawtransactions.py | ✓ Passed | 3 s txn_clone.py | ✓ Passed | 10 s txn_doublespend.py --mineblock | ✓ Passed | 10 s uptime.py | ✓ Passed | 3 s wallet-accounts.py | ✓ Passed | 3 s wallet-dump.py | ✓ Passed | 6 s wallet-encryption.py | ✓ Passed | 8 s wallet-hd.py | ✓ Passed | 18 s wallet.py | ✓ Passed | 69 s walletbackup.py | ✓ Passed | 130 s zapwallettxes.py | ✓ Passed | 15 s zmq_test.py | ○ Skipped | 0 s ALL | ✓ Passed | 936 s (accumulated) Runtime: 242 s 12:44:36 ``` Tree-SHA512: 6dfc4c11fd0caf7de6954c93679cf22c3df0acc6f432e616d1151062a61f456faa8ae2fe670b427868af55bb564802df84c8fd76e90b4b338750dbc23f46ad88
2017-08-15[tests] Introduce TestNodeJohn Newbery
TestNode is a class responsible for all state related to a bitcoind node under test. It stores local state, is responsible for tracking the bitcoind process and delegates unrecognised messages to the RPC connection. This commit changes start_nodes and stop_nodes to start and stop the bitcoind nodes in parallel, making test setup and teardown much faster.
2017-08-11Merge #11023: [tests] Add option to attach a python debugger if functional ↵MarcoFalke
test fails cc5d38f4b Add option to attach a python debugger if test fails (John Newbery) Pull request description: Adds a simple option to the test_framework to attach pdb if the test fails. Helpful for catching and debugging intermittent failures: Run the test in a loop with this option. The first failure will cause execution to pause and nodes will be left running for interactive debugging. @sdaftuar Tree-SHA512: 01cfae15fa3f04ed6ec6a99fef60a6c6a59723429309e81eacd6767caf12f5758f59b337804291ecab33a38a2958f36e2b513d201bee72a2eeb207a67046f952
2017-08-10Add option to attach a python debugger if test failsJohn Newbery
2017-08-07[tests] Test disconnecting unsupported service bits logic.John Newbery
In v0.15, we disconnect nodes that send us version messages with unsupported service bits (1 << 5 and 1 << 7). This commit adds a test that bitcoind will disconnect those nodes before August 1st 2018, and won't disconnect those nodes after August 1st 2018.
2017-07-17[tests] [wallet] Add wallet endpoint support to authproxyJohn Newbery
2017-07-06Fix multi_rpc test for hosts that dont default to utf8Matt Corallo
Otherwise the utf8 written to bitcoin.conf throws an exception when read from get_auth_cookie
2017-07-02[tests] nits in dbcrash.pyJohn Newbery
2017-06-29[tests] fix nits.John Newbery
Thanks to Marco Falke.
2017-06-29[tests] reorganize utils.py module (code move only)John Newbery
This commit re-organizes the utils.py module into logical sections.
2017-06-29[tests] move mocktime property and functions to BitcoinTestFrameworkJohn Newbery
2017-06-29[tests] Move stop_node and start_node methods to BitcoinTestFrameworkJohn Newbery
This commit moves functions start_node, start_nodes, stop_node and stop_nodes functions into the BitcoinTestFramework class. It also moves the bitcoind_processes dict and coverage variables into BitcoinTestFramework.
2017-06-29[tests] fix flake8 warnings in test_framework.py and util.pyJohn Newbery
2017-06-29[tests] remove unused imports from utils.pyJohn Newbery
2017-06-28Merge #10148: Use non-atomic flushing with block replayWladimir J. van der Laan
176c021 [qa] Test non-atomic chainstate writes (Suhas Daftuar) d6af06d Dont create pcoinsTip until after ReplayBlocks. (Matt Corallo) eaca1b7 Random db flush crash simulator (Pieter Wuille) 0580ee0 Adapt memory usage estimation for flushing (Pieter Wuille) 013a56a Non-atomic flushing using the blockchain as replay journal (Pieter Wuille) b3a279c [MOVEONLY] Move LastCommonAncestor to chain (Pieter Wuille) Tree-SHA512: 47ccc62303f9075c44d2a914be75bd6969ff881a857a2ff1227f05ec7def6f4c71c46680c5a28cb150c814999526797dc05cf2701fde1369c06169f46eccddee
2017-06-28Merge #10690: [qa] Bugfix: allow overriding extra_args in ↵MarcoFalke
ComparisonTestFramework 4ed3653 [qa] Bugfix: allow overriding extra_args in ComparisonTestFramework (Suhas Daftuar) Tree-SHA512: d8f724b3324aad73a7b15cf87ff394e8d615bf3cd5a394d5715347d833f2ae9ac745a944202986866255eca5fc105ea06ab3abe12e168b67de34482f751c68e2
2017-06-28[qa] Bugfix: allow overriding extra_args in ComparisonTestFrameworkSuhas Daftuar
2017-06-26[qa] Test non-atomic chainstate writesSuhas Daftuar
Adds new functional test, dbcrash.py, which uses -dbcrashratio to exercise the logic for recovering from a crash during chainstate flush. dbcrash.py is added to the extended tests, as it may take ~10 minutes to run Use _Exit() instead of exit() for crash simulation This eliminates stderr output such as: terminate called without an active exception or Assertion failed: (!pthread_mutex_destroy(&m)), function ~recursive_mutex, file /usr/local/include/boost/thread/pthread/recursive_mutex.hpp, line 104. Eliminating the stderr output on crash simulation allows testing with test_runner.py, which reports a test as failed if stderr is produced.
generated by cgit v1.2.3 (git 2.26.2) at 2024-10-30 00:33:14 +0000