aboutsummaryrefslogtreecommitdiff
path: root/test/functional/test_framework/test_node.py
AgeCommit message (Collapse)Author
2022-06-07test: Fix wait_for_debug_log UnicodeDecodeErrorMacroFake
2022-04-29test: stop node before calling assert_start_raises_init_errorMartin Zumsande
...in feature_coinstatsindex and feature_pruning. Also add an assert to assert_start_raises_init_error that the node is not already running.
2022-03-02Merge bitcoin/bitcoin#23304: wallet: Derive inactive HD chains in additional ↵laanwj
places c4d76c6faa3adf06f192649e169ca860ce420d30 tests: Tests for inactive HD chains (Andrew Chow) 8077862c5e8a3ed501f0baabc33536eb16922ceb wallet: Refactor TopUp to be able to top up inactive chains too (Andrew Chow) 70134eb34f58f0c572e7c3775e292d408f03b5ab wallet: Properly set hd chain counters when loading (Andrew Chow) 961b9e4e40019a87eaa11c8a9c3305870f7a6d75 wallet: Parse hdKeypath if key_origin is not available (Andrew Chow) 0652ee73ec880a66ec88bde007ee03c0b9d1b074 Add size check on meta.key_origin.path (Rob Fielding) Pull request description: Currently inactive HD chains are only derived from at the time a key in that chain is found to have been used. However, at that time, the wallet may not be able to derive keys (e.g. it is locked). Currently we would just move on and not derive any new keys, however this could result in missing funds. This PR resolves this problem by adding memory only variables to `CHDChain` which track the highest known index. `TopUp` is modified to always try to top up the inactive HD chains, and this process will use the new variables to determine how much to top up. In this way, after an encrypted wallet is unlocked, the inactive HD chains will be topped up and hopefully funds will not be missed. Note that because these variables are not persisted to disk (because `CHDChain`s for inactive HD chains are not written to disk), if an encrypted wallet is not unlocked in the same session as a key from an inactive chain is found to be used, then it will not be topped up later unless more keys are found. Additionally, wallets which do not have upgraded key metadata will not derive any keys from inactive HD chains. This is resolved by using the derivation path string in `CKeyMetadata.hdKeypath` to determine what indexes to derive. ACKs for top commit: laanwj: Code review ACK c4d76c6faa3adf06f192649e169ca860ce420d30 Tree-SHA512: b2b572ad7f1b1b2847edece09f7583543d63997e18ae32764e5a27ad608dd64b9bdb2d84ea27137894e986a8e82f047a3dba9c8015b74f5f179961911f0c4095
2022-02-22tests: Tests for inactive HD chainsAndrew Chow
test cases are added for inactive HD chains: a basic case, a case where the wallet is encrypted, and a case for the 21605 segfault.
2022-01-31test: Fix feature_init intermittent issuesMarcoFalke
2021-12-29test: add TestNode.wait_for_debug_logJames O'Beirne
2021-12-29test: introduce TestNode.debug_log_bytesJames O'Beirne
2021-12-14Merge bitcoin/bitcoin#22777: net processing: don't request tx relay on ↵MarcoFalke
feeler connections eaf6be0114a6d7763767da9496907fe8a670ff9e [net processing] Do not request transaction relay from feeler connections (John Newbery) 0220b834b175dc8c45a2c60213474a72c0ef8193 [test] Add testing for outbound feeler connections (John Newbery) Pull request description: Feelers are short-lived connections used to test the viability of peers. The bitcoind node will periodically open feeler connections to addresses in its addrman, wait for a `version` message from the peer, and then close the connection. Currently, we set `fRelay` to `1` in the `version` message for feeler connections, indicating that we want the peer to relay transactions to us. However, we close the connection immediately on receipt of the `version` message, and so never process any incoming transaction announcements. This PR changes that behaviour to instead set `fRelay` to `0` indicating that we do not wish to receive transaction announcements from the peer. This PR also extends the `addconnection` RPC to allow creating outbound feeler connections from the node to the test framework, and a test to verify that the node sets `fRelay` to `0` in the `version` message to feeler connections. ACKs for top commit: naumenkogs: ACK eaf6be0114a6d7763767da9496907fe8a670ff9e MarcoFalke: review ACK eaf6be0114a6d7763767da9496907fe8a670ff9e 🏃 Tree-SHA512: 1c56837dbd0a396fe404a5e39f7459864d15f666664d6b35ad109628b13158e077e417e586bf48946a23bd5cbe63716cb4bf22cdf8781b74dfce6047b87b465a
2021-11-30Merge bitcoin/bitcoin#23289: test: add stress tests for initializationW. J. van der Laan
d9803f7a0a33688f7429cf10384244f4770851ca test: add stress tests for initialization (James O'Beirne) 23f85616a8d9c9a1b054e492eca4d199028f34dc test: add node.chain_path and node.debug_log_path (James O'Beirne) Pull request description: In the course of coming up with a test plan for #23280, I thought it would be neat to include a Python snippet showing how I tested the initialization process. I quickly realized I was reinventing the functional test framework... so here's a new test. This change bangs init around like the Fonz hitting a jukebox. It adds some interesting (read: lazy and random) coverage for the initialization process by - interrupting init with SIGTERM after certain log statements, - interrupting init at random points, and - starting init with some essential data missing (block files, block indices, etc.) to test init error paths. As far as I can tell, some of these code paths are uncovered otherwise (namely the startup errors). --- Incidentally, I think I may have uncovered some kind of a bug or race condition with indexing initialization based on an intermittent failure in this testcase. This test sometimes fails after shutting down immediately after `loadblk` thread start: ``` 2021-10-15T21:14:51.295000Z TestFramework (INFO): Starting node and will exit after line 'loadblk thread start' 36 │ 2021-10-15T21:14:51.296000Z TestFramework.node0 (DEBUG): bitcoind started, waiting for RPC to come up 37 │ 2021-10-15T21:14:51.493000Z TestFramework (INFO): terminating node after 110 log lines seen 38 │ 2021-10-15T21:14:51.625000Z TestFramework (INFO): Starting node and will exit after line 'txindex thread start' 39 │ 2021-10-15T21:14:51.625000Z TestFramework.node0 (DEBUG): bitcoind started, waiting for RPC to come up ------> [[ FAILURE HERE ]] 2021-10-15T21:15:21.626000Z TestFramework (WARNING): missed line {bail_line}; bailing now after {num_lines} lines ``` and then fails to start up afterwards. Combined logs showing `Error: txindex best block of the index goes beyond pruned data`, when the node under test is not pruned: ``` node0 2021-10-15T21:16:51.848439Z [shutoff] [validationinterface.cpp:244] [ChainStateFlushed] Enqueuing ChainStateFlushed: block hash=1014bc4ff4917602ae53d10e9dfe230af4b7d52a6cdaa8a47798b9c288180907 node0 2021-10-15T21:16:51.848954Z [shutoff] [init.cpp:302] [Shutdown] Shutdown: done test 2021-10-15T21:16:51.882000Z TestFramework (ERROR): Unexpected exception caught during testing Traceback (most recent call last): File "/home/james/src/bitcoin/test/functional/test_framework/test_framework.py", line 132, in main self.run_test() File "/home/james/src/bitcoin/./test/functional/stress_init.py", line 87, in run_test check_clean_start() File "/home/james/src/bitcoin/./test/functional/stress_init.py", line 60, in check_clean_start node.wait_for_rpc_connection() File "/home/james/src/bitcoin/test/functional/test_framework/test_node.py", line 224, in wait_for_rpc_connection raise FailedToStartError(self._node_msg( test_framework.test_node.FailedToStartError: [node 0] bitcoind exited with status 1 during initialization test 2021-10-15T21:16:51.882000Z TestFramework (DEBUG): Closing down network thread test 2021-10-15T21:16:51.933000Z TestFramework (INFO): Stopping nodes test 2021-10-15T21:16:51.933000Z TestFramework.node0 (DEBUG): Stopping node node0 stderr Error: txindex best block of the index goes beyond pruned data. Please disable the index or reindex (which will download the whole blockchain again) node0 stderr Error: txindex best block of the index goes beyond pruned data. Please disable the index or reindex (which will download the whole blockchain again) node0 stderr Error: txindex best block of the index goes beyond pruned data. Please disable the index or reindex (which will download the whole blockchain again) node0 stderr Error: txindex best block of the index goes beyond pruned data. Please disable the index or reindex (which will download the whole blockchain again) node0 stderr Error: txindex best block of the index goes beyond pruned data. Please disable the index or reindex (which will download the whole blockchain again) ``` ACKs for top commit: laanwj: Code review ACK d9803f7a0a33688f7429cf10384244f4770851ca Tree-SHA512: 4d80dc399daf199a6222e81e47d12d830dc7af07355eddbb7f52479a676a645b8d3d45093ff54a9295f01a163b2f4fe0e038e83fc269969e03d4cfda69eaf111
2021-11-10scripted-diff: Bump copyright headersMarcoFalke
The previous diff touched most files in ./test/, so bump the headers to avoid having to touch them again for a bump later. -BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./test/ -END VERIFY SCRIPT-
2021-10-20test: add node.chain_path and node.debug_log_pathJames O'Beirne
To allow easier access to the node's datadir and debug logs.
2021-10-06test: Delete generate* calls from TestNodeMarcoFalke
2021-09-22[test] Add testing for outbound feeler connectionsJohn Newbery
Extend the addconnection RPC method to allow creating outbound feeler connections. Extend the test framework to accept those feeler connections.
2021-07-27test: add type annotations to util.get_rpc_proxyfanquake
Remove proxy.url assignment: error: "AuthServiceProxy" has no attribute "url"
2021-07-20Merge bitcoin/bitcoin#22096: p2p: AddrFetch - don't disconnect on ↵fanquake
self-announcements 5730a43703f7e5a5ca26245ba3b55fbdd027d0b6 test: Add functional test for AddrFetch connections (Martin Zumsande) c34ad3309f93979b274a37de013502b05d25fad8 net, rpc: Enable AddrFetch connections for functional testing (Martin Zumsande) 533500d9072b7d5a36a6491784bdeb9247e91fb0 p2p: Add timeout for AddrFetch peers (Martin Zumsande) b6c5d1e450dde6a54bd785504c923adfb45c7060 p2p: AddrFetch - don't disconnect on self-announcements (Martin Zumsande) Pull request description: AddrFetch connections (old name: oneshots) are intended to be short-lived connections on which we ask a peer for addresses via `getaddr` and disconnect after receiving them. This is done by disconnecting after receiving the first `addr`. However, it is no longer working as intended, because nowadays, the first `addr` a typical bitcoin core node sends is its self-announcement. So we'll disconnect before the peer gets a chance to answer our `getaddr`. I checked that this affects both `-seednode` peers specified manually, and DNS seeds when AddrFetch is used as a fallback if DNS doesn't work for us. The current behavior of getting peers via AddrFetch when starting with an empty addrman would be to connect to the peer, receive its self-announcement and add it to addrman, disconnect, reconnect to the same peer again as a full outbound (no other addresses in addrman) and then receive more `addr`. This is silly and not in line with AddrFetch peer being intended to be short-lived peers.  Fix this by only disconnecting after receiving an `addr` message of size > 1. [Edit] As per review discussion, this PR now also adds a timeout after which we disconnect if we haven't received any suitable `addr`, and a functional test. ACKs for top commit: amitiuttarwar: reACK 5730a43703f7e5a5ca26245ba3b55fbdd027d0b6 naumenkogs: ACK 5730a43703f7e5a5ca26245ba3b55fbdd027d0b6 jnewbery: ACK 5730a43703 Tree-SHA512: 8a81234f37e827705138eb254223f7f3b3bf44a06cb02126fc7990b0d231b9bd8f07d38d185cc30d55bf35548a6fdc286b69602498d875b937e7c58332158bf9
2021-07-12net, rpc: Enable AddrFetch connections for functional testingMartin Zumsande
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2021-06-03test: convert docs into type annotations in test_framework/test_node.pyfanquake
2021-04-29test: Assert that exit code indicates failureMarcoFalke
2021-02-23rpc: add external_signer option to createwalletSjors Provoost
2021-02-18Merge #19809: log: Prefix log messages with function name and source code ↵Wladimir J. van der Laan
location if -logsourcelocations is set b4511e2e2ed1a6077ae6826a9ee6b7a311293d08 log: Prefix log messages with function name if -logsourcelocations is set (practicalswift) Pull request description: Prefix log messages with function name if `-logfunctionnames` is set. Yes, exactly like `-logthreadnames` but for function names instead of thread names :) This is a small developer ergonomics improvement: I've found this to be a cheap/simple way to correlate log output and originating function. For me it beats the ordinary cycle of 1.) try to figure out a regexp matching the static part of the dynamic log message, 2.) `git grep -E 'Using .* MiB out of .* requested for signature cache'`, 3.) `mcedit filename.cpp` (`openemacs filename.cpp` works too!) and 4.) search for log message and scroll up to find the function name :) Without any logging parameters: ``` $ src/bitcoind -regtest 2020-08-25T03:29:04Z Using RdRand as an additional entropy source 2020-08-25T03:29:04Z Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements 2020-08-25T03:29:04Z Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements 2020-08-25T03:29:04Z Loaded best chain: hashBestChain=0fff88f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e22ff height=0 date=2011-02-02T23:16:42Z progress=1.000000 2020-08-25T03:29:04Z block tree size = 1 2020-08-25T03:29:04Z nBestHeight = 0 2020-08-25T03:29:04Z Imported mempool transactions from disk: 0 succeeded, 0 failed, 0 expired, 0 already there, 0 waiting for initial broadcast 2020-08-25T03:29:04Z 0 addresses found from DNS seeds ``` With `-logthreadnames` and `-logfunctionnames`: ``` $ src/bitcoind -regtest -logthreadnames -logfunctionnames 2020-08-25T03:29:04Z [init] [ReportHardwareRand] Using RdRand as an additional entropy source 2020-08-25T03:29:04Z [init] [InitSignatureCache] Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements 2020-08-25T03:29:04Z [init] [InitScriptExecutionCache] Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements 2020-08-25T03:29:04Z [init] [LoadChainTip] Loaded best chain: hashBestChain=0fff88f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e22ff height=0 date=2011-02-02T23:16:42Z progress=1.000000 2020-08-25T03:29:04Z [init] [AppInitMain] block tree size = 1 2020-08-25T03:29:04Z [init] [AppInitMain] nBestHeight = 0 2020-08-25T03:29:04Z [loadblk] [LoadMempool] Imported mempool transactions from disk: 0 succeeded, 0 failed, 0 expired, 0 already there, 0 waiting for initial broadcast 2020-08-25T03:29:04Z [dnsseed] [ThreadDNSAddressSeed] 0 addresses found from DNS seeds ``` ACKs for top commit: laanwj: Code review ACK b4511e2e2ed1a6077ae6826a9ee6b7a311293d08 MarcoFalke: review ACK b4511e2e2ed1a6077ae6826a9ee6b7a311293d08 🌃 Tree-SHA512: d100f5364630c323f31d275259864c597f7725e462d5f4bdedcc7033ea616d7fc0d16ef1b2af557e692f4deea73c6773ccfc681589e7bf6ba970b9ec169040c7
2021-02-17[test] Check user agent string from test framework connectionsJohn Newbery
Add a check that new connections from the test framework to the node have the correct user agent string. This makes bugs easier to detect if the user agent string ever changes.
2021-02-17[test] Move MY_SUBVERSION to p2p.pyJohn Newbery
The messages.py module should contain code and helpers for [de]serializing p2p messages. Specific usage of those messages should be in p2p.py. Therefore move MY_SUBVERSION to p2p.py. Also rename to P2P_SUBVERSION.
2021-01-23test: store subversion (user agent) as string in msg_versionSebastian Falbesoner
2021-01-15log: Prefix log messages with function name if -logsourcelocations is setpracticalswift
2021-01-07[test] Add test framework support to create outbound connections.Amiti Uttarwar
In the interest of increasing our P2P test coverage, add support to create full-relay or block-relay-only connections. To support this, a P2P connection spins up a listening thread & uses a callback to trigger the node initiating the connection. Co-authored-by: Anthony Towns <aj@erisian.com.au>
2020-12-17test: Fix restart node raceMarcoFalke
2020-12-10test: Use Popen.wait instead of RPC in assert_start_raises_init_errorMarcoFalke
2020-11-27[test] Fix wait condition in disconnect_p2psAmiti Uttarwar
MY_SUBVERSION is defined in messages.py as a byte string, but here we were comparing this value to the value returned by the RPC. Convert to ensure the types match.
2020-09-10[test] remove confusing p2p propertygzhao408
2020-08-27test: Rename wait until helper to wait_until_helperMarcoFalke
2020-08-25[test] Remove final references to mininodeJohn Newbery
2020-08-13Add loadwallet and createwallet RPC load_on_startup optionsRussell Yanofsky
This maintains a persistent list of wallets stored in settings that will automatically be loaded on startup. Being able to load a wallet automatically on startup will be more useful in the GUI when the option to create wallets is added in #15006, but it's reasonable to expose this feature by RPC as well.
2020-08-09Merge #19657: test: Wait until is_connected in add_p2p_connectionWladimir J. van der Laan
fa4dfd215f62e88d668311701735c332c264fa2a test: Wait until is_connected in add_p2p_connection (MarcoFalke) Pull request description: Moving the wait_until from the individual test scripts to the test framework simplifies two tests ACKs for top commit: jnewbery: Code review ACK fa4dfd215f62e88d668311701735c332c264fa2a theStack: ACK https://github.com/bitcoin/bitcoin/pull/19657/commits/fa4dfd215f62e88d668311701735c332c264fa2a :coffee: Tree-SHA512: 36eda7eb323614a4c4f9215f1d7b40b9f9c4036d1c08eb701ea705f3e2986fdabd2fc558965a6aadabeed861034aeaeef3c00f968ca17ed7a27e42e506cda87d
2020-08-04test: Wait until is_connected in add_p2p_connectionMarcoFalke
2020-07-30test: Catch decimal.InvalidOperation from TestNodeCLI#send_cliBen Woosley
decimal.InvalidOperation is a special case of a float parsing error, which presumably should be handled in the same way as a general parsing error, rather than blow up. Alternatives include: logging the error, or re-raising with more information. Example log output: File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 603, in sync_all self.sync_blocks(nodes) File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in sync_blocks best_hash = [x.getbestblockhash() for x in rpc_connections] File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in <listcomp> best_hash = [x.getbestblockhash() for x in rpc_connections] File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 571, in __call__ return self.cli.send_cli(self.command, *args, **kwargs) File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 639, in send_cli return json.loads(cli_stdout, parse_float=decimal.Decimal) File "/usr/lib64/python3.6/json/__init__.py", line 367, in loads return cls(**kw).decode(s) File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode obj, end = self.scan_once(s, idx) decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]
2020-06-16[test] wait for disconnect_p2ps to be reflected in getpeerinfogzhao408
-Waiting is important to avoid race conditions, especially if testing peer info through rpc later. -Wait for mininodes to be disconnected only, even though it's more complex, because we may still want to be connected to test nodes.
2020-05-19test: Fix intermittent ETIMEDOUT on FreeBSDMarcoFalke
2020-05-18test: Add capability to disable RPC timeout in functional tests.codeShark149
Modifies the existing --factor flag to --timeout-factor to better express intent. Adds rules to disable timeout if --timeout-factor is set to 0. Modfies --timeout-factor help doc to inform users about this feature.
2020-05-13Merge #18888: test: Remove RPCOverloadWrapper boilerplateMarcoFalke
faa26d374425f52e03efff3a575c391b7862abe5 test: Remove RPCOverloadWrapper boilerplate (MarcoFalke) Pull request description: There are too many wrappers in test_node already, so at least the code that implements the wrappers should be as minimal as possible. ACKs for top commit: laanwj: code review ACK faa26d374425f52e03efff3a575c391b7862abe5 Tree-SHA512: 94e593907de22187524e2445afb3101e40b3b599d4b4015aa8c6ca902d7586ff9daf520828759029d199a3af79e61b96b490a822a5a193ac7bf946beacb11a24
2020-05-05test: Fix intermittent sync_blocks failuresMarcoFalke
2020-05-05test: Remove RPCOverloadWrapper boilerplateMarcoFalke
2020-05-03test: add factor option to adjust test timeoutsHarris
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-04-27Merge #16528: Native Descriptor Wallets using DescriptorScriptPubKeyManSamuel Dobson
223588b1bbc63dc57098bbd0baa48635e0cc0b82 Add a --descriptors option to various tests (Andrew Chow) 869f7ab30aeb4d7fbd563c535b55467a8a0430cf tests: Add RPCOverloadWrapper which overloads some disabled RPCs (Andrew Chow) cf060628590fab87d73f278e744d70ef2d5d81db Correctly check for default wallet (Andrew Chow) 886e0d75f5fea2421190aa4812777d89f68962cc Implement CWallet::IsSpentKey for non-LegacySPKMans (Andrew Chow) 3c19fdd2a2fd5394fcfa75b2ba84ab2277cbdabf Return error when no ScriptPubKeyMan is available for specified type (Andrew Chow) 388ba94231f2f10a0be751c562cdd4650510a90a Change wallet_encryption.py to use signmessage instead of dumpprivkey (Andrew Chow) 1346e14831489f9c8f53a08f9dfed61d55d53c6f Functional tests for descriptor wallets (Andrew Chow) f193ea889ddb53d9a5c47647966681d525e38368 add importdescriptors RPC and tests for native descriptor wallets (Hugo Nguyen) ce24a944940019185efebcc5d85eac458ed26016 Add IsLegacy to CWallet so that the GUI knows whether to show watchonly (Andrew Chow) 1cb42b22b11c27e64462afc25a94b2fc50bfa113 Generate new descriptors when encrypting (Andrew Chow) 82ae02b1656819f4bd5023b8955447e1d4ea8692 Be able to create new wallets with DescriptorScriptPubKeyMans as backing (Andrew Chow) b713baa75a62335ab9c0eed9ef76a95bfec30668 Implement GetMetadata in DescriptorScriptPubKeyMan (Andrew Chow) 8b9603bd0b443e2f7984eb72bf2e21cf02af0bcb Change GetMetadata to use unique_ptr<CKeyMetadata> (Andrew Chow) 72a9540df96ffdb94f039b9c14eaacdc7d961196 Implement FillPSBT in DescriptorScriptPubKeyMan (Andrew Chow) 84b4978c02102171775c77a45f6ec198930f0a88 Implement SignMessage for descriptor wallets (Andrew Chow) bde7c9fa38775a81d53ac0484fa9c98076a0c7d1 Implement SignTransaction in DescriptorScriptPubKeyMan (Andrew Chow) d50c8ddd4190f20bf0debd410348b73408ec3143 Implement GetSolvingProvider for DescriptorScriptPubKeyMan (Andrew Chow) f1ca5feb4ad668a3e1ae543d0addd5f483f1a88f Implement GetKeypoolOldestTime and only display it if greater than 0 (Andrew Chow) 586b57a9a6b4b12a78f792785b63a5a1743bce0c Implement ReturnDestination in DescriptorScriptPubKeyMan (Andrew Chow) f866957979c23cefd41efa9dae9e53b9177818dc Implement GetReservedDestination in DescriptorScriptPubKeyMan (Andrew Chow) a775f7c7fd0b9094fcbeee6ba92206d5bbb19164 Implement Unlock and Encrypt in DescriptorScriptPubKeyMan (Andrew Chow) bfdd0734869a22217c15858d7a76d0dacc2ebc86 Implement GetNewDestination for DescriptorScriptPubKeyMan (Andrew Chow) 58c7651821b0eeff0a99dc61d78d2e9e07986580 Implement TopUp in DescriptorScriptPubKeyMan (Andrew Chow) e014886a342508f7c8d80323eee9a5f314eaf94c Implement SetupGeneration for DescriptorScriptPubKeyMan (Andrew Chow) 46dfb99768e7d03a3cf552812d5b41ceaebc06be Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file (Andrew Chow) 4cb9b69be031e1dc65d8964794781b347fd948f5 Implement several simple functions in DescriptorScriptPubKeyMan (Andrew Chow) d1ec3e4f19487b4b100f80ad02eac063c571777d Add IsSingleType to Descriptors (Andrew Chow) 953feb3d2724f5398dd48990c4957a19313d2c8c Implement loading of keys for DescriptorScriptPubKeyMan (Andrew Chow) 2363e9fcaa41b68bf11153f591b95f2d41ff9a1a Load the descriptor cache from the wallet file (Andrew Chow) 46c46aebb7943e1e2e96755e94dc6c197920bf75 Implement GetID for DescriptorScriptPubKeyMan (Andrew Chow) ec2f9e1178c8e38c0a5ca063fe81adac8f916348 Implement IsHDEnabled in DescriptorScriptPubKeyMan (Andrew Chow) 741122d4c1a62ced3e96d16d67f4eeb3a6522d99 Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan (Andrew Chow) 2db7ca765c8fb2c71dd6f7c4f29ad70e68ff1720 Implement IsMine for DescriptorScriptPubKeyMan (Andrew Chow) db7177af8c159abbcc209f2caafcd45d54c181c5 Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet (Andrew Chow) 78f8a92910d34247fa5d04368338c598d9908267 Implement SetType in DescriptorScriptPubKeyMan (Andrew Chow) 834de0300cde57ca3f662fb7aa5b1bdaed68bc8f Store WalletDescriptor in DescriptorScriptPubKeyMan (Andrew Chow) d8132669e10c1db9ae0c2ea0d3f822d7d2f01345 Add a lock cs_desc_man for DescriptorScriptPubKeyMan (Andrew Chow) 3194a7f88ac1a32997b390b4f188c4b6a4af04a5 Introduce WalletDescriptor class (Andrew Chow) 6b13cd3fa854dfaeb9e269bff3d67cacc0e5b5dc Create LegacyScriptPubKeyMan when not a descriptor wallet (Andrew Chow) aeac157c9dc141546b45e06ba9c2e641ad86083f Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet (Andrew Chow) 96accc73f067c7c95946e9932645dd821ef67f63 Add WALLET_FLAG_DESCRIPTORS (Andrew Chow) 6b8119af53ee2fdb4c4b5b24b4e650c0dc3bd27c Introduce DescriptorScriptPubKeyMan as a dummy class (Andrew Chow) 06620302c713cae65ee8e4ff9302e4c88e2a1285 Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it (Andrew Chow) Pull request description: Introducing the wallet of the glorious future (again): native descriptor wallets. With native descriptor wallets, addresses are generated from descriptors. Instead of generating keys and deriving addresses from keys, addresses come from the scriptPubKeys produced by a descriptor. Native descriptor wallets will be optional for now and can only be created by using `createwallet`. Descriptor wallets will store descriptors, master keys from the descriptor, and descriptor cache entries. Keys are derived from descriptors on the fly. In order to allow choosing different address types, 6 descriptors are needed for normal use. There is a pair of primary and change descriptors for each of the 3 address types. With the default keypool size of 1000, each descriptor has 1000 scriptPubKeys and descriptor cache entries pregenerated. This has a side effect of making wallets large since 6000 pubkeys are written to the wallet by default, instead of the current 2000. scriptPubKeys are kept only in memory and are generated every time a descriptor is loaded. By default, we use the standard BIP 44, 49, 84 derivation paths with an external and internal derivation chain for each. Descriptors can also be imported with a new `importdescriptors` RPC. Native descriptor wallets use the `ScriptPubKeyMan` interface introduced in #16341 to add a `DescriptorScriptPubKeyMan`. This defines a different IsMine which uses the simpler model of "does this scriptPubKey exist in this wallet". Furthermore, `DescriptorScriptPubKeyMan` does not have watchonly, so with native descriptor wallets, it is not possible to have a wallet with both watchonly and non-watchonly things. Rather a wallet with `disable_private_keys` needs to be used for watchonly things. A `--descriptor` option was added to some tests (`wallet_basic.py`, `wallet_encryption.py`, `wallet_keypool.py`, `wallet_keypool_topup.py`, and `wallet_labels.py`) to allow for these tests to use descriptor wallets. Additionally, several RPCs are disabled for descriptor wallets (`importprivkey`, `importpubkey`, `importaddress`, `importmulti`, `addmultisigaddress`, `dumpprivkey`, `dumpwallet`, `importwallet`, and `sethdseed`). ACKs for top commit: Sjors: utACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82 (rebased, nits addressed) jonatack: Code review re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82. fjahr: re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82 instagibbs: light re-ACK 223588b meshcollider: Code review ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82 Tree-SHA512: 59bc52aeddbb769ed5f420d5d240d8137847ac821b588eb616b34461253510c1717d6a70bab8765631738747336ae06f45ba39603ccd17f483843e5ed9a90986
2020-04-23Add a --descriptors option to various testsAndrew Chow
Adds a --descriptors option globally to the test framework. This will make the test create and use descriptor wallets. However some tests may not work with this. Some tests are modified to work with --descriptors and run with that option in test_runer: * wallet_basic.py * wallet_encryption.py * wallet_keypool.py * wallet_keypool_topup.py * wallet_labels.py * wallet_avoidreuse.py
2020-04-23tests: Add RPCOverloadWrapper which overloads some disabled RPCsAndrew Chow
RPCOverloadWrapper overloads some deprecated or disabled RPCs with an implementation using other RPCs to avoid having a ton of code churn around replacing those RPCs.
2020-04-21test: display command line options passed to send_cli() in debug logJon Atack
and fixup two cli calls from rpc commands to command line options. Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-04-19test: add wait_for_cookie_credentials() to test frameworkJon Atack
to be able to ensure the cookie file is written and auth credentials available when testing CLI/RPC commands before the RPC connection is up.
2020-04-19Merge #18633: test: Properly raise FailedToStartError when rpc shutdown ↵MarcoFalke
before warmup finished (take 2) fa03713e133e3017112fdd5c278e0c8643054578 test: Properly raise FailedToStartError when rpc shutdown before warmup finished (take 2) (MarcoFalke) Pull request description: actually (?) fix #18561 See most recent traceback https://travis-ci.org/github/bitcoin/bitcoin/jobs/674668692#L7062 I believe the reason the error is still there is that ConnectionResetError is derived from OSError: ConnectionResetError(ConnectionError(OSError)) And IOError is an alias for OSError since python 3.3, see https://docs.python.org/3/library/exceptions.html#IOError So fix that by renaming IOError to the alias OSError and move the less specific catch clause down a few lines. ACKs for top commit: jonatack: ACK fa03713e133e3017112fdd5c278e0c8643054578 Tree-SHA512: 6e5b214ed9101bf8ebe7472dcc1f9e9d128e2575c93ec00c8d0774ae1a9b52a8c2a653a45a0eab8d881570b08dd5ffeddf5aca88a10438c366e1f633253cb0b5
2020-04-18test: Properly raise FailedToStartError when rpc shutdown before warmup ↵MarcoFalke
finished (take 2)
2020-04-17test: Bump timeout in wallet_import_rescanMarcoFalke