aboutsummaryrefslogtreecommitdiff
path: root/test/functional/test_runner.py
AgeCommit message (Collapse)Author
2024-03-29Merge bitcoin/bitcoin#29130: wallet: Add `createwalletdescriptor` and ↵Ryan Ofsky
`gethdkeys` RPCs for adding new automatically generated descriptors 746b6d88395607abbd3c13bbdcdd4ca83e9bc9e4 test: Add test for createwalletdescriptor (Ava Chow) 2402b6306215a9ee8d5f4068ea81f4e7f324adeb wallet: Test upgrade of pre-taproot wallet to have tr() descriptors (Ava Chow) 460ae1bf67c0051033c1802d44787d173abb9248 wallet, rpc: Add createwalletdescriptor RPC (Ava Chow) 8e1a475062e62321e58a0624385cc3fa0885aa12 wallet: Be able to retrieve single key from descriptors (Ava Chow) 85b1fb19dd3a3f3c68da1c5e60a6eb911e1119a6 wallet: Add GetActiveHDPubKeys to retrieve xpubs from active descriptors (Ava Chow) 73926f2d31b61ff78d5f0c8f9b5e3130fb1f9620 wallet, descspkm: Refactor wallet descriptor generation to standalone func (Andrew Chow) 54e74f46ea10e479be682750c1279165f29bb2f4 wallet: Refactor function for single DescSPKM setup (Andrew Chow) 3b09d0eb7f2c1d6ebdab73d18db28e5bf7d74f18 tests: Test for gethdkeys (Ava Chow) 5febe28c9e131fb93fac9c35f80c42759654f150 wallet, rpc: Add gethdkeys RPC (Ava Chow) 66632e5c24c1b59afef1e89b562fbd0117ab6ef5 wallet: Add IsActiveScriptPubKeyMan (Ava Chow) fa6a259985b61235ebc21eae2a76014cc9437d5f desc spkm: Add functions to retrieve specific private keys (Ava Chow) fe67841464cc0f970a1c233caba92cb78e9c78dc descriptor: Be able to get the pubkeys involved in a descriptor (Ava Chow) ef6745879d87cdb6f1061337867a689167e965a1 key: Add constructor for CExtKey that takes CExtPubKey and CKey (Ava Chow) Pull request description: This PR adds a `createwalletdescriptor` RPC which allows users to add new automatically generated descriptors to their wallet, e.g. to upgrade a 0.21.x wallet to contain a taproot descriptor. This RPC takes 3 arguments: the output type to create a descriptor for, whether the descriptor will be internal or external, and the HD key to use if the user wishes to use a specific key. The HD key is an optional parameter. If it is not specified, the wallet will use the key shared by the active descriptors, if they are all single key. For most users in the expected upgrade scenario, this should be sufficient. In more advanced cases, the user must specify the HD key to use. Currently, specified HD keys must already exist in the wallet. To make it easier for the user to know, `gethdkeys` is also added to list out the HD keys in use by all of the descriptors in the wallet. This will include all HD keys, whether we have the private key, for it, which descriptors use it and their activeness, and optionally the extended private key. In this way, users with more complex wallets will be still be able to get HD keys from their wallet for use in other scenarios, and if they want to use `createwalletdescriptor`, they can easily get the keys that they can specify to it. See also https://github.com/bitcoin/bitcoin/pull/26728#issuecomment-1866961865 ACKs for top commit: Sjors: re-utACK 746b6d88395607abbd3c13bbdcdd4ca83e9bc9e4 furszy: ACK 746b6d8 ryanofsky: Code review ACK 746b6d88395607abbd3c13bbdcdd4ca83e9bc9e4, and this looks ready to merge. There were various suggested changes since last review where main change seems to be switching `gethdkeys` output to use normalized descriptors (removing hardened path components). Tree-SHA512: f2849101e6fbf1f59cb031eaaaee97af5b1ae92aaab54c5716940d210f08ab4fc952df2725b636596cd5747b8f5beb1a7a533425bc10d09da02659473516fbda
2024-03-20test: Add test for createwalletdescriptorAva Chow
2024-03-20tests: Test for gethdkeysAva Chow
2024-03-19Merge bitcoin/bitcoin#29279: test: p2p: check disconnect due to lack of ↵glozow
desirable service flags 2f23987849758537f76df7374d85a7e87b578b61 test: p2p: check limited peers desirability (depending on best block depth) (Sebastian Falbesoner) c4a67d396d0aa99f658cafe381e39622859eb0be test: p2p: check disconnect due to lack of desirable service flags (Sebastian Falbesoner) 405ac819af1eb0f6cf6d1805cb668f4e8ab4a6f3 test: p2p: support disconnect waiting for `add_outbound_p2p_connection` (Sebastian Falbesoner) Pull request description: This PR adds missing test coverage for disconnecting peers which don't offer the desirable service flags in their VERSION message: https://github.com/bitcoin/bitcoin/blob/5f3a0574c45477288bc678b15f24940486084576/src/net_processing.cpp#L3384-L3389 This check is relevant for the connection types "outbound-full-relay", "block-relay-only" and "addr-fetch" (see `CNode::ExpectServicesFromConn(...)`). Feeler connections always disconnect, which is also tested here. In lack of finding a proper file where this test would fit in, I created a new one. Happy to take suggestions there. ACKs for top commit: davidgumberg: reACK https://github.com/bitcoin/bitcoin/commit/2f23987849758537f76df7374d85a7e87b578b61 itornaza: tested ACK 2f23987849758537f76df7374d85a7e87b578b61 fjahr: re-utACK 2f23987849758537f76df7374d85a7e87b578b61 cbergqvist: re ACK 2f23987849758537f76df7374d85a7e87b578b61 stratospher: tested ACK 2f23987. 🚀 Tree-SHA512: cf75d9d4379d0f34fa1e13152e6a8d93cd51b9573466ab3a2fec32dc3e1ac49b174bd1063cae558bc736b111c8a6e7058b1b57a496df56255221bf367d29eb5d
2024-03-12test: simplify test_runner.pytdb3
Remove the num_running variable as it can be implied by the length of the jobs list. Remove the i variable as it can be implied by the length of the test_results list. Instead of counting results to determine if finished, make the queue object itself responsible (by looking at running jobs and jobs left). Originally proposed by @sipa in PR #23995. Co-authored-by: Pieter Wuille <pieter@wuille.net>
2024-03-11test: p2p: check disconnect due to lack of desirable service flagsSebastian Falbesoner
2024-03-11Merge bitcoin/bitcoin#29483: test, ci: add --v1transport option, add ↵Ava Chow
--v2transport to a CI task ecc036c5d63fb4bdff5caeede88baeb85bf62b05 ci: add --v2transport to an existing CI job (Martin Zumsande) 3a25a575f0c455b50abf95d85c0ba8de3cf53a87 test: ignore --v2transport for older versions instead of asserting (Martin Zumsande) 547aacff08a2a5d786cb0fa6c7bae022ea651f8c test: add -v1transport option and use it in test_runner (Martin Zumsande) Pull request description: This suggests a strategy to run the functional tests with both v1 and v2 transport in the CI. **Status Quo:** There is both the global `--v2transport` option for the `test_runner.py` (not enabled by default), plus the possibility to specify `--v2transport` for particular tests, which is used for a handful of tests. Currently, when running `test_runner.py --v2transport`, these tests are run twice with the same `--v2transport` configuration, as has been noted in https://github.com/bitcoin/bitcoin/pull/29358#discussion_r1485626063, which is wasteful. **Suggested Change:** Fix this by adding a `--v1transport` option and using it in `test_runner.py`, so that irrespective of the global `--v2transport` flag, the tests that run twice use v1 in one run and v2 in the other. Also add `--v2transport` to one CI task (`multiprocess, i686, DEBUG`). This means, that for each CI task, the majority of functional tests will run once using the global `--v2transport` option if provided, while a few selected tests will always run two times, once with `v1` and once with `v2`. **Rationale:** A simpler alternative would have been to remove all test-specific `--v2transport` commands from `test_runner.py` and just enable `--v2transport` option for a few CI tasks. I didn't do that because it would have meant that v2 would never be running in the CI for some platforms, and also be run a lot less locally by users and devs (who would have to actively enable the `--v2transport` option). ACKs for top commit: tdb3: ACK for ecc036c5d63fb4bdff5caeede88baeb85bf62b05. achow101: ACK ecc036c5d63fb4bdff5caeede88baeb85bf62b05 stratospher: ACK ecc036c. vasild: ACK ecc036c5d63fb4bdff5caeede88baeb85bf62b05 Tree-SHA512: 375b2293d49991f2fbd8e1bb646c0034004a09cee36063bc32996b721323eb19a43d7b2f36b3f9a3fdca846d74f48d8f1387565c03ef5d34b3481d2a0fe1d328
2024-03-06test: exit with code 1 when no fn tests are foundMax Edwards
Prevents the test_runner from exiting silently with code 0 when no tests were found.
2024-02-29test: add -v1transport option and use it in test_runnerMartin Zumsande
This option beats the --v2transport option and is meant to be used in test_runner.py. It applies these to a few tests that are particulary interesting in terms of the transport type. This ensures that these tests arei always run with both v1 and v2, irrespective of whether the global --v2transport test_runner option is set or not.
2024-02-28Merge bitcoin/bitcoin#29412: p2p: Don't process mutated blocksAva Chow
d8087adc7ebd4ea05dd3843e5a92e8115fd7bbcc [test] IsBlockMutated unit tests (dergoegge) 1ed2c9829700054526156297552bb47230406098 Add transaction_identifier::size to allow Span conversion (dergoegge) 1ec6bbeb8d27d31647d1433ccb87b362f6d81f90 [validation] Cache merkle root and witness commitment checks (dergoegge) 5bf4f5ba32da4627f152b54d266df9b2aa930457 [test] Add regression test for #27608 (dergoegge) 49257c0304828a185c273fcb99742c54bbef0c8e [net processing] Don't process mutated blocks (dergoegge) 2d8495e0800f5332748cd50eaad801ff77671bba [validation] Merkle root malleation should be caught by IsBlockMutated (dergoegge) 66abce1d98115e41f394bc4f4f52341960ddc839 [validation] Introduce IsBlockMutated (dergoegge) e7669e1343aec4298fd30d539847963e6fa5619c [refactor] Cleanup merkle root checks (dergoegge) 95bddb930aa72edd40fdff52cf447202995b0dce [validation] Isolate merkle root checks (dergoegge) Pull request description: This PR proposes to check for mutated blocks early as a defense-in-depth mitigation against attacks leveraging mutated blocks. We introduce `IsBlockMutated` which catches all known forms of block malleation and use it to do an early mutation check whenever we receive a `block` message. We have observed attacks that abused mutated blocks in the past, which could have been prevented by simply not processing mutated blocks (e.g. https://github.com/bitcoin/bitcoin/pull/27608 for which a regression test is included in this PR). ACKs for top commit: achow101: ACK d8087adc7ebd4ea05dd3843e5a92e8115fd7bbcc maflcko: ACK d8087adc7ebd4ea05dd3843e5a92e8115fd7bbcc 🏄 fjahr: Code review ACK d8087adc7ebd4ea05dd3843e5a92e8115fd7bbcc sr-gi: Code review ACK https://github.com/bitcoin/bitcoin/commit/d8087adc7ebd4ea05dd3843e5a92e8115fd7bbcc Tree-SHA512: 618ff4ea7f168e10f07504d3651290efbb1bb2ab3b838ffff3527c028caf6c52dedad18d04d3dbc627977479710930e200f2dfae18a08f627efe7e64a57e535f
2024-02-27[test] Add regression test for #27608dergoegge
2024-02-23test: Add option to skip unit tests for the test runnerMartin Zumsande
2024-02-09Merge bitcoin/bitcoin#28948: v3 transaction policy for anti-pinningAva Chow
29029df5c700e6940c712028303761d91ae15847 [doc] v3 signaling in mempool-replacements.md (glozow) e643ea795e4b6fea4a6bbb3d72870ee6a4c836b1 [fuzz] v3 transactions and sigop-adjusted vsize (glozow) 1fd16b5c62f54c7f4c60122acd65d852f63d1e8b [functional test] v3 transaction submission (glozow) 27c8786ba918a42c860e6a50eaee9fdf56d7c646 test framework: Add and use option for tx-version in MiniWallet methods (MarcoFalke) 9a1fea55b29fe025355b06b45e3d77d192acc635 [policy/validation] allow v3 transactions with certain restrictions (glozow) eb8d5a2e7d939dd3ee683486e98702079e0dfcc0 [policy] add v3 policy rules (glozow) 9a29d470fbb62bbb27d517efeafe46ff03c25f54 [rpc] return full string for package_msg and package-error (glozow) 158623b8e0726dff7eae4288138f1710e727db9c [refactor] change Workspace::m_conflicts and adjacent funcs/structs to use Txid (glozow) Pull request description: See #27463 for overall package relay tracking. Delving Bitcoin discussion thread: https://delvingbitcoin.org/t/v3-transaction-policy-for-anti-pinning/340 Delving Bitcoin discussion for LN usage: https://delvingbitcoin.org/t/lightning-transactions-with-v3-and-ephemeral-anchors/418 Rationale: - There are various pinning problems with RBF and our general ancestor/descendant limits. These policies help mitigate many pinning attacks and make package RBF feasible (see #28984 which implements package RBF on top of this). I would focus the most here on Rule 3 pinning. [1][2] - Switching to a cluster-based mempool (see #27677 and #28676) requires the removal of CPFP carve out, which applications depend on. V3 + package RBF + ephemeral anchors + 1-parent-1-child package relay provides an intermediate solution. V3 policy is for "Priority Transactions." [3][4] It allows users to opt in to more restrictive topological limits for shared transactions, in exchange for the more robust fee-bumping abilities that offers. Even though we don't have cluster limits, we are able to treat these transactions as having as having a maximum cluster size of 2. Immediate benefits: - You can presign a transaction with 0 fees (not just 1sat/vB!) and add a fee-bump later. - Rule 3 pinning is reduced by a significant amount, since the attacker can only attach a maximum of 1000vB to your shared transaction. This also enables some other cool things (again see #27463 for overall roadmap): - Ephemeral Anchors - Package RBF for these 1-parent-1-child packages. That means e.g. a commitment tx + child can replace another commitment tx using the child's fees. - We can transition to a "single anchor" universe without worrying about package limit pinning. So current users of CPFP carve out would have something else to use. - We can switch to a cluster-based mempool [5] (#27677 #28676), which removes CPFP carve out [6]. [1]: Original mailing list post and discussion about RBF pinning problems https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff, https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-January/019817.html [2]: A FAQ is "we need this for cluster mempool, but is this still necessary afterwards?" There are some pinning issues that are fixed here and not fully fixed in cluster mempool, so we will still want this or something similar afterward. [3]: Mailing list post for v3 https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-September/020937.html [4]: Original PR #25038 also contains a lot of the discussion [5]: https://delvingbitcoin.org/t/an-overview-of-the-cluster-mempool-proposal/393/7 [6]: https://delvingbitcoin.org/t/an-overview-of-the-cluster-mempool-proposal/393#the-cpfp-carveout-rule-can-no-longer-be-supported-12 ACKs for top commit: sdaftuar: ACK 29029df5c700e6940c712028303761d91ae15847 achow101: ACK 29029df5c700e6940c712028303761d91ae15847 instagibbs: ACK 29029df5c700e6940c712028303761d91ae15847 modulo that Tree-SHA512: 9664b078890cfdca2a146439f8835c9d9ab483f43b30af8c7cd6962f09aa557fb1ce7689d5e130a2ec142235dbc8f21213881baa75241c5881660f9008d68450
2024-02-08[functional test] v3 transaction submissionglozow
Co-authored-by: Suhas Daftuar <sdaftuar@gmail.com> Co-authored-by: Gregory Sanders <gsanders87@gmail.com>
2024-02-02test: fix intermittent failure in rpc_setban.py --v2transportMartin Zumsande
When initiating a v2 connection and being immediately disconnected, a node cannot know if the disconnect happens because the peer only supports v1, or because it has banned you, so it schedules to reconnect with v1. If the test doesn't wait for that, the reconnect can happen at a bad time, resulting in failure in a later connect_nodes call. Also add the test with --v2transport to the test runner.
2024-01-29Merge bitcoin/bitcoin#24748: test/BIP324: functional tests for v2 P2P encryptionAva Chow
bc9283c4415a932ec1eeb70ca2aa4399c80437b3 [test] Add functional test to test early key response behaviour in BIP 324 (stratospher) ffe6a56d75c0b47d0729e4e0b7225a827b43ad89 [test] Check whether v2 TestNode performs downgrading (stratospher) ba737358a37438c18f0fba723eab10ccfd9aae9b [test] Add functional tests to test v2 P2P behaviour (stratospher) 4115cf995647d1a513caecb54a4ff3f51927aa8e [test] Ignore BIP324 decoy messages (stratospher) 8c054aa04d33b247744b3747cd5bf3005a013e90 [test] Allow inbound and outbound connections supporting v2 P2P protocol (stratospher) 382894c3acd2dbf3e4198814f547c75b6fb17706 [test] Reconnect using v1 P2P when v2 P2P terminates due to magic byte mismatch (stratospher) a94e350ac0e5b65ef23a84b05fb10d1204c98c97 [test] Build v2 P2P messages (stratospher) bb7bffed799dc5ad8b606768164fce46d4cbf9d0 [test] Use lock for sending P2P messages in test framework (stratospher) 5b91fb14aba7d7fe45c9ac364526815bec742356 [test] Read v2 P2P messages (stratospher) 05bddb20f5cc9036fd680500bde8ece70dbf0646 [test] Perform initial v2 handshake (stratospher) a049d1bd08c8cdb3b693520f24f8a82572dcaab1 [test] Introduce EncryptedP2PState object in P2PConnection (stratospher) b89fa59e715a185d9fa7fce089dad4273d3b1532 [test] Construct class to handle v2 P2P protocol functions (stratospher) 8d6c848a48530893ca40be5c1285541b3e7a94f3 [test] Move MAGIC_BYTES to messages.py (stratospher) 595ad4b16880ae1f23463ca9985381c8eae945d8 [test/crypto] Add ECDH (stratospher) 4487b8051797173c7ab432e75efa370afb03b529 [rpc/net] Allow v2 p2p support in addconnection (stratospher) Pull request description: This PR introduces support for v2 P2P encryption(BIP 324) in the existing functional test framework and adds functional tests for the same. ### commits overview 1. introduces a new class `EncryptedP2PState` to store the keys, functions for performing the initial v2 handshake and encryption/decryption. 3. this class is used by `P2PConnection` in inbound/outbound connections to perform the initial v2 handshake before the v1 version handshake. Only after the initial v2 handshake is performed do application layer P2P messages(version, verack etc..) get exchanged. (in a v2 connection) - `v2_state` is the object of class `EncryptedP2PState` in `P2PConnection` used to store its keys, session-id etc. - a node [advertising](https://github.com/stratospher/blogosphere/blob/main/integration_test_bip324.md#advertising-to-support-v2-p2p) support for v2 P2P is different from a node actually [supporting v2 P2P](https://github.com/stratospher/blogosphere/blob/main/integration_test_bip324.md#supporting-v2-p2p) (differ when false advertisement of services occur) - introduce a boolean variable `supports_v2_p2p` in `P2PConnection` to denote if it supports v2 P2P. - introduce a boolean variable `advertises_v2_p2p` to denote whether `P2PConnection` which mimics peer behaviour advertises V2 P2P support. Default option is `False`. - In the test framework, you can create Inbound and Outbound connections to `TestNode` 1. During **Inbound Connections**, `P2PConnection` is the initiator [`TestNode` <--------- `P2PConnection`] - Case 1: - if the `TestNode` advertises/signals v2 P2P support (means `self.nodes[i]` set up with `"-v2transport=1"`), different behaviour will be exhibited based on whether: 1. `P2PConnection` supports v2 P2P 2. `P2PConnection` does not support v2 P2P - In a real world scenario, the initiator node would intrinsically know if they support v2 P2P based on whatever code they choose to run. However, in the test scenario where we mimic peer behaviour, we have no way of knowing if `P2PConnection` should support v2 P2P or not. So `supports_v2_p2p` boolean variable is used as an option to enable support for v2 P2P in `P2PConnection`. - Since the `TestNode` advertises v2 P2P support (using "-v2transport=1"), our initiator `P2PConnection` would send: 1. (if the `P2PConnection` supports v2 P2P) ellswift + garbage bytes to initiate the connection 2. (if the `P2PConnection` does not support v2 P2P) version message to initiate the connection - Case 2: - if the `TestNode` doesn't signal v2 P2P support; `P2PConnection` being the initiator would send version message to initiate a connection. 2. During **Outbound Connections** [TestNode --------> P2PConnection] - initiator `TestNode` would send: - (if the `P2PConnection` advertises v2 P2P) ellswift + garbage bytes to initiate the connection - (if the `P2PConnection` advertises v2 P2P) version message to initiate the connection - Suppose `P2PConnection` advertises v2 P2P support when it actually doesn't support v2 P2P (false advertisement scenario) - `TestNode` sends ellswift + garbage bytes - `P2PConnection` receives but can't process it and disconnects. - `TestNode` then tries using v1 P2P and sends version message - `P2PConnection` receives/processes this successfully and they communicate on v1 P2P 4. the encrypted P2P messages follow a different format - 3 byte length + 1-13 byte message_type + payload + 16 byte MAC 5. includes support for testing decoy messages and v2 connection downgrade(using false advertisement - when a v2 node makes an outbound connection to a node which doesn't support v2 but is advertised as v2 by some malicious intermediary) ### run the tests * functional test - `test/functional/p2p_v2_encrypted.py` `test/functional/p2p_v2_earlykeyresponse.py` I'm also super grateful to @ dhruv for his really valuable feedback on this branch. Also written a more elaborate explanation here - https://github.com/stratospher/blogosphere/blob/main/integration_test_bip324.md ACKs for top commit: naumenkogs: ACK bc9283c4415a932ec1eeb70ca2aa4399c80437b3 mzumsande: Code Review ACK bc9283c4415a932ec1eeb70ca2aa4399c80437b3 theStack: Code-review ACK bc9283c4415a932ec1eeb70ca2aa4399c80437b3 glozow: ACK bc9283c4415a932ec1eeb70ca2aa4399c80437b3 Tree-SHA512: 9b54ed27e925e1775e0e0d35e959cdbf2a9a1aab7bcf5d027e66f8b59780bdd0458a7a4311ddc7dd67657a4a2a2cd5034ead75524420d58a83f642a8304c9811
2024-01-25[test] Add functional test to test early key response behaviour in BIP 324stratospher
- A node initiates a v2 connection by sending 64 bytes ellswift - In BIP 324 "The responder waits until one byte is received which does not match the V1_PREFIX (16 bytes consisting of the network magic followed by "version\x00\x00\x00\x00\x00".)" - It's possible that the 64 bytes ellswift sent by an initiator starts with a prefix of V1_PREFIX - Example form of 64 bytes ellswift could be: 4 bytes network magic + 60 bytes which aren't prefixed with remaining V1_PREFIX - We test this behaviour: - when responder receives 4 byte network magic -> no response received by initiator - when first mismatch happens -> response received by initiator
2024-01-25[test] Add functional tests to test v2 P2P behaviourstratospher
2024-01-19test: add tests for fundrawtx and sendmany rpcsjosibake
If the serialized transaction passed to `fundrawtransaction` contains duplicates, they will be deserialized and added to the transaction. Add a test to ensure this behavior is not changed during the refactor. A user can pass any number of duplicated and unrelated addresses as an SFFO argument to `sendmany` and the RPC will not throw an error (note, all the rest of the RPCs which take SFFO as an argument will error if the user passes duplicates or specifies outputs not present in the transaction). Add a test to ensure this behavior is not changed during the refactor.
2024-01-16Merge bitcoin/bitcoin#29179: test: wallet rescan with reorged parent + ↵Ava Chow
IsFromMe child in mempool df30247705940c50c5eaafd74e2abbeb8b0cec07 [test] import descriptor wallet with reorged parent + IsFromMe child in mempool (glozow) c3d02be536ac3f35c10efa03653186a17ebbfc12 [test] rescan legacy wallet with reorged parent + IsFromMe child in mempool (Gloria Zhao) Pull request description: Originally motivated by #29019, which reverts back to having `requestMempoolTransactions` emit `transactionAddedToMempool` in `mapTx` default order instead of `GetSortedDepthAndScore` order. It's important that these notifications happen in topological order, otherwise the wallet rescan may miss transactions that belong to it. Notably, checking whether a transaction `IsFromMe` requires knowing its inputs, which may be from a mempool parent. When using `mapTx` order, a parent may come later than its child if it was added from a block disconnected in a reorg. This PR adds a test for this case. ACKs for top commit: achow101: ACK df30247705940c50c5eaafd74e2abbeb8b0cec07 furszy: Code review ACK df30247705, nits can be disregarded. Tree-SHA512: 2f1d9ef92313228adbbef94e634e5f7a9ec6e6a2c88e16aa343bdc95ffc9b9f9c82a569b412c9a3841db9d789e52f9283e8b9385731668d59355903e26e58a5d
2024-01-12[test] import descriptor wallet with reorged parent + IsFromMe child in mempoolglozow
Test that wallet rescans process transactions topologically, even if a parent's entry into the mempool is later than that of its child. This behavior is important because IsFromMe requires the ability to look up a transaction's inputs. Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
2024-01-11Merge bitcoin/bitcoin#29034: test: detect OS in functional tests ↵Ava Chow
consistently using `platform.system()` 878d914777a03a04ecb84217152e8b7fd73a5062 doc: test: mention OS detection preferences in style guideline (Sebastian Falbesoner) 4c65ac96f8b021c107783adce3e8afe4f8edee6e test: detect OS consistently using `platform.system()` (Sebastian Falbesoner) 37324ae3dfb0e50daaf752dc863a880559fa4637 test: use `skip_if_platform_not_linux` helper where possible (Sebastian Falbesoner) Pull request description: There are at least three ways to detect the operating system in Python3: - `os.name` (https://docs.python.org/3.9/library/os.html#os.name) - `sys.platform` (https://docs.python.org/3.9/library/sys.html#sys.platform) - `platform.system()` (https://docs.python.org/3.9/library/platform.html#platform.system) We are currently using all of them in functional tests (both in individual tests and shared test framework code), which seems a bit messy. This PR consolidates into using `platform.system()`, as it appears to be one most consistent and easy to read (see also [IRC discussion](https://bitcoin-irc.chaincode.com/bitcoin-core-dev/2023-12-08#989301;) and table below). `sys.platform` is inconsistent as it has the major version number encoded for BSD systems, which doesn't make much sense for e.g. OpenBSD, where there is no concept of major versions, but instead the version is simply increased by 0.1 on each release. Note that `os.name` is still useful to detect whether we are running a POSIX system (see `BitcoinTestFramework.skip_if_platform_not_posix`), so for this use-case it is kept as only exception. The following table shows values for common operating systems, found via ``` $ python3 -c "import os; import sys; import platform; print(os.name, sys.platform, platform.system())" ``` | OS | os.name | sys.platform | platform.system() | |--------------|---------|--------------|--------------------| | Linux 6.2.0 | posix | linux | Linux | | MacOS* | posix | darwin | Darwin | | OpenBSD 7.4 | posix | openbsd7 | OpenBSD | | Windows* | nt | win32 | Windows | \* = I neither have a MacOS nor a Windows machine available, so I extracted the values from documentation and our current code. Also I'm relying on CI for testing the relevant code-paths. Having reviewers to this this locally would be very appreciated, if this gets Concept ACKed. ACKs for top commit: kevkevinpal: ACK [878d914](https://github.com/bitcoin/bitcoin/pull/29034/commits/878d914777a03a04ecb84217152e8b7fd73a5062) achow101: ACK 878d914777a03a04ecb84217152e8b7fd73a5062 hebasto: ACK 878d914777a03a04ecb84217152e8b7fd73a5062, I have reviewed the code and it looks OK. pablomartin4btc: tACK 878d914777a03a04ecb84217152e8b7fd73a5062 Tree-SHA512: 24513d493e47f572028c843260b81c47c2c29bfb701991050255c9f9529cd19065ecbc7b3b6e15619da7f3f608b4825c345ce6fee30d8fd1eaadbd08cff400fc
2024-01-11Merge bitcoin/bitcoin#28838: test: add assumeutxo wallet testAva Chow
997b9a73e5166b4244f7c5b4fe144d524f3005f4 test: add assumeutxo wallet test (Sjors Provoost) Pull request description: Extracted from #28616, this adds a (very) basic wallet test for assume utxo. It checks some circumstances where a backup can and can't be loaded. ACKs for top commit: maflcko: lgtm ACK 997b9a73e5166b4244f7c5b4fe144d524f3005f4 achow101: ACK 997b9a73e5166b4244f7c5b4fe144d524f3005f4 theStack: Code-review ACK 997b9a73e5166b4244f7c5b4fe144d524f3005f4 Tree-SHA512: 69474e56c6a46bb4f30fc54f8e5844766ac2a5f8226bb0b168d11ae1e3d4eae58570c1f1b4cc2b2f6f51b5d0e055bbe2bbd11684265215e01d4eb81ab4b7b0bb
2023-12-14Merge bitcoin/bitcoin#28920: wallet: birth time update during tx scanningAva Chow
1ce45baed7dd2da3f1cb85c9c25110e5537451ae rpc: getwalletinfo, return wallet 'birthtime' (furszy) 83c66444d0604f0a9ec3bc3f89d4f1a810b7cda0 test: coverage for wallet birth time interaction with -reindex (furszy) 6f497377aa17cb8a590fd7717fa8ededf4249999 wallet: fix legacy spkm default birth time (furszy) 75fbf444c1e13c6ba0e79a34871534c845a13849 wallet: birth time update during tx scanning (furszy) b4306e3c8db6cbaedc8845c6d21c750b39f682bf refactor: rename FirstKeyTimeChanged to MaybeUpdateBirthTime (furszy) Pull request description: Fixing #28897. As the user may have imported a descriptor with a timestamp newer than the actual birth time of the first key (by setting 'timestamp=now'), the wallet needs to update the birth time when it detects a transaction older than the oldest descriptor timestamp. Testing Notes: Can cherry-pick the test commit on top of master. It will fail there. ACKs for top commit: Sjors: re-utACK 1ce45baed7dd2da3f1cb85c9c25110e5537451ae achow101: ACK 1ce45baed7dd2da3f1cb85c9c25110e5537451ae Tree-SHA512: 10c2382f87356ae9ea3fcb637d7edc5ed0e51e13cc2729c314c9ffb57c684b9ac3c4b757b85810c0a674020b7287c43d3be8273bcf75e2aff0cc1c037f1159f9
2023-12-12test: Actually fail when a python unit test failsMarcoFalke
2023-12-08test: detect OS consistently using `platform.system()`Sebastian Falbesoner
2023-12-05test: coverage for wallet birth time interaction with -reindexfurszy
Verifying the wallet updates the birth time accordingly when it detects a transaction with a time older than the oldest descriptor timestamp. This could happen when the user blindly imports a descriptor with 'timestamp=now'.
2023-11-28Merge bitcoin/bitcoin#28805: test: Make existing functional tests compatible ↵Andrew Chow
with --v2transport 35fb9930adb3501b29d3ad20d2e74c0114f2bcbe test: enable v2 transport for p2p_timeouts.py (Martin Zumsande) 2c1669c37a9759e15ff5f4e340aeaa8778a81b9a test: enable v2 transport for rpc_net.py (Sebastian Falbesoner) cc961c26956859850202f56191981b0306a65fcf test: enable v2 transport for p2p_node_network_limited.py (Sebastian Falbesoner) 3598a1b5c932634dc7ccb991cc83df5e1a1dcaa9 test: enable --v2transport in combination with --usecli (Martin Zumsande) 68a90017519874793e34e3b439a63e5aa3a6f6a7 test: persist -v2transport over restarts and respect -v2transport=0 (Martin Zumsande) Pull request description: This makes the functional test suite compatible with BIP324, so that `python3 test_runner.py --v2transport` should succeed (currently, 12 tests fail for me on master). Includes two commits by TheStack I found in an old discussion https://github.com/bitcoin/bitcoin/pull/28331#discussion_r1326714164 Note that even though all tests should pass, the python `p2p.py` module will do v2 connections only after the merge of #24748, so that for now only connections between two full nodes will actually run v2. Some of the fixed tests were added with `--v2transport` to the test runner. Though after #24748 we might also want to consider running the entire suite with `--v2transport` in some CI. ACKs for top commit: sipa: utACK 35fb9930adb3501b29d3ad20d2e74c0114f2bcbe. Thanks for taking care of this. achow101: ACK 35fb9930adb3501b29d3ad20d2e74c0114f2bcbe theStack: ACK 35fb9930adb3501b29d3ad20d2e74c0114f2bcbe stratospher: ACK 35fb993. Tree-SHA512: 80dc0bf211fa525ff1d092043aea9f222f14c02e5832a548fb8b83b9ede1fcee03c5e8ade0d05c331bdaa492af9c1cf3d0f0b15b846673c6eacea82dd4cefbc3
2023-11-13test: add assumeutxo wallet testSjors Provoost
Co-Authored-By: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
2023-11-08test: enable v2 transport for p2p_timeouts.pyMartin Zumsande
by skipping the part where we send a non-version message before the version - this message would be interpreted as part of the v2 handshake.
2023-11-08test: enable v2 transport for rpc_net.pySebastian Falbesoner
- "transport_protocol_type" of inbound peer before version handshake is "detecting" on p2p v2 nodes (as opposed to "v1" for p2p v1) - size of a ping/pong message is 29 bytes (as opposed to 32 for p2p v1) - for the sendmsgtopeer RPC sub-test, enforce p2p v1 connection to have a peer id of zero
2023-11-08test: enable v2 transport for p2p_node_network_limited.pySebastian Falbesoner
2023-11-07Merge bitcoin/bitcoin#28374: test: python cryptography required for BIP 324 ↵Andrew Chow
functional tests c534c0871038ded72dc9078cc91e030ceb746196 [test/crypto] Add FSChaCha20Poly1305 AEAD python implementation (stratospher) c2a458f1c2f54aca132da9959b3fca19fdf1f813 [test/crypto] Add FSChaCha20 python implementation (stratospher) c4ea5f628806ead16e21d2a1f639fc6acc0400f5 [test/crypto] Add RFC 8439's ChaCha20Poly1305 AEAD (stratospher) 9fc6e0355e5ab52e76e647e5dad7571f962b4d4c [test/crypto] Add Poly1305 python implementation (stratospher) fec2ca6c9a8a8e44b6e4d51c4ef7fa6eaca6e446 [test/crypto] Use chacha20_block function in `data_to_num3072` (stratospher) 0cde60da3a16a27697f1f9014fcd33ea92eb639b [test/crypto] Add ChaCha20 python implementation (stratospher) 69d3f50ab645ad0ade1d67c66d2058b48bcf943b [test/crypto] Add HMAC-based Key Derivation Function (HKDF) (stratospher) 08a4a56cbcfa54366c2c0bb52bb147fc2740edc5 [test] Move test framework crypto functions to crypto/ (stratospher) Pull request description: split off from #24748 to keep commits related to cryptography and functional test framework changes separate. This PR adds python implementation and unit tests for HKDF, ChaCha20, Poly1305, ChaCha20Poly1305 AEAD, FSChaCha20 and FSChaCha20Poly1305 AEAD. They're based on https://github.com/bitcoin/bips/blob/cc177ab7bc5abcdcdf9c956ee88afd1052053328/bip-0324/reference.py for easy review. ACKs for top commit: sipa: utACK c534c0871038ded72dc9078cc91e030ceb746196 achow101: ACK c534c0871038ded72dc9078cc91e030ceb746196 theStack: re-ACK c534c0871038ded72dc9078cc91e030ceb746196 Tree-SHA512: 08a0a422d2937eadcf0edfede37e535e6bc4c2e4b192441bbf9bc26dd3f03fa3388effd22f0527c55af173933d0b50e5b2b3d36f2b62d0aca3098728ef06970e
2023-11-05test: remove custom rpc timeout for `wallet_miniscript.py`, reorder in ↵Sebastian Falbesoner
test_runner
2023-10-30Remove feature_txindex_compatibility.py in V27Brandon Odiwuor
2023-10-08qa: bound testing for TapMiniscriptAntoine Poinsot
Make sure we can spend a maximum-sized Miniscript under Tapscript context.
2023-10-02test: enable v2 transport between nodes in some functional testsPieter Wuille
2023-10-02net: use V2Transport when NODE_P2P_V2 service flag is presentPieter Wuille
Co-authored-by: Dhruv Mehta <856960+dhruv@users.noreply.github.com>
2023-09-30test: add feature_assumeutxo functional testJames O'Beirne
Most ideas for test improvements (TODOs) provided by Russ Yanofsky.
2023-09-29[test/crypto] Add RFC 8439's ChaCha20Poly1305 AEADstratospher
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2023-09-29[test/crypto] Add Poly1305 python implementationstratospher
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2023-09-29[test/crypto] Add ChaCha20 python implementationstratospher
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2023-09-14Merge bitcoin/bitcoin#26152: Bump unconfirmed ancestor transactions to ↵Andrew Chow
target feerate f18f9ef4d31c70e2d71ab90a24511692821418c3 Amend bumpfee for inputs with overlapping ancestry (Murch) 2e35e944dab09eff30952233f8dfc0b12c4553d5 Bump unconfirmed parent txs to target feerate (Murch) 3e3e05241128f68cf12f73ee06ff997395643885 coinselection: Move GetSelectionWaste into SelectionResult (Andrew Chow) c57889da6650715f3e1153b6104bbdae15fcac90 [node] interface to get bump fees (glozow) c24851be945b2a633ee44ed3c8a501eee5580b62 Make MiniMinerMempoolEntry fields private (Murch) ac6030e4d8f7d578cd4a8593f41189efca548064 Remove unused imports (Murch) d2f90c31ef3b8dee5a3e0804ecc62fa1cfec7cd5 Fix calculation of ancestor set feerates in test (Murch) a1f7d986e0211e54e21a1d4a570e5f15294dca72 Match tx names to index in miniminer overlap test (Murch) Pull request description: Includes some commits to address follow-ups from #27021: https://github.com/bitcoin/bitcoin/pull/27021#issuecomment-1554675156 Reduces the effective value of unconfirmed UTXOs by the fees necessary to bump their ancestor transactions to the same feerate. While the individual UTXOs always account for their full ancestry before coin-selection, we can correct potential overestimates with a second pass where we establish the ancestry and bump fee for the whole input set collectively. Fixes #9645 Fixes #9864 Fixes #15553 ACKs for top commit: S3RK: ACK f18f9ef4d31c70e2d71ab90a24511692821418c3 ismaelsadeeq: ACK f18f9ef4d31c70e2d71ab90a24511692821418c3 achow101: ACK f18f9ef4d31c70e2d71ab90a24511692821418c3 brunoerg: crACK f18f9ef4d31c70e2d71ab90a24511692821418c3 t-bast: ACK https://github.com/bitcoin/bitcoin/pull/26152/commits/f18f9ef4d31c70e2d71ab90a24511692821418c3, I reviewed the latest changes and run e2e tests against eclair, everything looks good :+1: Tree-SHA512: b65180c4243b1f9d13c311ada7a1c9f2f055d530d6c533b78c2068b50b8c29ac1321e89e85675b15515760d4f1b653ebd9da77b37c7be52d9bc565a3538f0aa6
2023-09-14test: cover read-only blockstoreMatthew Zipkin
Co-authored-by: Andrew Chow <github@achow101.com>
2023-09-13Bump unconfirmed parent txs to target feerateMurch
When a transaction uses an unconfirmed input, preceding this commit it would not consider the feerate of the parent transaction. Given a parent transaction with a lower ancestor feerate, this resulted in the new transaction's ancestor feerate undershooting the target feerate. This commit changes how we calculate the effective value of unconfirmed UTXOs. The effective value of unconfirmed UTXOs is decreased by the fee necessary to bump its ancestry to the target feerate. This also impacts the calculation of the waste metric: since the estimate for the current fee is increased by the bump fees, unconfirmed UTXOs current fees appear less favorable compared to their unchanged long term fees. This has one caveat: if multiple UTXOs have overlapping ancestries, each of their individual estimates will account for bumping all ancestors.
2023-09-10[test] Move test framework crypto functions to crypto/stratospher
2023-08-24Merge bitcoin/bitcoin#28287: rpc, test: add `sendmsgtopeer` rpc and a test ↵Andrew Chow
for net-level deadlock situation b3a93b409e7fb33af77bd34a269a3eae71d3ba83 test: add functional test for deadlock situation (Martin Zumsande) 3557aa4d0ab59c18807661a49070b0e5cbfecde3 test: add basic tests for sendmsgtopeer to rpc_net.py (Martin Zumsande) a9a1d69391596f57851f545fae12f8851149d2c3 rpc: add test-only sendmsgtopeer rpc (Martin Zumsande) Pull request description: This adds a `sendmsgtopeer` rpc (for testing only) that allows a node to send a message (provided in hex) to a peer. While we would usually use a `p2p` object instead of a node for this in the test framework, that isn't possible in situations where this message needs to trigger an actual interaction of multiple nodes. Use this rpc to add test coverage for the bug fixed in #27981 (that just got merged): The test lets two nodes (almost) simultaneously send a single large (4MB) p2p message to each other, which would have caused a deadlock previously (making this test fail), but succeeds now. As can be seen from the discussion in #27981, it was not easy to reproduce this bug without `sendmsgtopeer`. I would imagine that `sendmsgtopeer` could also be helpful in various other test constellations. ACKs for top commit: ajtowns: ACK b3a93b409e7fb33af77bd34a269a3eae71d3ba83 sipa: ACK b3a93b409e7fb33af77bd34a269a3eae71d3ba83 achow101: ACK b3a93b409e7fb33af77bd34a269a3eae71d3ba83 Tree-SHA512: 6e22e72402f3c4dd70cddb9e96ea988444720f7a164031df159fbdd48056c8ac77ac53def045d9208a3ca07437c7c8e34f8b4ebc7066c0a84d81cd53f2f4fa5f
2023-08-22Merge bitcoin/bitcoin#28199: test: tx orphan handlingAndrew Chow
9eac5a0529f869075f0331e40d322c34fc8fc2af [functional test] transaction orphan handling (glozow) 61e77bb901bca038b46064dc4531e12c2ab50673 [test framework] make it easier to fast-forward setmocktime (glozow) Pull request description: I was doing some mutation testing (through reckless refactoring) locally and found some specific behaviors in orphan handling that weren't picked up by tests. Adding some of these test cases now can maybe help with reviewing refactors like #28031. - Parent requests aren't sent immediately. A delay is added and the requests are filtered by AlreadyHaveTx before they are sent, which means you can't use fake orphans to probe precise arrival timing of a tx. - Parent requests include all that are not AlreadyHaveTx. This means old confirmed parents may be requested. - The node does not give up on orphans if the peer responds to a parent request with notfound. This means that if a parent is an old confirmed transaction (in which notfound is expected), the orphan should still be resolved. - Rejected parents can cause an orphan to be dropped, but it depends on the reason and only based on txid. - Rejected parents can cause an orphan to be rejected too, by both wtxid and txid. - Requests for orphan parents should be de-duplicated with "regular" txrequest. If a missing parent has the same hash as an in-flight request, it shouldn't be requested. - Multiple orphans with overlapping parents should not cause duplicated parent requests. ACKs for top commit: instagibbs: reACK https://github.com/bitcoin/bitcoin/pull/28199/commits/9eac5a0529f869075f0331e40d322c34fc8fc2af dergoegge: reACK 9eac5a0529f869075f0331e40d322c34fc8fc2af achow101: ACK 9eac5a0529f869075f0331e40d322c34fc8fc2af fjahr: Code review ACK 9eac5a0529f869075f0331e40d322c34fc8fc2af Tree-SHA512: 85488dc6a3f62cf0c38e7dfe7839c01215b44b172d1755b18164d41d01038f3a749451241e4eba8b857fd344a445740b21d6382c45977234b21460e3f53b1b2a
2023-08-22test: add functional test for deadlock situationMartin Zumsande
2023-08-14[functional test] transaction orphan handlingglozow
generated by cgit v1.2.3 (git 2.26.2) at 2024-10-12 10:34:10 +0000