aboutsummaryrefslogtreecommitdiff
path: root/test/functional/test_framework
AgeCommit message (Collapse)Author
2021-05-17test: MiniWallet: add P2PK supportSebastian Falbesoner
2021-05-10test: use MiniWallet for feature_csv_activation.pySebastian Falbesoner
This test can now be run even with the Bitcoin Core wallet disabled.
2021-05-05Merge bitcoin/bitcoin#21754: test: Run feature_cltv with MiniWalletMarcoFalke
fa066f1b66ab75e6c88e11922bbd085c9d3211de test: Run feature_cltv with MiniWallet (MarcoFalke) fa5591d38cfc917235a3dffd61c36dbf1f781a4d test: Hide tx rehash in helper (MarcoFalke) fa5f938cfe4264054e9017fbe710c81e7b3bb150 test: Remove new_tx reference (MarcoFalke) Pull request description: Allows to run the test even with no wallet compiled in ACKs for top commit: theStack: ACK fa066f1b66ab75e6c88e11922bbd085c9d3211de 💽 Tree-SHA512: 3f659a178ba3ee0baffd70fddf8b8a68e5551d85626c7f254b234d7f75e6a16430a32a7952037db358b579f045b4d296b46156f72e5d226f3e80334dc635ca10
2021-04-30Merge bitcoin/bitcoin#19521: Coinstats IndexW. J. van der Laan
5f96d7d22d8e05876c6fc014e70488699950fe38 rpc: gettxoutsetinfo rejects hash_serialized_2 for specific height (Fabian Jahr) 23fe50436be641d7417152adc683192649ba206a test: Add test for coinstatsindex behavior in reorgs (Fabian Jahr) 90c966b0f3cfbd6bce5883f46d8527c6853a86a2 rpc: Allow gettxoutsetinfo and getblockstats for stale blocks (Fabian Jahr) b9362392aef2689bc106c20925859ede555d082b index, rpc: Add use_index option for gettxoutsetinfo (Fabian Jahr) bb7788b121a30489bc81a1f46dde6a9b19ae4ec1 test: Test coinstatsindex robustness across restarts (Fabian Jahr) e0938c29099635150014ffc9bb0cafa8049ec55a test: Add tests for block_info in gettxoutsetinfo (Fabian Jahr) 2501576eccb08af80471c7b7b843b189ad6758c0 rpc, index: Add verbose amounts tracking to Coinstats index (Fabian Jahr) 655d929836a71af23d2035d2e2e99ad8b8c340c3 test: add coinstatsindex getindexinfo coverage, improve current tests (Jon Atack) ca01bb8d689f93e1c7669b0ba7a4994c0206dabd rpc: Add Coinstats index to getindexinfo (Fabian Jahr) 57a026c30fef3138bb8db46e6865acb9dc2674f8 test: Add unit test for Coinstats index (Fabian Jahr) 6a4c0c09ab4d073a26c3c4a02783d5dcd88f6eef test: Add functional test for Coinstats index (Fabian Jahr) 3f166ecc125fce6ccd995687fa16572090a5d099 rpc: gettxoutsetinfo can be requested for specific blockheights (Fabian Jahr) 3c914d58ff323255b32e717d0ce28209ec0abdaa index: Coinstats index can be activated with command line flag (Fabian Jahr) dd58a4de21469d6d848ae309edc47f558628221d index: Add Coinstats index (Fabian Jahr) a8a46c4b3cfda4b95c92a36f8cebd3606377e57d refactor: Simplify ApplyStats and ApplyHash (Fabian Jahr) 9c8a265fd21a87228c18a1661df99fedc1866baf refactor: Pass hash_type to CoinsStats in stats object (Fabian Jahr) 2e2648a9021dfbb6e17dfa81472f057dacbc34e0 crypto: Make MuHash Remove method efficient (Fabian Jahr) Pull request description: This is part of the coinstats index project tracked in #18000 While the review of the new UTXO set hash algorithm (MuHash) takes longer recently #19328 was merged which added the possibility to run `gettxoutsetinfo` with a specific hash type. As the first type it added `hash_type=none` which skips the hashing of the UTXO set altogether. This alone did not make `gettxoutsetinfo` much faster but it allows the use of an index for the remaining coin statistics even before a new hashing algorithm has been added. Credit to Sjors for the idea to take this intermediate step. Features summary: - Users can start their node with the option `-coinstatsindex` which syncs the index in the background - After the index is synced the user can use `gettxoutsetinfo` with `hash_type=none` or `hash_type=muhash` and will get the response instantly out of the index - The user can specify a height or block hash when calling `gettxoutsetinfo` to see coin statistics at a specific block height ACKs for top commit: Sjors: re-tACK 5f96d7d22d8e05876c6fc014e70488699950fe38 jonatack: Code review re-ACK 5f96d7d22d8e05876c6fc014e70488699950fe38 per `git range-diff 13d27b4 07201d3 5f96d7d` promag: Tested ACK 5f96d7d22d8e05876c6fc014e70488699950fe38. Light code review ACK 5f96d7d22d8e05876c6fc014e70488699950fe38. Tree-SHA512: cbca78bee8e9605c19da4fbcd184625fb280200718396c694a56c7daab6f44ad23ca9fb5456d09f245d8b8d9659fdc2b3f3ce5e953c1c6cf4003dbc74c0463c2
2021-04-29test: Assert that exit code indicates failureMarcoFalke
2021-04-29test: Run feature_cltv with MiniWalletMarcoFalke
2021-04-29Merge bitcoin/bitcoin#21762: test: Speed up mempool_spend_coinbase.pyMarcoFalke
fa40eb5b6bfd151912c58d61771f6a6528f44e67 test: Speed up mempool_spend_coinbase.py (MarcoFalke) fa29382ab23d52b86bfda8a267195b6c51b713c2 test: Fix test cache issue (MarcoFalke) fa085b470a9647f3b261f506b46f4e7ca2baf0b0 test: Create MiniWallet.create_self_transfer (MarcoFalke) fa1bedb4944b513a3c9184ad549f58bfbe69e20e test: Add MiniWallet.sendrawtransaction (MarcoFalke) Pull request description: Locally the test will run 4 seconds faster with `--valgrind` (18s vs 14s) ACKs for top commit: mjdietzx: crACK https://github.com/bitcoin/bitcoin/pull/21762/commits/fa40eb5b6bfd151912c58d61771f6a6528f44e67 Tree-SHA512: ecfb60dda5ca5d7e6367bb9c6210390d95ebf6396ce657728901d118b75bb90c98f9351df3b01004d00682234448d6c6a13338d12097f7dced2cf7f1bd84d924
2021-04-28test: Fix intermittent issue in p2p_addr_relay.pyMarcoFalke
2021-04-23test: Fix test cache issueMarcoFalke
The documentation did not match the implementation, no coins were mined to the OP_TRUE address.
2021-04-23test: Create MiniWallet.create_self_transferMarcoFalke
2021-04-22test: Add MiniWallet.sendrawtransactionMarcoFalke
Can be reviewed with --ignore-all-space --color-moved=dimmed-zebra
2021-04-19test: Add tests for block_info in gettxoutsetinfoFabian Jahr
This additional data will automatically be returned if the coinstats index is used.
2021-04-19test: Add functional test for Coinstats indexFabian Jahr
2021-04-12tests: Skip SQLite fsyncs while testingAndrew Chow
Since we want tests to run quickly, and since tests do a lot more db operations than expected we expect to see in actual usage, we disable sqlite's syncing behavior to make db operations run much faster. This syncing behavior is necessary for normal operation as it helps guarantee that data won't become lost or corrupted, but in tests, we don't care about that.
2021-03-29Merge #20286: rpc: deprecate `addresses` and `reqSigs` from rpc outputsMarcoFalke
90ae3d8ca68334ec712d67b21a8d4721c6d79788 doc: Add release notes for -deprecatedrpc=addresses and bitcoin-tx (Michael Dietz) 085b3a729952896ccd0e40c17df569f4421f5493 rpc: deprecate `addresses` and `reqSigs` from rpc outputs (Michael Dietz) Pull request description: Considering the limited applicability of `reqSigs` and the confusing output of `1` in all cases except bare multisig, the `addresses` and `reqSigs` outputs are removed for all rpc commands. 1) add a new sane "address" field (for outputs that have an identifiable address, which doesn't include bare multisig) 2) with -deprecatedrpc: leave "reqSigs" and "addresses" intact (with all weird/wrong behavior they have now) 3) without -deprecatedrpc: drop "reqSigs" and "addresses" entirely always. Note: Some light refactoring done to allow us to very easily delete a few chunks of code (marked with TODOs) when we remove this deprecated behavior. Using `IsDeprecatedRPCEnabled` in core_write.cpp caused some circular dependencies involving core_io Circular dependencies were caused by rpc/util unnecessarily importing node/coinstats and node/transaction. Really what rpc/util needs are some fundamental type/helper-function definitions. So this was cleaned up to make more sense. This fixes #20102. ACKs for top commit: MarcoFalke: re-ACK 90ae3d8ca68334ec712d67b21a8d4721c6d79788 📢 Tree-SHA512: 8ffb617053b5f4a8b055da17c06711fd19632e0037d71c4c8135e50c8cd7a19163989484e4e0f17a6cc48bd597f04ecbfd609aef54b7d1d1e76a784214fcf72a
2021-03-26Merge #21390: test: Test improvements for UTXO set hash testsMarcoFalke
4f2653a89018fa4d24bd2a551832a7410b682600 test: Use deterministic chain in utxo set hash test (Fabian Jahr) 4973c5175c5fd1f4791ea26e8ddefd6fb11ac1c3 test: Remove wallet dependency of utxo set hash test (Fabian Jahr) 1a27af1d7b5ec18b4248ead1eaf0f381047b4b24 rpc: Improve gettxoutsetinfo help (Fabian Jahr) Pull request description: Follow-ups to #19145: - Small improvement on the help text of RPC gettxoutsetinfo - Using deterministic blockchain in the test `functional/feature_utxo_set_hash.py` - Removing wallet dependency in the test `functional/feature_utxo_set_hash.py` Split out of #19521. ACKs for top commit: MarcoFalke: review ACK 4f2653a89018fa4d24bd2a551832a7410b682600 👲 Tree-SHA512: 92927b3aa22b6324eb4fc9d346755313dec44d973aa69a0ebf80a8569b5f3a7cf3539721ebdba183737534b9e29b3e33f412515890f0d0b819878032a3bba8f9
2021-03-24Merge #21357: test: Unconditionally check for fRelay field in test frameworkMarcoFalke
39a9ec579f023ab262a1abd1f0c869be5b1f3f4d Unconditionally check for fRelay field in test framework (Troy Giorshev) Pull request description: picking up #20411 (rebased onto master) There is a discrepancy in the implementation of our p2p protocol between bitcoind and the testing framework. The fRelay field is an optional field at the end of a version message as of protocol version 70001. However, when deserializing a message in bitcoind, we don't check the version to see if it should have an fRelay field or not. Instead, we unconditionally attempt to deserialize into the field. This commit brings the testing framework in line with the implementation in core. This matters for a version message with the following fields: Version = 60000 fRelay = 1 Bitcoind would deserialize this into a version message with Version=60000 and fRelay=1, whereas (before this commit) our testing framework would deserialize this into a version message with Version=60000 and fRelay=0. ACKs for top commit: jnewbery: utACK 39a9ec579f023ab262a1abd1f0c869be5b1f3f4d Tree-SHA512: 13a23f1180b7121ba41cb85baa38094b41f4607a7c88b3384775177cb116e76faf5514760624f98a4e8a830767407c46753a7e0285158c33e0c6ce395de8f15c
2021-03-23Unconditionally check for fRelay field in test frameworkTroy Giorshev
There is a discrepancy in the implementation of our p2p protocol between bitcoind and the testing framework. The fRelay field is an optional field at the end of a version message as of protocol version 70001. However, when deserializing a message in bitcoind, we don't check the version to see if it should have an fRelay field or not. Instead we unconditionally attempt to deserialize into the field. This commit brings the testing framework in line with the implementation in core. This matters for a version message with the following fields: Version = 60000 fRelay = 1 Bitcoind would deserialize this into a version message with Version=60000 and fRelay=1, whereas (before this commit) our testing framework would deserialize this into a version message with Version=60000 and fRelay=0.
2021-03-23test: Remove wallet dependency of utxo set hash testFabian Jahr
2021-03-23rpc: deprecate `addresses` and `reqSigs` from rpc outputsMichael Dietz
1) add a new sane "address" field (for outputs that have an identifiable address, which doesn't include bare multisig) 2) with -deprecatedrpc: leave "reqSigs" and "addresses" intact (with all weird/wrong behavior they have now) 3) without -deprecatedrpc: drop "reqSigs" and "addresses" entirely, always.
2021-03-16Use Bech32m encoding for v1+ segwit addressesPieter Wuille
This also includes updates to the Python test framework implementation, test vectors, and release notes.
2021-03-05Merge #21246: doc: Correction for VerifyTaprootCommitment commentsfanquake
6a0a6e7d0509e48a6cc08c6604a25671d5f8cab4 Correction for VerifyTaprootCommitment comments (Russell O'Connor) Pull request description: According to BIP-341, 'p' is called the taproot *internal* key, not inner key. ACKs for top commit: sipa: ACK 6a0a6e7d0509e48a6cc08c6604a25671d5f8cab4 benthecarman: ACK 6a0a6e7d0509e48a6cc08c6604a25671d5f8cab4 theStack: ACK 6a0a6e7d0509e48a6cc08c6604a25671d5f8cab4 Tree-SHA512: 94f553476a8404bff4b2d5724a1a54c5f530b987a616cd00a3800095f245c06e3c7a9066c729976f32069a56029406859a70ba523151d333dc1ed874f242bce8
2021-03-01Correction for VerifyTaprootCommitment commentsRussell O'Connor
According to BIP-341, 'p' is called the taproot *internal* key, not inner key.
2021-02-25Merge #21254: test: Avoid connecting to real network when running testsMarcoFalke
fa730e915766b012ff145e3c2af782668defb21f test: Avoid connecting to real network when running tests (MarcoFalke) fa1b713941f71b077da07e9dedb743c7e99c3ca8 test: Assume node is running in subtests (MarcoFalke) Pull request description: Introduced in #19884 ACKs for top commit: Sjors: ACK fa730e915766b012ff145e3c2af782668defb21f Tree-SHA512: fe132a9ffe2fae1ab16857a3dec9839526fdf74d27a1ae794fbffca8356f639c4b916dc888b260281e9cc793916706c18d1687ebb5a076d4e1c481d218d308d3
2021-02-25Merge #21200: test: Speed up rpc_blockchain.py by removing miniwallet.generate()Wladimir J. van der Laan
faa137eb9eac5554504b062a6dc865ca87fd572b test: Speed up rpc_blockchain.py by removing miniwallet.generate() (MarcoFalke) fa1fe80c757df0adcbfaf41b5c5c8a468bc07b6f test: Change address type from P2PKH to P2WSH in rpc_blockchain (MarcoFalke) fa4d8f3169e38cbdbae20258efebe7070c49f522 test: Cache 25 mature coins for ADDRESS_BCRT1_P2WSH_OP_TRUE (MarcoFalke) fad25153f5c8e88f72cf666b16b0b0dbdc45d3b1 test: Remove unused bug workaround (MarcoFalke) faabce7d07c5776e4116b1a7ad1f6c408a4a4e46 test: Start only the number of nodes that are needed (MarcoFalke) Pull request description: Speed up various tests: * Remove unused nodes, which only consume time on start/stop * Remove unused "bug workarounds" * Remove the need for `miniwallet.generate()` by adding `miniwallet.scan_blocks()`. (On my system, with valgrind, generating 105 blocks takes 3.31 seconds. Rescanning 5 blocks takes 0.11 seconds.) ACKs for top commit: laanwj: Code review ACK faa137eb9eac5554504b062a6dc865ca87fd572b Tree-SHA512: ead1988d5aaa748ef9f8520af1e0bf812cf1d72e281ad22fbd172b7306d850053040526f8adbcec0b9a971c697a0ee7ee8962684644d65b791663eedd505a025
2021-02-23rpc: add external_signer option to createwalletSjors Provoost
2021-02-21test: framework: add skip_if_no_external_signerSjors Provoost
2021-02-21test: Avoid connecting to real network when running testsMarcoFalke
Can be reviewed with --word-diff-regex=.
2021-02-18Merge #19809: log: Prefix log messages with function name and source code ↵Wladimir J. van der Laan
location if -logsourcelocations is set b4511e2e2ed1a6077ae6826a9ee6b7a311293d08 log: Prefix log messages with function name if -logsourcelocations is set (practicalswift) Pull request description: Prefix log messages with function name if `-logfunctionnames` is set. Yes, exactly like `-logthreadnames` but for function names instead of thread names :) This is a small developer ergonomics improvement: I've found this to be a cheap/simple way to correlate log output and originating function. For me it beats the ordinary cycle of 1.) try to figure out a regexp matching the static part of the dynamic log message, 2.) `git grep -E 'Using .* MiB out of .* requested for signature cache'`, 3.) `mcedit filename.cpp` (`openemacs filename.cpp` works too!) and 4.) search for log message and scroll up to find the function name :) Without any logging parameters: ``` $ src/bitcoind -regtest 2020-08-25T03:29:04Z Using RdRand as an additional entropy source 2020-08-25T03:29:04Z Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements 2020-08-25T03:29:04Z Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements 2020-08-25T03:29:04Z Loaded best chain: hashBestChain=0fff88f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e22ff height=0 date=2011-02-02T23:16:42Z progress=1.000000 2020-08-25T03:29:04Z block tree size = 1 2020-08-25T03:29:04Z nBestHeight = 0 2020-08-25T03:29:04Z Imported mempool transactions from disk: 0 succeeded, 0 failed, 0 expired, 0 already there, 0 waiting for initial broadcast 2020-08-25T03:29:04Z 0 addresses found from DNS seeds ``` With `-logthreadnames` and `-logfunctionnames`: ``` $ src/bitcoind -regtest -logthreadnames -logfunctionnames 2020-08-25T03:29:04Z [init] [ReportHardwareRand] Using RdRand as an additional entropy source 2020-08-25T03:29:04Z [init] [InitSignatureCache] Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements 2020-08-25T03:29:04Z [init] [InitScriptExecutionCache] Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements 2020-08-25T03:29:04Z [init] [LoadChainTip] Loaded best chain: hashBestChain=0fff88f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e22ff height=0 date=2011-02-02T23:16:42Z progress=1.000000 2020-08-25T03:29:04Z [init] [AppInitMain] block tree size = 1 2020-08-25T03:29:04Z [init] [AppInitMain] nBestHeight = 0 2020-08-25T03:29:04Z [loadblk] [LoadMempool] Imported mempool transactions from disk: 0 succeeded, 0 failed, 0 expired, 0 already there, 0 waiting for initial broadcast 2020-08-25T03:29:04Z [dnsseed] [ThreadDNSAddressSeed] 0 addresses found from DNS seeds ``` ACKs for top commit: laanwj: Code review ACK b4511e2e2ed1a6077ae6826a9ee6b7a311293d08 MarcoFalke: review ACK b4511e2e2ed1a6077ae6826a9ee6b7a311293d08 🌃 Tree-SHA512: d100f5364630c323f31d275259864c597f7725e462d5f4bdedcc7033ea616d7fc0d16ef1b2af557e692f4deea73c6773ccfc681589e7bf6ba970b9ec169040c7
2021-02-17[test] Check user agent string from test framework connectionsJohn Newbery
Add a check that new connections from the test framework to the node have the correct user agent string. This makes bugs easier to detect if the user agent string ever changes.
2021-02-17[test] Add P2P_SERVICES to p2p.pyJohn Newbery
The messages.py module should contain code and helpers for [de]serializing p2p messages. Specific usage of those messages should be in p2p.py. Therefore specify the nServices value in the calling code, not in the messages.py module.
2021-02-17[test] Move MY_RELAY to p2p.pyJohn Newbery
messages.py is for message and primitive data structures. Specifics about the test framework's p2p implementation should be in p2p.py. Also rename to P2P_VERSION_RELAY. Also rename msg_version.nRelay to relay. In Bitcoin Core, this is referred to as fRelay, since it's a bool, so this field has always been misnamed.
2021-02-17[test] Move MY_SUBVERSION to p2p.pyJohn Newbery
The messages.py module should contain code and helpers for [de]serializing p2p messages. Specific usage of those messages should be in p2p.py. Therefore move MY_SUBVERSION to p2p.py. Also rename to P2P_SUBVERSION.
2021-02-17[test] Move MY_VERSION to p2p.pyJohn Newbery
The messages.py module should contain code and helpers for [de]serializing p2p messages. Specific usage of those messages should be in p2p.py. Therefore move MY_VERSION to p2p.py. Also rename to P2P_VERSION to distinguish it from other versioning used in Bitcoin/Bitcoin Core. Also always set the nVersion field in CBlockLocator to 0 and ignore the field in deserialized messages. The field is not currently used for anything in Bitcoin Core.
2021-02-17[test] Move MIN_VERSION_SUPPORTED to p2p.pyJohn Newbery
The messages.py module should contain code and helpers for [de]serializing p2p messages. Specific usage of those messages should be in p2p.py. Therefore move MIN_VERSION_SUPPORTED to p2p.py. Also rename to MIN_P2P_VERSION_SUPPORTED to distinguish it from other versioning used in Bitcoin/Bitcoin Core.
2021-02-17Merge #20993: test: store subversion (user agent) as string in msg_versionMarcoFalke
de85af5cce727981383ac0fe81f635451b331f23 test: store subversion (user agent) as string in msg_version (Sebastian Falbesoner) Pull request description: It seems more natural to treat the "subversion" field (=user agent string, see [BIP 14](https://github.com/bitcoin/bips/blob/master/bip-0014.mediawiki#Proposal)) of a node as pure string rather than a bytestring within the test framework. This is also suggested with the naming prefix in `msg_version.strSubVer`: one probably wouldn't expect a field starting with "str" to be a bytestring that needs further decoding to be useful. This PR moves the encoding/decoding parts to the serialization/deserialization routines so that the user doesn't have to bother with that anymore. Note that currently, in the master branch the `msg_version.strSubVer` is never read (only in `msg_version.__repr__`); However, one issue that is solved by this PR came up while testing #19509 (not merged yet): A decoding script for binary message capture files takes use of the functional test framework convert it into JSON format. Bytestrings will be convered to hexstrings, while pure strings will (surprise surprise) end up without modification in the file. So without this patch, we get: ``` $ jq . out.json | grep -m5 strSubVer "strSubVer": "2f5361746f7368693a32312e39392e302f" "strSubVer": "2f5361746f7368693a302e32302e312f" "strSubVer": "2f5361746f7368693a32312e39392e302f" "strSubVer": "2f5361746f7368693a302e32302e312f" "strSubVer": "2f5361746f7368693a32312e39392e302f" ``` After this patch: ``` $ jq . out2.json | grep -m5 strSubVer "strSubVer": "/Satoshi:21.99.0/" "strSubVer": "/Satoshi:0.20.1/" "strSubVer": "/Satoshi:21.99.0/" "strSubVer": "/Satoshi:0.20.1/" "strSubVer": "/Satoshi:21.99.0/" ``` ACKs for top commit: jnewbery: utACK de85af5cce727981383ac0fe81f635451b331f23 Tree-SHA512: ff23642705c858e8387a625537dfec82e6b8a15da6d99b8d12152560e52d243ba17431b602b26f60996d897e00e3f37dcf8dc8a303ffb1d544df29a5937080f9
2021-02-16test: Speed up rpc_blockchain.py by removing miniwallet.generate()MarcoFalke
2021-02-16test: Cache 25 mature coins for ADDRESS_BCRT1_P2WSH_OP_TRUEMarcoFalke
2021-02-15Merge #21100: test: remove unused function xor_bytesMarcoFalke
f64adc1eedff9d342b49d7e6428b2da21130c23c test: remove unused function xor_bytes (Sebastian Falbesoner) Pull request description: The function `xor_bytes` was introduced in commit 3c226639eb134314a0640d34e4ccb6148dbde22f (#19953, BIP340-342 validation), even [code-reviewed](https://github.com/bitcoin/bitcoin/pull/19953/files#r509383731), but actually never used. The [default signing algorithm in BIP340](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki#Default_Signing) needs a xor operation, but this step is currently done by a single xor operation on large integer operands: ``` t = (sec ^ int.from_bytes(TaggedHash("BIP0340/aux", aux), 'big')).to_bytes(32, 'big') ``` Alternatively, we could keep the function and as well use it: ```diff --- a/test/functional/test_framework/key.py +++ b/test/functional/test_framework/key.py @@ -492,7 +492,7 @@ def sign_schnorr(key, msg, aux=None, flip_p=False, flip_r=False): P = SECP256K1.affine(SECP256K1.mul([(SECP256K1_G, sec)])) if SECP256K1.has_even_y(P) == flip_p: sec = SECP256K1_ORDER - sec - t = (sec ^ int.from_bytes(TaggedHash("BIP0340/aux", aux), 'big')).to_bytes(32, 'big') + t = xor_bytes(sec.to_bytes(32, 'big'), TaggedHash("BIP0340/aux", aux)) kp = int.from_bytes(TaggedHash("BIP0340/nonce", t + P[0].to_bytes(32, 'big') + msg), 'big') % SECP256K1_ORDER assert kp != 0 R = SECP256K1.affine(SECP256K1.mul([(SECP256K1_G, kp)])) ``` ACKs for top commit: practicalswift: cr ACK f64adc1eedff9d342b49d7e6428b2da21130c23c: untested unused code should be removed Tree-SHA512: e9afae303488f19c6f6f44874d3537ed1c8164a197490e2b4e34853db886b858825b719648fa1a30b95177dcee9cf241f94ee9b835f0a2cae07024ce38a8c090
2021-02-09test: remove unused function xor_bytesSebastian Falbesoner
2021-02-09test: remove unnecessary assignment in bdbBruno Garcia
2021-02-08test: remove type: comments in favour of actual annotationsfanquake
Now that we require Python 3.6+, we should be using variable type annotations directly rather than # type: comments. Also takes care of the discarded value issue in p2p_message_capture.py. See: https://github.com/bitcoin/bitcoin/pull/19509/files#r571674446.
2021-02-05Merge #20267: Disable and fix tests for when BDB is not compiledWladimir J. van der Laan
49797c3ccfbb9f7ac9c1fbb574d35b315c103805 tests: Disable bdb dump test when no bdb (Andrew Chow) 1194cf9269e6c4bf67b09c4766f42bf173d12c0a Fix wallet_send.py wallet setup to work with descriptors (Andrew Chow) fbaea7bfe44822710a36601c6b0febbd5e33dfbd Require legacy wallet for wallet_upgradewallet.py (Andrew Chow) b1b679e0ab7a9981e3e78424fe8836edd59abf6f Explicitly mark legacy wallet tests as such (Andrew Chow) 09514e1bef46444a67cde9ff13e76bd4b9f8c7ac Setup wallets for interface_zmq.py (Andrew Chow) 4d03ef9a73ceb5ef4e9d184a135bca6bdeb8c311 Use MiniWallet in rpc_net.py (Andrew Chow) 4de23824b0c711ece68f9fc007ffac12126710aa Setup wallets for interface_bitcoin_cli.py (Andrew Chow) 7c71c627d28f0cddaf2349a55336278a681c27c2 Setup wallets with descriptors for feature_notifications (Andrew Chow) 1f1bef8dbab7225884d769a45477ee11d0ebf654 Have feature_filelock.py test both bdb and sqlite, depending on compiled (Andrew Chow) c77975abc0123b29b0eb3481b8916e7c025b7c4c Disable upgrades tests that require BDB if BDB is not compiled (Andrew Chow) 1f20cac9d41e507901a2811d6db7147d7ab0321b Disable wallet_descriptor.py bdb format check if BDB is not compiled (Andrew Chow) 3641597d7ef6f5097a9e93cab3ef7e0f9c820296 tests: Don't make any wallets unless wallet is required (Andrew Chow) b9b88f57a9b9a28e0f0614c12ae3012cf5050b10 Skip legacy wallet reliant tests if BDB is not compiled (Andrew Chow) 6f36242389bd3e7eacf594ce90491e8ccca70f3a tests: Set descriptors default based on compilation (Andrew Chow) Pull request description: This PR fixes tests for when BDB is not compiled. Tests which rely on or test legacy wallet behavior are disabled and skipped when BDB is not compiled. For the components of some tests that are for legacy wallet things, those parts of the tests are skipped. For the majority of tests, changes are made so that they can be run with either legacy wallets or descriptor wallets without materially effecting the test. Most tests only need the wallet for balance and transactions, so the type of wallet is not an important part of those tests. Additionally, some tests are wallet agnostic and modified to instead use the test framework's MiniWallet. ACKs for top commit: laanwj: ACK 49797c3ccfbb9f7ac9c1fbb574d35b315c103805 ryanofsky: Code review ACK 49797c3ccfbb9f7ac9c1fbb574d35b315c103805. Only change since last review is dropping last commit. Previous review w/ suggestions for future followup is https://github.com/bitcoin/bitcoin/pull/20267#pullrequestreview-581508843 Tree-SHA512: 69659f8a81fb437ecbca962f4082c12835282dbf1fba7d9952f727a49e01981d749af9b09feda1c8ca737516c7d7a08ef17e782795df3fa69892d5021b41c1ed
2021-02-02Merge #19509: Per-Peer Message CaptureMarcoFalke
bff7c66e67aa2f18ef70139338643656a54444fe Add documentation to contrib folder (Troy Giorshev) 381f77be858d7417209b6de0b7cd23cb7eb99261 Add Message Capture Test (Troy Giorshev) e4f378a505922c0f544b4cfbfdb169e884e02be9 Add capture parser (Troy Giorshev) 4d1a582549bc982d55e24585b0ba06f92f21e9da Call CaptureMessage at appropriate locations (Troy Giorshev) f2a77ff97bec09dd5fcc043d8659d8ec5dfb87c2 Add CaptureMessage (Troy Giorshev) dbf779d5deb04f55c6e8493ce4e12ed4628638f3 Clean PushMessage and ProcessMessages (Troy Giorshev) Pull request description: This PR introduces per-peer message capture into Bitcoin Core. 📓 ## Purpose The purpose and scope of this feature is intentionally limited. It answers a question anyone new to Bitcoin's P2P protocol has had: "Can I see what messages my node is sending and receiving?". ## Functionality When a new debug-only command line argument `capturemessages` is set, any message that the node receives or sends is captured. The capture occurs in the MessageHandler thread. When receiving a message, it is captured as soon as the MessageHandler thread takes the message off of the vProcessMsg queue. When sending, the message is captured just before the message is pushed onto the vSendMsg queue. The message capture is as minimal as possible to reduce the performance impact on the node. Messages are captured to a new `message_capture` folder in the datadir. Each node has their own subfolder named with their IP address and port. Inside, received and sent messages are captured into two binary files, msgs_recv.dat and msgs_sent.dat, like so: ``` message_capture/203.0.113.7:56072/msgs_recv.dat message_capture/203.0.113.7:56072/msgs_sent.dat ``` Because the messages are raw binary dumps, included in this PR is a Python parsing tool to convert the binary files into human-readable JSON. This script has been placed on its own and out of the way in the new `contrib/message-capture` folder. Its usage is simple and easily discovered by the autogenerated `-h` option. ## Future Maintenance I sympathize greatly with anyone who says "the best code is no code". The future maintenance of this feature will be minimal. The logic to deserialize the payload of the p2p messages exists in our testing framework. As long as our testing framework works, so will this tool. Additionally, I hope that the simplicity of this tool will mean that it gets used frequently, so that problems will be discovered and solved when they are small. ## FAQ "Why not just use Wireshark" Yes, Wireshark has the ability to filter and decode Bitcoin messages. However, the purpose of the message capture added in this PR is to assist with debugging, primarily for new developers looking to improve their knowledge of the Bitcoin Protocol. This drives the design in a different direction than Wireshark, in two different ways. First, this tool must be convenient and simple to use. Using an external tool, like Wireshark, requires setup and interpretation of the results. To a new user who doesn't necessarily know what to expect, this is unnecessary difficulty. This tool, on the other hand, "just works". Turn on the command line flag, run your node, run the script, read the JSON. Second, because this tool is being used for debugging, we want it to be as close to the true behavior of the node as possible. A lot can happen in the SocketHandler thread that would be missed by Wireshark. Additionally, if we are to use Wireshark, we are at the mercy of whoever it maintaining the protocol in Wireshark, both as to it being accurate and recent. As can be seen by the **many** previous attempts to include Bitcoin in Wireshark (google "bitcoin dissector") this is easier said than done. Lastly, I truly believe that this tool will be used significantly more by being included in the codebase. It's just that much more discoverable. ACKs for top commit: MarcoFalke: re-ACK bff7c66e67aa2f18ef70139338643656a54444fe only some minor changes: 👚 jnewbery: utACK bff7c66e67aa2f18ef70139338643656a54444fe theStack: re-ACK bff7c66e67aa2f18ef70139338643656a54444fe Tree-SHA512: e59e3160422269221f70f98720b47842775781c247c064071d546c24fa7a35a0e5534e8baa4b4591a750d7eb16de6b4ecf54cbee6d193b261f4f104e28c15f47
2021-01-31Merge #20954: test: Declare `nodes` type in test_framework.py.MarcoFalke
5353b0c64d32e44fc411464e080d4b00fae7124e Change type definitions for "chain" and "setup_clean_chain" from type comments to Python 3.6+ types. Additionally, set type for "nodes". (Kiminuo) Pull request description: ### Motivation When I wanted to understand better https://github.com/bitcoin/bitcoin/pull/19145/files#diff-4bebbd3b112dc222ea7e75ef051838ceffcee63b9e9234a98a4cc7251d34451b test, I noticed that navigation in PyCharm/VS Code did not work for `nodes` variable. I think this is frustrating, especially for newcomers. ### Summary * This PR modifies Python 3.5 [type comments](https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html#variables) to Python 3.6+ types and adds a proper type for `nodes` [instance attribute](https://mypy.readthedocs.io/en/stable/class_basics.html#instance-and-class-attributes). * This PR does not change behavior. * This PR is intentionally very small, if the concept is accepted, a follow-up PRs can be more ambitious. ### End result 1. Open `test/functional/feature_abortnode.py` 2. Move your caret to: `self.nodes[0].generate[caret here](3)` 3. Use "Go to definition" [F12] should work now. I have tested this on PyCharm (Windows, Ubuntu) and VS Code (Windows, Ubuntu). Note: Some `TestNode` methods (e.g. `self.nodes[0].getblock(...)` ) use `__call__` mechanism and navigation does not work for them even with this PR. ACKs for top commit: laanwj: ACK 5353b0c64d32e44fc411464e080d4b00fae7124e theStack: ACK 5353b0c64d32e44fc411464e080d4b00fae7124e Tree-SHA512: 821773f052ab9b2889dc357d38c59407a4af09e3b86d7134fcca7d78e5edf3a5ede9bfb37595ea97caf9ebfcbda372bcf73763b7f89b0677670f21b3e396a12b
2021-01-27tests: Don't make any wallets unless wallet is requiredAndrew Chow
2021-01-27Skip legacy wallet reliant tests if BDB is not compiledAndrew Chow
2021-01-27tests: Set descriptors default based on compilationAndrew Chow
Determines whether descriptors should be used based on whether the --descriptor or --legacy-wallet option is set, and the compiled support. If no option is set and both BDB and SQLite are available, it defaults to legacy. This is used to switch descriptor agnostic tests between descriptors and legacy wallet.
2021-01-23Add capture parserTroy Giorshev
This commit adds contrib/message-capture/message-capture-parser.py, a python script to be used alongside -capturemessages to parse the captured messages. It is complete with arguments and will parse any file given, sorting the messages in the files when creating the output. If an output file is specified with -o or --output, it will dump the messages in json format to that file, otherwise it will print to stdout. The small change to the unused msg_generic is to bring it in line with the other message classes, purely to avoid a bug in the future.
2021-01-23test: store subversion (user agent) as string in msg_versionSebastian Falbesoner
generated by cgit v1.2.3 (git 2.26.2) at 2024-09-03 00:29:13 +0000