aboutsummaryrefslogtreecommitdiff
path: root/test/functional/test_framework/messages.py
AgeCommit message (Collapse)Author
2020-08-06refactor: test: use _ variable for unused loop countersSebastian Falbesoner
substitutes "for x in range(N):" by "for _ in range(N):" indicates to the reader that a block is just repeated N times, and that the loop counter is not used in the body
2020-07-19test: Update test framework p2p protocol version to 70016Fabian Jahr
This new p2p protocol version allows to use WTXIDs for tx relay.
2020-07-10[protocol] Remove unused CADDR_TIME_VERSIONJohn Newbery
Add comments to CAddress serialization code explaining why it's no longer needed.
2020-06-19test: hoist p2p values to test framework constantsJon Atack
2020-05-26[test] Add test for cfilters.Jim Posen
2020-05-22[test] Add test for cfheadersJim Posen
2020-05-20Merge #18764: refactor: test: replace inv type magic numbers by constantsMarcoFalke
4a614ff88ac71d6457513a4f7aa8a3235874ff3f test: explicit imports from test_framework.messages in p2p_invalid_messages.py (Sebastian Falbesoner) b35e1d2471c3d9e0b6cc4b94f4726a7e6ea32206 test: add inventory type constant MSG_CMPCT_BLOCK (Sebastian Falbesoner) eeaaa58d2ccc243a7506d34bca47c678867d6dd5 test: replace inv type magic numbers by constants (Sebastian Falbesoner) Pull request description: Many functional tests still use magic numbers for inventory types, either passed to the `CInv` constructor or for comparing the `type` member of `CInv`. This PR replaces all of those by constants in the module `test_framework.messages` that have been introduced in commit c32cf9f62285b5cd18a5064aee91f0802f0f87a8: `MSG_TX` (1) or `MSG_BLOCK` (2). It also introduces a new constant `MSG_CMPCT_BLOCK` (naming as in `src/protocol.h`) and uses it to replace the remaining magic numbers. The occurences of the magic numbers were identified through `grep`ing for `CInv(` and `type ==`. The idea was first to create a scripted-diff, but since also adding missing `import`s is needed, this would be non-trivial. Besides, also some unneeded comments like `# 2 == "Block"` could be removed. ACKs for top commit: gzhao408: ACK [`4a614ff`](https://github.com/bitcoin/bitcoin/pull/18764/commits/4a614ff88ac71d6457513a4f7aa8a3235874ff3f) Tree-SHA512: 4ba4fdef9f3eef7fd5ac72cb03ca3524863d1ae292161c550424a4c1047283fa2d2e7e03017d1fbae3652b3cb14f08b8d4b368403f3f209993aef3f2e2b22784
2020-05-08[test] Add test for cfcheckptJim Posen
2020-05-07test: add inventory type constant MSG_CMPCT_BLOCKSebastian Falbesoner
2020-04-25test: use zero-argument super() shortcut (Python 3.0+)Sebastian Falbesoner
as defined in PEP 3135: "The new syntax: super() is equivalent to: super(__class__, <firstarg>) where __class__ is the class that the method was defined in, and <firstarg> is the first parameter of the method (normally self for instance methods, and cls for class methods)."
2020-04-20test: add further BIP37 size limit checks to p2p_filter.pySebastian Falbesoner
also unified method of detecting misbehaviour (using assert_debug_log instead of checking peer's banscore)
2020-04-19Merge #18610: scripted-diff: test: replace command with msgtype (naming)MarcoFalke
9df32e820d83aa74e2f175d8d63b5666b8b4ef0e scripted-diff: test: replace command with msgtype (Sebastian Falbesoner) Pull request description: This is a follow-up PR to https://github.com/bitcoin/bitcoin/pull/18533, which changed the naming of `strCommand` to `msg_type` in the network processing code. The same approach is done here for the function test framework, to get rid of the wrong "command" terminology for network mesage types. (Commands are usually used in the CLI or RPC context, so using the same name in the network message context would only be confusing.) The commit was created through the following steps: 1. search for all occurences of the string "command" within the folder `test/functional` ```git grep -i command test/functional > command_finds``` 2. manually sort out all false-positives, i.e. occurences of "command" which describe commands in the correct sense (mostly CLI or RPC related, also some with Socks5) 3. put the remaining occurences into a scripted-diff (a quite simple one, actually) that renames "command" to "msgtype" in the concerned files. The name `msgtype` was intentionally chosen without the underscore `_` as classes beginning with `msg_` define concrete types of messages. ACKs for top commit: MarcoFalke: ACK 9df32e820d83aa74e2f175d8d63b5666b8b4ef0e . Makes sense that tests use the same naming as Bitcoin Core. See `NetMsgType` here: https://doxygen.bitcoincore.org/namespace_net_msg_type.html Tree-SHA512: cd0ee08a382910b7f10ce583acdaf4f8a39f9ba4a22434a914415727eedd98bac538de9bf6633574d5eb86f62558bc8dcb638a3289d99b04f8481f34e7a9a0c7
2020-04-16scripted-diff: Bump copyright headersMarcoFalke
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT-
2020-04-15scripted-diff: test: replace command with msgtypeSebastian Falbesoner
This is the functional test framework pendant for 7777e3624fabe4718675b2be8b088697b7ad4d0d, which renamed "strCommand" with "msg_type" in the network processing code. -BEGIN VERIFY SCRIPT- # Rename in test framework sed -i 's/command/msgtype/g' ./test/functional/test_framework/messages.py ./test/functional/test_framework/mininode.py # Rename in individual tests sed -i 's/command/msgtype/g' ./test/functional/p2p_invalid_messages.py ./test/functional/p2p_leak.py -END VERIFY SCRIPT-
2020-04-11test: complete impl. of msg_merkleblock and wait_for_merkleblockSebastian Falbesoner
Implements the missing initialization/serialization methods for msg_merkleblock, based on the already present class CMerkleBlock. Also changes the method wait_for_merkleblock() to be more precise by waiting for a merkleblock with a specified blockhash instead of an arbitrary one. In the BIP37 test p2p_filter.py, this new method is used to make the test of receiving merkleblock and tx if a filter is set to be more precise, by checking if they also arrive in the right order.
2020-04-03test: add BIP37 remote crash bug [CVE-2013-5700] test to p2p_filter.pySebastian Falbesoner
2020-03-31test: add BIP37 'filterclear' test to p2p_filter.pySebastian Falbesoner
2020-03-25test: Add basic test for BIP 37MarcoFalke
2020-03-03test: Add bad-txns-*-toolarge test cases to invalid_txsMarcoFalke
2020-01-23test: Make msg_tx a witness txMarcoFalke
2019-10-02p2p: Remove BIP61 reject messagesMarcoFalke
2019-09-11tests: add a test for the 'servicesnames' RPC fielddarosior
In getpeerinfo and getnetworkinfo
2019-08-26Avoid using mutable default parameter valuespracticalswift
2019-07-19Merge #16152: Disable bloom filtering by default.fanquake
bead32e31e399090af30b2ee3539995d4105a66d Add release notes for DEFAULT_BLOOM change (Matt Corallo) f27309f55c4fa2b115525d72abb280757a568709 Move DEFAULT_PEERBLOOMFILTERS from validation.h to net_processing.h (Matt Corallo) 5efcb772838e404ca5757818d5548efcb872724b Disable bloom filtering by default. (Matt Corallo) Pull request description: BIP 37 bloom filters have been well-known to be a significant DoS target for some time. However, in order to provide continuity for SPV clients relying on it, the NODE_BLOOM service flag was added, and left as a default, to ensure sufficient nodes exist with such a flag. NODE_BLOOM is, at this point, well-established and, as long as there exist 0.18 nodes with default config (which I'd anticipate will be true for many years), will be available from some peers. By that time, the continued slowdown of BIP 37-based filtering will likely have rendered it useless (though this is already largely the case). Further, BIP 37 was deliberately never updated to support witness-based filtering as newer wallets are expected to migrate to some yet-to-be-network-exposed filters. ACKs for top commit: jnewbery: ACK bead32e31e399090af30b2ee3539995d4105a66d kallewoof: ACK bead32e31e399090af30b2ee3539995d4105a66d Tree-SHA512: ecd901898e8efe1a7c82b471af0acc2373c2282ac633eb58d9aae7c35deda1999d0f79fb0485e6cecbda7246aeda00206cd82c7fa36866e2ac64705ba93f9390
2019-06-06Disable bloom filtering by default.Matt Corallo
BIP 37 bloom filters have been well-known to be a significant DoS target for some time. However, in order to provide continuity for SPV clients relying on it, the NODE_BLOOM service flag was added, and left as a default, to ensure sufficient nodes exist with such a flag. NODE_BLOOM is, at this point, well-established and, as long as there exist 0.18 nodes with default config (which I'd anticipate will be true for many years), will be available from some peers. By that time, the continued slowdown of BIP 37-based filtering will likely have rendered it useless (though this is already largely the case). Further, BIP 37 was deliberately never updated to support witness-based filtering as newer wallets are expected to migrate to some yet-to-be-network-exposed filters.
2019-05-08tests: Make msg_block a witness blockMarcoFalke
This diff has been generated with the following script, but is better reviewed without looking at the script. # -BEGIN VERIFY SCRIPT- echo "Use msg_witness_block everywhere, except for tests that require msg_block" # This could be a separate commit, but it is combined with the # following scripts to reduce the overall diff sed -i -e 's/msg_block/msg_witness_block/g' ./test/functional/{feature_assumevalid,feature_cltv,feature_dersig,feature_versionbits_warning,p2p_fingerprint,p2p_sendheaders,p2p_unrequested_blocks,example_test,rpc_blockchain}.py echo "Rename msg_block to msg_no_witness_block" # Rename msg_block to msg_no_witness_block in all tests (not the # framework) sed -i -e 's/msg_block/msg_no_witness_block/g' $(git grep -l msg_block ./test/functional/*.py) # Derive msg_no_witness_block from msg_block # Make msg_block a witness block in messages.py patch -p1 --fuzz 0 << EOF diff --git a/test/functional/test_framework/messages.py b/test/functional/test_framework/messages.py index 00190e4cbd..e454ed5987 100755 --- a/test/functional/test_framework/messages.py +++ b/test/functional/test_framework/messages.py @@ -1133 +1133 @@ class msg_block: - return self.block.serialize(with_witness=False) + return self.block.serialize() @@ -1155 +1155 @@ class msg_generic: -class msg_witness_block(msg_block): +class msg_no_witness_block(msg_block): @@ -1158,2 +1158 @@ class msg_witness_block(msg_block): - r = self.block.serialize() - return r + return self.block.serialize(with_witness=False) @@ -1445 +1444 @@ class msg_blocktxn: - r += self.block_transactions.serialize(with_witness=False) + r += self.block_transactions.serialize() @@ -1452 +1451 @@ class msg_blocktxn: -class msg_witness_blocktxn(msg_blocktxn): +class msg_no_witness_blocktxn(msg_blocktxn): @@ -1456,3 +1455 @@ class msg_witness_blocktxn(msg_blocktxn): - r = b"" - r += self.block_transactions.serialize() - return r + return self.block_transactions.serialize(with_witness=False) EOF # Conclude rename of msg_block to msg_no_witness_block sed -i -e 's/msg_witness_block/msg_block/g' $(git grep -l msg_witness_block) # -END VERIFY SCRIPT-
2019-05-08test: Remove True argument to CBlock::serializeMarcoFalke
Unnamed arguments are confusing as to what they mean without looking up the function signature. Since segwit is active by default in regtest, and all blocks are serialized with witness (#15664, c459c5f), remove the argument `with_witness=True` from all calls to `CBlock::serialize` and `BlockTransactions::serialize`. This diff has been created with a script, but is better reviewed without a scripted diff. sed -i --regexp-extended -e 's/block(_?[2a-z]*)\.serialize\([a-z_]*=?True/block\1.serialize(/g' $(git grep -l serialize ./test)
2019-04-02change default Python block serialization to witnessGregory Sanders
2019-03-02scripted-diff: Update copyright in ./testMarcoFalke
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./test/ -END VERIFY SCRIPT-
2019-03-02scripted-diff: test: Use py3.5 bytes::hex() methodMarcoFalke
-BEGIN VERIFY SCRIPT- sed -i -e "s/def bytes_to_hex_str/def b_2_x/g" $(git grep -l bytes_to_hex_str) export RE_B_0="[^()]*" # match no bracket export RE_B_1="${RE_B_0}\(${RE_B_0}\)${RE_B_0}" # match exactly one () export RE_B_2="${RE_B_0}\(${RE_B_1}\)${RE_B_0}" # match wrapped (()) export RE_M="(b2x|bytes_to_hex_str)\(((${RE_B_0}|${RE_B_1}|${RE_B_2})*)\)" sed -i --regexp-extended -e "s/${RE_M}/\2.hex()/g" $(git grep -l -E '(b2x|bytes_to_hex_str)') sed -i --regexp-extended -e "/ +bytes_to_hex_str( as b2x)?,/d" $(git grep -l bytes_to_hex_str) sed -i --regexp-extended -e "s/ +bytes_to_hex_str( as b2x)?,//g" $(git grep -l bytes_to_hex_str) sed -i --regexp-extended -e "s/, bytes_to_hex_str( as b2x)?//g" $(git grep -l bytes_to_hex_str) export RE_M="(binascii\.)?hexlify\(((${RE_B_0}|${RE_B_1}|${RE_B_2})*)\).decode\(${RE_B_0}\)" sed -i --regexp-extended -e "s/${RE_M}/\2.hex()/g" $(git grep -l hexlify -- ':(exclude)share') sed -i --regexp-extended -e "/from binascii import hexlify$/d" $(git grep -l hexlify -- ':(exclude)share') sed -i --regexp-extended -e "s/(from binascii import) .*hexlify/\1 unhexlify/g" $(git grep -l hexlify -- ':(exclude)share') sed -i -e 's/ignore-names "/ignore-names "b_2_x,/g' ./test/lint/lint-python-dead-code.sh -END VERIFY SCRIPT-
2019-01-24remove some magic mining constants in functional testsGregory Sanders
2018-12-13Merge #14926: test: consensus: Check that final transactions are validWladimir J. van der Laan
aaaa8eb1edba2a28916d5da6001d421c1b1b253b test: consensus: Check that final transactions are valid (MarcoFalke) fae3617d79deee73dd375dc3ea5f4204a74420c5 test: Correctly deserialize without witness (MarcoFalke) Pull request description: There is no check that checks that final transactions are valid, i.e. the consensus rules could be changed (accidentally) with none of the tests failing. Tree-SHA512: 48f4c24bfcc525ddbc1bfe8c37131953b464823428c1f7a278ba6d98b98827b6b84a8eb2b33396bfb5b8cc4012b7cc1cd771637f405ea20beddae001c22aa290
2018-12-11test: Correctly deserialize without witnessMarcoFalke
2018-12-10Compare to None with is/is notDaniel Ingram
2018-11-07tests: Remove unused testing codepracticalswift
2018-10-26[tests] Test that nodes respond to getdata with notfoundMarcoFalke
If a node has not announced a tx at all, then it should respond to getdata messages for that tx with notfound, to avoid leaking tx origination privacy.
2018-09-26Strictly enforce instance attrs in critical functional test classes.Justin Turner Arthur
Additionally, removed redundant parentheses and added PEP-8 compliant spacing around those classes.
2018-08-23qa: Remove unused deserialization code in msg_versionMarcoFalke
2018-08-22p2p: Remove dead code for nVersion=10300MarcoFalke
2018-08-11Merge #13915: [qa] Add test for max number of entries in locatorMarcoFalke
fa85c985ed qa: Add p2p_invalid_locator test (MarcoFalke) Pull request description: Should not be merged *before* #13907 Tree-SHA512: a67ca407854c421ed20a184d0b0dc90085aed3e3431d9652a107fa3022244767e67f67e50449b7e95721f56906836b134615875f28a21e8a012eb22cfe6a66a5
2018-08-10qa: Add p2p_invalid_locator testMarcoFalke
2018-07-27Update copyright headers to 2018DrahtBot
2018-07-11Remove dead service bits codeMarcoFalke
2018-06-22[qa] Add a test for merkle proof malleationSuhas Daftuar
2018-03-24rpc: Add testmempoolacceptMarcoFalke
2018-03-21Fix typosDimitris Apostolou
2018-03-01Merge #10387: Eventually connect to NODE_NETWORK_LIMITED peersWladimir J. van der Laan
eb91835 Add setter for g_initial_block_download_completed (Jonas Schnelli) 3f56df5 [QA] add NODE_NETWORK_LIMITED address relay and sync test (Jonas Schnelli) 158e1a6 [QA] fix mininode CAddress ser/deser (Jonas Schnelli) fa999af [QA] Allow addrman loopback tests (add debug option -addrmantest) (Jonas Schnelli) 6fe57bd Connect to peers signaling NODE_NETWORK_LIMITED when out-of-IBD (Jonas Schnelli) 31c45a9 Accept addresses with NODE_NETWORK_LIMITED flag (Jonas Schnelli) Pull request description: Eventually connect to peers signalling NODE_NETWORK_LIMITED if we are out of IBD. Accept and relay NODE_NETWORK_LIMITED peers in addrman. Tree-SHA512: 8a238fc97f767f81cae1866d6cc061390f23a72af4a711d2f7158c77f876017986abb371d213d1c84019eef7be4ca951e8e6f83fda36769c4e1a1d763f787037
2018-02-09[QA] fix mininode CAddress ser/deserJonas Schnelli
2018-01-30[tests] Add P2PDataStore classJohn Newbery
P2PDataStore subclasses NodeConnCB. This class keeps a store of txs and blocks and responds to getdata requests from the node-under-test.
2018-01-10Merge #11403: SegWit wallet supportJonas Schnelli
b224a47a1 Add address_types test (Pieter Wuille) 7ee54fd7c Support downgrading after recovered keypool witness keys (Pieter Wuille) 940a21932 SegWit wallet support (Pieter Wuille) f37c64e47 Implicitly know about P2WPKH redeemscripts (Pieter Wuille) 57273f2b3 [test] Serialize CTransaction with witness by default (Pieter Wuille) cf2c0b6f5 Support P2WPKH and P2SH-P2WPKH in dumpprivkey (Pieter Wuille) 37c03d3e0 Support P2WPKH addresses in create/addmultisig (Pieter Wuille) 3eaa003c8 Extend validateaddress information for P2SH-embedded witness (Pieter Wuille) 30a27dc5b Expose method to find key for a single-key destination (Pieter Wuille) 985c79552 Improve witness destination types and use them more (Pieter Wuille) cbe197470 [refactor] GetAccount{PubKey,Address} -> GetAccountDestination (Pieter Wuille) 0c8ea6380 Abstract out IsSolvable from Witnessifier (Pieter Wuille) Pull request description: This implements a minimum viable implementation of SegWit wallet support, based on top of #11389, and includes part of the functionality from #11089. Two new configuration options are added: * `-addresstype`, with options `legacy`, `p2sh`, and `bech32`. It controls what kind of addresses are produced by `getnewaddress`, `getaccountaddress`, and `createmultisigaddress`. * `-changetype`, with the same options, and by default equal to `-addresstype`, that controls what kind of change is used. All wallet private and public keys can be used for any type of address. Support for address types dependent on different derivation paths will need a major overhaul of how our internal detection of outputs work. I expect that that will happen for a next major version. The above also applies to imported keys, as having a distinction there but not for normal operations is a disaster for testing, and probably for comprehension of users. This has some ugly effects, like needing to associate the provided label to `importprivkey` with each style address for the corresponding key. To deal with witness outputs requiring a corresponding redeemscript in wallet, three approaches are used: * All SegWit addresses created through `getnewaddress` or multisig RPCs explicitly get their redeemscripts added to the wallet file. This means that downgrading after creating a witness address will work, as long as the wallet file is up to date. * All SegWit keys in the wallet get an _implicit_ redeemscript added, without it being written to the file. This means recovery of an old backup will work, as long as you use new software. * All keypool keys that are seen used in transactions explicitly get their redeemscripts added to the wallet files. This means that downgrading after recovering from a backup that includes a witness address will work. These approaches correspond to solutions 3a, 1a, and 5a respectively from https://gist.github.com/sipa/125cfa1615946d0c3f3eec2ad7f250a2. As argued there, there is no full solution for dealing with the case where you both downgrade and restore a backup, so that's also not implemented. `dumpwallet`, `importwallet`, `importmulti`, `signmessage` and `verifymessage` don't work with SegWit addresses yet. They're remaining TODOs, for this PR or a follow-up. Because of that, several tests unexpectedly run with `-addresstype=legacy` for now. Tree-SHA512: d425dbe517c0422061ab8dacdc3a6ae47da071450932ed992c79559d922dff7b2574a31a8c94feccd3761c1dffb6422c50055e6dca8e3cf94a169bc95e39e959