aboutsummaryrefslogtreecommitdiff
path: root/test/functional/test_framework/messages.py
AgeCommit message (Collapse)Author
2024-03-11test: p2p: check disconnect due to lack of desirable service flagsSebastian Falbesoner
2024-02-01test: Fix CPartialMerkleTree.nTransactions signednessMarcoFalke
2024-01-30Merge bitcoin/bitcoin#29067: test: Treat msg_version.relay as unsigned, ↵glozow
Remove `struct` packing in messages.py 55556a64a8e4e6238f990cf66295c3b9594c2c3d test: Remove struct import from messages.py (MarcoFalke) fa3fa86ddaaa2246e873b7a3f19bc589a3f46c11 scripted-diff: test: Use int from_bytes and to_bytes over struct packing (MarcoFalke) fafc0d68eef9c9381b1a3d8e160aad9eeb8540a7 test: Use int from_bytes and to_bytes over struct packing (MarcoFalke) fa3886b7c69cbbe564478f30bb2c35e9e6b1cffa test: Treat msg_version.relay as unsigned (MarcoFalke) Pull request description: `struct` has many issues in messages.py: * For unpacking, it requires to specify the length a second time, even when it is already clear from the `f.read(num_bytes)` context. * For unpacking, it is designed to support a long format string and returning a tuple of many values. However, except for 3 instances in `messages.py`, usually only a single value is unpacked and all those cases require an `[0]` access. * For packing and unpacking of a single value, the format string consists of characters that may be confusing and may need to be looked up in the documentation, as opposed to using easy to understand self-documenting code. I presume the above issues lead to accidentally treat `msg_version.relay` as a "signed bool", which is fine, but confusing. Fix all issues by using the built-in `int` helpers `to_bytes` and `from_bytes` via a scripted diff. Review notes: * `struct.unpack` throws an error if the number of bytes passed is incorrect. `int.from_bytes` doesn't know about "missing" bytes and treats an empty byte array as `int(0)`. "Extraneous" bytes should never happen, because all `read` calls are limited in this file. If it is important to keep this error behavior, a helper `int_from_stream(stream, num_bytes, bytes, byteorder, *, **kwargs)` can be added, which checks the number of bytes read from the stream. * For `struct.pack` and `int.to_bytes` the error behavior is the same, although the error messages are not identical. ACKs for top commit: stickies-v: ACK 55556a64a8e4e6238f990cf66295c3b9594c2c3d theStack: re-ACK 55556a64a8e4e6238f990cf66295c3b9594c2c3d Tree-SHA512: 1cef8cdfd763fb424ed4b8be850a834b83fd0ef47fbea626a29784eb4f4832d44e42c4fe05b298b6070a933ef278b0222289a9955a97c86707e091e20bbb247a
2024-01-29test: Remove struct import from messages.pyMarcoFalke
2024-01-29scripted-diff: test: Use int from_bytes and to_bytes over struct packingMarcoFalke
-BEGIN VERIFY SCRIPT- sed -i --regexp-extended 's!struct.unpack\("(|<|>)B", (.*)\)\[0\]!int.from_bytes(\2, "little")!g' ./test/functional/test_framework/messages.py sed -i --regexp-extended 's!struct.unpack\("<(H|I|Q)", (.*)\)\[0\]!int.from_bytes(\2, "little")!g' ./test/functional/test_framework/messages.py sed -i --regexp-extended 's!struct.unpack\("<(h|i|q)", (.*)\)\[0\]!int.from_bytes(\2, "little", signed=True)!g' ./test/functional/test_framework/messages.py sed -i --regexp-extended 's!struct.unpack\(">(H|I|Q)", (.*)\)\[0\]!int.from_bytes(\2, "big")!g' ./test/functional/test_framework/messages.py sed -i --regexp-extended 's!struct.pack\("<?B", (.*)\)!\1.to_bytes(1, "little")!g' ./test/functional/test_framework/messages.py sed -i --regexp-extended 's!struct.pack\("<I", (.*)\)!\1.to_bytes(4, "little")!g' ./test/functional/test_framework/messages.py sed -i --regexp-extended 's!struct.pack\("<i", (.*)\)!\1.to_bytes(4, "little", signed=True)!g' ./test/functional/test_framework/messages.py sed -i --regexp-extended 's!struct.pack\("<Q", (.*)\)!\1.to_bytes(8, "little")!g' ./test/functional/test_framework/messages.py sed -i --regexp-extended 's!struct.pack\("<q", (.*)\)!\1.to_bytes(8, "little", signed=True)!g' ./test/functional/test_framework/messages.py sed -i --regexp-extended 's!struct.pack\(">H", (.*)\)!\1.to_bytes(2, "big")!g' ./test/functional/test_framework/messages.py -END VERIFY SCRIPT-
2024-01-29test: Use int from_bytes and to_bytes over struct packingMarcoFalke
This is done in prepration for the scripted diff, which can not deal with the 0 literal int.
2024-01-29test: Treat msg_version.relay as unsignedMarcoFalke
The C++ code treats bool as uint8_t, so the python tests should as well. This also allows to simplify the code, because converting an empty byte array to int gives int(0). >>> int.from_bytes(b'') 0
2024-01-23[test] Move MAGIC_BYTES to messages.pystratospher
This avoids circular dependency happening when importing MAGIC_BYTES. Before, p2p.py <--import for EncryptedP2PState-- v2_p2p.py | ^ | | └---------import for MAGIC_BYTES----------┘ Now, MAGIC_BYTES are kept separately in messages.py Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-11-07Merge bitcoin/bitcoin#28374: test: python cryptography required for BIP 324 ↵Andrew Chow
functional tests c534c0871038ded72dc9078cc91e030ceb746196 [test/crypto] Add FSChaCha20Poly1305 AEAD python implementation (stratospher) c2a458f1c2f54aca132da9959b3fca19fdf1f813 [test/crypto] Add FSChaCha20 python implementation (stratospher) c4ea5f628806ead16e21d2a1f639fc6acc0400f5 [test/crypto] Add RFC 8439's ChaCha20Poly1305 AEAD (stratospher) 9fc6e0355e5ab52e76e647e5dad7571f962b4d4c [test/crypto] Add Poly1305 python implementation (stratospher) fec2ca6c9a8a8e44b6e4d51c4ef7fa6eaca6e446 [test/crypto] Use chacha20_block function in `data_to_num3072` (stratospher) 0cde60da3a16a27697f1f9014fcd33ea92eb639b [test/crypto] Add ChaCha20 python implementation (stratospher) 69d3f50ab645ad0ade1d67c66d2058b48bcf943b [test/crypto] Add HMAC-based Key Derivation Function (HKDF) (stratospher) 08a4a56cbcfa54366c2c0bb52bb147fc2740edc5 [test] Move test framework crypto functions to crypto/ (stratospher) Pull request description: split off from #24748 to keep commits related to cryptography and functional test framework changes separate. This PR adds python implementation and unit tests for HKDF, ChaCha20, Poly1305, ChaCha20Poly1305 AEAD, FSChaCha20 and FSChaCha20Poly1305 AEAD. They're based on https://github.com/bitcoin/bips/blob/cc177ab7bc5abcdcdf9c956ee88afd1052053328/bip-0324/reference.py for easy review. ACKs for top commit: sipa: utACK c534c0871038ded72dc9078cc91e030ceb746196 achow101: ACK c534c0871038ded72dc9078cc91e030ceb746196 theStack: re-ACK c534c0871038ded72dc9078cc91e030ceb746196 Tree-SHA512: 08a0a422d2937eadcf0edfede37e535e6bc4c2e4b192441bbf9bc26dd3f03fa3388effd22f0527c55af173933d0b50e5b2b3d36f2b62d0aca3098728ef06970e
2023-10-02net: advertise NODE_P2P_V2 if CLI arg -v2transport is onPieter Wuille
Co-authored-by: Dhruv Mehta <856960+dhruv@users.noreply.github.com>
2023-09-10[test] Move test framework crypto functions to crypto/stratospher
2023-07-10test: add support for all networks in CAddress in messages.pybrunoerg
Also removes TorV2 from messages.py See https://github.com/bitcoin/bitcoin/pull/22050 Co-authored-by: Matthew Zipkin <pinheadmz@gmail.com>
2023-04-23test: simplify uint256 (de)serialization routinesSebastian Falbesoner
These routines look fancy, but do nothing more than converting between byte objects of length 32 to/from integers in little endian byte order and can be replaced by simple one-liners, using the int.{from,to}_bytes methods (available since Python 3.2).
2022-12-24scripted-diff: Bump copyright headersHennadii Stepanov
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT- Commits of previous years: - 2021: f47dda2c58b5d8d623e0e7ff4e74bc352dfa83d7 - 2020: fa0074e2d82928016a43ca408717154a1c70a4db - 2019: aaaaad6ac95b402fe18d019d67897ced6b316ee0
2022-11-10p2p: Drop roles from sendtxrcnclGleb Naumenko
This feature was currently redundant (although could have provided more flexibility in the future), and already been causing confusion.
2022-10-17test: Add functional tests for sendtxrcncl from inboundGleb Naumenko
2022-08-29test: Test that an unconfirmed not-in-mempool chain is rebroadcastAndrew Chow
The test checks that parent txs are broadcast before child txs. The previous behavior is that the rebroadcasting would simply iterate mapWallet. As mapWallet is a std::unsorted_map, the child can sometimes come before the parent and thus be rebroadcast in the wrong order and fail the test.
2022-08-11test: add tests for `datacarrier` and `datacarriersize` optionsw0xlt
Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
2022-08-09test: refactor: deduplicate `DEFAULT_{ANCESTOR,DESCENDANT}_LIMIT` constantsSebastian Falbesoner
2022-07-19refactor: move `from_binary` helper from signet miner to test frameworkSebastian Falbesoner
Can be easily reviewed with `--color-moved=dimmed-zebra`.
2022-07-12scripted-diff: [test] Rename BIP125_SEQUENCE_NUMBER to MAX_BIP125_RBF_SEQUENCEMacroFake
-BEGIN VERIFY SCRIPT- sed -i 's:BIP125_SEQUENCE_NUMBER:MAX_BIP125_RBF_SEQUENCE:g' $(git grep -l BIP125_SEQUENCE_NUMBER ./test) -END VERIFY SCRIPT-
2022-05-13test: make BIP157 messages default-constructible (MESSAGEMAP compatibility)Sebastian Falbesoner
In order to deserialize received or read messages via lookup in MESSAGEMAP (e.g.: `t = MESSAGEMAP[msgtype]()`), the messages must have a default constructor, i.e. there needs to be the possibility to initialize them with zero arguments.
2022-01-11test: create txs with current `nVersion` (2) by defaultSebastian Falbesoner
This enables testing of BIP68 without the need of explicitly setting nVersion to 2. This is e.g. useful for transactions created with MiniWallet.
2022-01-11test: refactor: add constant for sequence number `SEQUENCE_FINAL`Sebastian Falbesoner
2021-12-09test: Replace hashlib.new with named constructorMarcoFalke
2021-11-10scripted-diff: Bump copyright headersMarcoFalke
The previous diff touched most files in ./test/, so bump the headers to avoid having to touch them again for a bump later. -BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./test/ -END VERIFY SCRIPT-
2021-08-16refactor: replace remaining binascii method calls Zero-1729
2021-08-03Merge bitcoin/bitcoin#16333: test: Set BIP34Height = 2 for regtestMarcoFalke
222290f54388270937cb6c174195717e2214ec0d test: Set BIP34Height = 2 for regtest (MarcoFalke) fac90c55be478f0323eafa1d560ea2c56f04fb23 test: Create all blocks with version 4 or higher (MarcoFalke) Pull request description: BIP34 is active on the current tip of mainnet, so all miners must obey it. It would be nice if it also was active in fresh regtest instances from the earliest time possible. I changed the BIP34 height to `2`, so that the block at height=1 may be used to mine a duplicate coinbase. (Needed to test mainnet behaviour) This pull is done in two commits: * test: Create all blocks with version 4 or higher: Now that BIP34 is activated earlier, we need to create blocks with a higher version number. Just bump it to 4 instead of 2 to avoid having to bump it again later. * test: Set BIP34Height = 2 for regtest: This fixes the BIP34 implementation in the tests (to match the one of the Core codebase) and updates the tests where needed ACKs for top commit: ajtowns: ACK 222290f54388270937cb6c174195717e2214ec0d jonatack: ACK 222290f54388270937cb6c174195717e2214ec0d tested and reviewed rebased to current master 5e213822f86d theStack: Tested ACK 222290f54388270937cb6c174195717e2214ec0d Tree-SHA512: d69c637a62a64b8e87de8c7f0b305823d8f4d115c1852514b923625dbbcf9a4854b5bb3771ff41702ebf47c4c182a4442c6d7c0b9f282c95a34b83e56a73939b
2021-08-02Merge bitcoin/bitcoin#22378: test: remove confusing `MAX_BLOCK_BASE_SIZE`MarcoFalke
607076d01bf23c69ac21950c17b01fb4e1130774 test: remove confusing `MAX_BLOCK_BASE_SIZE` (Sebastian Falbesoner) 4af97c74edcda56cd15523bf3a335adea2bad14a test: introduce `get_weight()` helper for CBlock (Sebastian Falbesoner) a084ebe1330bcec15715e08b0f65319142927ad1 test: introduce `get_weight()` helper for CTransaction (Sebastian Falbesoner) Pull request description: This is a very late follow-up PR to #10618, which removed the constant `MAX_BLOCK_BASE_SIZE` from the core implementation about four years ago (see also #10608 in why it was considered confusing and superfluous). Since there is also no point in still keeping it in the functional test framework, the PR switches to weight-based accounting on the relevant test code parts and use `MAX_BLOCK_WEIGHT` instead for the block limit checks. To prepare that, the first two commits introduce `get_weight()` helpers for the classes CTransaction and CBlock, respectively. ACKs for top commit: MarcoFalke: review ACK 607076d01bf23c69ac21950c17b01fb4e1130774 🚴 Tree-SHA512: d59aa0b6b3dfd0a849b8063e66de275d252f705f99e25cd3bf6daec028b47d946777ee5b42a060f5283cb18e917ac073119c2c0e11bbc21211f69ef0a6ed335a
2021-08-01test: refactor: remove `hex_str_to_bytes` helperSebastian Falbesoner
Use the built-in class method bytes.fromhex() instead, which is available since Python 3.0.
2021-07-15Merge bitcoin/bitcoin#22211: net: relay I2P addresses even if not reachable ↵W. J. van der Laan
(by us) 7593b06bd1262f438bf34769ecc00e9c22328e56 test: ensure I2P addresses are relayed (Vasil Dimov) e7468139a1dddd4946bc70697ec38816b3fa8f9b test: make CAddress in functional tests comparable (Vasil Dimov) 33e211d2a442e4555ff3401f92af4ee2f7552568 test: implement ser/unser of I2P addresses in functional tests (Vasil Dimov) 86742811ce3662789ac85334008090a3b54babe3 test: use NODE_* constants instead of magic numbers (Vasil Dimov) ba45f0270815d54ae3290efc16324c2ff1984565 net: relay I2P addresses even if not reachable (by us) (Vasil Dimov) Pull request description: Nodes that can reach the I2P network (have set `-i2psam=`) will relay I2P addresses even without this patch. However, nodes that can't reach the I2P network will not. This was done as a precaution in https://github.com/bitcoin/bitcoin/pull/20119 before anybody could connect to I2P because then, for sure, it would have been useless. Now, however, we have I2P support and a bunch of I2P nodes, so get all nodes on the network to relay I2P addresses to help with propagation, similarly to what we do with Tor addresses. ACKs for top commit: jonatack: ACK 7593b06bd1262f438bf34769ecc00e9c22328e56 naumenkogs: ACK 7593b06bd1262f438bf34769ecc00e9c22328e56. laanwj: Code review ACK 7593b06bd1262f438bf34769ecc00e9c22328e56 kristapsk: ACK 7593b06bd1262f438bf34769ecc00e9c22328e56. Code looks correct, tested that functional test suite passes and also that `test/functional/p2p_addrv2_replay.py` fails if I undo changes in `IsRelayable()`. Tree-SHA512: c9feec4a9546cc06bc2fec6d74f999a3c0abd3d15b7c421c21fcf2d610eb94611489e33d61bdcd5a4f42041a6d84aa892f7ae293b0d4f755309a8560b113b735
2021-07-03test: remove confusing `MAX_BLOCK_BASE_SIZE`Sebastian Falbesoner
The constant `MAX_BLOCK_BASE_SIZE` has been removed from the core implementation years ago due to being confusing and superfluous, as it is implied by the block weight limit (see PRs #10618 and #10608). Since there is also no point in still keeping it in the functional test framework, we switch to weight-based accounting on the relevant test code parts and use `MAX_BLOCK_WEIGHT` instead for the block limit checks.
2021-07-01test: introduce `get_weight()` helper for CBlockSebastian Falbesoner
2021-07-01test: introduce `get_weight()` helper for CTransactionSebastian Falbesoner
2021-06-21test: doc: improve doc for `from_hex` helper (mention `to_hex` alternative)Sebastian Falbesoner
2021-06-21scripted-diff: test: rename `FromHex` to `from_hex`Sebastian Falbesoner
-BEGIN VERIFY SCRIPT- sed -i 's/\<FromHex\>/from_hex/g' $(git grep -l FromHex) -END VERIFY SCRIPT- Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2021-06-21test: remove `ToHex` helper, use .serialize().hex() insteadSebastian Falbesoner
2021-06-21test: introduce `tx_from_hex` helper for tx deserializationSebastian Falbesoner
`FromHex` is mostly used for transactions, so we introduce a shortcut `tx_from_hex` for `FromHex(CTransaction, hex_str)`.
2021-06-18test: Create all blocks with version 4 or higherMarcoFalke
2021-06-11test: make CAddress in functional tests comparableVasil Dimov
This way we can compare CAddress objects using `==` or even arrays of CAddress using `array1 == array2`.
2021-06-11test: implement ser/unser of I2P addresses in functional testsVasil Dimov
2021-06-10test: Use MiniWallet in test_no_inherited_signaling RBF testMarcoFalke
2021-03-23Unconditionally check for fRelay field in test frameworkTroy Giorshev
There is a discrepancy in the implementation of our p2p protocol between bitcoind and the testing framework. The fRelay field is an optional field at the end of a version message as of protocol version 70001. However, when deserializing a message in bitcoind, we don't check the version to see if it should have an fRelay field or not. Instead we unconditionally attempt to deserialize into the field. This commit brings the testing framework in line with the implementation in core. This matters for a version message with the following fields: Version = 60000 fRelay = 1 Bitcoind would deserialize this into a version message with Version=60000 and fRelay=1, whereas (before this commit) our testing framework would deserialize this into a version message with Version=60000 and fRelay=0.
2021-02-17[test] Add P2P_SERVICES to p2p.pyJohn Newbery
The messages.py module should contain code and helpers for [de]serializing p2p messages. Specific usage of those messages should be in p2p.py. Therefore specify the nServices value in the calling code, not in the messages.py module.
2021-02-17[test] Move MY_RELAY to p2p.pyJohn Newbery
messages.py is for message and primitive data structures. Specifics about the test framework's p2p implementation should be in p2p.py. Also rename to P2P_VERSION_RELAY. Also rename msg_version.nRelay to relay. In Bitcoin Core, this is referred to as fRelay, since it's a bool, so this field has always been misnamed.
2021-02-17[test] Move MY_SUBVERSION to p2p.pyJohn Newbery
The messages.py module should contain code and helpers for [de]serializing p2p messages. Specific usage of those messages should be in p2p.py. Therefore move MY_SUBVERSION to p2p.py. Also rename to P2P_SUBVERSION.
2021-02-17[test] Move MY_VERSION to p2p.pyJohn Newbery
The messages.py module should contain code and helpers for [de]serializing p2p messages. Specific usage of those messages should be in p2p.py. Therefore move MY_VERSION to p2p.py. Also rename to P2P_VERSION to distinguish it from other versioning used in Bitcoin/Bitcoin Core. Also always set the nVersion field in CBlockLocator to 0 and ignore the field in deserialized messages. The field is not currently used for anything in Bitcoin Core.
2021-02-17[test] Move MIN_VERSION_SUPPORTED to p2p.pyJohn Newbery
The messages.py module should contain code and helpers for [de]serializing p2p messages. Specific usage of those messages should be in p2p.py. Therefore move MIN_VERSION_SUPPORTED to p2p.py. Also rename to MIN_P2P_VERSION_SUPPORTED to distinguish it from other versioning used in Bitcoin/Bitcoin Core.
2021-02-17Merge #20993: test: store subversion (user agent) as string in msg_versionMarcoFalke
de85af5cce727981383ac0fe81f635451b331f23 test: store subversion (user agent) as string in msg_version (Sebastian Falbesoner) Pull request description: It seems more natural to treat the "subversion" field (=user agent string, see [BIP 14](https://github.com/bitcoin/bips/blob/master/bip-0014.mediawiki#Proposal)) of a node as pure string rather than a bytestring within the test framework. This is also suggested with the naming prefix in `msg_version.strSubVer`: one probably wouldn't expect a field starting with "str" to be a bytestring that needs further decoding to be useful. This PR moves the encoding/decoding parts to the serialization/deserialization routines so that the user doesn't have to bother with that anymore. Note that currently, in the master branch the `msg_version.strSubVer` is never read (only in `msg_version.__repr__`); However, one issue that is solved by this PR came up while testing #19509 (not merged yet): A decoding script for binary message capture files takes use of the functional test framework convert it into JSON format. Bytestrings will be convered to hexstrings, while pure strings will (surprise surprise) end up without modification in the file. So without this patch, we get: ``` $ jq . out.json | grep -m5 strSubVer "strSubVer": "2f5361746f7368693a32312e39392e302f" "strSubVer": "2f5361746f7368693a302e32302e312f" "strSubVer": "2f5361746f7368693a32312e39392e302f" "strSubVer": "2f5361746f7368693a302e32302e312f" "strSubVer": "2f5361746f7368693a32312e39392e302f" ``` After this patch: ``` $ jq . out2.json | grep -m5 strSubVer "strSubVer": "/Satoshi:21.99.0/" "strSubVer": "/Satoshi:0.20.1/" "strSubVer": "/Satoshi:21.99.0/" "strSubVer": "/Satoshi:0.20.1/" "strSubVer": "/Satoshi:21.99.0/" ``` ACKs for top commit: jnewbery: utACK de85af5cce727981383ac0fe81f635451b331f23 Tree-SHA512: ff23642705c858e8387a625537dfec82e6b8a15da6d99b8d12152560e52d243ba17431b602b26f60996d897e00e3f37dcf8dc8a303ffb1d544df29a5937080f9
2021-01-23Add capture parserTroy Giorshev
This commit adds contrib/message-capture/message-capture-parser.py, a python script to be used alongside -capturemessages to parse the captured messages. It is complete with arguments and will parse any file given, sorting the messages in the files when creating the output. If an output file is specified with -o or --output, it will dump the messages in json format to that file, otherwise it will print to stdout. The small change to the unused msg_generic is to bring it in line with the other message classes, purely to avoid a bug in the future.
generated by cgit v1.2.3 (git 2.26.2) at 2024-09-19 19:31:31 +0000