aboutsummaryrefslogtreecommitdiff
path: root/test/functional/p2p_invalid_messages.py
AgeCommit message (Collapse)Author
2021-07-15Rate limit the processing of incoming addr messagesPieter Wuille
While limitations on the influence of attackers on addrman already exist (affected buckets are restricted to a subset based on incoming IP / network group), there is no reason to permit them to let them feed us addresses at more than a multiple of the normal network rate. This commit introduces a "token bucket" rate limiter for the processing of addresses in incoming ADDR and ADDRV2 messages. Every connection gets an associated token bucket. Processing an address in an ADDR or ADDRV2 message from non-whitelisted peers consumes a token from the bucket. If the bucket is empty, the address is ignored (it is not forwarded or processed). The token counter increases at a rate of 0.1 tokens per second, and will accrue up to a maximum of 1000 tokens (the maximum we accept in a single ADDR or ADDRV2). When a GETADDR is sent to a peer, it immediately gets 1000 additional tokens, as we actively desire many addresses from such peers (this may temporarily cause the token count to exceed 1000). The rate limit of 0.1 addr/s was chosen based on observation of honest nodes on the network. Activity in general from most nodes is either 0, or up to a maximum around 0.025 addr/s for recent Bitcoin Core nodes. A few (self-identified, through subver) crawler nodes occasionally exceed 0.1 addr/s.
2021-05-07net: Clarify message header validation errorsW. J. van der Laan
Make the errors less shouty and more descriptive.
2021-05-06net: Sanitize message type for loggingW. J. van der Laan
- Use `SanitizeString` when logging message errors to make sure that the message type is sanitized. - For the `MESSAGESTART` error don't inspect and log header details at all: receiving invalid start bytes makes it likely that the packet isn't even formatted as valid P2P message. Logging the four unexpected start bytes should be enough. - Update `p2p_invalid_messages.py` test to check this. Issue reported by gmaxwell.
2020-11-19p2p: Ignore version msgs after initial version msgMarcoFalke
Sending a version message after the intial version message is peer misbehavior. Though, it seems arbitrary and confusing to disconnect only after exactly 100 version messages. Duplicate version messages affect us no different than any other unknown message. So remove the Misbehaving and ignore the redundant msgs.
2020-10-09net: advertise support for ADDRv2 via new messageVasil Dimov
Introduce a new message `sendaddrv2` to signal support for ADDRv2. Send the new message immediately after sending the `VERACK` message. Add support for receiving and parsing ADDRv2 messages. Send ADDRv2 messages (instead of ADDR) to a peer if he has advertised support for it. Co-authored-by: Carl Dong <contact@carldong.me>
2020-09-29Merge #19107: p2p: Move all header verification into the network layer, ↵fanquake
extend logging deb52711a17236d0fca302701b5af585341ab42a Remove header checks out of net_processing (Troy Giorshev) 52d4ae46ab822d0f54e246a6f2364415cda149bd Give V1TransportDeserializer CChainParams& member (Troy Giorshev) 5bceef6b12fa16d20287693be377dace3dfec3e5 Change CMessageHeader Constructor (Troy Giorshev) 1ca20c1af8f08f07c407c3183c37b467ddf0f413 Add doxygen comment for ReceiveMsgBytes (Troy Giorshev) 890b1d7c2b8312d41d048d2db124586c5dbc8a49 Move checksum check from net_processing to net (Troy Giorshev) 2716647ebf60cea05fc9edce6a18dcce4e7727ad Give V1TransportDeserializer an m_node_id member (Troy Giorshev) Pull request description: Inspired by #15206 and #15197, this PR moves all message header verification from the message processing layer and into the network/transport layer. In the previous PRs there is a change in behavior, where we would disconnect from peers upon a single failed checksum check. In various discussions there was concern over whether this was the right choice, and some expressed a desire to see how this would look if it was made to be a pure refactor. For more context, see https://bitcoincore.reviews/15206.html#l-81. This PR improves the separation between the p2p layers, helping improvements like [BIP324](https://github.com/bitcoin/bitcoin/pull/18242) and #18989. ACKs for top commit: ryanofsky: Code review ACK deb52711a17236d0fca302701b5af585341ab42a just rebase due to conflict on adjacent line jnewbery: Code review ACK deb52711a17236d0fca302701b5af585341ab42a. Tree-SHA512: 1a3b7ae883b020cfee1bef968813e04df651ffdad9dd961a826bd80654f2c98676ce7f4721038a1b78d8790e4cebe8060419e3d8affc97ce2b9b4e4b72e6fa9f
2020-09-28test: Check that invalid peer traffic is accounted forMarcoFalke
2020-09-28test: Remove confusing cast to same type (int to int)MarcoFalke
2020-09-22Remove header checks out of net_processingTroy Giorshev
This moves header size and netmagic checking out of net_processing and into net. This check now runs in ReadHeader, so that net can exit early out of receiving bytes from the peer. IsValid is now slimmed down, so it no longer needs a MessageStartChars& parameter. Additionally this removes the rest of the m_valid_* members from CNetMessage.
2020-08-26test: Update wait_until usage in tests not to use the one from utilsSeleme Topuz
Replace "wait_until()" usage from utils, with the ones from BitcoinTestFramework and P2PInterface. closes #19080
2020-08-21scripted-diff: Rename mininode to p2pJohn Newbery
-BEGIN VERIFY SCRIPT- sed -i 's/\.mininode/\.p2p/g' $(git grep -l "mininode") git mv test/functional/test_framework/mininode.py test/functional/test_framework/p2p.py -END VERIFY SCRIPT-
2020-06-19test: hoist p2p values to test framework constantsJon Atack
2020-06-19test: improve msg sends and p2p disconnections in p2p_invalid_messagesJon Atack
- call disconnect_p2ps() outside of the assert_debug_log scopes - send messages directly from the p2p conn rather than via nodes[0].p2p - add an assertion
2020-06-19test: refactor test_large_inv() into 3 tests with common methodJon Atack
2020-06-19test: add p2p_invalid_messages loggingJon Atack
2020-06-19net: update misbehavior logging for oversized messagesJon Atack
so that oversized ADDR, GETDATA, HEADERS and INV messages print the same consistent debug logs.
2020-06-17Test buffered valid messageTroy Giorshev
A message can be broken across two buffers, with the split inside its header. Usually this will occur when sending many messages, such that the first buffer fills. This test uses the RPC to verify that the message is actually being received in two pieces. There is a very rare chance of a race condition where the test framework sends a message in between the two halves of the message under test. In this case the peer will almost certainly disconnect and the test will fail. An assert has been added to help debugging that rare case.
2020-06-13[tests] Don't import asyncio to test magic bytesJohn Newbery
2020-06-08Refactor resource exhaustion testTroy Giorshev
This is a simple refactor of the specified test. It is now brought in line with the rest of the tests in the module. This should make things easier to debug, as all of the tests are now grouped together at the top.
2020-06-08Fix "invalid message size" testTroy Giorshev
This test originally made a message with an invalid stated length, and an invalid checksum. This was because only the header was changed, but the checksum stayed the same. This was fine for now because we check the header first to see if it has a valid stated size, and we disconnect if it does not, so we never end up checking for the checksum. If this behavior was to change, this test would become a problem. (Indeed I discovered this when playing around with this behavior). By instead creating a message with an oversized payload from the start, we create a message with an invalid stated length but a valid checksum, as intended. Additionally, this takes advantage to the newly module-global VALID_DATA_LIMIT as opposed to the magic 0x02000000. Yes, 4MB < 32MiB, but at the moment when receiving a message we check both, so this makes the test tighter.
2020-06-08Move size limits to module-globalTroy Giorshev
As well, this renames those variables to match PEP8 and this clears up the comment relating to VALID_DATA_LIMIT. Admittedly, this commit is mainly to make the following ones cleaner.
2020-06-08Remove two unneeded testsTroy Giorshev
Test 1 is a duplicate of test_size() later in the file. Inexplicably, this test does not work on macOS, whereas test_size() does. Test 2 is problematic for two reasons. First, it always fails with an invalid checksum, which is probably not what was intended. Second, it's not defined at this layer what the behavior should be. Hypothetically, if this test was fixed so that it gave messages with valid checksums, then the message would pass successfully thought the network layer and fail only in the processing layer. A priori the network layer has no idea what the size of a message "actually" is. The "Why does behavior change at 78 bytes" is because of the following: print(len(node.p2p.build_message(msg))) # 125 => Payload size = 125 - 24 = 101 If we take 77 bytes, then there are 101 - 77 = 24 left That's exactly the size of a header So, bitcoind deserializes the header and rejects it for some other reason (Almost always an invalid size (too large)) But, if we take 78 bytes, then there are 101 - 78 = 23 left That's not enough to fill a header, so the socket stays open waiting for more data. That's why we sometimes have to push additional data in order for the peer to disconnect. Additionally, both of these tests use the "conn" variable. For fun, go look at where it's declared. (Hint: test_large_inv(). Don't we all love python's idea of scope?)
2020-05-07test: explicit imports from test_framework.messages in p2p_invalid_messages.pySebastian Falbesoner
2020-05-07test: replace inv type magic numbers by constantsSebastian Falbesoner
2020-04-19Merge #18610: scripted-diff: test: replace command with msgtype (naming)MarcoFalke
9df32e820d83aa74e2f175d8d63b5666b8b4ef0e scripted-diff: test: replace command with msgtype (Sebastian Falbesoner) Pull request description: This is a follow-up PR to https://github.com/bitcoin/bitcoin/pull/18533, which changed the naming of `strCommand` to `msg_type` in the network processing code. The same approach is done here for the function test framework, to get rid of the wrong "command" terminology for network mesage types. (Commands are usually used in the CLI or RPC context, so using the same name in the network message context would only be confusing.) The commit was created through the following steps: 1. search for all occurences of the string "command" within the folder `test/functional` ```git grep -i command test/functional > command_finds``` 2. manually sort out all false-positives, i.e. occurences of "command" which describe commands in the correct sense (mostly CLI or RPC related, also some with Socks5) 3. put the remaining occurences into a scripted-diff (a quite simple one, actually) that renames "command" to "msgtype" in the concerned files. The name `msgtype` was intentionally chosen without the underscore `_` as classes beginning with `msg_` define concrete types of messages. ACKs for top commit: MarcoFalke: ACK 9df32e820d83aa74e2f175d8d63b5666b8b4ef0e . Makes sense that tests use the same naming as Bitcoin Core. See `NetMsgType` here: https://doxygen.bitcoincore.org/namespace_net_msg_type.html Tree-SHA512: cd0ee08a382910b7f10ce583acdaf4f8a39f9ba4a22434a914415727eedd98bac538de9bf6633574d5eb86f62558bc8dcb638a3289d99b04f8481f34e7a9a0c7
2020-04-16scripted-diff: Bump copyright headersMarcoFalke
-BEGIN VERIFY SCRIPT- ./contrib/devtools/copyright_header.py update ./ -END VERIFY SCRIPT-
2020-04-15scripted-diff: test: replace command with msgtypeSebastian Falbesoner
This is the functional test framework pendant for 7777e3624fabe4718675b2be8b088697b7ad4d0d, which renamed "strCommand" with "msg_type" in the network processing code. -BEGIN VERIFY SCRIPT- # Rename in test framework sed -i 's/command/msgtype/g' ./test/functional/test_framework/messages.py ./test/functional/test_framework/mininode.py # Rename in individual tests sed -i 's/command/msgtype/g' ./test/functional/p2p_invalid_messages.py ./test/functional/p2p_leak.py -END VERIFY SCRIPT-
2020-04-13test: Add various low-level p2p testsMarcoFalke
2020-03-31test: Remove redundant sync_with_ping after add_p2p_connectionMarcoFalke
Also replace the two-line (send_message + sync_with_ping) with the one-line send_and_ping
2020-01-25Merge #17770: test: bump test timeouts so that functional tests run in valgrindMarcoFalke
2d23082cbe4641175d752a5969f67cdadf1afcea bump test timeouts so that functional tests run in valgrind (Micky Yun Chan) Pull request description: ci/tests: Bump timeouts so all functional tests run on travis in valgrind #17763 Top commit has no ACKs. Tree-SHA512: 5a8c6e2ea02b715facfcb58c761577be15ae58c45a61654beb98c2c2653361196c2eec521bcae4a9a1bab8e409d6807de771ef4c46d3d05996ae47a22d499d54
2020-01-25bump test timeouts so that functional tests run in valgrindMicky Yun Chan
2020-01-15Replace coroutine with async def in p2p_invalid_messages.pyElichai Turkel
2019-11-14test: Remove fragile assert_memory_usage_stableMarcoFalke
2019-10-23Remove oversized message detection from log and interfacePieter Wuille
2019-10-18Refactor: split network transport deserializing from message containerJonas Schnelli
2019-09-16test: Bump timeouts in slow running testsMarcoFalke
2019-07-23test: Skip flaky p2p_invalid_messages test on macOSFabian Jahr
2019-04-04qa: Make swap_magic_bytes in p2p_invalid_messages atomicMarcoFalke
2019-02-02test: Fix race in p2p_invalid_messagesMarcoFalke
2019-01-24qa: Add tests for invalid message headersMarcoFalke
2018-12-20test: Bump timeout to run tests in travis thread sanitizerMarcoFalke
2018-11-26Make test p2p_invalid_messages.py pass: Allow for expected Travis ASAN ↵practicalswift
memory increase
2018-11-26qa: clean up assert_memory_usage_stable utilityJames O'Beirne
2018-11-26qa: fix p2p_invalid_messages on macOSJames O'Beirne
2018-11-06Send fewer spam messages in p2p_invalid_messagesJames O'Beirne
Builds on travis are failing because the test node isn't able to drop all the bad messages sent within the given timeout. Reduce the number of bad messages we're sending and increase the timeout to avoid failures on travis.
2018-11-01tests: add tests for invalid P2P messagesJames O'Beirne
E.g., ensure that we can't DoS a node by sending it a bunch of large, unrecognized messages.