diff options
author | fanquake <fanquake@gmail.com> | 2020-10-11 08:16:41 +0800 |
---|---|---|
committer | fanquake <fanquake@gmail.com> | 2020-10-11 08:51:57 +0800 |
commit | 0b2abaa666d6f3331e3246ffd64dd47946e9dcdf (patch) | |
tree | d1b254fd219179db7fb0657a8299f5707eb5bbea | |
parent | 12a1c3ad1a43634d2a98717e49e3f02c4acea2fe (diff) | |
parent | dcf0cb477699d11afd0ff37c8bfb2b1b4f7f1ee5 (diff) |
Merge #19954: Complete the BIP155 implementation and upgrade to TORv3
dcf0cb477699d11afd0ff37c8bfb2b1b4f7f1ee5 tor: make a TORv3 hidden service instead of TORv2 (Vasil Dimov)
353a3fdaad055eea42a0baf7326bdd591f541170 net: advertise support for ADDRv2 via new message (Vasil Dimov)
201a4596d92d640d5eb7e76cc8d959228fa09dbb net: CAddress & CAddrMan: (un)serialize as ADDRv2 (Vasil Dimov)
1d3ec2a1fda7446323786a52da1fd109c01aa6fb Support bypassing range check in ReadCompactSize (Pieter Wuille)
Pull request description:
This PR contains the two remaining commits from #19031 to complete the [BIP155](https://github.com/bitcoin/bips/blob/master/bip-0155.mediawiki) implementation:
`net: CAddress & CAddrMan: (un)serialize as ADDRv2`
`net: advertise support for ADDRv2 via new message`
plus one more commit:
`tor: make a TORv3 hidden service instead of TORv2`
ACKs for top commit:
jonatack:
re-ACK dcf0cb477699d11afd0ff37c8bfb2b1b4f7f1ee5 per `git diff 9b56a68 dcf0cb4` only change since last review is an update to the release notes which partially picked up the suggested text. Running a node on this branch and addnode-ing to 6 other Tor v3 nodes, I see "addrv2" and "sendaddrv2" messages in getpeerinfo in both the "bytesrecv_per_msg" and "bytessent_per_msg" JSON objects.
sipa:
ACK dcf0cb477699d11afd0ff37c8bfb2b1b4f7f1ee5
hebasto:
re-ACK dcf0cb477699d11afd0ff37c8bfb2b1b4f7f1ee5, the node works flawlessly in all of the modes: Tor-only, clearnet-only, mixed.
laanwj:
Edit: I have to retract this ACK for now, I'm having some problems with this PR on a FreeBSD node. It drops all outgoing connections with this dcf0cb477699d11afd0ff37c8bfb2b1b4f7f1ee5 merged on master (12a1c3ad1a43634d2a98717e49e3f02c4acea2fe).
ariard:
Code Review ACK dcf0cb4
Tree-SHA512: 28d4d0d817b8664d2f4b18c0e0f31579b2f0f2d23310ed213f1f436a4242afea14dfbf99e07e15889bc5c5c71ad50056797e9307ff8a90e96704f588a6171308
-rw-r--r-- | doc/files.md | 3 | ||||
-rw-r--r-- | doc/release-notes.md | 19 | ||||
-rw-r--r-- | src/addrman.h | 60 | ||||
-rw-r--r-- | src/net.h | 12 | ||||
-rw-r--r-- | src/net_processing.cpp | 45 | ||||
-rw-r--r-- | src/netaddress.cpp | 29 | ||||
-rw-r--r-- | src/netaddress.h | 6 | ||||
-rw-r--r-- | src/protocol.cpp | 4 | ||||
-rw-r--r-- | src/protocol.h | 24 | ||||
-rw-r--r-- | src/serialize.h | 22 | ||||
-rw-r--r-- | src/streams.h | 1 | ||||
-rw-r--r-- | src/test/net_tests.cpp | 17 | ||||
-rw-r--r-- | src/test/netbase_tests.cpp | 105 | ||||
-rw-r--r-- | src/torcontrol.cpp | 7 | ||||
-rwxr-xr-x | test/functional/p2p_addrv2_relay.py | 79 | ||||
-rwxr-xr-x | test/functional/p2p_invalid_messages.py | 95 | ||||
-rwxr-xr-x | test/functional/test_framework/messages.py | 103 | ||||
-rwxr-xr-x | test/functional/test_framework/p2p.py | 12 | ||||
-rwxr-xr-x | test/functional/test_runner.py | 1 |
19 files changed, 596 insertions, 48 deletions
diff --git a/doc/files.md b/doc/files.md index c5c612114b..64cff43d3f 100644 --- a/doc/files.md +++ b/doc/files.md @@ -58,7 +58,7 @@ Subdirectory | File(s) | Description `./` | `guisettings.ini.bak` | Backup of former [GUI settings](#gui-settings) after `-resetguisettings` option is used `./` | `ip_asn.map` | IP addresses to Autonomous System Numbers (ASNs) mapping used for bucketing of the peers; path can be specified with the `-asmap` option `./` | `mempool.dat` | Dump of the mempool's transactions -`./` | `onion_private_key` | Cached Tor onion service private key for `-listenonion` option +`./` | `onion_v3_private_key` | Cached Tor onion service private key for `-listenonion` option `./` | `peers.dat` | Peer IP address database (custom format) `./` | `settings.json` | Read-write settings set through GUI or RPC interfaces, augmenting manual settings from [bitcoin.conf](bitcoin-conf.md). File is created automatically if read-write settings storage is not disabled with `-nosettings` option. Path can be specified with `-settings` option `./` | `.cookie` | Session RPC authentication cookie; if used, created at start and deleted on shutdown; can be specified by `-rpccookiefile` option @@ -100,6 +100,7 @@ Path | Description | Repository notes `blkindex.dat` | Blockchain index BDB database; replaced by {`chainstate/`, `blocks/index/`, `blocks/revNNNNN.dat`<sup>[\[2\]](#note2)</sup>} in 0.8.0 | [PR #1677](https://github.com/bitcoin/bitcoin/pull/1677) `blk000?.dat` | Block data (custom format, 2 GiB per file); replaced by `blocks/blkNNNNN.dat`<sup>[\[2\]](#note2)</sup> in 0.8.0 | [PR #1677](https://github.com/bitcoin/bitcoin/pull/1677) `addr.dat` | Peer IP address BDB database; replaced by `peers.dat` in [0.7.0](https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.7.0.md) | [PR #1198](https://github.com/bitcoin/bitcoin/pull/1198), [`928d3a01`](https://github.com/bitcoin/bitcoin/commit/928d3a011cc66c7f907c4d053f674ea77dc611cc) +`onion_private_key` | Cached Tor onion service private key for `-listenonion` option. Was used for Tor v2 services; replaced by `onion_v3_private_key` in [0.21.0](https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.21.0.md) | [PR #19954](https://github.com/bitcoin/bitcoin/pull/19954) ## Notes diff --git a/doc/release-notes.md b/doc/release-notes.md index ba5c3c2c1b..aef021a29d 100644 --- a/doc/release-notes.md +++ b/doc/release-notes.md @@ -60,6 +60,14 @@ From Bitcoin Core 0.20.0 onwards, macOS versions earlier than 10.12 are no longer supported. Additionally, Bitcoin Core does not yet change appearance when macOS "dark mode" is activated. +The node's known peers are persisted to disk in a file called `peers.dat`. The +format of this file has been changed in a backwards-incompatible way in order to +accommodate the storage of Tor v3 and other BIP155 addresses. This means that if +the file is modified by 0.21.0 or newer then older versions will not be able to +read it. Those old versions, in the event of a downgrade, will log an error +message that deserialization has failed and will continue normal operation +as if the file was missing, creating a new empty one. (#19954) + Notable changes =============== @@ -74,6 +82,17 @@ P2P and network changes node using P2P relay. This version reduces the initial broadcast guarantees for wallet transactions submitted via P2P to a node running the wallet. (#18038) +- The Tor onion service that is automatically created by setting the + `-listenonion` configuration parameter will now be created as a Tor v3 service + instead of Tor v2. The private key that was used for Tor v2 (if any) will be + left untouched in the `onion_private_key` file in the data directory (see + `-datadir`) and can be removed if not needed. Bitcoin Core will no longer + attempt to read it. The private key for the Tor v3 service will be saved in a + file named `onion_v3_private_key`. To use the deprecated Tor v2 service (not + recommended), then `onion_private_key` can be copied over + `onion_v3_private_key`, e.g. + `cp -f onion_private_key onion_v3_private_key`. (#19954) + Updated RPCs ------------ diff --git a/src/addrman.h b/src/addrman.h index ca045b91cd..b4089dc894 100644 --- a/src/addrman.h +++ b/src/addrman.h @@ -12,6 +12,7 @@ #include <random.h> #include <sync.h> #include <timedata.h> +#include <tinyformat.h> #include <util/system.h> #include <fs.h> @@ -264,6 +265,14 @@ protected: void SetServices_(const CService &addr, ServiceFlags nServices) EXCLUSIVE_LOCKS_REQUIRED(cs); public: + //! Serialization versions. + enum class Format : uint8_t { + V0_HISTORICAL = 0, //!< historic format, before commit e6b343d88 + V1_DETERMINISTIC = 1, //!< for pre-asmap files + V2_ASMAP = 2, //!< for files including asmap version + V3_BIP155 = 3, //!< same as V2_ASMAP plus addresses are in BIP155 format + }; + // Compressed IP->ASN mapping, loaded from a file when a node starts. // Should be always empty if no file was provided. // This mapping is then used for bucketing nodes in Addrman. @@ -285,8 +294,8 @@ public: /** - * serialized format: - * * version byte (1 for pre-asmap files, 2 for files including asmap version) + * Serialized format. + * * version byte (@see `Format`) * * 0x20 + nKey (serialized as if it were a vector, for backward compatibility) * * nNew * * nTried @@ -313,13 +322,16 @@ public: * We don't use SERIALIZE_METHODS since the serialization and deserialization code has * very little in common. */ - template<typename Stream> - void Serialize(Stream &s) const + template <typename Stream> + void Serialize(Stream& s_) const { LOCK(cs); - unsigned char nVersion = 2; - s << nVersion; + // Always serialize in the latest version (currently Format::V3_BIP155). + + OverrideStream<Stream> s(&s_, s_.GetType(), s_.GetVersion() | ADDRV2_FORMAT); + + s << static_cast<uint8_t>(Format::V3_BIP155); s << ((unsigned char)32); s << nKey; s << nNew; @@ -370,14 +382,34 @@ public: s << asmap_version; } - template<typename Stream> - void Unserialize(Stream& s) + template <typename Stream> + void Unserialize(Stream& s_) { LOCK(cs); Clear(); - unsigned char nVersion; - s >> nVersion; + + Format format; + s_ >> Using<CustomUintFormatter<1>>(format); + + static constexpr Format maximum_supported_format = Format::V3_BIP155; + if (format > maximum_supported_format) { + throw std::ios_base::failure(strprintf( + "Unsupported format of addrman database: %u. Maximum supported is %u. " + "Continuing operation without using the saved list of peers.", + static_cast<uint8_t>(format), + static_cast<uint8_t>(maximum_supported_format))); + } + + int stream_version = s_.GetVersion(); + if (format >= Format::V3_BIP155) { + // Add ADDRV2_FORMAT to the version so that the CNetAddr and CAddress + // unserialize methods know that an address in addrv2 format is coming. + stream_version |= ADDRV2_FORMAT; + } + + OverrideStream<Stream> s(&s_, s_.GetType(), stream_version); + unsigned char nKeySize; s >> nKeySize; if (nKeySize != 32) throw std::ios_base::failure("Incorrect keysize in addrman deserialization"); @@ -386,7 +418,7 @@ public: s >> nTried; int nUBuckets = 0; s >> nUBuckets; - if (nVersion != 0) { + if (format >= Format::V1_DETERMINISTIC) { nUBuckets ^= (1 << 30); } @@ -449,7 +481,7 @@ public: supplied_asmap_version = SerializeHash(m_asmap); } uint256 serialized_asmap_version; - if (nVersion > 1) { + if (format >= Format::V2_ASMAP) { s >> serialized_asmap_version; } @@ -457,13 +489,13 @@ public: CAddrInfo &info = mapInfo[n]; int bucket = entryToBucket[n]; int nUBucketPos = info.GetBucketPosition(nKey, true, bucket); - if (nVersion == 2 && nUBuckets == ADDRMAN_NEW_BUCKET_COUNT && vvNew[bucket][nUBucketPos] == -1 && + if (format >= Format::V2_ASMAP && nUBuckets == ADDRMAN_NEW_BUCKET_COUNT && vvNew[bucket][nUBucketPos] == -1 && info.nRefCount < ADDRMAN_NEW_BUCKETS_PER_ADDRESS && serialized_asmap_version == supplied_asmap_version) { // Bucketing has not changed, using existing bucket positions for the new table vvNew[bucket][nUBucketPos] = n; info.nRefCount++; } else { - // In case the new table data cannot be used (nVersion unknown, bucket count wrong or new asmap), + // In case the new table data cannot be used (format unknown, bucket count wrong or new asmap), // try to give them a reference based on their primary source address. LogPrint(BCLog::ADDRMAN, "Bucketing method was updated, re-bucketing addrman entries from disk\n"); bucket = info.GetNewBucket(nKey, m_asmap); @@ -867,6 +867,11 @@ public: bool m_legacyWhitelisted{false}; bool fClient{false}; // set by version message bool m_limited_node{false}; //after BIP159, set by version message + /** + * Whether the peer has signaled support for receiving ADDRv2 (BIP155) + * messages, implying a preference to receive ADDRv2 instead of ADDR ones. + */ + std::atomic_bool m_wants_addrv2{false}; std::atomic_bool fSuccessfullyConnected{false}; // Setting fDisconnect to true will cause the node to be disconnected the // next time DisconnectNodes() runs @@ -1114,11 +1119,16 @@ public: void PushAddress(const CAddress& _addr, FastRandomContext &insecure_rand) { + // Whether the peer supports the address in `_addr`. For example, + // nodes that do not implement BIP155 cannot receive Tor v3 addresses + // because they require ADDRv2 (BIP155) encoding. + const bool addr_format_supported = m_wants_addrv2 || _addr.IsAddrV1Compatible(); + // Known checking here is only to save space from duplicates. // SendMessages will filter it again for knowns that were added // after addresses were pushed. assert(m_addr_known); - if (_addr.IsValid() && !m_addr_known->contains(_addr.GetKey())) { + if (_addr.IsValid() && !m_addr_known->contains(_addr.GetKey()) && addr_format_supported) { if (vAddrToSend.size() >= MAX_ADDR_TO_SEND) { vAddrToSend[insecure_rand.randrange(vAddrToSend.size())] = _addr; } else { diff --git a/src/net_processing.cpp b/src/net_processing.cpp index 490ae8db82..951b8a1811 100644 --- a/src/net_processing.cpp +++ b/src/net_processing.cpp @@ -23,6 +23,7 @@ #include <random.h> #include <reverse_iterator.h> #include <scheduler.h> +#include <streams.h> #include <tinyformat.h> #include <txmempool.h> #include <util/check.h> // For NDEBUG compile time check @@ -2435,11 +2436,16 @@ void PeerManager::ProcessMessage(CNode& pfrom, const std::string& msg_type, CDat pfrom.SetCommonVersion(greatest_common_version); pfrom.nVersion = nVersion; + const CNetMsgMaker msg_maker(greatest_common_version); + if (greatest_common_version >= WTXID_RELAY_VERSION) { - m_connman.PushMessage(&pfrom, CNetMsgMaker(greatest_common_version).Make(NetMsgType::WTXIDRELAY)); + m_connman.PushMessage(&pfrom, msg_maker.Make(NetMsgType::WTXIDRELAY)); } - m_connman.PushMessage(&pfrom, CNetMsgMaker(greatest_common_version).Make(NetMsgType::VERACK)); + m_connman.PushMessage(&pfrom, msg_maker.Make(NetMsgType::VERACK)); + + // Signal ADDRv2 support (BIP155). + m_connman.PushMessage(&pfrom, msg_maker.Make(NetMsgType::SENDADDRV2)); pfrom.nServices = nServices; pfrom.SetAddrLocal(addrMe); @@ -2608,16 +2614,25 @@ void PeerManager::ProcessMessage(CNode& pfrom, const std::string& msg_type, CDat return; } - if (msg_type == NetMsgType::ADDR) { + if (msg_type == NetMsgType::ADDR || msg_type == NetMsgType::ADDRV2) { + int stream_version = vRecv.GetVersion(); + if (msg_type == NetMsgType::ADDRV2) { + // Add ADDRV2_FORMAT to the version so that the CNetAddr and CAddress + // unserialize methods know that an address in v2 format is coming. + stream_version |= ADDRV2_FORMAT; + } + + OverrideStream<CDataStream> s(&vRecv, vRecv.GetType(), stream_version); std::vector<CAddress> vAddr; - vRecv >> vAddr; + + s >> vAddr; if (!pfrom.RelayAddrsWithConn()) { return; } if (vAddr.size() > MAX_ADDR_TO_SEND) { - Misbehaving(pfrom.GetId(), 20, strprintf("addr message size = %u", vAddr.size())); + Misbehaving(pfrom.GetId(), 20, strprintf("%s message size = %u", msg_type, vAddr.size())); return; } @@ -2661,6 +2676,11 @@ void PeerManager::ProcessMessage(CNode& pfrom, const std::string& msg_type, CDat return; } + if (msg_type == NetMsgType::SENDADDRV2) { + pfrom.m_wants_addrv2 = true; + return; + } + if (msg_type == NetMsgType::SENDHEADERS) { LOCK(cs_main); State(pfrom.GetId())->fPreferHeaders = true; @@ -4117,6 +4137,17 @@ bool PeerManager::SendMessages(CNode* pto) std::vector<CAddress> vAddr; vAddr.reserve(pto->vAddrToSend.size()); assert(pto->m_addr_known); + + const char* msg_type; + int make_flags; + if (pto->m_wants_addrv2) { + msg_type = NetMsgType::ADDRV2; + make_flags = ADDRV2_FORMAT; + } else { + msg_type = NetMsgType::ADDR; + make_flags = 0; + } + for (const CAddress& addr : pto->vAddrToSend) { if (!pto->m_addr_known->contains(addr.GetKey())) @@ -4126,14 +4157,14 @@ bool PeerManager::SendMessages(CNode* pto) // receiver rejects addr messages larger than MAX_ADDR_TO_SEND if (vAddr.size() >= MAX_ADDR_TO_SEND) { - m_connman.PushMessage(pto, msgMaker.Make(NetMsgType::ADDR, vAddr)); + m_connman.PushMessage(pto, msgMaker.Make(make_flags, msg_type, vAddr)); vAddr.clear(); } } } pto->vAddrToSend.clear(); if (!vAddr.empty()) - m_connman.PushMessage(pto, msgMaker.Make(NetMsgType::ADDR, vAddr)); + m_connman.PushMessage(pto, msgMaker.Make(make_flags, msg_type, vAddr)); // we only send the big addr message once if (pto->vAddrToSend.capacity() > 40) pto->vAddrToSend.shrink_to_fit(); diff --git a/src/netaddress.cpp b/src/netaddress.cpp index 0c4c0a339b..147c775e61 100644 --- a/src/netaddress.cpp +++ b/src/netaddress.cpp @@ -474,6 +474,26 @@ bool CNetAddr::IsInternal() const return m_net == NET_INTERNAL; } +bool CNetAddr::IsAddrV1Compatible() const +{ + switch (m_net) { + case NET_IPV4: + case NET_IPV6: + case NET_INTERNAL: + return true; + case NET_ONION: + return m_addr.size() == ADDR_TORV2_SIZE; + case NET_I2P: + case NET_CJDNS: + return false; + case NET_UNROUTABLE: // m_net is never and should not be set to NET_UNROUTABLE + case NET_MAX: // m_net is never and should not be set to NET_MAX + assert(false); + } // no default case, so the compiler can warn about missing cases + + assert(false); +} + enum Network CNetAddr::GetNetwork() const { if (IsInternal()) @@ -744,9 +764,12 @@ std::vector<unsigned char> CNetAddr::GetGroup(const std::vector<bool> &asmap) co std::vector<unsigned char> CNetAddr::GetAddrBytes() const { - uint8_t serialized[V1_SERIALIZATION_SIZE]; - SerializeV1Array(serialized); - return {std::begin(serialized), std::end(serialized)}; + if (IsAddrV1Compatible()) { + uint8_t serialized[V1_SERIALIZATION_SIZE]; + SerializeV1Array(serialized); + return {std::begin(serialized), std::end(serialized)}; + } + return std::vector<unsigned char>(m_addr.begin(), m_addr.end()); } uint64_t CNetAddr::GetHash() const diff --git a/src/netaddress.h b/src/netaddress.h index 78e7e1b4b3..3bcf6d35ca 100644 --- a/src/netaddress.h +++ b/src/netaddress.h @@ -177,6 +177,12 @@ class CNetAddr bool IsRoutable() const; bool IsInternal() const; bool IsValid() const; + + /** + * Check if the current object can be serialized in pre-ADDRv2/BIP155 format. + */ + bool IsAddrV1Compatible() const; + enum Network GetNetwork() const; std::string ToString() const; std::string ToStringIP() const; diff --git a/src/protocol.cpp b/src/protocol.cpp index 84b6e96aee..dc8f795a0c 100644 --- a/src/protocol.cpp +++ b/src/protocol.cpp @@ -14,6 +14,8 @@ namespace NetMsgType { const char *VERSION="version"; const char *VERACK="verack"; const char *ADDR="addr"; +const char *ADDRV2="addrv2"; +const char *SENDADDRV2="sendaddrv2"; const char *INV="inv"; const char *GETDATA="getdata"; const char *MERKLEBLOCK="merkleblock"; @@ -52,6 +54,8 @@ const static std::string allNetMessageTypes[] = { NetMsgType::VERSION, NetMsgType::VERACK, NetMsgType::ADDR, + NetMsgType::ADDRV2, + NetMsgType::SENDADDRV2, NetMsgType::INV, NetMsgType::GETDATA, NetMsgType::MERKLEBLOCK, diff --git a/src/protocol.h b/src/protocol.h index 9a44a1626c..309fac621c 100644 --- a/src/protocol.h +++ b/src/protocol.h @@ -77,6 +77,18 @@ extern const char* VERACK; */ extern const char* ADDR; /** + * The addrv2 message relays connection information for peers on the network just + * like the addr message, but is extended to allow gossiping of longer node + * addresses (see BIP155). + */ +extern const char *ADDRV2; +/** + * The sendaddrv2 message signals support for receiving ADDRV2 messages (BIP155). + * It also implies that its sender can encode as ADDRV2 and would send ADDRV2 + * instead of ADDR to a peer that has signaled ADDRV2 support by sending SENDADDRV2. + */ +extern const char *SENDADDRV2; +/** * The inv message (inventory message) transmits one or more inventories of * objects known to the transmitting peer. */ @@ -351,7 +363,8 @@ class CAddress : public CService public: CAddress() : CService{} {}; - explicit CAddress(CService ipIn, ServiceFlags nServicesIn) : CService{ipIn}, nServices{nServicesIn} {}; + CAddress(CService ipIn, ServiceFlags nServicesIn) : CService{ipIn}, nServices{nServicesIn} {}; + CAddress(CService ipIn, ServiceFlags nServicesIn, uint32_t nTimeIn) : CService{ipIn}, nTime{nTimeIn}, nServices{nServicesIn} {}; SERIALIZE_METHODS(CAddress, obj) { @@ -370,7 +383,14 @@ public: // nTime. READWRITE(obj.nTime); } - READWRITE(Using<CustomUintFormatter<8>>(obj.nServices)); + if (nVersion & ADDRV2_FORMAT) { + uint64_t services_tmp; + SER_WRITE(obj, services_tmp = obj.nServices); + READWRITE(Using<CompactSizeFormatter<false>>(services_tmp)); + SER_READ(obj, obj.nServices = static_cast<ServiceFlags>(services_tmp)); + } else { + READWRITE(Using<CustomUintFormatter<8>>(obj.nServices)); + } READWRITEAS(CService, obj); } diff --git a/src/serialize.h b/src/serialize.h index 7a94e704b2..d9ca984f9c 100644 --- a/src/serialize.h +++ b/src/serialize.h @@ -24,7 +24,11 @@ #include <prevector.h> #include <span.h> -static const unsigned int MAX_SIZE = 0x02000000; +/** + * The maximum size of a serialized object in bytes or number of elements + * (for eg vectors) when the size is encoded as CompactSize. + */ +static constexpr uint64_t MAX_SIZE = 0x02000000; /** Maximum amount of memory (in bytes) to allocate at once when deserializing vectors. */ static const unsigned int MAX_VECTOR_ALLOCATE = 5000000; @@ -304,8 +308,14 @@ void WriteCompactSize(Stream& os, uint64_t nSize) return; } +/** + * Decode a CompactSize-encoded variable-length integer. + * + * As these are primarily used to encode the size of vector-like serializations, by default a range + * check is performed. When used as a generic number encoding, range_check should be set to false. + */ template<typename Stream> -uint64_t ReadCompactSize(Stream& is) +uint64_t ReadCompactSize(Stream& is, bool range_check = true) { uint8_t chSize = ser_readdata8(is); uint64_t nSizeRet = 0; @@ -331,8 +341,9 @@ uint64_t ReadCompactSize(Stream& is) if (nSizeRet < 0x100000000ULL) throw std::ios_base::failure("non-canonical ReadCompactSize()"); } - if (nSizeRet > (uint64_t)MAX_SIZE) + if (range_check && nSizeRet > MAX_SIZE) { throw std::ios_base::failure("ReadCompactSize(): size too large"); + } return nSizeRet; } @@ -466,7 +477,7 @@ static inline Wrapper<Formatter, T&> Using(T&& t) { return Wrapper<Formatter, T& #define VARINT_MODE(obj, mode) Using<VarIntFormatter<mode>>(obj) #define VARINT(obj) Using<VarIntFormatter<VarIntMode::DEFAULT>>(obj) -#define COMPACTSIZE(obj) Using<CompactSizeFormatter>(obj) +#define COMPACTSIZE(obj) Using<CompactSizeFormatter<true>>(obj) #define LIMITED_STRING(obj,n) Using<LimitedStringFormatter<n>>(obj) /** Serialization wrapper class for integers in VarInt format. */ @@ -529,12 +540,13 @@ struct CustomUintFormatter template<int Bytes> using BigEndianFormatter = CustomUintFormatter<Bytes, true>; /** Formatter for integers in CompactSize format. */ +template<bool RangeCheck> struct CompactSizeFormatter { template<typename Stream, typename I> void Unser(Stream& s, I& v) { - uint64_t n = ReadCompactSize<Stream>(s); + uint64_t n = ReadCompactSize<Stream>(s, RangeCheck); if (n < std::numeric_limits<I>::min() || n > std::numeric_limits<I>::max()) { throw std::ios_base::failure("CompactSize exceeds limit of type"); } diff --git a/src/streams.h b/src/streams.h index 6ce8065da8..c22f5936fd 100644 --- a/src/streams.h +++ b/src/streams.h @@ -60,6 +60,7 @@ public: int GetVersion() const { return nVersion; } int GetType() const { return nType; } size_t size() const { return stream->size(); } + void ignore(size_t size) { return stream->ignore(size); } }; /* Minimal stream for overwriting and/or appending to an existing byte vector diff --git a/src/test/net_tests.cpp b/src/test/net_tests.cpp index 92792569a7..121a16e548 100644 --- a/src/test/net_tests.cpp +++ b/src/test/net_tests.cpp @@ -212,6 +212,7 @@ BOOST_AUTO_TEST_CASE(cnetaddr_basic) BOOST_REQUIRE(addr.IsIPv4()); BOOST_CHECK(addr.IsBindAny()); + BOOST_CHECK(addr.IsAddrV1Compatible()); BOOST_CHECK_EQUAL(addr.ToString(), "0.0.0.0"); // IPv4, INADDR_NONE @@ -220,6 +221,7 @@ BOOST_AUTO_TEST_CASE(cnetaddr_basic) BOOST_REQUIRE(addr.IsIPv4()); BOOST_CHECK(!addr.IsBindAny()); + BOOST_CHECK(addr.IsAddrV1Compatible()); BOOST_CHECK_EQUAL(addr.ToString(), "255.255.255.255"); // IPv4, casual @@ -228,6 +230,7 @@ BOOST_AUTO_TEST_CASE(cnetaddr_basic) BOOST_REQUIRE(addr.IsIPv4()); BOOST_CHECK(!addr.IsBindAny()); + BOOST_CHECK(addr.IsAddrV1Compatible()); BOOST_CHECK_EQUAL(addr.ToString(), "12.34.56.78"); // IPv6, in6addr_any @@ -236,6 +239,7 @@ BOOST_AUTO_TEST_CASE(cnetaddr_basic) BOOST_REQUIRE(addr.IsIPv6()); BOOST_CHECK(addr.IsBindAny()); + BOOST_CHECK(addr.IsAddrV1Compatible()); BOOST_CHECK_EQUAL(addr.ToString(), "::"); // IPv6, casual @@ -244,6 +248,7 @@ BOOST_AUTO_TEST_CASE(cnetaddr_basic) BOOST_REQUIRE(addr.IsIPv6()); BOOST_CHECK(!addr.IsBindAny()); + BOOST_CHECK(addr.IsAddrV1Compatible()); BOOST_CHECK_EQUAL(addr.ToString(), "1122:3344:5566:7788:9900:aabb:ccdd:eeff"); // IPv6, scoped/link-local. See https://tools.ietf.org/html/rfc4007 @@ -271,6 +276,7 @@ BOOST_AUTO_TEST_CASE(cnetaddr_basic) BOOST_REQUIRE(addr.IsTor()); BOOST_CHECK(!addr.IsBindAny()); + BOOST_CHECK(addr.IsAddrV1Compatible()); BOOST_CHECK_EQUAL(addr.ToString(), "6hzph5hv6337r6p2.onion"); // TORv3 @@ -280,6 +286,7 @@ BOOST_AUTO_TEST_CASE(cnetaddr_basic) BOOST_REQUIRE(addr.IsTor()); BOOST_CHECK(!addr.IsBindAny()); + BOOST_CHECK(!addr.IsAddrV1Compatible()); BOOST_CHECK_EQUAL(addr.ToString(), torv3_addr); // TORv3, broken, with wrong checksum @@ -304,6 +311,7 @@ BOOST_AUTO_TEST_CASE(cnetaddr_basic) BOOST_REQUIRE(addr.IsInternal()); BOOST_CHECK(!addr.IsBindAny()); + BOOST_CHECK(addr.IsAddrV1Compatible()); BOOST_CHECK_EQUAL(addr.ToString(), "esffpvrt3wpeaygy.internal"); // Totally bogus @@ -398,6 +406,7 @@ BOOST_AUTO_TEST_CASE(cnetaddr_unserialize_v2) s >> addr; BOOST_CHECK(addr.IsValid()); BOOST_CHECK(addr.IsIPv4()); + BOOST_CHECK(addr.IsAddrV1Compatible()); BOOST_CHECK_EQUAL(addr.ToString(), "1.2.3.4"); BOOST_REQUIRE(s.empty()); @@ -434,6 +443,7 @@ BOOST_AUTO_TEST_CASE(cnetaddr_unserialize_v2) s >> addr; BOOST_CHECK(addr.IsValid()); BOOST_CHECK(addr.IsIPv6()); + BOOST_CHECK(addr.IsAddrV1Compatible()); BOOST_CHECK_EQUAL(addr.ToString(), "102:304:506:708:90a:b0c:d0e:f10"); BOOST_REQUIRE(s.empty()); @@ -445,6 +455,7 @@ BOOST_AUTO_TEST_CASE(cnetaddr_unserialize_v2) // sha256(name)[0:10] s >> addr; BOOST_CHECK(addr.IsInternal()); + BOOST_CHECK(addr.IsAddrV1Compatible()); BOOST_CHECK_EQUAL(addr.ToString(), "zklycewkdo64v6wc.internal"); BOOST_REQUIRE(s.empty()); @@ -480,6 +491,7 @@ BOOST_AUTO_TEST_CASE(cnetaddr_unserialize_v2) s >> addr; BOOST_CHECK(addr.IsValid()); BOOST_CHECK(addr.IsTor()); + BOOST_CHECK(addr.IsAddrV1Compatible()); BOOST_CHECK_EQUAL(addr.ToString(), "6hzph5hv6337r6p2.onion"); BOOST_REQUIRE(s.empty()); @@ -501,6 +513,7 @@ BOOST_AUTO_TEST_CASE(cnetaddr_unserialize_v2) s >> addr; BOOST_CHECK(addr.IsValid()); BOOST_CHECK(addr.IsTor()); + BOOST_CHECK(!addr.IsAddrV1Compatible()); BOOST_CHECK_EQUAL(addr.ToString(), "pg6mmjiyjmcrsslvykfwnntlaru7p5svn6y2ymmju6nubxndf4pscryd.onion"); BOOST_REQUIRE(s.empty()); @@ -522,6 +535,8 @@ BOOST_AUTO_TEST_CASE(cnetaddr_unserialize_v2) "f98232ae42d4b6fd2fa81952dfe36a87")); s >> addr; BOOST_CHECK(addr.IsValid()); + BOOST_CHECK(addr.IsI2P()); + BOOST_CHECK(!addr.IsAddrV1Compatible()); BOOST_CHECK_EQUAL(addr.ToString(), "ukeu3k5oycgaauneqgtnvselmt4yemvoilkln7jpvamvfx7dnkdq.b32.i2p"); BOOST_REQUIRE(s.empty()); @@ -543,6 +558,8 @@ BOOST_AUTO_TEST_CASE(cnetaddr_unserialize_v2) )); s >> addr; BOOST_CHECK(addr.IsValid()); + BOOST_CHECK(addr.IsCJDNS()); + BOOST_CHECK(!addr.IsAddrV1Compatible()); BOOST_CHECK_EQUAL(addr.ToString(), "fc00:1:2:3:4:5:6:7"); BOOST_REQUIRE(s.empty()); diff --git a/src/test/netbase_tests.cpp b/src/test/netbase_tests.cpp index 6681c92bb5..eb0d95a373 100644 --- a/src/test/netbase_tests.cpp +++ b/src/test/netbase_tests.cpp @@ -4,9 +4,13 @@ #include <net_permissions.h> #include <netbase.h> +#include <protocol.h> +#include <serialize.h> +#include <streams.h> #include <test/util/setup_common.h> #include <util/strencodings.h> #include <util/translation.h> +#include <version.h> #include <string> @@ -443,4 +447,105 @@ BOOST_AUTO_TEST_CASE(netbase_dont_resolve_strings_with_embedded_nul_characters) BOOST_CHECK(!LookupSubNet(std::string("5wyqrzbvrdsumnok.onion\0example.com\0", 35), ret)); } +// Since CNetAddr (un)ser is tested separately in net_tests.cpp here we only +// try a few edge cases for port, service flags and time. + +static const std::vector<CAddress> fixture_addresses({ + CAddress( + CService(CNetAddr(in6addr_loopback), 0 /* port */), + NODE_NONE, + 0x4966bc61U /* Fri Jan 9 02:54:25 UTC 2009 */ + ), + CAddress( + CService(CNetAddr(in6addr_loopback), 0x00f1 /* port */), + NODE_NETWORK, + 0x83766279U /* Tue Nov 22 11:22:33 UTC 2039 */ + ), + CAddress( + CService(CNetAddr(in6addr_loopback), 0xf1f2 /* port */), + static_cast<ServiceFlags>(NODE_WITNESS | NODE_COMPACT_FILTERS | NODE_NETWORK_LIMITED), + 0xffffffffU /* Sun Feb 7 06:28:15 UTC 2106 */ + ) +}); + +// fixture_addresses should equal to this when serialized in V1 format. +// When this is unserialized from V1 format it should equal to fixture_addresses. +static constexpr const char* stream_addrv1_hex = + "03" // number of entries + + "61bc6649" // time, Fri Jan 9 02:54:25 UTC 2009 + "0000000000000000" // service flags, NODE_NONE + "00000000000000000000000000000001" // address, fixed 16 bytes (IPv4 embedded in IPv6) + "0000" // port + + "79627683" // time, Tue Nov 22 11:22:33 UTC 2039 + "0100000000000000" // service flags, NODE_NETWORK + "00000000000000000000000000000001" // address, fixed 16 bytes (IPv6) + "00f1" // port + + "ffffffff" // time, Sun Feb 7 06:28:15 UTC 2106 + "4804000000000000" // service flags, NODE_WITNESS | NODE_COMPACT_FILTERS | NODE_NETWORK_LIMITED + "00000000000000000000000000000001" // address, fixed 16 bytes (IPv6) + "f1f2"; // port + +// fixture_addresses should equal to this when serialized in V2 format. +// When this is unserialized from V2 format it should equal to fixture_addresses. +static constexpr const char* stream_addrv2_hex = + "03" // number of entries + + "61bc6649" // time, Fri Jan 9 02:54:25 UTC 2009 + "00" // service flags, COMPACTSIZE(NODE_NONE) + "02" // network id, IPv6 + "10" // address length, COMPACTSIZE(16) + "00000000000000000000000000000001" // address + "0000" // port + + "79627683" // time, Tue Nov 22 11:22:33 UTC 2039 + "01" // service flags, COMPACTSIZE(NODE_NETWORK) + "02" // network id, IPv6 + "10" // address length, COMPACTSIZE(16) + "00000000000000000000000000000001" // address + "00f1" // port + + "ffffffff" // time, Sun Feb 7 06:28:15 UTC 2106 + "fd4804" // service flags, COMPACTSIZE(NODE_WITNESS | NODE_COMPACT_FILTERS | NODE_NETWORK_LIMITED) + "02" // network id, IPv6 + "10" // address length, COMPACTSIZE(16) + "00000000000000000000000000000001" // address + "f1f2"; // port + +BOOST_AUTO_TEST_CASE(caddress_serialize_v1) +{ + CDataStream s(SER_NETWORK, PROTOCOL_VERSION); + + s << fixture_addresses; + BOOST_CHECK_EQUAL(HexStr(s), stream_addrv1_hex); +} + +BOOST_AUTO_TEST_CASE(caddress_unserialize_v1) +{ + CDataStream s(ParseHex(stream_addrv1_hex), SER_NETWORK, PROTOCOL_VERSION); + std::vector<CAddress> addresses_unserialized; + + s >> addresses_unserialized; + BOOST_CHECK(fixture_addresses == addresses_unserialized); +} + +BOOST_AUTO_TEST_CASE(caddress_serialize_v2) +{ + CDataStream s(SER_NETWORK, PROTOCOL_VERSION | ADDRV2_FORMAT); + + s << fixture_addresses; + BOOST_CHECK_EQUAL(HexStr(s), stream_addrv2_hex); +} + +BOOST_AUTO_TEST_CASE(caddress_unserialize_v2) +{ + CDataStream s(ParseHex(stream_addrv2_hex), SER_NETWORK, PROTOCOL_VERSION | ADDRV2_FORMAT); + std::vector<CAddress> addresses_unserialized; + + s >> addresses_unserialized; + BOOST_CHECK(fixture_addresses == addresses_unserialized); +} + BOOST_AUTO_TEST_SUITE_END() diff --git a/src/torcontrol.cpp b/src/torcontrol.cpp index 666e7a37a5..8ebe3d750d 100644 --- a/src/torcontrol.cpp +++ b/src/torcontrol.cpp @@ -537,8 +537,9 @@ void TorController::auth_cb(TorControlConnection& _conn, const TorControlReply& } // Finally - now create the service - if (private_key.empty()) // No private key, generate one - private_key = "NEW:RSA1024"; // Explicitly request RSA1024 - see issue #9214 + if (private_key.empty()) { // No private key, generate one + private_key = "NEW:ED25519-V3"; // Explicitly request key type - see issue #9214 + } // Request onion service, redirect port. // Note that the 'virtual' port is always the default port to avoid decloaking nodes using other ports. _conn.Command(strprintf("ADD_ONION %s Port=%i,%s", private_key, Params().GetDefaultPort(), m_target.ToStringIPPort()), @@ -723,7 +724,7 @@ void TorController::Reconnect() fs::path TorController::GetPrivateKeyFile() { - return GetDataDir() / "onion_private_key"; + return GetDataDir() / "onion_v3_private_key"; } void TorController::reconnect_cb(evutil_socket_t fd, short what, void *arg) diff --git a/test/functional/p2p_addrv2_relay.py b/test/functional/p2p_addrv2_relay.py new file mode 100755 index 0000000000..23ce3e5d04 --- /dev/null +++ b/test/functional/p2p_addrv2_relay.py @@ -0,0 +1,79 @@ +#!/usr/bin/env python3 +# Copyright (c) 2020 The Bitcoin Core developers +# Distributed under the MIT software license, see the accompanying +# file COPYING or http://www.opensource.org/licenses/mit-license.php. +""" +Test addrv2 relay +""" + +import time + +from test_framework.messages import ( + CAddress, + msg_addrv2, + NODE_NETWORK, + NODE_WITNESS, +) +from test_framework.p2p import P2PInterface +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import assert_equal + +ADDRS = [] +for i in range(10): + addr = CAddress() + addr.time = int(time.time()) + i + addr.nServices = NODE_NETWORK | NODE_WITNESS + addr.ip = "123.123.123.{}".format(i % 256) + addr.port = 8333 + i + ADDRS.append(addr) + + +class AddrReceiver(P2PInterface): + addrv2_received_and_checked = False + + def __init__(self): + super().__init__(support_addrv2 = True) + + def on_addrv2(self, message): + for addr in message.addrs: + assert_equal(addr.nServices, 9) + assert addr.ip.startswith('123.123.123.') + assert (8333 <= addr.port < 8343) + self.addrv2_received_and_checked = True + + def wait_for_addrv2(self): + self.wait_until(lambda: "addrv2" in self.last_message) + + +class AddrTest(BitcoinTestFramework): + def set_test_params(self): + self.setup_clean_chain = True + self.num_nodes = 1 + + def run_test(self): + self.log.info('Create connection that sends addrv2 messages') + addr_source = self.nodes[0].add_p2p_connection(P2PInterface()) + msg = msg_addrv2() + + self.log.info('Send too-large addrv2 message') + msg.addrs = ADDRS * 101 + with self.nodes[0].assert_debug_log(['addrv2 message size = 1010']): + addr_source.send_and_ping(msg) + + self.log.info('Check that addrv2 message content is relayed and added to addrman') + addr_receiver = self.nodes[0].add_p2p_connection(AddrReceiver()) + msg.addrs = ADDRS + with self.nodes[0].assert_debug_log([ + 'Added 10 addresses from 127.0.0.1: 0 tried', + 'received: addrv2 (131 bytes) peer=0', + 'sending addrv2 (131 bytes) peer=1', + ]): + addr_source.send_and_ping(msg) + self.nodes[0].setmocktime(int(time.time()) + 30 * 60) + addr_receiver.wait_for_addrv2() + + assert addr_receiver.addrv2_received_and_checked + + +if __name__ == '__main__': + AddrTest().main() diff --git a/test/functional/p2p_invalid_messages.py b/test/functional/p2p_invalid_messages.py index fbe58c5e2f..db72a361d9 100755 --- a/test/functional/p2p_invalid_messages.py +++ b/test/functional/p2p_invalid_messages.py @@ -4,6 +4,9 @@ # file COPYING or http://www.opensource.org/licenses/mit-license.php. """Test node responses to invalid network messages.""" +import struct +import time + from test_framework.messages import ( CBlockHeader, CInv, @@ -22,7 +25,10 @@ from test_framework.p2p import ( P2PInterface, ) from test_framework.test_framework import BitcoinTestFramework -from test_framework.util import assert_equal +from test_framework.util import ( + assert_equal, + hex_str_to_bytes, +) VALID_DATA_LIMIT = MAX_PROTOCOL_MESSAGE_LENGTH - 5 # Account for the 5-byte length prefix @@ -42,6 +48,11 @@ class msg_unrecognized: return "{}(data={})".format(self.msgtype, self.str_data) +class SenderOfAddrV2(P2PInterface): + def wait_for_sendaddrv2(self): + self.wait_until(lambda: 'sendaddrv2' in self.last_message) + + class InvalidMessagesTest(BitcoinTestFramework): def set_test_params(self): self.num_nodes = 1 @@ -53,6 +64,10 @@ class InvalidMessagesTest(BitcoinTestFramework): self.test_checksum() self.test_size() self.test_msgtype() + self.test_addrv2_empty() + self.test_addrv2_no_addresses() + self.test_addrv2_too_long_address() + self.test_addrv2_unrecognized_network() self.test_oversized_inv_msg() self.test_oversized_getdata_msg() self.test_oversized_headers_msg() @@ -127,6 +142,84 @@ class InvalidMessagesTest(BitcoinTestFramework): assert_equal(self.nodes[0].getpeerinfo()[0]['bytesrecv_per_msg']['*other*'], 26) self.nodes[0].disconnect_p2ps() + def test_addrv2(self, label, required_log_messages, raw_addrv2): + node = self.nodes[0] + conn = node.add_p2p_connection(SenderOfAddrV2()) + + # Make sure bitcoind signals support for ADDRv2, otherwise this test + # will bombard an old node with messages it does not recognize which + # will produce unexpected results. + conn.wait_for_sendaddrv2() + + self.log.info('Test addrv2: ' + label) + + msg = msg_unrecognized(str_data=b'') + msg.msgtype = b'addrv2' + with node.assert_debug_log(required_log_messages): + # override serialize() which would include the length of the data + msg.serialize = lambda: raw_addrv2 + conn.send_raw_message(conn.build_message(msg)) + conn.sync_with_ping() + + node.disconnect_p2ps() + + def test_addrv2_empty(self): + self.test_addrv2('empty', + [ + 'received: addrv2 (0 bytes)', + 'ProcessMessages(addrv2, 0 bytes): Exception', + 'end of data', + ], + b'') + + def test_addrv2_no_addresses(self): + self.test_addrv2('no addresses', + [ + 'received: addrv2 (1 bytes)', + ], + hex_str_to_bytes('00')) + + def test_addrv2_too_long_address(self): + self.test_addrv2('too long address', + [ + 'received: addrv2 (525 bytes)', + 'ProcessMessages(addrv2, 525 bytes): Exception', + 'Address too long: 513 > 512', + ], + hex_str_to_bytes( + '01' + # number of entries + '61bc6649' + # time, Fri Jan 9 02:54:25 UTC 2009 + '00' + # service flags, COMPACTSIZE(NODE_NONE) + '01' + # network type (IPv4) + 'fd0102' + # address length (COMPACTSIZE(513)) + 'ab' * 513 + # address + '208d')) # port + + def test_addrv2_unrecognized_network(self): + now_hex = struct.pack('<I', int(time.time())).hex() + self.test_addrv2('unrecognized network', + [ + 'received: addrv2 (25 bytes)', + 'IP 9.9.9.9 mapped', + 'Added 1 addresses', + ], + hex_str_to_bytes( + '02' + # number of entries + # this should be ignored without impeding acceptance of subsequent ones + now_hex + # time + '01' + # service flags, COMPACTSIZE(NODE_NETWORK) + '99' + # network type (unrecognized) + '02' + # address length (COMPACTSIZE(2)) + 'ab' * 2 + # address + '208d' + # port + # this should be added: + now_hex + # time + '01' + # service flags, COMPACTSIZE(NODE_NETWORK) + '01' + # network type (IPv4) + '04' + # address length (COMPACTSIZE(4)) + '09' * 4 + # address + '208d')) # port + def test_oversized_msg(self, msg, size): msg_type = msg.msgtype.decode('ascii') self.log.info("Test {} message of size {} is logged as misbehaving".format(msg_type, size)) diff --git a/test/functional/test_framework/messages.py b/test/functional/test_framework/messages.py index 00cf1ef66d..ff7f73bdf4 100755 --- a/test/functional/test_framework/messages.py +++ b/test/functional/test_framework/messages.py @@ -136,12 +136,17 @@ def uint256_from_compact(c): return v -def deser_vector(f, c): +# deser_function_name: Allow for an alternate deserialization function on the +# entries in the vector. +def deser_vector(f, c, deser_function_name=None): nit = deser_compact_size(f) r = [] for _ in range(nit): t = c() - t.deserialize(f) + if deser_function_name: + getattr(t, deser_function_name)(f) + else: + t.deserialize(f) r.append(t) return r @@ -204,38 +209,82 @@ def ToHex(obj): class CAddress: - __slots__ = ("ip", "nServices", "pchReserved", "port", "time") + __slots__ = ("net", "ip", "nServices", "port", "time") + + # see https://github.com/bitcoin/bips/blob/master/bip-0155.mediawiki + NET_IPV4 = 1 + + ADDRV2_NET_NAME = { + NET_IPV4: "IPv4" + } + + ADDRV2_ADDRESS_LENGTH = { + NET_IPV4: 4 + } def __init__(self): self.time = 0 self.nServices = 1 - self.pchReserved = b"\x00" * 10 + b"\xff" * 2 + self.net = self.NET_IPV4 self.ip = "0.0.0.0" self.port = 0 def deserialize(self, f, *, with_time=True): + """Deserialize from addrv1 format (pre-BIP155)""" if with_time: # VERSION messages serialize CAddress objects without time - self.time = struct.unpack("<i", f.read(4))[0] + self.time = struct.unpack("<I", f.read(4))[0] self.nServices = struct.unpack("<Q", f.read(8))[0] - self.pchReserved = f.read(12) + # We only support IPv4 which means skip 12 bytes and read the next 4 as IPv4 address. + f.read(12) + self.net = self.NET_IPV4 self.ip = socket.inet_ntoa(f.read(4)) self.port = struct.unpack(">H", f.read(2))[0] def serialize(self, *, with_time=True): + """Serialize in addrv1 format (pre-BIP155)""" + assert self.net == self.NET_IPV4 r = b"" if with_time: # VERSION messages serialize CAddress objects without time - r += struct.pack("<i", self.time) + r += struct.pack("<I", self.time) r += struct.pack("<Q", self.nServices) - r += self.pchReserved + r += b"\x00" * 10 + b"\xff" * 2 + r += socket.inet_aton(self.ip) + r += struct.pack(">H", self.port) + return r + + def deserialize_v2(self, f): + """Deserialize from addrv2 format (BIP155)""" + self.time = struct.unpack("<I", f.read(4))[0] + + self.nServices = deser_compact_size(f) + + self.net = struct.unpack("B", f.read(1))[0] + assert self.net == self.NET_IPV4 + + address_length = deser_compact_size(f) + assert address_length == self.ADDRV2_ADDRESS_LENGTH[self.net] + + self.ip = socket.inet_ntoa(f.read(4)) + + self.port = struct.unpack(">H", f.read(2))[0] + + def serialize_v2(self): + """Serialize in addrv2 format (BIP155)""" + assert self.net == self.NET_IPV4 + r = b"" + r += struct.pack("<I", self.time) + r += ser_compact_size(self.nServices) + r += struct.pack("B", self.net) + r += ser_compact_size(self.ADDRV2_ADDRESS_LENGTH[self.net]) r += socket.inet_aton(self.ip) r += struct.pack(">H", self.port) return r def __repr__(self): - return "CAddress(nServices=%i ip=%s port=%i)" % (self.nServices, - self.ip, self.port) + return ("CAddress(nServices=%i net=%s addr=%s port=%i)" + % (self.nServices, self.ADDRV2_NET_NAME[self.net], self.ip, self.port)) class CInv: @@ -1064,6 +1113,40 @@ class msg_addr: return "msg_addr(addrs=%s)" % (repr(self.addrs)) +class msg_addrv2: + __slots__ = ("addrs",) + msgtype = b"addrv2" + + def __init__(self): + self.addrs = [] + + def deserialize(self, f): + self.addrs = deser_vector(f, CAddress, "deserialize_v2") + + def serialize(self): + return ser_vector(self.addrs, "serialize_v2") + + def __repr__(self): + return "msg_addrv2(addrs=%s)" % (repr(self.addrs)) + + +class msg_sendaddrv2: + __slots__ = () + msgtype = b"sendaddrv2" + + def __init__(self): + pass + + def deserialize(self, f): + pass + + def serialize(self): + return b"" + + def __repr__(self): + return "msg_sendaddrv2()" + + class msg_inv: __slots__ = ("inv",) msgtype = b"inv" diff --git a/test/functional/test_framework/p2p.py b/test/functional/test_framework/p2p.py index 5f9b316b18..6846d31221 100755 --- a/test/functional/test_framework/p2p.py +++ b/test/functional/test_framework/p2p.py @@ -33,6 +33,7 @@ from test_framework.messages import ( MAX_HEADERS_RESULTS, MIN_VERSION_SUPPORTED, msg_addr, + msg_addrv2, msg_block, MSG_BLOCK, msg_blocktxn, @@ -56,6 +57,7 @@ from test_framework.messages import ( msg_notfound, msg_ping, msg_pong, + msg_sendaddrv2, msg_sendcmpct, msg_sendheaders, msg_tx, @@ -75,6 +77,7 @@ logger = logging.getLogger("TestFramework.p2p") MESSAGEMAP = { b"addr": msg_addr, + b"addrv2": msg_addrv2, b"block": msg_block, b"blocktxn": msg_blocktxn, b"cfcheckpt": msg_cfcheckpt, @@ -97,6 +100,7 @@ MESSAGEMAP = { b"notfound": msg_notfound, b"ping": msg_ping, b"pong": msg_pong, + b"sendaddrv2": msg_sendaddrv2, b"sendcmpct": msg_sendcmpct, b"sendheaders": msg_sendheaders, b"tx": msg_tx, @@ -285,7 +289,7 @@ class P2PInterface(P2PConnection): Individual testcases should subclass this and override the on_* methods if they want to alter message handling behaviour.""" - def __init__(self): + def __init__(self, support_addrv2=False): super().__init__() # Track number of messages of each type received. @@ -303,6 +307,8 @@ class P2PInterface(P2PConnection): # The network services received from the peer self.nServices = 0 + self.support_addrv2 = support_addrv2 + def peer_connect(self, *args, services=NODE_NETWORK|NODE_WITNESS, send_version=True, **kwargs): create_conn = super().peer_connect(*args, **kwargs) @@ -345,6 +351,7 @@ class P2PInterface(P2PConnection): pass def on_addr(self, message): pass + def on_addrv2(self, message): pass def on_block(self, message): pass def on_blocktxn(self, message): pass def on_cfcheckpt(self, message): pass @@ -365,6 +372,7 @@ class P2PInterface(P2PConnection): def on_merkleblock(self, message): pass def on_notfound(self, message): pass def on_pong(self, message): pass + def on_sendaddrv2(self, message): pass def on_sendcmpct(self, message): pass def on_sendheaders(self, message): pass def on_tx(self, message): pass @@ -389,6 +397,8 @@ class P2PInterface(P2PConnection): if message.nVersion >= 70016: self.send_message(msg_wtxidrelay()) self.send_message(msg_verack()) + if self.support_addrv2: + self.send_message(msg_sendaddrv2()) self.nServices = message.nServices # Connection helper methods diff --git a/test/functional/test_runner.py b/test/functional/test_runner.py index cb86305cc6..2e757d7090 100755 --- a/test/functional/test_runner.py +++ b/test/functional/test_runner.py @@ -155,6 +155,7 @@ BASE_SCRIPTS = [ 'feature_proxy.py', 'rpc_signrawtransaction.py', 'wallet_groups.py', + 'p2p_addrv2_relay.py', 'p2p_disconnect_ban.py', 'rpc_decodescript.py', 'rpc_blockchain.py', |