aboutsummaryrefslogtreecommitdiff
path: root/src/test
AgeCommit message (Collapse)Author
2023-10-27refactor: Add LIFETIMEBOUND to all (w)txid gettersMarcoFalke
Then, use the compiler warnings to create copies only where needed. Also, fix iwyu includes while touching the includes.
2023-10-26Merge bitcoin/bitcoin#26078: p2p: return `CSubNet` in `LookupSubNet`Andrew Chow
fb3e812277041f239b97b88689a5076796d75b9b p2p: return `CSubNet` in `LookupSubNet` (brunoerg) Pull request description: Analyzing the usage of `LookupSubNet`, noticed that most cases uses check if the subnet is valid by calling `subnet.IsValid()`, and the boolean returned by `LookupSubNet` hasn't been used so much, see: https://github.com/bitcoin/bitcoin/blob/29d540b7ada890dd588c4825d40c27c5e6f20061/src/httpserver.cpp#L172-L174 https://github.com/bitcoin/bitcoin/blob/29d540b7ada890dd588c4825d40c27c5e6f20061/src/net_permissions.cpp#L114-L116 It makes sense to return `CSubNet` instead of `bool`. ACKs for top commit: achow101: ACK fb3e812277041f239b97b88689a5076796d75b9b vasild: ACK fb3e812277041f239b97b88689a5076796d75b9b theStack: Code-review ACK fb3e812277041f239b97b88689a5076796d75b9b stickies-v: Concept ACK, but Approach ~0 (for now). Reviewed the code (fb3e812277041f239b97b88689a5076796d75b9b) and it all looks good to me. Tree-SHA512: ba50d6bd5d58dfdbe1ce1faebd80dd8cf8c92ac53ef33519860b83399afffab482d5658cb6921b849d7a3df6d5cea911412850e08f3f4e27f7af510fbde4b254
2023-10-26Merge bitcoin/bitcoin#28107: util: Type-safe transaction identifiersAndrew Chow
940a49978c70453e1aaf2c4a0bcb382872b844a5 Use type-safe txid types in orphanage (dergoegge) ed70e6501648466b9ca91a39b83775363e9a726d Introduce types for txids & wtxids (dergoegge) cdb14d79e809bf7d1612b21b554a9fcfb2ab1c91 [net processing] Use HasWitness over comparing (w)txids (dergoegge) Pull request description: We currently have two different identifiers for transactions: `txid` (refering to the hash of a transaction without witness data) and `wtxid` (referring to the hash of a transaction including witness data). Both are typed as `uint256` which could lead to type-safety bugs in which one transaction identifier type is passed where the other would be expected. This PR introduces explicit `Txid` and `Wtxid` types that (if used) would cause compilation errors for such type confusion bugs. (Only the orphanage is converted to use these types in this PR) ACKs for top commit: achow101: ACK 940a49978c70453e1aaf2c4a0bcb382872b844a5 stickies-v: ACK 940a49978c70453e1aaf2c4a0bcb382872b844a5 hebasto: ACK 940a49978c70453e1aaf2c4a0bcb382872b844a5, I have reviewed the code and it looks OK. instagibbs: re-ACK 940a49978c70453e1aaf2c4a0bcb382872b844a5 BrandonOdiwuor: re-ACK 940a49978c70453e1aaf2c4a0bcb382872b844a5 glozow: reACK 940a49978c Tree-SHA512: 55298d1c2bb82b7a6995e96e554571c22eaf4a89fb2a4d7a236d70e0f625e8cca62ff2490e1c179c47bd93153fe6527b56870198f026f5ee7753d64d7a424c92
2023-10-20coinstats: Fix hash_serialized2 calculationFabian Jahr
The legacy serialization was vulnerable to maleation and is fixed by adopting the same serialization procedure as was already in use for MuHash. This also includes necessary test fixes where the hash_serialized2 was hardcoded as well as correction of the regtest chainparams. Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
2023-10-20Merge bitcoin/bitcoin#28692: fuzz: Delete i2p fuzz testfanquake
dd4dcbd4cda31f67d014a93340a6d1ba1c245b0f [fuzz] Delete i2p target (dergoegge) Pull request description: closes #28665 The target is buggy and doesn't reach basic coverage. ACKs for top commit: maflcko: lgtm ACK dd4dcbd4cda31f67d014a93340a6d1ba1c245b0f glozow: ACK dd4dcbd4cda31f67d014a93340a6d1ba1c245b0f, agree it's better to delete this test until somebody wants to write a better one Tree-SHA512: b6ca6cad1773b1ceb6e5ac0fd501ea615f66507ef811745799deaaa4460f1700d96ae03cf55b740a96ed8cd2283b3d6738cd580ba97f2af619197d6c4414ca21
2023-10-20[fuzz] Delete i2p targetdergoegge
2023-10-19Merge bitcoin/bitcoin#28077: I2P: also sleep after errors in Accept() & ↵Andrew Chow
destroy the session if we get an unexpected error 5c8e15c451ec870b9dd4eb843ec6ca3ba64cda4f i2p: destroy the session if we get an unexpected error from the I2P router (Vasil Dimov) 762404a68c114e8831cdfae937627174544b55a7 i2p: also sleep after errors in Accept() (Vasil Dimov) Pull request description: ### Background In the `i2p::sam::Session` class: `Listen()` does: * if the session is not created yet * create the control socket and on it: * `HELLO` * `SESSION CREATE ID=sessid` * leave the control socked opened * create a new socket and on it: * `HELLO` * `STREAM ACCEPT ID=sessid` * read reply (`STREAM STATUS`), `Listen()` only succeeds if it contains `RESULT=OK` Then a wait starts, for a peer to connect. When connected, `Accept()` does: * on the socket from `STREAM ACCEPT` from `Listen()`: read the Base64 identification of the connecting peer ### Problem The I2P router may be in such a state that this happens in a quick succession (many times per second, see https://github.com/bitcoin/bitcoin/issues/22759#issuecomment-1609907115): `Listen()`-succeeds, `Accept()`-fails. `Accept()` fails because the I2P router sends something that is not Base64 on the socket: `STREAM STATUS RESULT=I2P_ERROR MESSAGE="Session was closed"` We only sleep after failed `Listen()` because the assumption was that if `Accept()` fails then the next `Listen()` will also fail. ### Solution Avoid filling the log with "Error accepting:" messages and sleep also after a failed `Accept()`. ### Extra changes * Reset the error waiting time after one successful connection. Otherwise the timer will remain high due to problems that have been solved long time in the past. * Increment the wait time less aggressively. * Handle the unexpected "Session was closed" message more gracefully (don't log stupid messages like `Cannot decode Base64: "STREAM STATUS...`) and destroy the session right way. ACKs for top commit: achow101: ACK 5c8e15c451ec870b9dd4eb843ec6ca3ba64cda4f jonatack: re-ACK 5c8e15c451ec870b9dd4eb843ec6ca3ba64cda4f Tree-SHA512: 1d47958c50eeae9eefcb668b8539fd092adead93328e4bf3355267819304b99ab41cbe1b5dbedbc3452c2bc389dc8330c0e27eb5ccb880e33dc46930a1592885
2023-10-19Merge bitcoin/bitcoin#27071: Handle CJDNS from LookupSubNet()Andrew Chow
0e6f6ebc064c5fb425fc3699efe760ec6cd4b6af net: remove unused CConnman::FindNode(const CSubNet&) (Vasil Dimov) 9482cb780fe04c1f1d9050edd1b8e549e52c86ce netbase: possibly change the result of LookupSubNet() to CJDNS (Vasil Dimov) 53afa68026ffa1313ae4aba3664de7791d23b1c8 net: move MaybeFlipIPv6toCJDNS() from net to netbase (Vasil Dimov) 6e308651c441cbf8763c67cc099c538c333c2872 net: move IsReachable() code to netbase and encapsulate it (Vasil Dimov) c42ded3d9bda8b273780a4a81490bbf1b9e9c261 fuzz: ConsumeNetAddr(): avoid IPv6 addresses that look like CJDNS (Vasil Dimov) 64d6f77907afd461d9b14ee10ab32335f4454734 net: put CJDNS prefix byte in a constant (Vasil Dimov) Pull request description: `LookupSubNet()` would treat addresses that start with `fc` as IPv6 even if `-cjdnsreachable` is set. This creates the following problems where it is called: * `NetWhitelistPermissions::TryParse()`: otherwise `-whitelist=` fails to white list CJDNS addresses: when a CJDNS peer connects to us, it will be matched against IPv6 `fc...` subnet and the match will never succeed. * `BanMapFromJson()`: CJDNS bans are stored as just IPv6 addresses in `banlist.json`. Upon reading from disk they have to be converted back to CJDNS, otherwise, after restart, a ban entry like (`fc00::1`, IPv6) would not match a peer (`fc00::1`, CJDNS). * `RPCConsole::unbanSelectedNode()`: in the GUI the ban entries go through `CSubNet::ToString()` and back via `LookupSubNet()`. Then it must match whatever is stored in `BanMan`, otherwise it is impossible to unban via the GUI. These were uncovered by https://github.com/bitcoin/bitcoin/pull/26859. Thus, flip the result of `LookupSubNet()` to CJDNS if the network base address starts with `fc` and `-cjdnsreachable` is set. Since subnetting/masking does not make sense for CJDNS (the address is "random" bytes, like Tor and I2P, there is no hierarchy) treat `fc.../mask` as an invalid `CSubNet`. To achieve that, `MaybeFlipIPv6toCJDNS()` has to be moved from `net` to `netbase` and thus also `IsReachable()`. In the process of moving `IsReachable()`, `SetReachable()` and `vfLimited[]` encapsulate those in a class. ACKs for top commit: jonatack: Code review ACK 0e6f6ebc064c5fb425fc3699efe760ec6cd4b6af achow101: ACK 0e6f6ebc064c5fb425fc3699efe760ec6cd4b6af mzumsande: re-ACK 0e6f6ebc064c5fb425fc3699efe760ec6cd4b6af Tree-SHA512: 4767a60dc882916de4c8b110ce8de208ff3f58daaa0b560e6547d72e604d07c4157e72cf98b237228310fc05c0a3922f446674492e2ba02e990a272d288bd566
2023-10-17Merge bitcoin/bitcoin#28651: Make miniscript GetWitnessSize accurate for ↵Andrew Chow
tapscript b22810887b3840ad0fcb424ea7e16d2d195767d9 miniscript: make GetWitnessSize accurate for tapscript (Pieter Wuille) 8be98514080ab816fcb2498ea4bc6f211a2b05e0 test: add tests for miniscript GetWitnessSize (Pieter Wuille) 7ed2b2d430e4dc0d3ba62a30f814df2c7c0c0651 test: remove mutable global contexts in miniscript fuzzer/test (Pieter Wuille) Pull request description: So far, the same algorithm is used to compute an (upper bound on) the maximum witness size for both P2WSH and P2TR miniscript. That's unfortunate, because it means fee estimations for P2TR miniscript will miss out on the generic savings brought by P2TR witnesses (smaller signatures and public keys, specifically). Fix this by making the algorithm use script context specification calculations, and add tests for it. Also included is a cleanup for the tests to avoid mutable globals, as I found it hard to reason about what exactly was being tested. ACKs for top commit: achow101: ACK b22810887b3840ad0fcb424ea7e16d2d195767d9 darosior: ACK b22810887b3840ad0fcb424ea7e16d2d195767d9 Tree-SHA512: e4bda7376628f3e91cfc74917cefc554ca16eb5f2a0e1adddc33eb8717c4aaa071e56a40f85a2041ae74ec445a7bd0129bba48994c203e0e6e4d25af65954d9e
2023-10-16Merge bitcoin/bitcoin#28539: lib: add taproot support to libconsensusAndrew Chow
ff8e2fc2e2416f6f3b84cdb40db8ac168596b579 fuzz: add coverage for `bitcoinconsensus_verify_script_with_spent_outputs` (brunoerg) c5f2a757d736f14d27ac5256a9df887cd2f174f1 docs: add release notes for #28539 (brunoerg) de54882348502d860cf1e504100aa8fb1e52aa88 docs: add docs for additional libconsensus functions (Jake Rawsthorne) 70106e0689546fee497814c63a6a4747e0937b36 docs: link to rust-bitcoinconsensus (Jake Rawsthorne) fb0db07e414fec3318b3af683167ebef9c82fc84 lib: add Taproot support to libconsensus (Jake Rawsthorne) Pull request description: Grabbed from #21158. Closes #21133. ACKs for top commit: achow101: ACK ff8e2fc2e2416f6f3b84cdb40db8ac168596b579 theStack: ACK ff8e2fc2e2416f6f3b84cdb40db8ac168596b579 darosior: re-ACK ff8e2fc2e2416f6f3b84cdb40db8ac168596b579 Tree-SHA512: bf6f500c7e8c9ff6884137c2cd9b4522c586e52848dd639b774b94d998b0516b877498d24f3a6cc7425aedf81d18b0d30c1ccf19e2d527fdfdfa3955ca49b6e7
2023-10-16Merge bitcoin/bitcoin#28583: refactor: [tidy] modernize-use-emplacefanquake
fa05a726c225dc65dee79367bb67f099ae4f99e6 tidy: modernize-use-emplace (MarcoFalke) Pull request description: Constructing a temporary unnamed object only to copy or move it into a container seems both verbose in code and a strict performance penalty. Fix both issues via the `modernize-use-emplace` tidy check. ACKs for top commit: Sjors: re-utACK fa05a726c2 hebasto: ACK fa05a726c225dc65dee79367bb67f099ae4f99e6. TheCharlatan: ACK fa05a726c225dc65dee79367bb67f099ae4f99e6 Tree-SHA512: 4408a094f406e7bf6c1468c2b0798f68f4d952a1253cf5b20bdc648ad7eea4a2c070051fed46d66fd37bce2ce6f85962484a1d32826b7ab8c9baba431eaa2765
2023-10-13miniscript: make GetWitnessSize accurate for tapscriptPieter Wuille
2023-10-13test: add tests for miniscript GetWitnessSizePieter Wuille
2023-10-13test: remove mutable global contexts in miniscript fuzzer/testPieter Wuille
2023-10-13fuzz: add coverage for `bitcoinconsensus_verify_script_with_spent_outputs`brunoerg
Co-authored-by: Antonie Poinsot <darosior@protonmail.com>
2023-10-13lib: add Taproot support to libconsensusJake Rawsthorne
Co-authored-by: Bruno Garcia <brunoely.gc@gmail.com>
2023-10-12Use type-safe txid types in orphanagedergoegge
2023-10-12Introduce types for txids & wtxidsdergoegge
2023-10-12ci: Drop no longer needed `NOLINTNEXTLINE`Hennadii Stepanov
2023-10-12tidy: modernize-use-emplaceMarcoFalke
2023-10-09test: Unit test for inferring scripts with hybrid and uncompressed keysAndrew Chow
2023-10-08script/sign: Miniscript support in TapscriptAntoine Poinsot
We make the Satisfier a base in which to store the common methods between the Tapscript and P2WSH satisfier, and from which they both inherit. A field is added to SignatureData to be able to satisfy pkh() under Tapscript context (to get the pubkey hash preimage) without wallet data. For instance in `finalizepsbt` RPC. See also the next commits for a functional test that exercises this.
2023-10-08descriptor: parse Miniscript expressions within Taproot descriptorsAntoine Poinsot
2023-10-08fuzz: miniscript: higher sensitivity for max stack size limit under TapscriptAntoine Poinsot
In order to exacerbate a mistake in the stack size tracking logic, sometimes pad the witness to make the script execute at the brink of the stack size limit. This way if the stack size is underestimated for a script it would immediately fail `VerifyScript`.
2023-10-08qa: test Miniscript max stack size trackingAntoine Poinsot
2023-10-08fuzz: adapt Miniscript targets to TapscriptAntoine Poinsot
We introduce another global that dictates the script context under which to operate when running the target. For miniscript_script, just consume another byte to set the context. This should only affect existing seeds to the extent they contain a CHECKMULTISIG. However it would not invalidate them entirely as they may contain a NUMEQUAL or a CHECKSIGADD, and this still exercises a bit of the parser. For miniscript_string, reduce the string size by one byte and use the last byte to determine the context. This is the change that i think would invalidate the lowest number of existing seeds. For miniscript_stable, we don't want to invalidate any seed. Instead of creating a new miniscript_stable_tapscript, simply run the target once for P2WSH and once for Tapscript (with the same seed). For miniscript_smart, consume one byte before generating a pseudo-random node to set the context. We have less regard for seed stability for this target anyways.
2023-10-08qa: Tapscript-Miniscript unit testsAntoine Poinsot
Adapt the test data and the parsing context to support x-only keys. Adapt the Test() helper to test existing cases under both Tapscript and P2WSH context, asserting what needs to be valid or not in each. Finally, add more cases that exercise the logic that was added in the previous commits (multi_a, different resource checks and keys serialization under Tapscript, different properties for 'd:' fragment, ..).
2023-10-08miniscript: account for keys as being 32 bytes under Taproot contextAntoine Poinsot
2023-10-08miniscript: make 'd:' have the 'u' property under Tapscript contextAntoine Poinsot
In Tapscript MINIMALIF is a consensus rule, so we can rely on the fact that the `DUP IF [X] ENDIF` will always put an exact 1 on the stack upon satisfaction.
2023-10-08miniscript: introduce a multi_a fragmentAntoine Poinsot
It is the equivalent of multi() but for Tapscript, using CHECKSIGADD instead of CHECKMULTISIG. It shares the same properties as multi() but for 'n', since a threshold multi_a() may have an empty vector as the top element of its satisfaction. It could also have the 'o' property when it only has a single key, but in this case a 'pk()' is always preferable anyways.
2023-10-08miniscript: store the script context within the Node structureAntoine Poinsot
Some checks will be different depending on the script context (for instance the maximum script size).
2023-10-08miniscript: introduce a MsContext() helper to contextsAntoine Poinsot
We are going to introduce Tapscript support in Miniscript, for which some of Miniscript rules and properties change (new or modified fragments, different typing rules, different resources consumption, ..).
2023-10-06chain: Rename HaveTxsDownloaded to HaveNumChainTxsFabian Jahr
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2023-10-06validation, test: Improve and document nChainTx check for testabilityFabian Jahr
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-10-05Merge bitcoin/bitcoin#27609: rpc: allow submitpackage to be called outside ↵Andrew Chow
of regtest 5b878be742dbfcd232d949d2df1fff4743aec3d8 [doc] add release note for submitpackage (glozow) 7a9bb2a2a59ba49f80519c8435229abec2432486 [rpc] allow submitpackage to be called outside of regtest (glozow) 5b9087a9a7da2602485e85e0b163dc3cbd2daf31 [rpc] require package to be a tree in submitpackage (glozow) e32ba1599c599e75b1da3393f71f633de860505f [txpackages] IsChildWithParentsTree() (glozow) b4f28cc345ef9c5261c4a8d743654a44784c7802 [doc] parent pay for child in aggregate CheckFeeRate (glozow) Pull request description: Permit (restricted topology) submitpackage RPC outside of regtest. Suggested in https://github.com/bitcoin/bitcoin/pull/26933#issuecomment-1510851570 This RPC should be safe but still experimental - interface may change, not all features (e.g. package RBF) are implemented, etc. If a miner wants to expose this to people, they can effectively use "package relay" before the p2p changes are implemented. However, please note **this is not package relay**; transactions submitted this way will not relay to other nodes if the feerates are below their mempool min fee. Users should put this behind some kind of rate limit or permissions. ACKs for top commit: instagibbs: ACK 5b878be742dbfcd232d949d2df1fff4743aec3d8 achow101: ACK 5b878be742dbfcd232d949d2df1fff4743aec3d8 dergoegge: Code review ACK 5b878be742dbfcd232d949d2df1fff4743aec3d8 ajtowns: ACK 5b878be742dbfcd232d949d2df1fff4743aec3d8 ariard: Code Review ACK 5b878be742. Though didn’t manually test the PR. Tree-SHA512: 610365c0b2ffcccd55dedd1151879c82de1027e3319712bcb11d54f2467afaae4d05dca5f4b25f03354c80845fef538d3938b958174dda8b14c10670537a6524
2023-10-05Merge bitcoin/bitcoin#28587: descriptors: disallow hybrid public keysAndrew Chow
c1e6c542af6d89a499e2a65465865aec651c4d67 descriptors: disallow hybrid public keys (Pieter Wuille) Pull request description: Fixes #28511 The descriptor documentation (`doc/descriptors.md`) and [BIP380](https://github.com/bitcoin/bips/blob/master/bip-0380.mediawiki) explicitly require that hex-encoded public keys start with 02 or 03 (compressed) or 04 (uncompressed). However, the current parsing/inference code permit 06 and 07 (hybrid) encoding as well. Fix this. ACKs for top commit: darosior: ACK c1e6c542af6d89a499e2a65465865aec651c4d67 achow101: ACK c1e6c542af6d89a499e2a65465865aec651c4d67 Tree-SHA512: 23b674fb420619b2536d12da10008bb87cf7bc0333ec59e618c0d02c3574b468cc71248475ece37f76658d743ef51e68566948e903bca79fda5f7d75416fea4d
2023-10-05net: move IsReachable() code to netbase and encapsulate itVasil Dimov
`vfLimited`, `IsReachable()`, `SetReachable()` need not be in the `net` module. Move them to `netbase` because they will be needed in `LookupSubNet()` to possibly flip the result to CJDNS (if that network is reachable). In the process, encapsulate them in a class. `NET_UNROUTABLE` and `NET_INTERNAL` are no longer ignored when adding or removing reachable networks. This was unnecessary.
2023-10-05fuzz: ConsumeNetAddr(): avoid IPv6 addresses that look like CJDNSVasil Dimov
The fuzz testing framework runs as if `-cjdnsreachable` is set and in this case addresses like `{net=IPv6, addr=fc...}` are not possible.
2023-10-05Merge bitcoin/bitcoin#28558: Make PeerManager own a FastRandomContextfanquake
4cafe9f176e93ebb6c38abb12140e8d8be005cbf [test] Make PeerManager's rng deterministic in tests (dergoegge) fecec3e1c661ba273470ecc5ef12d4c070b53050 [net processing] FeeFilterRounder doesn't own a FastRandomContext (dergoegge) 47520ed209d9341702a0fb6006bee6f63f7da42e [net processing] Make fee filter rounder non-global (dergoegge) 77506f4ac6b3a3d7396a3a6101345019e05b3b10 [net processing] Addr shuffle uses PeerManager's rng (dergoegge) a648dd79e5ebfdb627d0221b1207862efb664dfc [net processing] PushAddress uses PeerManager's rng (dergoegge) 87c706713e5d1c78bad943a42bf7c69047d28ea5 [net processing] PeerManager holds a FastRandomContext (dergoegge) Pull request description: This lets us avoid some non-determinism in tests (also see #28537). ACKs for top commit: MarcoFalke: re-ACK 4cafe9f176e93ebb6c38abb12140e8d8be005cbf 🕗 glozow: concept && light code review ACK 4cafe9f176e93ebb6c38abb12140e8d8be005cbf Tree-SHA512: 3c18700773d0bc547ccb6442c41567e6f26b0b50fab5b79620da417ec91b9c0ae1395d15258da3aa4a91447b8ce560145dd135e39fbbd0610749e528e665b111
2023-10-05i2p: destroy the session if we get an unexpected error from the I2P routerVasil Dimov
From https://geti2p.net/en/docs/api/samv3: If SILENT=false was passed, which is the default value, the SAM bridge sends the client a ASCII line containing the base64 public destination key of the requesting peer So, `Accept()` is supposed to receive a Base64 encoded destination of the connecting peer, but if it receives something like this instead: STREAM STATUS RESULT=I2P_ERROR MESSAGE="Session was closed" then destroy the session.
2023-10-04descriptors: disallow hybrid public keysPieter Wuille
The descriptor documentation (doc/descriptors.md) and BIP380 explicitly require that hex-encoded public keys start with 02 or 03 (compressed) or 04 (uncompressed). However, the current parsing/inference code permit 06 and 07 (hybrid) encoding as well. Fix this.
2023-10-04[test] Make PeerManager's rng deterministic in testsdergoegge
2023-10-04[net processing] FeeFilterRounder doesn't own a FastRandomContextdergoegge
2023-10-03Merge bitcoin/bitcoin#28523: rpc: add hidden getrawaddrman RPC to list ↵Andrew Chow
addrman table entries 352d5eb2a9e89cff4a2815d94a9d81fcc20c4b2c test: getrawaddrman RPC (0xb10c) da384a286bd84a97e7ebe7a64654c5be20ab2df1 rpc: getrawaddrman for addrman entries (0xb10c) Pull request description: Inspired by `getaddrmaninfo` (#27511), this adds a hidden/test-only `getrawaddrman` RPC. The RPC returns information on all addresses in the address manager new and tried tables. Addrman table contents can be used in tests and during development. The RPC result encodes the `bucket` and `position`, the internal location of addresses in the tables, in the address object's string key. This allows users to choose to consume or to ignore the location information. If the internals of the address manager implementation change, the location encoding might change too. ``` getrawaddrman EXPERIMENTAL warning: this call may be changed in future releases. Returns information on all address manager entries for the new and tried tables. Result: { (json object) "table" : { (json object) buckets with addresses in the address manager table ( new, tried ) "bucket/position" : { (json object) the location in the address manager table (<bucket>/<position>) "address" : "str", (string) The address of the node "port" : n, (numeric) The port number of the node "network" : "str", (string) The network (ipv4, ipv6, onion, i2p, cjdns) of the address "services" : n, (numeric) The services offered by the node "time" : xxx, (numeric) The UNIX epoch time when the node was last seen "source" : "str", (string) The address that relayed the address to us "source_network" : "str" (string) The network (ipv4, ipv6, onion, i2p, cjdns) of the source address }, ... }, ... } Examples: > bitcoin-cli getrawaddrman > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "getrawaddrman", "params": []}' -H 'content-type: text/plain;' http://127.0.0.1:8332/ ``` ACKs for top commit: willcl-ark: reACK 352d5eb2a9 amitiuttarwar: reACK 352d5eb2a9e stratospher: reACK 352d5eb. achow101: ACK 352d5eb2a9e89cff4a2815d94a9d81fcc20c4b2c Tree-SHA512: cc462666b5c709617c66b0e3e9a17c4c81e9e295f91bdd9572492d1cb6466fc9b6d48ee805ebe82f9f16010798370effe5c8f4db15065b8c7c0d8637675d615e
2023-10-03Merge bitcoin/bitcoin#26312: Remove Sock::Get() and Sock::Sock()Ryan Ofsky
7df450836969b81e98322c9a09c08b35d1095a25 test: improve sock_tests/move_assignment (Vasil Dimov) 5086a99b84367a45706af7197da1016dd966e6d9 net: remove Sock default constructor, it's not necessary (Vasil Dimov) 7829272f7826511241defd34954e6040ea963f07 net: remove now unnecessary Sock::Get() (Vasil Dimov) 944b21b70ae490a5a746bcc1810a5074d74e9d34 net: don't check if the socket is valid in ConnectSocketDirectly() (Vasil Dimov) aeac68d036e3cff57ce155f1a904d77f98b357d4 net: don't check if the socket is valid in GetBindAddress() (Vasil Dimov) 5ac1a51ee5a57da59f1ff1986b7d9054484d3c80 i2p: avoid using Sock::Get() for checking for a valid socket (Vasil Dimov) Pull request description: _This is a piece of #21878, chopped off to ease review._ Peeking at the underlying socket file descriptor of `Sock` and checkig if it is `INVALID_SOCKET` is bad encapsulation and stands in the way of testing/mocking/fuzzing. Instead use an empty `unique_ptr` to denote that there is no valid socket where appropriate or outright remove such checks where they are not necessary. The default constructor `Sock::Sock()` is unnecessary now after recent changes, thus remove it. ACKs for top commit: ajtowns: ACK 7df450836969b81e98322c9a09c08b35d1095a25 jonatack: ACK 7df450836969b81e98322c9a09c08b35d1095a25 Tree-SHA512: 9742aeeeabe8690530bf74caa6ba296787028c52f4a3342afd193b05dbbb1f6645935c33ba0a5230199a09af01c666bd3c7fb16b48692a0d185356ea59a8ddbf
2023-10-02net: expose transport types/session IDs of connections in RPC and logsPieter Wuille
Co-authored-by: Dhruv Mehta <856960+dhruv@users.noreply.github.com>
2023-10-02rpc: addnode arg to use BIP324 v2 p2pdhruv
Co-authored-by: Pieter Wuille <bitcoin-dev@wuille.net>
2023-10-02net: advertise NODE_P2P_V2 if CLI arg -v2transport is onPieter Wuille
Co-authored-by: Dhruv Mehta <856960+dhruv@users.noreply.github.com>
2023-10-02Merge bitcoin/bitcoin#27596: assumeutxo (2)Andrew Chow
edbed31066e3674ba52b8c093ab235625527f383 chainparams: add signet assumeutxo param at height 160_000 (Sjors Provoost) b8cafe38713cbf10d15459042f7f911bcc1b1e4e chainparams: add testnet assumeutxo param at height 2_500_000 (Sjors Provoost) 99839bbfa7110c7abf22e587ae2f72c9c57d3c85 doc: add note about confusing HaveTxsDownloaded name (James O'Beirne) 7ee46a755f1d57ce9d51975d3b54dc9ac3d08d52 contrib: add script to demo/test assumeutxo (James O'Beirne) 42cae39356fd20d521aaf99aff1ed85856f3c9f3 test: add feature_assumeutxo functional test (James O'Beirne) 0f64bac6030334d798ae205cd7af4bf248feddd9 rpc: add getchainstates (James O'Beirne) bb0585779472962f40d9cdd9c6532132850d371c refuse to activate a UTXO snapshot if mempool not empty (James O'Beirne) ce585a9a158476b0ad3296477b922e79f308e795 rpc: add loadtxoutset (James O'Beirne) 62ac519e718eb7a31dca1102a96ba219fbc7f95d validation: do not activate snapshot if behind active chain (James O'Beirne) 9511fb3616b7bbe1d0d2f54a45ea0a650ba0367b validation: assumeutxo: swap m_mempool on snapshot activation (James O'Beirne) 7fcd21544a333ffdf1910b65c573579860be6a36 blockstorage: segment normal/assumedvalid blockfiles (James O'Beirne) 4c3b8ca35c2e4a441264749bb312df2bd054b5b8 validation: populate nChainTx value for assumedvalid chainstates (James O'Beirne) 49ef778158c43859946a592e11ec34fe1b93a5b6 test: adjust chainstate tests to use recognized snapshot base (James O'Beirne) 1019c399825b0d512c1fd751c376d46fed4992b9 validation: pruning for multiple chainstates (James O'Beirne) 373cf91531b84bfdd06fdf8abf4dca228029ce6b validation: indexing changes for assumeutxo (James O'Beirne) 1fffdd76a1bca908f55d73b64983655b14cf7432 net_processing: validationinterface: ignore some events for bg chain (James O'Beirne) fbe0a7d7ca680358237b6c2369b3fd2b43221113 wallet: validationinterface: only handle active chain notifications (James O'Beirne) f073917a9e7ba423643dcae0339776470b628f65 validationinterface: only send zmq notifications for active (James O'Beirne) 4d8f4dcb450d31e4847804e62bf91545b949fa14 validation: pass ChainstateRole for validationinterface calls (James O'Beirne) 1e59acdf17309f567c370885f0cf02605e2baa58 validation: only call UpdatedBlockTip for active chainstate (James O'Beirne) c6af23c5179cc383f8e6c275373af8d11e6a989f validation: add ChainstateRole (James O'Beirne) 9f2318c76cc6986d48e13831cf5bd8dab194fdf4 validation: MaybeRebalanceCaches when chain leaves IBD (James O'Beirne) 434495a8c1496ca23fe35b84499f3daf668d76b8 chainparams: add blockhash to AssumeutxoData (James O'Beirne) c711ca186f8d8a28810be0beedcb615ddcf93163 assumeutxo: remove snapshot during -reindex{-chainstate} (James O'Beirne) c93ef43e4fd4fbc1263cdc9e98ae5856830fe89e bugfix: correct is_snapshot_cs in VerifyDB (James O'Beirne) b73d3bbd23220857bf17cbb6401275bf58013b72 net_processing: Request assumeutxo background chain blocks (Suhas Daftuar) Pull request description: - Background and FAQ: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal - Prior progress/project: https://github.com/bitcoin/bitcoin/projects/11 - Replaces https://github.com/bitcoin/bitcoin/pull/15606, which was closed due to Github slowness. Original description and commentary can be found there. --- This changeset finishes the first phase of the assumeutxo project. It makes UTXO snapshots loadable via RPC (`loadtxoutset`) and adds `assumeutxo` parameters to chainparams. It contains all the remaining changes necessary to both use an assumedvalid snapshot chainstate and do a full validation sync in the background. This may look like a lot to review, but note that - ~200 lines are a (non-essential) demo shell script - Many lines are functional test, documentation, and relatively dilute RPC code. So it shouldn't be as burdensome to review as the linecount might suggest. - **P2P**: minor changes are made to `init.cpp` and `net_processing.cpp` to make simultaneous IBD across multiple chainstates work. - **Pruning**: implement correct pruning behavior when using a background chainstate - **Blockfile separation**: to prevent "fragmentation" in blockfile storage, have background chainstates use separate blockfiles from active snapshot chainstates to avoid interleaving heights and impairing pruning. - **Indexing**: some `CValidationInterface` events are given with an additional parameter, ChainstateRole, and all indexers ignore events from ChainstateRole::ASSUMEDVALID so that indexation only happens sequentially. - Have `-reindex` properly wipe snapshot chainstates. - **RPC**: introduce RPC commands `loadtxoutset` and (hidden) `getchainstates`. - **Release docs & first assumeutxo commitment**: add notes and a particular assumeutxo hash value for first AU-enabled release. - This will complete the project and allow use of UTXO snapshots for faster node bootstrap. The next phase, if it were to be pursued, would be coming up with a way to distribute the UTXO snapshots over the P2P network. --- ### UTXO snapshots Create your own with `./contrib/devtools/utxo_snapshot.sh`, e.g. ```shell ./contrib/devtools/utxo_snapshot.sh 788000 utxo.dat ./src/bitcoin-cli -datadir=$(pwd)/testdata`) ``` or use the pre-generated ones listed below. - Testnet: **2'500'000** (Sjors): - torrent: `magnet:?xt=urn:btih:511e09f4bf853aefab00de5c070b1e031f0ecbe9&dn=utxo-testnet-2500000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969` - sha256: `79db4b025448cc0ac388d8589a28eab02de53055d181e34eb47391717aa16388` - Signet: **160'000** (Sjors): - torrent: `magnet:?xt=urn:btih:9da986cb27b3980ea7fd06b21e199b148d486880&dn=utxo-signet-160000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969` - sha256: `eeeca845385ba91e84ef58c09d38f98f246a24feadaad57fe1e5874f3f92ef8c` - Mainnet: **800'000** (Sjors): - Note: this needs the following commit cherry-picked in: https://github.com/Sjors/bitcoin/commit/24deb2022b822f22fba9fcbee201e37a83225eb2 - torrent: `magnet:?xt=urn:btih:50ee955bef37f5ec3e5b0df4cf0288af3d715a2e&dn=utxo-800000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969` ### Testing #### For fun (~5min) If you want to do a quick test, you can run `./contrib/devtools/test_utxo_snapshots.sh` and follow the instructions. This is mostly obviated by the functional tests, though. #### For real (longer) If you'd like to experience a real usage of assumeutxo, you can do that too. I've cut a new snapshot at height 788'000 (http://img.jameso.be/utxo-788000.dat - but you can do it yourself with `./contrib/devtools/utxo_snapshot.sh` if you want). Download that, and then create a datadir for testing: ```sh $ cd ~/src/bitcoin # or whatever # get the snapshot $ curl http://img.jameso.be/utxo-788000.dat > utxo-788000.dat # you'll want to do this if you like copy/pasting $ export AU_DATADIR=/home/${USER}/au-test # or wherever $ mkdir ${AU_DATADIR} $ vim ${AU_DATADIR}/bitcoin.conf dbcache=8000 # or, you know, something high blockfilterindex=1 coinstatsindex=1 prune=3000 logthreadnames=1 ``` Obtain this branch, build it, and then start bitcoind: ```sh $ git remote add jamesob https://github.com/jamesob/bitcoin $ git fetch jamesob assumeutxo $ git checkout jamesob/assumeutxo $ ./configure $conf_args && make # (whatever you like to do here) # start 'er up and watch the logs $ ./src/bitcoind -datadir=${AU_DATADIR} ``` Then, in some other window, load the snapshot ```sh $ ./src/bitcoin-cli -datadir=${AU_DATADIR} loadtxoutset $(pwd)/utxo-788000.dat ``` You'll see some log messages about headers retrieval and waiting to see the snapshot in the headers chain. Once you get the full headers chain, you'll spend a decent amount of time (~10min) loading the snapshot, checking it, and flushing it to disk. After all that happens, you should be syncing to tip in pretty short order, and you'll see the occasional `[background validation]` log message go by. In yet another window, you can check out chainstate status with ```sh $ ./src/bitcoin-cli -datadir=${AU_DATADIR} getchainstates ``` as well as usual favorites like `getblockchaininfo`. ACKs for top commit: achow101: ACK edbed31066e3674ba52b8c093ab235625527f383 Tree-SHA512: 6086fb9a38dc7df85fedc76b30084dd8154617a2a91e89a84fb41326d34ef8e7d7ea593107afba01369093bf8cc91770621d98f0ea42a5b3b99db868d2f14dc2
2023-10-02rpc: getrawaddrman for addrman entries0xb10c
Exposing address manager table entries in a hidden RPC allows to introspect addrman tables in tests and during development. As response JSON object the following FORMAT1 is choosen: { "table": { "<bucket>/<position>": { "address": "..", "port": .., ... }, "<bucket>/<position>": { "address": "..", "port": .., ... }, "<bucket>/<position>": { "address": "..", "port": .., ... }, ... } } An alternative would be FORMAT2 { "table": { "bucket": { "position": { "address": "..", "port": .., ... }, "position": { "address": "..", "port": .., ... }, .. }, "bucket": { "position": { "address": "..", "port": .., ... }, .. }, } } FORMAT1 and FORMAT2 have different encodings for the location of the address in the address manager. While FORMAT2 might be easier to process for downstream tools, it also mimics internal addrman mappings, which might change at some point. Users not interested in the address location can ignore the location key. They don't have to adapt to a new RPC response format, when the internal addrman layout changes. Additionally, FORMAT1 is also slightly easier to to iterate in downstream tools. The RPC response-building implemenation complexcity is lower with FORMAT1 as we can more easily build a "<bucket>/<position>" key than a multiple "bucket" objects with multiple "position" objects (FORMAT2).
generated by cgit v1.2.3 (git 2.26.2) at 2024-11-07 21:29:21 +0000