aboutsummaryrefslogtreecommitdiff
path: root/src/test/fuzz
AgeCommit message (Collapse)Author
2024-03-07fuzz: restrict fopencookie usage to Linux & FreeBSDfanquake
Should fix the GCC compilation portion of #29517: https://github.com/bitcoin/bitcoin/issues/29517#issuecomment-1973573314. See also: https://www.gnu.org/software/gnulib/manual/html_node/fopencookie.html but note that FreeBSD has supported this function since 11.x. Github-Pull: #29529 Rebased-From: 312f3381a2d3a7afb7c81b4662214d4d67b4e84a
2023-10-20Merge bitcoin/bitcoin#28692: fuzz: Delete i2p fuzz testfanquake
dd4dcbd4cda31f67d014a93340a6d1ba1c245b0f [fuzz] Delete i2p target (dergoegge) Pull request description: closes #28665 The target is buggy and doesn't reach basic coverage. ACKs for top commit: maflcko: lgtm ACK dd4dcbd4cda31f67d014a93340a6d1ba1c245b0f glozow: ACK dd4dcbd4cda31f67d014a93340a6d1ba1c245b0f, agree it's better to delete this test until somebody wants to write a better one Tree-SHA512: b6ca6cad1773b1ceb6e5ac0fd501ea615f66507ef811745799deaaa4460f1700d96ae03cf55b740a96ed8cd2283b3d6738cd580ba97f2af619197d6c4414ca21
2023-10-20[fuzz] Delete i2p targetdergoegge
2023-10-19Merge bitcoin/bitcoin#27071: Handle CJDNS from LookupSubNet()Andrew Chow
0e6f6ebc064c5fb425fc3699efe760ec6cd4b6af net: remove unused CConnman::FindNode(const CSubNet&) (Vasil Dimov) 9482cb780fe04c1f1d9050edd1b8e549e52c86ce netbase: possibly change the result of LookupSubNet() to CJDNS (Vasil Dimov) 53afa68026ffa1313ae4aba3664de7791d23b1c8 net: move MaybeFlipIPv6toCJDNS() from net to netbase (Vasil Dimov) 6e308651c441cbf8763c67cc099c538c333c2872 net: move IsReachable() code to netbase and encapsulate it (Vasil Dimov) c42ded3d9bda8b273780a4a81490bbf1b9e9c261 fuzz: ConsumeNetAddr(): avoid IPv6 addresses that look like CJDNS (Vasil Dimov) 64d6f77907afd461d9b14ee10ab32335f4454734 net: put CJDNS prefix byte in a constant (Vasil Dimov) Pull request description: `LookupSubNet()` would treat addresses that start with `fc` as IPv6 even if `-cjdnsreachable` is set. This creates the following problems where it is called: * `NetWhitelistPermissions::TryParse()`: otherwise `-whitelist=` fails to white list CJDNS addresses: when a CJDNS peer connects to us, it will be matched against IPv6 `fc...` subnet and the match will never succeed. * `BanMapFromJson()`: CJDNS bans are stored as just IPv6 addresses in `banlist.json`. Upon reading from disk they have to be converted back to CJDNS, otherwise, after restart, a ban entry like (`fc00::1`, IPv6) would not match a peer (`fc00::1`, CJDNS). * `RPCConsole::unbanSelectedNode()`: in the GUI the ban entries go through `CSubNet::ToString()` and back via `LookupSubNet()`. Then it must match whatever is stored in `BanMan`, otherwise it is impossible to unban via the GUI. These were uncovered by https://github.com/bitcoin/bitcoin/pull/26859. Thus, flip the result of `LookupSubNet()` to CJDNS if the network base address starts with `fc` and `-cjdnsreachable` is set. Since subnetting/masking does not make sense for CJDNS (the address is "random" bytes, like Tor and I2P, there is no hierarchy) treat `fc.../mask` as an invalid `CSubNet`. To achieve that, `MaybeFlipIPv6toCJDNS()` has to be moved from `net` to `netbase` and thus also `IsReachable()`. In the process of moving `IsReachable()`, `SetReachable()` and `vfLimited[]` encapsulate those in a class. ACKs for top commit: jonatack: Code review ACK 0e6f6ebc064c5fb425fc3699efe760ec6cd4b6af achow101: ACK 0e6f6ebc064c5fb425fc3699efe760ec6cd4b6af mzumsande: re-ACK 0e6f6ebc064c5fb425fc3699efe760ec6cd4b6af Tree-SHA512: 4767a60dc882916de4c8b110ce8de208ff3f58daaa0b560e6547d72e604d07c4157e72cf98b237228310fc05c0a3922f446674492e2ba02e990a272d288bd566
2023-10-17Merge bitcoin/bitcoin#28651: Make miniscript GetWitnessSize accurate for ↵Andrew Chow
tapscript b22810887b3840ad0fcb424ea7e16d2d195767d9 miniscript: make GetWitnessSize accurate for tapscript (Pieter Wuille) 8be98514080ab816fcb2498ea4bc6f211a2b05e0 test: add tests for miniscript GetWitnessSize (Pieter Wuille) 7ed2b2d430e4dc0d3ba62a30f814df2c7c0c0651 test: remove mutable global contexts in miniscript fuzzer/test (Pieter Wuille) Pull request description: So far, the same algorithm is used to compute an (upper bound on) the maximum witness size for both P2WSH and P2TR miniscript. That's unfortunate, because it means fee estimations for P2TR miniscript will miss out on the generic savings brought by P2TR witnesses (smaller signatures and public keys, specifically). Fix this by making the algorithm use script context specification calculations, and add tests for it. Also included is a cleanup for the tests to avoid mutable globals, as I found it hard to reason about what exactly was being tested. ACKs for top commit: achow101: ACK b22810887b3840ad0fcb424ea7e16d2d195767d9 darosior: ACK b22810887b3840ad0fcb424ea7e16d2d195767d9 Tree-SHA512: e4bda7376628f3e91cfc74917cefc554ca16eb5f2a0e1adddc33eb8717c4aaa071e56a40f85a2041ae74ec445a7bd0129bba48994c203e0e6e4d25af65954d9e
2023-10-16Merge bitcoin/bitcoin#28539: lib: add taproot support to libconsensusAndrew Chow
ff8e2fc2e2416f6f3b84cdb40db8ac168596b579 fuzz: add coverage for `bitcoinconsensus_verify_script_with_spent_outputs` (brunoerg) c5f2a757d736f14d27ac5256a9df887cd2f174f1 docs: add release notes for #28539 (brunoerg) de54882348502d860cf1e504100aa8fb1e52aa88 docs: add docs for additional libconsensus functions (Jake Rawsthorne) 70106e0689546fee497814c63a6a4747e0937b36 docs: link to rust-bitcoinconsensus (Jake Rawsthorne) fb0db07e414fec3318b3af683167ebef9c82fc84 lib: add Taproot support to libconsensus (Jake Rawsthorne) Pull request description: Grabbed from #21158. Closes #21133. ACKs for top commit: achow101: ACK ff8e2fc2e2416f6f3b84cdb40db8ac168596b579 theStack: ACK ff8e2fc2e2416f6f3b84cdb40db8ac168596b579 darosior: re-ACK ff8e2fc2e2416f6f3b84cdb40db8ac168596b579 Tree-SHA512: bf6f500c7e8c9ff6884137c2cd9b4522c586e52848dd639b774b94d998b0516b877498d24f3a6cc7425aedf81d18b0d30c1ccf19e2d527fdfdfa3955ca49b6e7
2023-10-16Merge bitcoin/bitcoin#28583: refactor: [tidy] modernize-use-emplacefanquake
fa05a726c225dc65dee79367bb67f099ae4f99e6 tidy: modernize-use-emplace (MarcoFalke) Pull request description: Constructing a temporary unnamed object only to copy or move it into a container seems both verbose in code and a strict performance penalty. Fix both issues via the `modernize-use-emplace` tidy check. ACKs for top commit: Sjors: re-utACK fa05a726c2 hebasto: ACK fa05a726c225dc65dee79367bb67f099ae4f99e6. TheCharlatan: ACK fa05a726c225dc65dee79367bb67f099ae4f99e6 Tree-SHA512: 4408a094f406e7bf6c1468c2b0798f68f4d952a1253cf5b20bdc648ad7eea4a2c070051fed46d66fd37bce2ce6f85962484a1d32826b7ab8c9baba431eaa2765
2023-10-13test: add tests for miniscript GetWitnessSizePieter Wuille
2023-10-13test: remove mutable global contexts in miniscript fuzzer/testPieter Wuille
2023-10-13fuzz: add coverage for `bitcoinconsensus_verify_script_with_spent_outputs`brunoerg
Co-authored-by: Antonie Poinsot <darosior@protonmail.com>
2023-10-12ci: Drop no longer needed `NOLINTNEXTLINE`Hennadii Stepanov
2023-10-12tidy: modernize-use-emplaceMarcoFalke
2023-10-08fuzz: miniscript: higher sensitivity for max stack size limit under TapscriptAntoine Poinsot
In order to exacerbate a mistake in the stack size tracking logic, sometimes pad the witness to make the script execute at the brink of the stack size limit. This way if the stack size is underestimated for a script it would immediately fail `VerifyScript`.
2023-10-08fuzz: adapt Miniscript targets to TapscriptAntoine Poinsot
We introduce another global that dictates the script context under which to operate when running the target. For miniscript_script, just consume another byte to set the context. This should only affect existing seeds to the extent they contain a CHECKMULTISIG. However it would not invalidate them entirely as they may contain a NUMEQUAL or a CHECKSIGADD, and this still exercises a bit of the parser. For miniscript_string, reduce the string size by one byte and use the last byte to determine the context. This is the change that i think would invalidate the lowest number of existing seeds. For miniscript_stable, we don't want to invalidate any seed. Instead of creating a new miniscript_stable_tapscript, simply run the target once for P2WSH and once for Tapscript (with the same seed). For miniscript_smart, consume one byte before generating a pseudo-random node to set the context. We have less regard for seed stability for this target anyways.
2023-10-08miniscript: account for keys as being 32 bytes under Taproot contextAntoine Poinsot
2023-10-08miniscript: make 'd:' have the 'u' property under Tapscript contextAntoine Poinsot
In Tapscript MINIMALIF is a consensus rule, so we can rely on the fact that the `DUP IF [X] ENDIF` will always put an exact 1 on the stack upon satisfaction.
2023-10-08miniscript: introduce a multi_a fragmentAntoine Poinsot
It is the equivalent of multi() but for Tapscript, using CHECKSIGADD instead of CHECKMULTISIG. It shares the same properties as multi() but for 'n', since a threshold multi_a() may have an empty vector as the top element of its satisfaction. It could also have the 'o' property when it only has a single key, but in this case a 'pk()' is always preferable anyways.
2023-10-08miniscript: store the script context within the Node structureAntoine Poinsot
Some checks will be different depending on the script context (for instance the maximum script size).
2023-10-08miniscript: introduce a MsContext() helper to contextsAntoine Poinsot
We are going to introduce Tapscript support in Miniscript, for which some of Miniscript rules and properties change (new or modified fragments, different typing rules, different resources consumption, ..).
2023-10-06chain: Rename HaveTxsDownloaded to HaveNumChainTxsFabian Jahr
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2023-10-05fuzz: ConsumeNetAddr(): avoid IPv6 addresses that look like CJDNSVasil Dimov
The fuzz testing framework runs as if `-cjdnsreachable` is set and in this case addresses like `{net=IPv6, addr=fc...}` are not possible.
2023-10-05Merge bitcoin/bitcoin#28558: Make PeerManager own a FastRandomContextfanquake
4cafe9f176e93ebb6c38abb12140e8d8be005cbf [test] Make PeerManager's rng deterministic in tests (dergoegge) fecec3e1c661ba273470ecc5ef12d4c070b53050 [net processing] FeeFilterRounder doesn't own a FastRandomContext (dergoegge) 47520ed209d9341702a0fb6006bee6f63f7da42e [net processing] Make fee filter rounder non-global (dergoegge) 77506f4ac6b3a3d7396a3a6101345019e05b3b10 [net processing] Addr shuffle uses PeerManager's rng (dergoegge) a648dd79e5ebfdb627d0221b1207862efb664dfc [net processing] PushAddress uses PeerManager's rng (dergoegge) 87c706713e5d1c78bad943a42bf7c69047d28ea5 [net processing] PeerManager holds a FastRandomContext (dergoegge) Pull request description: This lets us avoid some non-determinism in tests (also see #28537). ACKs for top commit: MarcoFalke: re-ACK 4cafe9f176e93ebb6c38abb12140e8d8be005cbf 🕗 glozow: concept && light code review ACK 4cafe9f176e93ebb6c38abb12140e8d8be005cbf Tree-SHA512: 3c18700773d0bc547ccb6442c41567e6f26b0b50fab5b79620da417ec91b9c0ae1395d15258da3aa4a91447b8ce560145dd135e39fbbd0610749e528e665b111
2023-10-04[net processing] FeeFilterRounder doesn't own a FastRandomContextdergoegge
2023-10-03Merge bitcoin/bitcoin#28523: rpc: add hidden getrawaddrman RPC to list ↵Andrew Chow
addrman table entries 352d5eb2a9e89cff4a2815d94a9d81fcc20c4b2c test: getrawaddrman RPC (0xb10c) da384a286bd84a97e7ebe7a64654c5be20ab2df1 rpc: getrawaddrman for addrman entries (0xb10c) Pull request description: Inspired by `getaddrmaninfo` (#27511), this adds a hidden/test-only `getrawaddrman` RPC. The RPC returns information on all addresses in the address manager new and tried tables. Addrman table contents can be used in tests and during development. The RPC result encodes the `bucket` and `position`, the internal location of addresses in the tables, in the address object's string key. This allows users to choose to consume or to ignore the location information. If the internals of the address manager implementation change, the location encoding might change too. ``` getrawaddrman EXPERIMENTAL warning: this call may be changed in future releases. Returns information on all address manager entries for the new and tried tables. Result: { (json object) "table" : { (json object) buckets with addresses in the address manager table ( new, tried ) "bucket/position" : { (json object) the location in the address manager table (<bucket>/<position>) "address" : "str", (string) The address of the node "port" : n, (numeric) The port number of the node "network" : "str", (string) The network (ipv4, ipv6, onion, i2p, cjdns) of the address "services" : n, (numeric) The services offered by the node "time" : xxx, (numeric) The UNIX epoch time when the node was last seen "source" : "str", (string) The address that relayed the address to us "source_network" : "str" (string) The network (ipv4, ipv6, onion, i2p, cjdns) of the source address }, ... }, ... } Examples: > bitcoin-cli getrawaddrman > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "getrawaddrman", "params": []}' -H 'content-type: text/plain;' http://127.0.0.1:8332/ ``` ACKs for top commit: willcl-ark: reACK 352d5eb2a9 amitiuttarwar: reACK 352d5eb2a9e stratospher: reACK 352d5eb. achow101: ACK 352d5eb2a9e89cff4a2815d94a9d81fcc20c4b2c Tree-SHA512: cc462666b5c709617c66b0e3e9a17c4c81e9e295f91bdd9572492d1cb6466fc9b6d48ee805ebe82f9f16010798370effe5c8f4db15065b8c7c0d8637675d615e
2023-10-03Merge bitcoin/bitcoin#26312: Remove Sock::Get() and Sock::Sock()Ryan Ofsky
7df450836969b81e98322c9a09c08b35d1095a25 test: improve sock_tests/move_assignment (Vasil Dimov) 5086a99b84367a45706af7197da1016dd966e6d9 net: remove Sock default constructor, it's not necessary (Vasil Dimov) 7829272f7826511241defd34954e6040ea963f07 net: remove now unnecessary Sock::Get() (Vasil Dimov) 944b21b70ae490a5a746bcc1810a5074d74e9d34 net: don't check if the socket is valid in ConnectSocketDirectly() (Vasil Dimov) aeac68d036e3cff57ce155f1a904d77f98b357d4 net: don't check if the socket is valid in GetBindAddress() (Vasil Dimov) 5ac1a51ee5a57da59f1ff1986b7d9054484d3c80 i2p: avoid using Sock::Get() for checking for a valid socket (Vasil Dimov) Pull request description: _This is a piece of #21878, chopped off to ease review._ Peeking at the underlying socket file descriptor of `Sock` and checkig if it is `INVALID_SOCKET` is bad encapsulation and stands in the way of testing/mocking/fuzzing. Instead use an empty `unique_ptr` to denote that there is no valid socket where appropriate or outright remove such checks where they are not necessary. The default constructor `Sock::Sock()` is unnecessary now after recent changes, thus remove it. ACKs for top commit: ajtowns: ACK 7df450836969b81e98322c9a09c08b35d1095a25 jonatack: ACK 7df450836969b81e98322c9a09c08b35d1095a25 Tree-SHA512: 9742aeeeabe8690530bf74caa6ba296787028c52f4a3342afd193b05dbbb1f6645935c33ba0a5230199a09af01c666bd3c7fb16b48692a0d185356ea59a8ddbf
2023-10-02net: expose transport types/session IDs of connections in RPC and logsPieter Wuille
Co-authored-by: Dhruv Mehta <856960+dhruv@users.noreply.github.com>
2023-10-02rpc: addnode arg to use BIP324 v2 p2pdhruv
Co-authored-by: Pieter Wuille <bitcoin-dev@wuille.net>
2023-10-02Merge bitcoin/bitcoin#27596: assumeutxo (2)Andrew Chow
edbed31066e3674ba52b8c093ab235625527f383 chainparams: add signet assumeutxo param at height 160_000 (Sjors Provoost) b8cafe38713cbf10d15459042f7f911bcc1b1e4e chainparams: add testnet assumeutxo param at height 2_500_000 (Sjors Provoost) 99839bbfa7110c7abf22e587ae2f72c9c57d3c85 doc: add note about confusing HaveTxsDownloaded name (James O'Beirne) 7ee46a755f1d57ce9d51975d3b54dc9ac3d08d52 contrib: add script to demo/test assumeutxo (James O'Beirne) 42cae39356fd20d521aaf99aff1ed85856f3c9f3 test: add feature_assumeutxo functional test (James O'Beirne) 0f64bac6030334d798ae205cd7af4bf248feddd9 rpc: add getchainstates (James O'Beirne) bb0585779472962f40d9cdd9c6532132850d371c refuse to activate a UTXO snapshot if mempool not empty (James O'Beirne) ce585a9a158476b0ad3296477b922e79f308e795 rpc: add loadtxoutset (James O'Beirne) 62ac519e718eb7a31dca1102a96ba219fbc7f95d validation: do not activate snapshot if behind active chain (James O'Beirne) 9511fb3616b7bbe1d0d2f54a45ea0a650ba0367b validation: assumeutxo: swap m_mempool on snapshot activation (James O'Beirne) 7fcd21544a333ffdf1910b65c573579860be6a36 blockstorage: segment normal/assumedvalid blockfiles (James O'Beirne) 4c3b8ca35c2e4a441264749bb312df2bd054b5b8 validation: populate nChainTx value for assumedvalid chainstates (James O'Beirne) 49ef778158c43859946a592e11ec34fe1b93a5b6 test: adjust chainstate tests to use recognized snapshot base (James O'Beirne) 1019c399825b0d512c1fd751c376d46fed4992b9 validation: pruning for multiple chainstates (James O'Beirne) 373cf91531b84bfdd06fdf8abf4dca228029ce6b validation: indexing changes for assumeutxo (James O'Beirne) 1fffdd76a1bca908f55d73b64983655b14cf7432 net_processing: validationinterface: ignore some events for bg chain (James O'Beirne) fbe0a7d7ca680358237b6c2369b3fd2b43221113 wallet: validationinterface: only handle active chain notifications (James O'Beirne) f073917a9e7ba423643dcae0339776470b628f65 validationinterface: only send zmq notifications for active (James O'Beirne) 4d8f4dcb450d31e4847804e62bf91545b949fa14 validation: pass ChainstateRole for validationinterface calls (James O'Beirne) 1e59acdf17309f567c370885f0cf02605e2baa58 validation: only call UpdatedBlockTip for active chainstate (James O'Beirne) c6af23c5179cc383f8e6c275373af8d11e6a989f validation: add ChainstateRole (James O'Beirne) 9f2318c76cc6986d48e13831cf5bd8dab194fdf4 validation: MaybeRebalanceCaches when chain leaves IBD (James O'Beirne) 434495a8c1496ca23fe35b84499f3daf668d76b8 chainparams: add blockhash to AssumeutxoData (James O'Beirne) c711ca186f8d8a28810be0beedcb615ddcf93163 assumeutxo: remove snapshot during -reindex{-chainstate} (James O'Beirne) c93ef43e4fd4fbc1263cdc9e98ae5856830fe89e bugfix: correct is_snapshot_cs in VerifyDB (James O'Beirne) b73d3bbd23220857bf17cbb6401275bf58013b72 net_processing: Request assumeutxo background chain blocks (Suhas Daftuar) Pull request description: - Background and FAQ: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal - Prior progress/project: https://github.com/bitcoin/bitcoin/projects/11 - Replaces https://github.com/bitcoin/bitcoin/pull/15606, which was closed due to Github slowness. Original description and commentary can be found there. --- This changeset finishes the first phase of the assumeutxo project. It makes UTXO snapshots loadable via RPC (`loadtxoutset`) and adds `assumeutxo` parameters to chainparams. It contains all the remaining changes necessary to both use an assumedvalid snapshot chainstate and do a full validation sync in the background. This may look like a lot to review, but note that - ~200 lines are a (non-essential) demo shell script - Many lines are functional test, documentation, and relatively dilute RPC code. So it shouldn't be as burdensome to review as the linecount might suggest. - **P2P**: minor changes are made to `init.cpp` and `net_processing.cpp` to make simultaneous IBD across multiple chainstates work. - **Pruning**: implement correct pruning behavior when using a background chainstate - **Blockfile separation**: to prevent "fragmentation" in blockfile storage, have background chainstates use separate blockfiles from active snapshot chainstates to avoid interleaving heights and impairing pruning. - **Indexing**: some `CValidationInterface` events are given with an additional parameter, ChainstateRole, and all indexers ignore events from ChainstateRole::ASSUMEDVALID so that indexation only happens sequentially. - Have `-reindex` properly wipe snapshot chainstates. - **RPC**: introduce RPC commands `loadtxoutset` and (hidden) `getchainstates`. - **Release docs & first assumeutxo commitment**: add notes and a particular assumeutxo hash value for first AU-enabled release. - This will complete the project and allow use of UTXO snapshots for faster node bootstrap. The next phase, if it were to be pursued, would be coming up with a way to distribute the UTXO snapshots over the P2P network. --- ### UTXO snapshots Create your own with `./contrib/devtools/utxo_snapshot.sh`, e.g. ```shell ./contrib/devtools/utxo_snapshot.sh 788000 utxo.dat ./src/bitcoin-cli -datadir=$(pwd)/testdata`) ``` or use the pre-generated ones listed below. - Testnet: **2'500'000** (Sjors): - torrent: `magnet:?xt=urn:btih:511e09f4bf853aefab00de5c070b1e031f0ecbe9&dn=utxo-testnet-2500000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969` - sha256: `79db4b025448cc0ac388d8589a28eab02de53055d181e34eb47391717aa16388` - Signet: **160'000** (Sjors): - torrent: `magnet:?xt=urn:btih:9da986cb27b3980ea7fd06b21e199b148d486880&dn=utxo-signet-160000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969` - sha256: `eeeca845385ba91e84ef58c09d38f98f246a24feadaad57fe1e5874f3f92ef8c` - Mainnet: **800'000** (Sjors): - Note: this needs the following commit cherry-picked in: https://github.com/Sjors/bitcoin/commit/24deb2022b822f22fba9fcbee201e37a83225eb2 - torrent: `magnet:?xt=urn:btih:50ee955bef37f5ec3e5b0df4cf0288af3d715a2e&dn=utxo-800000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969` ### Testing #### For fun (~5min) If you want to do a quick test, you can run `./contrib/devtools/test_utxo_snapshots.sh` and follow the instructions. This is mostly obviated by the functional tests, though. #### For real (longer) If you'd like to experience a real usage of assumeutxo, you can do that too. I've cut a new snapshot at height 788'000 (http://img.jameso.be/utxo-788000.dat - but you can do it yourself with `./contrib/devtools/utxo_snapshot.sh` if you want). Download that, and then create a datadir for testing: ```sh $ cd ~/src/bitcoin # or whatever # get the snapshot $ curl http://img.jameso.be/utxo-788000.dat > utxo-788000.dat # you'll want to do this if you like copy/pasting $ export AU_DATADIR=/home/${USER}/au-test # or wherever $ mkdir ${AU_DATADIR} $ vim ${AU_DATADIR}/bitcoin.conf dbcache=8000 # or, you know, something high blockfilterindex=1 coinstatsindex=1 prune=3000 logthreadnames=1 ``` Obtain this branch, build it, and then start bitcoind: ```sh $ git remote add jamesob https://github.com/jamesob/bitcoin $ git fetch jamesob assumeutxo $ git checkout jamesob/assumeutxo $ ./configure $conf_args && make # (whatever you like to do here) # start 'er up and watch the logs $ ./src/bitcoind -datadir=${AU_DATADIR} ``` Then, in some other window, load the snapshot ```sh $ ./src/bitcoin-cli -datadir=${AU_DATADIR} loadtxoutset $(pwd)/utxo-788000.dat ``` You'll see some log messages about headers retrieval and waiting to see the snapshot in the headers chain. Once you get the full headers chain, you'll spend a decent amount of time (~10min) loading the snapshot, checking it, and flushing it to disk. After all that happens, you should be syncing to tip in pretty short order, and you'll see the occasional `[background validation]` log message go by. In yet another window, you can check out chainstate status with ```sh $ ./src/bitcoin-cli -datadir=${AU_DATADIR} getchainstates ``` as well as usual favorites like `getblockchaininfo`. ACKs for top commit: achow101: ACK edbed31066e3674ba52b8c093ab235625527f383 Tree-SHA512: 6086fb9a38dc7df85fedc76b30084dd8154617a2a91e89a84fb41326d34ef8e7d7ea593107afba01369093bf8cc91770621d98f0ea42a5b3b99db868d2f14dc2
2023-10-02rpc: getrawaddrman for addrman entries0xb10c
Exposing address manager table entries in a hidden RPC allows to introspect addrman tables in tests and during development. As response JSON object the following FORMAT1 is choosen: { "table": { "<bucket>/<position>": { "address": "..", "port": .., ... }, "<bucket>/<position>": { "address": "..", "port": .., ... }, "<bucket>/<position>": { "address": "..", "port": .., ... }, ... } } An alternative would be FORMAT2 { "table": { "bucket": { "position": { "address": "..", "port": .., ... }, "position": { "address": "..", "port": .., ... }, .. }, "bucket": { "position": { "address": "..", "port": .., ... }, .. }, } } FORMAT1 and FORMAT2 have different encodings for the location of the address in the address manager. While FORMAT2 might be easier to process for downstream tools, it also mimics internal addrman mappings, which might change at some point. Users not interested in the address location can ignore the location key. They don't have to adapt to a new RPC response format, when the internal addrman layout changes. Additionally, FORMAT1 is also slightly easier to to iterate in downstream tools. The RPC response-building implemenation complexcity is lower with FORMAT1 as we can more easily build a "<bucket>/<position>" key than a multiple "bucket" objects with multiple "position" objects (FORMAT2).
2023-10-02Merge bitcoin/bitcoin#28508: refactor: Remove SER_GETHASH, hard-code client ↵fanquake
version in CKeyPool serialize fac29a0ab19fda457b55d7a0a37c5cd3d9680f82 Remove SER_GETHASH, hard-code client version in CKeyPool serialize (MarcoFalke) fa72f09d6ff8ee204f331a69d3f5e825223c9e11 Remove CHashWriter type (MarcoFalke) fa4a9c0f4334678fb80358ead667807bf2a0a153 Remove unused GetType() from OverrideStream, CVectorWriter, SpanReader (MarcoFalke) Pull request description: Removes a bunch of redundant, dead or duplicate code. Uses the idea from and finishes the idea https://github.com/bitcoin/bitcoin/pull/28428 by theuni ACKs for top commit: ajtowns: ACK fac29a0ab19fda457b55d7a0a37c5cd3d9680f82 kevkevinpal: added one nit but otherwise ACK [fac29a0](https://github.com/bitcoin/bitcoin/pull/28508/commits/fac29a0ab19fda457b55d7a0a37c5cd3d9680f82) Tree-SHA512: cc805e2f38e73869a6691fdb5da09fa48524506b87fc93f05d32c336ad3033425a2d7608e317decd3141fde3f084403b8de280396c0c39132336fe0f7510af9e
2023-09-30rpc: add getchainstatesJames O'Beirne
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-09-30rpc: add loadtxoutsetJames O'Beirne
Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
2023-09-28Merge bitcoin/bitcoin#28450: Add package evaluation fuzzerglozow
262ab8ef7860d43cebc9d04721e3a075b4edf06e Add package evaluation fuzzer (Greg Sanders) Pull request description: This fuzzer target caught the issue in https://github.com/bitcoin/bitcoin/pull/28251 within 5 minutes on master branch, and an additional issue which I've applied a preliminary patch to cover. Fuzzer target does the following: 1) Picks mempool confgs, including max package size, count, mempool size, etc 2) Generates 1 to 26 transactions with arbitrary coins/fees, the first N-1 spending only confirmed outpoints 3) Nth transaction, if >1, sweeps all unconfirmed outpoints in mempool 4) If N==1, it may submit it through single-tx submission path, to allow for more interesting topologies 5) Otherwise submits through package submission interface 6) Repeat 1-5 a few hundred times per mempool instance In other words, it ends up building chains of txns in the mempool using parents-and-children packages, which is currently the topology supported on master. The test itself is a direct rip of tx_pool.cpp, with a number of assertions removed because they were failing for unknown reasons, likely due to the notification changes of single tx submission to package, which is used to track addition/removal of transactions in the test. I'll continue working on re-adding these assertions for further invariant testing. ACKs for top commit: murchandamus: ACK 262ab8ef7860d43cebc9d04721e3a075b4edf06e glozow: reACK 262ab8ef7860d43cebc9d04721e3a075b4edf06e dergoegge: tACK 262ab8ef7860d43cebc9d04721e3a075b4edf06e Tree-SHA512: 190784777d0f2361b051b3271db8f79b7927e3cab88596d2c30e556da721510bd17f6cc96f6bb03403bbf0589ad3f799fa54e63c1b2bd92a2084485b5e3e96a5
2023-09-27Add package evaluation fuzzerGreg Sanders
2023-09-26Merge bitcoin/bitcoin#28483: refactor: Return CAutoFile from ↵fanquake
BlockManager::Open*File() fa56c421be04af846f479c30749b17e6663ab418 Return CAutoFile from BlockManager::Open*File() (MarcoFalke) 9999b89cd37fb2a23c5ebcd91d9cb31d69375933 Make BufferedFile to be a CAutoFile wrapper (MarcoFalke) fa389d902fbf5fac19fba8c5d0c5e0a25f15ca63 refactor: Drop unused fclose() from BufferedFile (MarcoFalke) Pull request description: This is required for https://github.com/bitcoin/bitcoin/pull/28052, but makes sense on its own, because offloading logic to `CAutoFile` instead of re-implementing it allows to delete code and complexity. ACKs for top commit: TheCharlatan: Re-ACK fa56c421be04af846f479c30749b17e6663ab418 willcl-ark: tACK fa56c421be Tree-SHA512: fe4638f3a6bd3f9d968cfb9ae3259c9d6cd278fe2912cbc90289851311c8c781099db4c160e775960975c4739098d9af801a8d2d12603f371f8edfe134d8f85a
2023-09-20Merge bitcoin/bitcoin#27511: rpc: Add test-only RPC getaddrmaninfo for ↵Andrew Chow
new/tried table address count 28bac81a346c0b68273fa73af924f7096cb3f41d test: add functional test for getaddrmaninfo (stratospher) c8eb8dae51039aa1938e7040001a149210e87275 rpc: Introduce getaddrmaninfo for count of addresses stored in new/tried table (stratospher) Pull request description: implements https://github.com/bitcoin/bitcoin/issues/26907. split off from #26988 to keep RPC, CLI discussions separate. This PR introduces a new RPC `getaddrmaninfo`which returns the count of addresses in the new/tried table of a node's addrman broken down by network type. This would be useful for users who want to see the distribution of addresses from different networks across new/tried table in the addrman. ```jsx $ getaddrmaninfo Result: { (json object) json object with network type as keys "network" : { (json object) The network (ipv4, ipv6, onion, i2p, cjdns) "new" : n, (numeric) number of addresses in new table "tried" : n, (numeric) number of addresses in tried table "total" : n (numeric) total number of addresses in both new/tried tables from a network }, ... } ``` ### additional context from [original PR](https://github.com/bitcoin/bitcoin/pull/26988) 1. network coverage tests were skipped because there’s a small chance that addresses from different networks could hash to the same bucket and cause count of different network addresses in the tests to fail. see https://github.com/bitcoin/bitcoin/pull/26988#discussion_r1137596851. 2. #26988 uses this RPC in -addrinfo CLI. Slight preference for keeping the RPC hidden since this info will mostly be useful to only super users. see https://github.com/bitcoin/bitcoin/pull/26988#discussion_r1173964808. ACKs for top commit: 0xB10C: ACK 28bac81a346c0b68273fa73af924f7096cb3f41d willcl-ark: reACK 28bac81a346c0b68273fa73af924f7096cb3f41d achow101: ACK 28bac81a346c0b68273fa73af924f7096cb3f41d brunoerg: reACK 28bac81a346c0b68273fa73af924f7096cb3f41d theStack: Code-review ACK 28bac81a346c0b68273fa73af924f7096cb3f41d Tree-SHA512: 346390167e1ebed7ca5c79328ea452633736aff8b7feefea77460e04d4489059334ae78a3f757f32f5fb7827b309d7186bebab3c3760b3dfb016d564a647371a
2023-09-20Merge bitcoin/bitcoin#28470: fuzz: Rework addr fuzzingfanquake
fad52baf1e9bf9d55a300922e73d3bc3169a8843 fuzz: Rework addr fuzzing (MarcoFalke) fa5b6d29ee90911271d4304a6f39c38743a84f33 fuzz: Drop unused params from serialize helpers (MarcoFalke) Pull request description: Some minor fixups to addr fuzzing ACKs for top commit: dergoegge: utACK fad52baf1e9bf9d55a300922e73d3bc3169a8843 Tree-SHA512: 6a2b07fb1a65cf855d5e7c0a52bfcb81d46dbc5d4b3e72cef359987cbd28dbfeb2fc54f210e9737cb131b40ac5f88a90e9af284e441e0b37196121590bbaf015
2023-09-19Merge bitcoin/bitcoin#28246: wallet: Use CTxDestination in CRecipient ↵fanquake
instead of just scriptPubKey ad0c469d98c51931b98b7fd937c6ac3eeaed024e wallet: Use CTxDestination in CRecipient rather than scriptPubKey (Andrew Chow) 07d3bdf4ebc06825ea24ab6f7c87aef6a22238c6 Add PubKeyDestination for P2PK scripts (Andrew Chow) 1a98a51c666e9ae77364115775ec2e0ba984e8e0 Allow CNoDestination to represent a raw script (Andrew Chow) 8dd067088d41f021b357d7db5fa5f0a9f61edddc Make WitnessUnknown members private (Andrew Chow) Pull request description: For silent payments, we want to provide a `SilentPaymentsDestination` to be used as the recipient, which requires `CRecipient` to use something other than just the `scriptPubKey` as we cannot know the output script for a silent payment prior to transaction creation. `CTxDestination` seems like the obvious place to add a `SilentPaymentsDestination` as it is our internal representation of an address. In order to still allow paying to arbitrary scriptPubKeys (e.g. for data carrier outputs, or the user hand crafted a raw transaction that they have given to `fundrawtransaction`), `CNoDestination` is changed to contain raw scripts. Additionally, P2PK scripts are now interpreted as a new `PubKeyDestination` rather than `PKHash`. This results in some things that would have given an address for P2PK scripts to no longer do so. This is arguably more correct. `ExtractDestination`'s behavior is slightly changed for the above. It now returns `true` for those destinations that have addresses, so P2PK scripts now result in `false`. Even though it returns false for `CNoDestination`, the script will now be included in that `CNoDestination`. Builds on #28244 ACKs for top commit: josibake: ACK https://github.com/bitcoin/bitcoin/pull/28246/commits/ad0c469d98c51931b98b7fd937c6ac3eeaed024e Tree-SHA512: ef3f8f3c7284779d9806c77c85b21caf910a79a1f7e7f1b51abcc0d7e074f14e00abf30f625a13075e41d94dad6202c10ddff462c0ee74c2ca4aab585b145a52
2023-09-19Remove unused GetType() from OverrideStream, CVectorWriter, SpanReaderMarcoFalke
GetType() is never called, so it is completely unused and can be removed.
2023-09-19fuzz: Add missing PROVIDE_FUZZ_MAIN_FUNCTION guard to __AFL_FUZZ_INITMarcoFalke
2023-09-19rpc: Introduce getaddrmaninfo for count of addresses stored in new/tried tablestratospher
2023-09-15Merge bitcoin/bitcoin#28473: refactor: Serialization parameter cleanupsfanquake
fb6a2ab63e310d8b600352ef41aab6dafccfbff0 scripted-diff: use SER_PARAMS_OPFUNC (Anthony Towns) 5e5c8f86b60a8018e8801fb44bbe56ce97d9deef serialize: add SER_PARAMS_OPFUNC (Anthony Towns) 33203f59b482bddfe0bbe7d497cb8731ce8334a4 serialize: specify type for ParamsWrapper not ref (Anthony Towns) bf147bfffa1afb11721f30e83eec1fa829f64d5f serialize: move ser_action functions out of global namespace (Anthony Towns) Pull request description: Cleanups after #25284: * ser_action namespacing - https://github.com/bitcoin/bitcoin/pull/25284#discussion_r1316189977 * make reference implicit - https://github.com/bitcoin/bitcoin/pull/25284#discussion_r1316277030 * function notation - https://github.com/bitcoin/bitcoin/pull/25284#issuecomment-1710714821 ACKs for top commit: MarcoFalke: lgtm ACK fb6a2ab63e310d8b600352ef41aab6dafccfbff0 💨 TheCharlatan: ACK fb6a2ab63e310d8b600352ef41aab6dafccfbff0 Tree-SHA512: aacca2ee9cfec360ade6b394606e13d1dfe05bc29c5fbdd48a4e6992bd420312d4ed0d32218d95c560646af326e9977728dc2e759990636298e326947f6f9526
2023-09-15Make BufferedFile to be a CAutoFile wrapperMarcoFalke
This refactor allows to forward some calls to the underlying CAutoFile, instead of re-implementing the logic in the buffered file.
2023-09-15refactor: Drop unused fclose() from BufferedFileMarcoFalke
This was only explicitly used in the tests, where it can be replaced by wrapping the original raw file pointer into a CAutoFile on creation and then calling CAutoFile::fclose(). Also, it was used in LoadExternalBlockFile(), where it can also be replaced by the (implicit call to the) CAutoFile destructor after wrapping the original raw file pointer in a CAutoFile.
2023-09-14[fuzz] Don't use afl++ deferred forkserver modedergoegge
Deferring the forkserver initialization doesn't make sense for some of our targets since they involve state that can't be forked (e.g. threads). We therefore remove the use of __AFL_INIT entirely. We also increase the __AFL_LOOP count to 100000. Our fuzz targets are meant to all be deterministic and stateless therefore this should be fine.
2023-09-14Merge bitcoin/bitcoin#28460: fuzz: Use afl++ shared-memory fuzzingfanquake
97e2e1d641016cd7b74848b9560e3771f092c1ea [fuzz] Use afl++ shared-memory fuzzing (dergoegge) Pull request description: Using shared-memory is faster than reading from stdin, see https://github.com/AFLplusplus/AFLplusplus/blob/7d2122e0596132f9344a5d0896020ebc79cd33db/instrumentation/README.persistent_mode.md ACKs for top commit: MarcoFalke: review ACK 97e2e1d641016cd7b74848b9560e3771f092c1ea Tree-SHA512: 7e71b5f84835e41531c19ee959be2426da245869757de8e5dd1c730ae83ead650e2ef75f4d594d7965f661821a4ffbd27be84d3ce623702991501b34a8d02fc3
2023-09-14Merge bitcoin/bitcoin#28423: kernel: Remove protocol.h/netaddress.h/compat.h ↵fanquake
from kernel headers d5067651991f3e6daf456ba13c7036ddc4545352 [refactor] Remove compat.h from kernel headers (TheCharlatan) 36193af47c8dcff53e59498c416b85b59e0d0f91 [refactor] Remove netaddress.h from kernel headers (TheCharlatan) 2b08c55f01996e0b05763f05eac50b83ba9d5a8e [refactor] Add CChainParams member to CConnman (TheCharlatan) f0d1d8b35c3aa9f2f923f74e3dbbf1e5ece4cd2f [refactor] Add missing includes for next commit (TheCharlatan) 534b314a7401d44f51aabd4565f97be9ee411740 kernel: Move MessageStartChars to its own file (TheCharlatan) 9be330b654cfbd792620295f3867f592059d6a7a [refactor] Define MessageStartChars as std::array (TheCharlatan) 37e2b011136ca1cf00dfb9e575d12f0d035a6a2c [refactor] Allow std::array<std::byte, N> in serialize.h (MarcoFalke) Pull request description: This removes the non-consensus critical `protocol.h` and `netaddress.h` headers from the kernel headers. With this patch, they are no longer required to include in order to use the libbitcoinkernel library. This also allows for the removal of the `compat.h` header from the kernel headers. As an added future benefit it also reduces the number of of kernel headers that include the platform specific `bitcoin-config.h`. For those interested, the currently required kernel headers can be inspected visually with the [sourcetrail](https://github.com/CoatiSoftware/Sourcetrail) tool by looking at the required includes of `bitcoin-chainstate.cpp`. --- This is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587), namely its stage 1 step 3: Decouple most non-consensus headers from libbitcoinkernel. ACKs for top commit: stickies-v: re-ACK d506765 hebasto: ACK d5067651991f3e6daf456ba13c7036ddc4545352. ajtowns: utACK d5067651991f3e6daf456ba13c7036ddc4545352 MarcoFalke: lgtm ACK d5067651991f3e6daf456ba13c7036ddc4545352 🍛 Tree-SHA512: 6f90ea510a302c2927e84d16900e89997c39b8ff3ce9d4effeb8a134bd29cc52bd9e81e51aaa11f7496bad00025b78a58b88c5a9e0bb3f4ebbe9a76309215fb7
2023-09-14scripted-diff: use SER_PARAMS_OPFUNCAnthony Towns
-BEGIN VERIFY SCRIPT- sed -i 's/WithParams(\(CAddress::V[12]_[A-Z]*\) *, */\1(/g' $(git grep -l 'WithParams' src/) sed -i 's/WithParams(\(CNetAddr::V[12]\) *, */\1(/g' $(git grep -l 'WithParams' src/) sed -i 's@\(CNetAddr::V1.CService{}.*\) //@\1 //@' src/test/util/net.cpp -END VERIFY SCRIPT-
2023-09-13fuzz: Rework addr fuzzingMarcoFalke
* Replace ConsumeDeserializationParams with V1, because V2 is unconditionally checked as well. * Also fuzz CAddress::Format::Disk in the address_deserialize fuzz target.
2023-09-13fuzz: Drop unused params from serialize helpersMarcoFalke
With the ser-type and ser-version going away, it seems unlikely that there is need for them in the future, so just remove them.
generated by cgit v1.2.3 (git 2.26.2) at 2025-03-17 04:41:57 +0000