aboutsummaryrefslogtreecommitdiff
path: root/src/Makefile.test.include
AgeCommit message (Collapse)Author
2024-04-01Merge bitcoin/bitcoin#29648: Remove libbitcoinconsensusfanquake
80f8b92f4f2311b9e9a25361c9dd973244e6f95c remove libbitcoinconsensus (fanquake) Pull request description: This was deprecated in `v27.0`, for removal in `v28.0`. See discussion in PR #29189. ACKs for top commit: theuni: Concept ACK and light review ACK 80f8b92f4f2311b9e9a25361c9dd973244e6f95c. My only hesitation here is that (afaics?) there's now nothing keeping undesired features like threading or globals from working their way into the interpreter in future commits. m3dwards: Concept ACK https://github.com/bitcoin/bitcoin/pull/29648/commits/80f8b92f4f2311b9e9a25361c9dd973244e6f95c TheCharlatan: ACK 80f8b92f4f2311b9e9a25361c9dd973244e6f95c hebasto: ACK 80f8b92f4f2311b9e9a25361c9dd973244e6f95c, I have reviewed the code and it looks OK. Tree-SHA512: 17a62118aeb088f2695c892bb32794dfea3061e3cb7d9e8e9f1c06c3ff6f63a7587fa532e37edbb91fbc5a19b12c9a0f8e05fa9e8864aa07f92665375d847e80
2024-03-18remove libbitcoinconsensusfanquake
This was deprecated in v27.0, for removal in v28.0. See discussion in PR #29189.
2024-03-18Add fuzz test for FeeFracGreg Sanders
2024-03-18fuzz: fuzz diagram creation and comparisonGreg Sanders
Co-authored-by: Suhas Daftuar <sdaftuar@chaincode.com> Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2024-03-18Add FeeFrac unit testsGreg Sanders
Co-authored-by: Suhas Daftuar <sdaftuar@chaincode.com>
2024-02-03test: Add makefile target for running unit testsTheCharlatan
make check runs a bunch of other subtree tests that exercise code that is hardly ever changed and have a comparatively long runtime. There seems to be no target for running just the unit tests, so add one.
2024-01-31Merge bitcoin/bitcoin#28170: p2p: adaptive connections services flagsAva Chow
27f260aa6e04f82dad78e9a06d58927546143a27 net: remove now unused global 'g_initial_block_download_completed' (furszy) aff7d92b1500e2478ce36a7e86ae47df47dda178 test: add coverage for peerman adaptive connections service flags (furszy) 6ed53602ac7c565273b5722de167cb2569a0e381 net: peer manager, dynamically adjust desirable services flag (furszy) 9f36e591c551ec2e58a6496334541bfdae8fdfe5 net: move state dependent peer services flags (furszy) f9ac96b8d6f4eba23c88f302b22a2c676e351263 net: decouple state independent service flags from desirable ones (furszy) 97df4e38879d2644aeec34c1eef241fed627333e net: store best block tip time inside PeerManager (furszy) Pull request description: Derived from #28120 discussion. By relocating the peer desirable services flags into the peer manager, we allow the connections acceptance process to handle post-IBD potential stalling scenarios. The peer manager will be able to dynamically adjust the services flags based on the node's proximity to the tip (back and forth). Allowing the node to recover from the following post-IBD scenario: Suppose the node has successfully synced the chain, but later experienced dropped connections and remained inactive for a duration longer than the limited peers threshold (the timeframe within which limited peers can provide blocks). In such cases, upon reconnecting to the network, the node might only establish connections with limited peers, filling up all available outbound slots. Resulting in an inability to synchronize the chain (because limited peers will not provide blocks older than the `NODE_NETWORK_LIMITED_MIN_BLOCKS` threshold). ACKs for top commit: achow101: ACK 27f260aa6e04f82dad78e9a06d58927546143a27 vasild: ACK 27f260aa6e04f82dad78e9a06d58927546143a27 naumenkogs: ACK 27f260aa6e04f82dad78e9a06d58927546143a27 mzumsande: Light Code Review ACK 27f260aa6e04f82dad78e9a06d58927546143a27 andrewtoth: ACK 27f260aa6e04f82dad78e9a06d58927546143a27 Tree-SHA512: 07befb9bcd0b60a4e7c45e4429c02e7b6c66244f0910f4b2ad97c9b98258b6f46c914660a717b5ed4ef4814d0dbfae6e18e6559fe9bec7d0fbc2034109200953
2024-01-23Merge bitcoin/bitcoin#28921: multiprocess: Add basic type conversion hooksAva Chow
6acec6b9ff02b91de132bb1575d75908a8a2d27b multiprocess: Add type conversion code for UniValue types (Ryan Ofsky) 0cc74fce72e0c79849109ee5d7afe707991b3512 multiprocess: Add type conversion code for serializable types (Ryan Ofsky) 4aaee239211a5287fbc361c0eb158b105ae8c8db test: add ipc test to test multiprocess type conversion code (Ryan Ofsky) Pull request description: Add type conversion hooks to allow `UniValue` objects, and objects that have `CDataStream` `Serialize` and `Unserialize` methods to be used as arguments and return values in Cap'nProto interface methods. Also add unit test to verify the hooks are working and data can be round-tripped correctly. The non-test code in this PR was previously part of #10102 and has been split off for easier review, but the test code is new. --- This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722). ACKs for top commit: achow101: ACK 6acec6b9ff02b91de132bb1575d75908a8a2d27b dergoegge: reACK 6acec6b9ff02b91de132bb1575d75908a8a2d27b Tree-SHA512: 5d2cbc5215d488b876d34420adf91205dabf09b736183dcc85aa86255e3804c2bac5bab6792dacd585ef99a1d92cf29c8afb3eb65e4d953abc7ffe41994340c6
2024-01-23test: add coverage for peerman adaptive connections service flagsfurszy
2024-01-09Revert "build: Fix undefined reference to __mulodi4"MarcoFalke
This reverts commit e4c8bb62e4a6873c45f42d0d2a24927cb241a0ea.
2023-11-28test: add ipc test to test multiprocess type conversion codeRyan Ofsky
Add unit test to test IPC method calls and type conversion between bitcoin c++ types and capnproto messages. Right now there are custom type hooks in bitcoin IPC code, so the test is simple, but in upcoming commits, code will be added to convert bitcoin types to capnproto messages, and the test will be expanded.
2023-11-20fuzz: add target for `DescriptorScriptPubKeyMan`brunoerg
2023-11-13Merge bitcoin/bitcoin#28721: multiprocess compatibility updatesfanquake
3b70f7b6156cb110c47a6e482791cf337bb6ad6d doc: fix broken doc/design/multiprocess.md links after #24352 (Ryan Ofsky) 6d43aad742c7ea28303cf2799528188938e7ce32 span: Make Span template deduction guides work in SFINAE context (Ryan Ofsky) 8062c3bdb9dd3062597ed8299e99151b612d32b7 util: Add ArgsManager SetConfigFilePath method (Ryan Ofsky) 441d00c60f0a67889d23f8556190ff99dde488bc interfaces: Rename CalculateBumpFees methods to be compatible with capn'proto (Ryan Ofsky) 156f49d682ef025fb942c997a6c5475e18eef9cf interfaces: Change getUnspentOutput return type to avoid multiprocess segfault (Ryan Ofsky) 4978754c0058bbdfbcd492f25fa49ef211e11d6e interfaces: Add schedulerMockForward method so mockscheduler RPC can work across processes (Ryan Ofsky) 924327eaf3ada45a603e80aa4a3ab38a0f8c8673 interfaces: Fix const virtual method that breaks multiprocess support (Ryan Ofsky) 82a379eca8251c736b4de6e7a2516582641ce397 streams: Add SpanReader ignore method (Russell Yanofsky) Pull request description: This is a collection of small changes to interfaces and code which were needed as part of multiprocess PR #10102, but have been moved here to make that PR smaller. All of these changes are refactoring changes which do not affect behavior of current code --- This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722). ACKs for top commit: achow101: ACK 3b70f7b6156cb110c47a6e482791cf337bb6ad6d naumenkogs: ACK 3b70f7b6156cb110c47a6e482791cf337bb6ad6d maflcko: re-ACK 3b70f7b6156cb110c47a6e482791cf337bb6ad6d 🎆 Tree-SHA512: 2368772b887056ad8a9f84c299cfde76ba45943770e3b5353130580900afa9611302195b899ced7b6e303b11f053ff204cae7c28ff4e12c55562fcc81119ba4c
2023-11-08Merge bitcoin/bitcoin#28155: net: improves addnode / m_added_nodes logicglozow
0420f99f429ce2382057e101859067f40de47be0 Create net_peer_connection unit tests (Jon Atack) 4b834f649921aceb44d3e0b5a2ffd7847903f9f7 Allow unit tests to access additional CConnman members (Jon Atack) 34b9ef443bc2655a85c8802edc5d5d48d792a286 net/rpc: Makes CConnman::GetAddedNodeInfo able to return only non-connected address on request (Sergi Delgado Segura) 94e8882d820969ddc83f24f4cbe1515a886da4ea rpc: Prevents adding the same ip more than once when formatted differently (Sergi Delgado Segura) 2574b7e177ef045e64f1dd48cb000640ff5103d3 net/rpc: Check all resolved addresses in ConnectNode rather than just one (Sergi Delgado Segura) Pull request description: ## Rationale Currently, `addnode` has a couple of corner cases that allow it to either connect to the same peer more than once, hence wasting outbound connection slots, or add redundant information to `m_added_nodes`, hence making Bitcoin iterate through useless data on a regular basis. ### Connecting to the same node more than once In general, connecting to the same node more than once is something we should try to prevent. Currently, this is possible via `addnode` in two different ways: 1. Calling `addnode` more than once in a short time period, using two equivalent but distinct addresses 2. Calling `addnode add` using an IP, and `addnode onetry` after with an address that resolved to the same IP For the former, the issue boils down to `CConnman::ThreadOpenAddedConnections` calling `CConnman::GetAddedNodeInfo` once, and iterating over the result to open connections (`CConman::OpenNetworkConnection`) on the same loop for all addresses.`CConnman::ConnectNode` only checks a single address, at random, when resolving from a hostname, and uses it to check whether we are already connected to it. An example to test this would be calling: ``` bitcoin-cli addnode "127.0.0.1:port" add bitcoin-cli addnode "localhost:port" add ``` And check how it allows us to perform both connections some times, and some times it fails. The latter boils down to the same issue, but takes advantage of `onetry` bypassing the `CConnman::ThreadOpenAddedConnections` logic and calling `CConnman::OpenNetworkConnection` straightaway. A way to test this would be: ``` bitcoin-cli addnode "127.0.0.1:port" add bitcoin-cli addnode "localhost:port" onetry ``` ### Adding the same peer with two different, yet equivalent, addresses The current implementation of `addnode` is pretty naive when checking what data is added to `m_added_nodes`. Given the collection stores strings, the checks at `CConnman::AddNode()` basically check wether the exact provided string is already in the collection. If so, the data is rejected, otherwise, it is accepted. However, ips can be formatted in several ways that would bypass those checks. Two examples would be `127.0.0.1` being equal to `127.1` and `[::1]` being equal to `[0:0:0:0:0:0:0:1]`. Adding any pair of these will be allowed by the rpc command, and both will be reported as connected by `getaddednodeinfo`, given they map to the same `CService`. This is less severe than the previous issue, since even tough both nodes are reported as connected by `getaddednodeinfo`, there is only a single connection to them (as properly reported by `getpeerinfo`). However, this adds redundant data to `m_added_nodes`, which is undesirable. ### Parametrize `CConnman::GetAddedNodeInfo` Finally, this PR also parametrizes `CConnman::GetAddedNodeInfo` so it returns either all added nodes info, or only info about the nodes we are **not** connected to. This method is used both for `rpc`, in `getaddednodeinfo`, in which we are reporting all data to the user, so the former applies, and to check what nodes we are not connected to, in `CConnman::ThreadOpenAddedConnections`, in which we are currently returning more data than needed and then actively filtering using `CService.fConnected()` ACKs for top commit: jonatack: re-ACK 0420f99f429ce2382057e101859067f40de47be0 kashifs: > > tACK [0420f9](https://github.com/bitcoin/bitcoin/commit/0420f99f429ce2382057e101859067f40de47be0) sr-gi: > > > tACK [0420f9](https://github.com/bitcoin/bitcoin/commit/0420f99f429ce2382057e101859067f40de47be0) mzumsande: Tested ACK 0420f99f429ce2382057e101859067f40de47be0 Tree-SHA512: a3a10e748c12d98d439dfb193c75bc8d9486717cda5f41560f5c0ace1baef523d001d5e7eabac9fa466a9159a30bb925cc1327c2d6c4efb89dcaf54e176d1752
2023-11-02Merge bitcoin/bitcoin#28530: tests, bug fix: DisconnectedBlockTransactions ↵glozow
rewrite followups 9b3da70bd06b45482e7211aa95637a72bd115553 [test] DisconnectedBlockTransactions::DynamicMemoryUsage (glozow) b2d04479647af64ad7cf5ebfb6175251b2f6b72e bugfix: correct DisconnectedBlockTransactions memory usage (stickies-v) f4254e209801d6a790b5f0c251c0b32154a4e3cc assume duplicate transactions are not added to `iters_by_txid` (ismaelsadeeq) 29eb219c1247993378fce06c8f71aab20736c237 move only: move implementation code to disconnected_transactions.cpp (ismaelsadeeq) 81dfeddea70ae5feeaf79062585c2ff9f33c0ca3 refactor: update `MAX_DISCONNECTED_TX_POOL` from kb to bytes (ismaelsadeeq) Pull request description: This PR is a follow-up to fix review comments and a bugfix from #28385 The PR - Updated `DisconnectedBlockTransactions`'s `MAX_DISCONNECTED_TX_POOL` from kb to bytes. - Moved `DisconnectedBlockTransactions` implementation code to `kernel/disconnected_transactions.cpp`. - `AddTransactionsFromBlock` now assume duplicate transactions are not passed by asserting after inserting each transaction to `iters_by_txid`. - Included a Bug fix: In the current master we are underestimating the memory usage of `DisconnectedBlockTransactions`. * When adding and subtracting `cachedInnerUsage` we call `RecursiveDynamicUsage` with `CTransaction` which invokes this [`RecursiveDynamicUsage(const CTransaction& tx)`](https://github.com/bitcoin/bitcoin/blob/6e721c923c87abdb8d99674093d08d8c17bc52c2/src/core_memusage.h#L32) version of `RecursiveDynamicUsage`, the output of that call only account for the memory usage of the inputs and outputs of the `CTransaction`, this omits the memory usage of the `CTransaction` object and the control block. * This PR fixes this bug by calling `RecursiveDynamicUsage` with `CTransactionRef` when adding and subtracting `cachedInnerUsage` which invokes [`RecursiveDynamicUsage(const std::shared_ptr<X>& p)`](https://github.com/bitcoin/bitcoin/blob/6e721c923c87abdb8d99674093d08d8c17bc52c2/src/core_memusage.h#L67) version of `RecursiveDynamicUsage` the output of the calculation accounts for the` CTransaction` object, the control blocks, inputs and outputs memory usage. * see [comment ](https://github.com/bitcoin/bitcoin/pull/28385#discussion_r1322948452) - Added test for DisconnectedBlockTransactions memory limit. ACKs for top commit: stickies-v: ACK 9b3da70bd06b45482e7211aa95637a72bd115553 - nice work! BrandonOdiwuor: re ACK 9b3da70bd06b45482e7211aa95637a72bd115553 glozow: ACK 9b3da70bd06b45482e7211aa95637a72bd115553 Tree-SHA512: 69b9595d09f4d0209038f97081d790cea92ccf63efb94e9e372749979fcbe527f7f17a8e454720cedd12021be0c8e11cf99874625d3dafd9ec602b12dbeb4098
2023-10-31Create net_peer_connection unit testsJon Atack
for initial partial unit test coverage of these CConnman class methods: - AddNode() - ConnectNode() - GetAddedNodeInfo() - AlreadyConnectedToAddress() - ThreadOpenAddedConnections() and of the GetAddedNodeInfo() call in RPC addnode.
2023-10-20span: Make Span template deduction guides work in SFINAE contextRyan Ofsky
Also add test to make sure this doesn't get broken in the future. This was breaking vector<bool> serialization in multiprocess code because template current deduction guides would make it appear like vector<bool> could be converted to a span, but then the actual conversion to span would fail.
2023-10-20[fuzz] Delete i2p targetdergoegge
2023-10-19[test] DisconnectedBlockTransactions::DynamicMemoryUsageglozow
2023-09-27Add package evaluation fuzzerGreg Sanders
2023-08-10Merge bitcoin/bitcoin#28008: BIP324 ciphersuitefanquake
1c7582ead6e1119899922041c1af2b4169b0bc74 tests: add decryption test to bip324_tests (Pieter Wuille) 990f0f8da92a2d11828a7c05ca93bf0520b2a95e Add BIP324Cipher, encapsulating key agreement, derivation, and stream/AEAD ciphers (Pieter Wuille) c91cedf281e5207fb5fd2ca81feec9760f7c2ed0 crypto: support split plaintext in ChaCha20Poly1305 Encrypt/Decrypt (Pieter Wuille) af2b44c76e5de8ce880381e5535ead37ab0b3ba9 bench: add benchmark for FSChaCha20Poly1305 (Pieter Wuille) aa8cee93342ee857931afec9af3ff5dbd8ce7749 crypto: add FSChaCha20Poly1305, rekeying wrapper around ChaCha20Poly1305 (Pieter Wuille) 0fee267792eb8cbdd48ad78f1712420b5d8d905b crypto: add FSChaCha20, a rekeying wrapper around ChaCha20 (Pieter Wuille) 9ff0768bdcca06836ccc673eacfa648e801930cb crypto: add the ChaCha20Poly1305 AEAD as specified in RFC8439 (Pieter Wuille) 9fd085a1a49d317abcaf1492b71c48bf1a1b3007 crypto: remove outdated variant of ChaCha20Poly1305 AEAD (Pieter Wuille) Pull request description: Depends on #27985 and #27993, based on and partially replaces #25361, part of #27634. Draft while dependencies are not merged. This adds implementations of: * The ChaCha20Poly1305 AEAD from [RFC8439 section 2.8](https://datatracker.ietf.org/doc/html/rfc8439#section-2.8), including test vectors. * The FSChaCha20 stream cipher as specified in [BIP324](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#rekeying-wrappers-fschacha20poly1305-and-fschacha20), a rekeying wrapper around ChaCha20. * The FSChaCha20Poly1305 AEAD as specified in [BIP324](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#rekeying-wrappers-fschacha20poly1305-and-fschacha20), a rekeying wrapper around ChaCha20Poly1305. * A BIP324Cipher class that encapsulates key agreement, key derivation, and stream ciphers and AEADs for [BIP324 packet encoding](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#overall-packet-encryption-and-decryption-pseudocode). The ChaCha20Poly1305 and FSChaCha20Poly1305 implementations are new, taking advance of the improvements in #27993. ACKs for top commit: jamesob: reACK 1c7582e theStack: ACK 1c7582ead6e1119899922041c1af2b4169b0bc74 stratospher: tested ACK 1c7582e. Tree-SHA512: 06728b4b95b21c5b732ed08faf40e94d0583f9d86ff4db3b92dd519dcd9fbfa0f310bc66ef1e59c9e49dd844ba8c5ac06e2001762a804fb5aa97027816045a46
2023-07-27Remove unused raw-pointer read helper from univalueMarcoFalke
2023-07-26Add BIP324Cipher, encapsulating key agreement, derivation, and stream/AEAD ↵Pieter Wuille
ciphers Co-authored-by: dhruv <856960+dhruv@users.noreply.github.com>
2023-07-26crypto: remove outdated variant of ChaCha20Poly1305 AEADPieter Wuille
Remove the variant of ChaCha20Poly1305 AEAD that was previously added in anticipation of BIP324 using it. BIP324 was updated to instead use rekeying wrappers around otherwise unmodified versions of the ChaCha20 stream cipher and the ChaCha20Poly1305 AEAD as specified in RFC8439.
2023-06-17fuzz: wallet, add target for CoinControlAyush Singh
2023-06-14fuzz: wallet, add target for `fees`brunoerg
2023-05-19Merge bitcoin/bitcoin#27021: Implement Mini version of BlockAssembler to ↵glozow
calculate mining scores 6b605b91c1faf2c7f7cc0c9d39b4fcfd66dc2965 [fuzz] Add MiniMiner target + diff fuzz against BlockAssembler (glozow) 3f3f2d59ea2946a7b7cc8cb0222fb602d62645d0 [unit test] GatherClusters and MiniMiner unit tests (glozow) 59afcc83548ea67a863dac7b75d000bc8f6a7023 Implement Mini version of BlockAssembler to calculate mining scores (glozow) 56484f0fdc44261e723563f59df886d5acdd851f [mempool] find connected mempool entries with GatherClusters(…) (glozow) Pull request description: Implement Mini version of BlockAssembler to calculate mining scores Run the mining algorithm on a subset of the mempool, only disturbing the mempool to copy out fee information for relevant entries. Intended to be used by wallet to calculate amounts needed for fee-bumping unconfirmed transactions. From comments of sipa and glozow below: > > In what way does the code added here differ from the real block assembly code? > > * Only operates on the relevant transactions rather than full mempool > * Has the ability to remove transactions that will be replaced so they don't impact their ancestors > * Does not hold mempool lock outside of the constructor, makes copies of the entries it needs instead (though I'm not sure if this has an effect in practice) > * Doesn't do the sanity checks like keeping weight within max block weight and `IsFinalTx()` > * After the block template is built, additionally calculates fees to bump remaining ancestor packages to target feerate ACKs for top commit: achow101: ACK 6b605b91c1faf2c7f7cc0c9d39b4fcfd66dc2965 Xekyo: > ACK [6b605b9](https://github.com/bitcoin/bitcoin/commit/6b605b91c1faf2c7f7cc0c9d39b4fcfd66dc2965) modulo `miniminer_overlap` test. furszy: ACK 6b605b91 modulo `miniminer_overlap` test. theStack: Code-review ACK 6b605b91c1faf2c7f7cc0c9d39b4fcfd66dc2965 Tree-SHA512: f86a8b4ae0506858a7b15d90f417ebceea5038b395c05c825e3796123ad3b6cb8a98ebb948521316802a4c6d60ebd7041093356b1e2c2922a06b3b96b3b8acb6
2023-05-16Merge bitcoin/bitcoin#27041: Build: Improve handling of suppressed logging ↵fanquake
in Makefiles 1b1ffbd014b931afb9435ec10911b9a7c130d3e5 Build: Log when test -f fails in Makefile (TheCharlatan) 541012e621386cd824eed81295206a34ba3ba497 Build: Use AM_V_GEN in Makefiles where appropriate (TheCharlatan) Pull request description: This PR triages some behavior around Makefile recipe echoing suppression. When generating new files as part of the Makefile the recipe is sometimes suppressed with $(AM_V_GEN) and sometimes with `@`. We should prefer $(AM_V_GEN), since this also prints the lines in silent mode. This is arguably more in style with the current recipe echoing. Before: `Generated test/data/script_tests.json.h` Now: ` GEN test/data/script_tests.json.h` A side effect of this change is that the recipe for generating build.h is now echoed on each make run. Arguably this makes its generation more transparent. Sometimes the error emitted by `test -f` is currently thrown without any logging. This makes it a bit harder to debug. Instead, print a helpful log message to point the developer in the right direction. Alternatively this could have been implemented by just removing the recipe echo suppression (@), but the subsequent make output became too noisy. ACKs for top commit: fanquake: ACK 1b1ffbd014b931afb9435ec10911b9a7c130d3e5 Tree-SHA512: e31869fab25e72802b692ce6735f9561912caea903c1577101b64c9cb115c98de01a59300e8ffe7b05b998345c1b64a79226231d7d1453236ac338c62dc9fbb3
2023-05-05fuzz: BIP 42, BIP 30, CVE-2018-17144MarcoFalke
2023-04-20Merge bitcoin/bitcoin#25325: Add pool based memory resourceAndrew Chow
9f947fc3d4b779f017332135323b34e8f216f613 Use PoolAllocator for CCoinsMap (Martin Leitner-Ankerl) 5e4ac5abf54f8e6d6330df0c73119aa0cca4c103 Call ReallocateCache() on each Flush() (Martin Leitner-Ankerl) 1afca6b663bb54022afff193fd9d83856606b189 Add PoolResource fuzzer (Martin Leitner-Ankerl) e19943f049ed8aa4f32a1d8440a9fbf160367f0f Calculate memory usage correctly for unordered_maps that use PoolAllocator (Martin Leitner-Ankerl) b8401c3281978beed6198b2f9782b6a8dd35cbd7 Add pool based memory resource & allocator (Martin Leitner-Ankerl) Pull request description: A memory resource similar to `std::pmr::unsynchronized_pool_resource`, but optimized for node-based containers. The goal is to be able to cache more coins with the same memory usage, and allocate/deallocate faster. This is a reimplementation of #22702. The goal was to implement it in a way that is simpler to review & test * There is now a generic `PoolResource` for allocating/deallocating memory. This has practically the same API as `std::pmr::memory_resource`. (Unfortunately I cannot use std::pmr because libc++ simply doesn't implement that API). * Thanks to sipa there is now a fuzzer for PoolResource! On a fast machine I ran it for ~770 million executions without finding any issue. * The estimation of the correct node size is now gone, PoolResource now has multiple pools and just needs to be created large enough to have space for the unordered_map nodes. I ran benchmarks with #22702, mergebase, and this PR. Frequency locked Intel i7-8700, clang++ 13.0.1 to reindex up to block 690000. ```sh bitcoind -dbcache=5000 -assumevalid=00000000000000000002a23d6df20eecec15b21d32c75833cce28f113de888b7 -reindex-chainstate -printtoconsole=0 -stopatheight=690000 ``` The performance is practically identical with #22702, just 0.4% slower. It's ~21% faster than master: ![Progress in Million Transactions over Time(2)](https://user-images.githubusercontent.com/14386/173288685-91952ade-f304-4825-8bfb-0725a71ca17b.png) ![Size of Cache in MiB over Time](https://user-images.githubusercontent.com/14386/173291421-e6b410be-ac77-479b-ad24-5fafcebf81eb.png) Note that on cache drops mergebase's memory doesnt go so far down because it does not free the `CCoinsMap` bucket array. ![Size of Cache in Million tx over Time(1)](https://user-images.githubusercontent.com/14386/173288703-a80c9c9e-93c8-4a16-9df8-610c89c61cc4.png) ACKs for top commit: LarryRuane: ACK 9f947fc3d4b779f017332135323b34e8f216f613 achow101: re-ACK 9f947fc3d4b779f017332135323b34e8f216f613 john-moffett: ACK 9f947fc3d4b779f017332135323b34e8f216f613 jonatack: re-ACK 9f947fc3d4b779f017332135323b34e8f216f613 Tree-SHA512: 48caf57d1775875a612b54388ef64c53952cd48741cacfe20d89049f2fb35301b5c28e69264b7d659a3ca33d4c714d47bafad6fd547c4075f08b45acc87c0f45
2023-04-11Merge bitcoin/bitcoin#26662: fuzz: Add HeadersSyncState targetfanquake
3153e7d779ac284f86e433af033d63f13f361b6f [fuzz] Add HeadersSyncState target (dergoegge) 53552affca381cdb5103ecdbcc7f3fb562e66ac4 [headerssync] Make m_commit_offset protected (dergoegge) Pull request description: This adds a fuzz target for the `HeadersSyncState` class. I am unsure how well this is able to cover the logic since it is just processing unserialized CBlockHeaders straight from the fuzz input (headers are sometimes made continuous). However, it does manage to get to the redownload phase so i thought it is better then not having fuzzing at all. It would also be nice to fuzz the p2p logic that is using `HeadersSyncState` (e.g. `TryLowWorkHeadersSync`, `IsContinuationOfLowWorkHeadersSync`) but that likely requires some more work (refactoring👻). ACKs for top commit: mzumsande: ACK 3153e7d779ac284f86e433af033d63f13f361b6f Tree-SHA512: 8a4630ceeeb30e4eeabaa8eb5491d98f0bf900efe7cda07384eaac9f2afaccfbcaa979cc1cc7f0b6ca297a8f5c17a7759f94809dd87eb87d35348d847c83e8ab
2023-04-06[fuzz] Add MiniMiner target + diff fuzz against BlockAssemblerglozow
Co-authored-by: dergoegge <n.goeggi@gmail.com> Co-authored-by: mzumsande <mzumsande@gmail.com> Co-authored-by: Murch <murch@murch.one>
2023-04-06[unit test] GatherClusters and MiniMiner unit testsglozow
Co-authored-by: Murch <murch@murch.one> Co-authored-by: theStack <sebastian.falbesoner@gmail.com> Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
2023-03-23Add PoolResource fuzzerMartin Leitner-Ankerl
Fuzzes PoolResource with random allocations/deallocations, and multiple asserts. Co-Authored-By: Pieter Wuille <pieter@wuille.net>
2023-03-23Add pool based memory resource & allocatorMartin Leitner-Ankerl
A memory resource similar to std::pmr::unsynchronized_pool_resource, but optimized for node-based containers. Co-Authored-By: Pieter Wuille <pieter@wuille.net>
2023-03-07Merge bitcoin/bitcoin#27150: Deduplicate bitcoind and bitcoin-qt init codeAndrew Chow
802cc1ef536e11944608fe9ab782d3e962037703 Deduplicate bitcoind and bitcoin-qt init code (Ryan Ofsky) d172b5c6718b69200c8ad211fe709860081bd692 Add InitError(error, details) overload (Ryan Ofsky) 3db2874bd71d2391747b7385cabcbfef67218c4c Extend bilingual_str support for tinyformat (Ryan Ofsky) c361df90b9fd34e50bbf1db43b866930e5498357 scripted-diff: Remove double newlines after some init errors (Ryan Ofsky) Pull request description: Add common InitConfig function to deduplicate bitcoind and bitcoin-qt code reading config files and creating the datadir. Noticed the duplicate code while reviewing #27073 and want to remove it because difference in bitcoin-qt and bitcoind behavior make it hard to evaluate changes like #27073 There are a few minor changes in behavior: - In bitcoin-qt, when there is a problem reading the configuration file, the GUI error text has changed from "Error: Cannot parse configuration file:" to "Error reading configuration file:" to be consistent with bitcoind. - In bitcoind, when there is a problem reading the settings.json file, the error text has changed from "Failed loading settings file" to "Settings file could not be read" to be consistent with bitcoin-qt. - In bitcoind, when there is a problem writing the settings.json file, the error text has changed from "Failed saving settings file" to "Settings file could not be written" to be consistent with bitcoin-qt. - In bitcoin-qt, if there datadir is not accessible (e.g. no permission to read), there is an normal error dialog showing "Error: filesystem error: status: Permission denied [.../settings.json]", instead of an uncaught exception. ACKs for top commit: Sjors: Light review ACK 802cc1ef536e11944608fe9ab782d3e962037703 TheCharlatan: ACK 802cc1ef536e11944608fe9ab782d3e962037703 achow101: ACK 802cc1ef536e11944608fe9ab782d3e962037703 Tree-SHA512: 9c78d277e9ed595fa8ce286b97d2806e1ec06ddbbe7bd3434bd9dd7b456faf8d989f71231e97311f36edb9caaec645a50c730bd7514b8e0fe6e6f7741b13d981
2023-03-06test: wallet, add coverage for outputs grouping processfurszy
The following scenarios are covered: 1) 10 UTXO with the same script: partial spends is enabled --> outputs must not be grouped. 2) 10 UTXO with the same script: partial spends disabled --> outputs must be grouped. 3) 20 UTXO, 10 one from scriptA + 10 from scriptB: a) if partial spends is enabled --> outputs must not be grouped. b) if partial spends is not enabled --> 2 output groups expected (one per script). 3) Try to add a negative output (value - fee < 0): a) if "positive_only" is enabled --> negative output must be skipped. b) if "positive_only" is disabled --> negative output must be added. 4) Try to add a non-eligible UTXO (due not fulfilling the min depth target for "not mine" UTXOs) --> it must not be added to any group 5) Try to add a non-eligible UTXO (due not fulfilling the min depth target for "mine" UTXOs) --> it must not be added to any group 6) Surpass the 'OUTPUT_GROUP_MAX_ENTRIES' size and verify that a second partial group gets created.
2023-02-28Extend bilingual_str support for tinyformatRyan Ofsky
Previous bilingual_str tinyformat::format accepted bilingual format strings, but not bilingual arguments. Extend it to accept both. This is useful when embedding one translated string inside another translated string, for example: `strprintf(_("Error: %s"), message)` which would fail previously if `message` was a bilingual_str.
2023-02-15Merge bitcoin/bitcoin#26153: Reduce wasted pseudorandom bytes in ChaCha20 + ↵fanquake
various improvements 511aa4f1c7508f15cab8d7e58007900ad6fd3d5d Add unit test for ChaCha20's new caching (Pieter Wuille) fb243d25f754da8f01793b41e2d225b917f3e5d7 Improve test vectors for ChaCha20 (Pieter Wuille) 93aee8bbdad808b7009279b67470d496cc26b936 Inline ChaCha20 32-byte specific constants (Pieter Wuille) 62ec713961ade7b58e90c905395558a41e8a59f0 Only support 32-byte keys in ChaCha20{,Aligned} (Pieter Wuille) f21994a02e1cc46d41995581b54222abc655be93 Use ChaCha20Aligned in MuHash3072 code (Pieter Wuille) 5d16f757639e2cc6e81db6e07bc1d5dd74abca6c Use ChaCha20 caching in FastRandomContext (Pieter Wuille) 38eaece67b1bc37b2f502348c5d7537480a34346 Add fuzz test for testing that ChaCha20 works as a stream (Pieter Wuille) 5f05b27841af0bed1b6e7de5f46ffe33e5919e4d Add xoroshiro128++ PRNG (Martin Leitner-Ankerl) 12ff72476ac0dbf8add736ad3fb5fad2eeab156c Make unrestricted ChaCha20 cipher not waste keystream bytes (Pieter Wuille) 6babf402130a8f3ef3058594750aeaa50b8f5044 Rename ChaCha20::Seek -> Seek64 to clarify multiple of 64 (Pieter Wuille) e37bcaa0a6dbb334ab6e817efcb609ccee6edc39 Split ChaCha20 into aligned/unaligned variants (Pieter Wuille) Pull request description: This is an alternative to #25354 (by my benchmarking, somewhat faster), subsumes #25712, and adds additional test vectors. It separates the multiple-of-64-bytes-only "core" logic (which becomes simpler) from a layer around which performs caching/slicing to support arbitrary byte amounts. Both have their uses (in particular, the MuHash3072 code can benefit from multiple-of-64-bytes assumptions), plus the separation results in more readable code. Also, since FastRandomContext effectively had its own (more naive) caching on top of ChaCha20, that can be dropped in favor of ChaCha20's new built-in caching. I thought about rebasing #25712 on top of this, but the changes before are fairly extensive, so redid it instead. ACKs for top commit: ajtowns: ut reACK 511aa4f1c7508f15cab8d7e58007900ad6fd3d5d dhruv: tACK crACK 511aa4f1c7 Tree-SHA512: 3aa80971322a93e780c75a8d35bd39da3a9ea570fbae4491eaf0c45242f5f670a24a592c50ad870d5fd09b9f88ec06e274e8aa3cefd9561d623c63f7198cf2c7
2023-02-03Build: Use AM_V_GEN in Makefiles where appropriateTheCharlatan
When generating new files as part of the Makefile the recipe is sometimes suppressed with $(AM_V_GEN) and sometimes with `@`. We should prefer $(AM_V_GEN), since this also prints the lines in silent mode. This is arguably more in style with the current recipe echoing. Before: Generated test/data/script_tests.json.h Now: GEN test/data/script_tests.json.h A side effect of this change is that the recipe for generating build.h is now echoed on each make run. Arguably this makes its generation more transparent.
2023-02-01Add simulation-based CCoinsViewCache fuzzerPieter Wuille
The fuzzer goes through a sequence of operations that get applied to both a real stack of CCoinsViewCache objects, and to simulation data, comparing the two at the end.
2023-01-30Add xoroshiro128++ PRNGMartin Leitner-Ankerl
Xoroshiro128++ is a fast non-cryptographic random generator. Reference implementation is available at https://prng.di.unimi.it/ Co-Authored-By: Pieter Wuille <pieter@wuille.net>
2023-01-23[fuzz] Add PartiallyDownloadedBlock targetdergoegge
2023-01-03Merge bitcoin/bitcoin#25789: test: clean and extend availablecoins_tests ↵Andrew Chow
coverage 9622fe64b8785430c71d4abc8637075026dc690c test: move coins result test to wallet_tests.cpp (furszy) f69347d0588647ff9a4e986c7be987827a0417f4 test: extend and simplify availablecoins_tests (furszy) 212ccdf2c2b70d973b18ae78f0158ec5f0c3bbb4 wallet: AvailableCoins, add arg to include/skip locked coins (furszy) Pull request description: Negative PR with extended test coverage :). 1) Cleaned duplicated code and added coverage for the 'AvailableCoins' incremental result. 2) The class `AvailableCoinsTestingSetup` inside `availablecoins_tests.cpp` is a plain copy of `ListCoinsTestingSetup` that is inside `wallet_tests.cpp`. So, deleted the file and moved the `BasicOutputTypesTest` test case to `wallet_tests.cpp`. 3) Added arg to include/skip locked coins from the `AvailableCoins` result. This is needed for point (1) as otherwise the wallet will spend the coins that we recently created due its closeness to the recipient amount. Note: this last point comes from #25659 where I'm using the same functionality to clean/speedup another flow as well. ACKs for top commit: achow101: ACK 9622fe64b8785430c71d4abc8637075026dc690c theStack: ACK 9622fe64b8785430c71d4abc8637075026dc690c aureleoules: reACK 9622fe64b8785430c71d4abc8637075026dc690c, nice cleanup! Tree-SHA512: 1ed9133120bfe8815455d1ad317bb0ff96e11a0cc34ee8098716ab9b001749168fa649212b2fa14b330c1686cb1f29039ff1f88ae306db68881b0428c038f388
2022-12-29Merge bitcoin/bitcoin#26481: bench: Suppress output when running with ↵MarcoFalke
`-sanity-check` option f1e89597c803001ab9d5afd7e173184fe6886d1d test: Drop no longer required bench output redirection (Hennadii Stepanov) 4dbcdf26a301f54c60c85ceab1aaa4dae43f6aeb bench: Suppress output when running with `-sanity-check` option (Hennadii Stepanov) Pull request description: This change allows to simplify CI tests, and makes it easier to integrate the `bench_bitcoin` binary into CMake custom [targets](https://cmake.org/cmake/help/latest/command/add_custom_target.html) or [commands](https://cmake.org/cmake/help/latest/command/add_custom_command.html), as `COMMAND` does not support output redirection. ACKs for top commit: aureleoules: tACK f1e89597c803001ab9d5afd7e173184fe6886d1d. Ran as expected and is more practical than using an output redirection. Tree-SHA512: 29086d428cccedcfd031c0b4514213cbc1670e35f955e8fd35cee212bc6f9616cf9f20d0cb984495390c4ae2c50788ace616aea907d44e0d6a905b9dda1685d8
2022-12-14test: move coins result test to wallet_tests.cppfurszy
The class `AvailableCoinsTestingSetup` inside `availablecoins_tests.cpp` is a plain copy of `ListCoinsTestingSetup` that is inside wallet_tests.cpp.
2022-12-12[fuzz] Add HeadersSyncState targetdergoegge
2022-11-21refactor: unify test/util/wallet.h with wallet/test/util.hfurszy
files share the same purpose, and we shouldn't have wallet code inside the test directory. This later is needed to use wallet util functions in the bench and test binaries without be forced to duplicate them.
2022-11-14test: Split overly large util_tests.cpp fileMacroFake
2022-11-10test: Drop no longer required bench output redirectionHennadii Stepanov
generated by cgit v1.2.3 (git 2.26.2) at 2024-09-13 12:14:59 +0000