aboutsummaryrefslogtreecommitdiff
path: root/src/test
AgeCommit message (Collapse)Author
2023-05-20kernel: Add notification interfaceTheCharlatan
This commit is part of the libbitcoinkernel project and seeks to remove the ChainstateManager's and, more generally, the kernel library's dependency on interface_ui with options methods in this and the following few commits. By removing interface_ui from the kernel library, its dependency on boost is reduced to just boost::multi_index. Define a new kernel notification class with virtual methods for notifying about internal kernel events. Create a new file in the node library for defining a function creating the default set of notification methods such that these do not need to be re-defined all over the codebase. As a first step, add a `blockTip` method, wrapping `uiInterface.NotifyBlockTip`.
2023-05-19Merge bitcoin/bitcoin#27021: Implement Mini version of BlockAssembler to ↵glozow
calculate mining scores 6b605b91c1faf2c7f7cc0c9d39b4fcfd66dc2965 [fuzz] Add MiniMiner target + diff fuzz against BlockAssembler (glozow) 3f3f2d59ea2946a7b7cc8cb0222fb602d62645d0 [unit test] GatherClusters and MiniMiner unit tests (glozow) 59afcc83548ea67a863dac7b75d000bc8f6a7023 Implement Mini version of BlockAssembler to calculate mining scores (glozow) 56484f0fdc44261e723563f59df886d5acdd851f [mempool] find connected mempool entries with GatherClusters(…) (glozow) Pull request description: Implement Mini version of BlockAssembler to calculate mining scores Run the mining algorithm on a subset of the mempool, only disturbing the mempool to copy out fee information for relevant entries. Intended to be used by wallet to calculate amounts needed for fee-bumping unconfirmed transactions. From comments of sipa and glozow below: > > In what way does the code added here differ from the real block assembly code? > > * Only operates on the relevant transactions rather than full mempool > * Has the ability to remove transactions that will be replaced so they don't impact their ancestors > * Does not hold mempool lock outside of the constructor, makes copies of the entries it needs instead (though I'm not sure if this has an effect in practice) > * Doesn't do the sanity checks like keeping weight within max block weight and `IsFinalTx()` > * After the block template is built, additionally calculates fees to bump remaining ancestor packages to target feerate ACKs for top commit: achow101: ACK 6b605b91c1faf2c7f7cc0c9d39b4fcfd66dc2965 Xekyo: > ACK [6b605b9](https://github.com/bitcoin/bitcoin/commit/6b605b91c1faf2c7f7cc0c9d39b4fcfd66dc2965) modulo `miniminer_overlap` test. furszy: ACK 6b605b91 modulo `miniminer_overlap` test. theStack: Code-review ACK 6b605b91c1faf2c7f7cc0c9d39b4fcfd66dc2965 Tree-SHA512: f86a8b4ae0506858a7b15d90f417ebceea5038b395c05c825e3796123ad3b6cb8a98ebb948521316802a4c6d60ebd7041093356b1e2c2922a06b3b96b3b8acb6
2023-05-17index: Enable reindex-chainstate with active indexesMartin Zumsande
This is achieved by letting the index sync thread wait until reindex-chainstate is finished. This also disables the pruning check when reindexing the chainstate (which is incompatible with prune mode) because there would be no chain at this point in init.
2023-05-11Merge bitcoin/bitcoin#27125: refactor, kernel: Decouple ArgsManager from ↵fanquake
blockstorage 5ff63a09a9edd1204b2cc56cf6f48a44adab7bb3 refactor, blockstorage: Replace stopafterblockimport arg (TheCharlatan) 18e5ba7c8002bcd473ee29ce4b5bfc56df6142a4 refactor, blockstorage: Replace blocksdir arg (TheCharlatan) 02a0899527ba3d31329e56c791c9dbf36075bb84 refactor, BlockManager: Replace fastprune from arg with options (TheCharlatan) a498d699e3fdac5bfdb33020a1fd6c4a79989752 refactor/iwyu: Complete includes for blockmanager_args (TheCharlatan) f0bb1021f0d60f5f19176e67a66fcf7c325f88d1 refactor: Move functions to BlockManager methods (TheCharlatan) cfbb2124939822e95265a39242ffca3d86bac6e8 zmq: Pass lambda to zmq's ZMQPublishRawBlockNotifier (TheCharlatan) 8ed4ff8e05d61a8e954d72cebdc2e1d1ab24fb84 refactor: Declare g_zmq_notification_interface as unique_ptr (TheCharlatan) Pull request description: The libbitcoin_kernel library should not rely on the `ArgsManager`, but rather use option structs that can be passed to the various classes it uses. This PR removes reliance on the `ArgsManager` from the `blockstorage.*` files. Like similar prior work, it uses the options struct in the `BlockManager` that can be populated with `ArgsManager` values. Some related prior work: https://github.com/bitcoin/bitcoin/pull/26889 https://github.com/bitcoin/bitcoin/pull/25862 https://github.com/bitcoin/bitcoin/pull/25527 https://github.com/bitcoin/bitcoin/pull/25487 Related PR removing blockstorage globals: https://github.com/bitcoin/bitcoin/pull/25781 ACKs for top commit: ryanofsky: Code review ACK 5ff63a09a9edd1204b2cc56cf6f48a44adab7bb3. Since last ACK just added std::move and fixed commit title. Sorry for the noise! mzumsande: Code Review ACK 5ff63a09a9edd1204b2cc56cf6f48a44adab7bb3 Tree-SHA512: 4bde8fd140a40b97eca923e9016d85dcea6fad6fd199731f158376294af59c3e8b163a0725aa47b4be3519b61828044e0a042deea005e0c28de21d8b6c3e1ea7
2023-05-10Merge bitcoin/bitcoin#19690: util: improve FindByte() performanceAndrew Chow
72efc26439da9a1344a19569fb0cab01f82ae7d1 util: improve streams.h:FindByte() performance (Larry Ruane) 604df63f6c70b9692b067777ddb38d946ac0b2fc [bench] add streams findbyte (gzhao408) Pull request description: This PR is strictly a performance improvement; there is no functional change. The `CBufferedFile::FindByte()` method searches for the next occurrence of the given byte in the file. Currently, this is done by explicitly inspecting each byte in turn. This PR takes advantage of `std::find()` to do the same more efficiently, improving its CPU runtime by a factor of about 25 in typical use. ACKs for top commit: achow101: re-ACK 72efc26439da9a1344a19569fb0cab01f82ae7d1 stickies-v: re-ACK 72efc26439da9a1344a19569fb0cab01f82ae7d1 Tree-SHA512: ddf0bff335cc8aa34f911aa4e0558fa77ce35d963d602e4ab1c63090b4a386faf074548daf06ee829c7f2c760d06eed0125cf4c34e981c6129cea1804eb3b719
2023-05-10refactor, blockstorage: Replace blocksdir argTheCharlatan
Add a blocks_dir field to the BlockManager options. Move functions relying on the global gArgs to get the blocks_dir into the BlockManager class. This should eventually allow users of the BlockManager to not rely on the global Args and instead pass in their own options.
2023-05-10refactor: Move functions to BlockManager methodsTheCharlatan
This is a commit in preparation for the next few commits. The functions are moved to methods to avoid their re-declaration for the purpose of passing in BlockManager options. The functions that were now moved into the BlockManager should no longer use the params as an argument, but instead use the member variable. In the moved ReadBlockFromDisk and UndoReadFromDisk, change the function signature to accept a reference to a CBlockIndex instead of a raw pointer. The pointer is expected to be non-null, so reflect that in the type. To allow for the move of functions to BlockManager methods all call sites require an instantiated BlockManager, or a callback to one.
2023-05-10Merge bitcoin/bitcoin#27605: refactor: Replace global find_value function ↵Andrew Chow
with UniValue::find_value method fa266c4bbf564308ddbc12653527226506902084 Temporarily work around gcc-13 warning bug in interfaces_tests (MarcoFalke) fa28850562bda0c2049aaff42103dfa6f05395dc Fix clang-tidy performance-unnecessary-copy-initialization warnings (MarcoFalke) faaa60a30e7738a1490c9c2bb8963420f53c7f2d Remove unused find_value global function (MarcoFalke) fa422aeec2909df0151177816dc1ff5eb5a1fbab scripted-diff: Use UniValue::find_value method (MarcoFalke) fa548ac872c094edc94c2afda5cc9b0d84f73af0 Add UniValue::find_value method (MarcoFalke) Pull request description: The global function has issues: * It causes gcc-13 warnings, see https://github.com/bitcoin/bitcoin/issues/26926 * There is no rationale for it being a global function, when it acts like a member function * `performance-unnecessary-copy-initialization` clang-tidy isn't run on it Fix all issues by making it a member function. ACKs for top commit: achow101: ACK fa266c4bbf564308ddbc12653527226506902084 hebasto: re-ACK fa266c4bbf564308ddbc12653527226506902084 Tree-SHA512: 6c4e25da3122cd3b91c376bef73ea94fb3beb7bf8ef5cb3853c5128d95bfbacbcbfb16cc843eb7b1a7ebd350c2b6311f8085eeacf9aeeab3366987037d209e44
2023-05-10refactor: Use ChainType enum exhaustivelyTheCharlatan
This is a follow up of https://github.com/bitcoin/bitcoin/pull/27491, more concretely https://github.com/bitcoin/bitcoin/pull/27491#discussion_r1188847896, for not using default cases (as per the style guide), and https://github.com/bitcoin/bitcoin/pull/27491#discussion_r1188852707 and https://github.com/bitcoin/bitcoin/pull/27491#discussion_r1188851857 for avoiding dead code. Also change chain name to chain type in docstrings
2023-05-09Temporarily work around gcc-13 warning bug in interfaces_testsMarcoFalke
This can be reverted once gcc excludes lambdas with decltype(auto) return type from its -Wdangling-reference analysis.
2023-05-09scripted-diff: Use UniValue::find_value methodMarcoFalke
-BEGIN VERIFY SCRIPT- sed --regexp-extended -i 's/find_value\(([^ ,]+), /\1.find_value(/g' $(git grep -l find_value) -END VERIFY SCRIPT-
2023-05-09Merge bitcoin/bitcoin#27491: refactor: Move chain constants to the util libraryfanquake
d168458d1ff987e0d741c75ac1d4b63ae0cfb7e7 scripted-diff: Remove unused chainparamsbase includes (TheCharlatan) e9ee8aaf3acdf6dce2b339916d4c602484570050 Add missing definitions in prep for scripted diff (TheCharlatan) ba8fc7d788932b25864fb260ca14983aa2398c23 refactor: Replace string chain name constants with ChainTypes (TheCharlatan) 401453df419af35957ec711423ac3d93ad512fe8 refactor: Introduce ChainType getters for ArgsManager (TheCharlatan) bfc21c31b2186f7d30fc9a9ca7d6887ab61c6fb9 refactor: Create chaintype files (TheCharlatan) Pull request description: This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". It is also a follow up to #26177. It replaces pull request https://github.com/bitcoin/bitcoin/pull/27294, which just moved the constants to a new file, but did not re-declare them as enums. The code move of the chain name constants out of the `chainparamsbase` to their own separate header allows the kernel `chainparams` to no longer include `chainparamsbase`. The `chainparamsbase` contain references to the `ArgsManager` and networking related options that should not belong to the kernel library. Besides this move, the constants are re-declared as enums with helper functions facilitating string conversions. ACKs for top commit: ryanofsky: Code review ACK d168458d1ff987e0d741c75ac1d4b63ae0cfb7e7. Just suggested changes since last review. Tree-SHA512: ac2fbe5cbbab4f52eae1e30af1f16700b6589eb4764c328a151a712adfc37f326cc94a65c385534c57d4bc92cc1a13bf1777d92bc924a20dbb30440e7380b316
2023-05-09scripted-diff: Remove unused chainparamsbase includesTheCharlatan
This is a follow-up to previous commits moving the chain constants out of chainparamsbase. The script removes the chainparamsbase header in all files where it is included, but not used. This is done by filtering against all defined symbols of the header as well as its respective .cpp file. The kernel chainparams now no longer relies on chainparamsbase. -BEGIN VERIFY SCRIPT- sed -i '/#include <chainparamsbase.h>/d' $( git grep -l 'chainparamsbase.h' | xargs grep -L 'CBaseChainParams\|CreateBaseChainParams\|SetupChainParamsBaseOptions\|BaseParams\|SelectBaseParams\|chainparamsbase.cpp' ) -END VERIFY SCRIPT-
2023-05-09refactor: Replace string chain name constants with ChainTypesTheCharlatan
This commit effectively moves the definition of these constants out of the chainparamsbase to their own file. Using the ChainType enums provides better type safety compared to passing around strings. The commit is part of an ongoing effort to decouple the libbitcoinkernel library from the ArgsManager and other functionality that should not be part of the kernel library.
2023-05-08Merge bitcoin/bitcoin#26076: Switch hardened derivation marker to hAndrew Chow
fe49f06c0e91b96feb8d8f1bd478c3173f14782c doc: clarify PR 26076 release note (Sjors Provoost) bd13dc2f46ea10302a928fcf0f53b7aed77ad260 Switch hardened derivation marker to h in descriptors (Sjors Provoost) Pull request description: This makes it easier to handle descriptor strings manually, especially when importing from another Bitcoin Core wallet. For example the `importdescriptors` RPC call is easiest to use `h` as the marker: `'["desc": ".../0h/..."]'`, avoiding the need for escape characters. With this change `listdescriptors` will use `h`, so you can copy-paste the result, without having to add escape characters or switch `'` to 'h' manually. Both markers can still be parsed. The `hdkeypath` field in `getaddressinfo` is also impacted by this change, except for legacy wallets. The latter is to prevent accidentally breaking ancient software that uses our legacy wallet. See discussion in #15740 ACKs for top commit: achow101: ACK fe49f06c0e91b96feb8d8f1bd478c3173f14782c darosior: re-ACK fe49f06c0e91b96feb8d8f1bd478c3173f14782c Tree-SHA512: f78bc873b24a6f7a2bf38f5dd58f2b723e35e6b10e4d65c36ec300e2d362d475eeca6e5afa04b3037ab4bee0bf8ebc93ea5fc18102a2111d3d88fc873c08dc89
2023-05-06Merge bitcoin/bitcoin#17860: fuzz: BIP 30, CVE-2018-17144fanquake
fa2d8b61f9343d350b67357a12f39b613c8ee8ad fuzz: BIP 42, BIP 30, CVE-2018-17144 (MarcoFalke) faae7d5c00c99b0f3e99a1fbffbf369645716dd1 Move LoadVerifyActivateChainstate to ChainTestingSetup (MarcoFalke) fa26e3462a0fb1a9ad116ed58afa6897798f2c24 Avoid dereferencing interruption_point if it is nullptr (MarcoFalke) fa846ee074822160077f3f7476b2af62a876dec7 test: Add util to mine invalid blocks (MarcoFalke) Pull request description: Add a validation fuzz test for BIP 30 and CVE-2018-17144 ACKs for top commit: dergoegge: Code review ACK fa2d8b61f9343d350b67357a12f39b613c8ee8ad mzumsande: Tested ACK fa2d8b61f9343d350b67357a12f39b613c8ee8ad Tree-SHA512: 1f4620cc078709487abff24b304a6bb4eeab2e7628b392e2bc6de9cc0ce6745c413388ede6e93025d0c56eec905607ba9786633ef183e5779bf5183cc9ff92c0
2023-05-06Merge bitcoin/bitcoin#27405: util: Use steady clock instead of system clock ↵fanquake
to measure durations fa83fb31619c19a1a30b4181486601a944941b16 wallet: Use steady clock to calculate number of derive iterations (MarcoFalke) fa2c099cec1780a498e198750be0cf5bf0ca315a wallet: Use steady clock to measure scanning duration (MarcoFalke) fa976218044f3ff244abbd797b183a1408375c74 qt: Use steady clock to throttle GUI notifications (MarcoFalke) fa1d8044abc2cd0f149a2d526b3b03441443cdb0 test: Use steady clock in index tests (MarcoFalke) fa454dcb20b9e7943cc25e6eeea72912b5f1c7b5 net: Use steady clock in InterruptibleRecv (MarcoFalke) Pull request description: `GetTimeMillis` has multiple issues: * It doesn't denote the underlying clock type * It isn't type-safe * It is used incorrectly in places that should use a steady clock Fix all issues here. ACKs for top commit: willcl-ark: ACK fa83fb3161 martinus: Code review ACK https://github.com/bitcoin/bitcoin/commit/fa83fb31619c19a1a30b4181486601a944941b16, also ran all tests. All usages of the steady_clock are just for duration measurements, so the change to a different epoch is ok. Tree-SHA512: 5ec4fede8c7f97e2e08863c011856e8304f16ba30a68fdeb42f96a50a04961092cbe46ccf9ea6ac99ff5203c09f9e0924eb483eb38d7df0759addc85116c8a9f
2023-05-05util: improve streams.h:FindByte() performanceLarry Ruane
Avoid use of the expensive mod operator (%) when calculating the buffer offset. No functional difference. Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2023-05-05fuzz: BIP 42, BIP 30, CVE-2018-17144MarcoFalke
2023-05-05Move LoadVerifyActivateChainstate to ChainTestingSetupMarcoFalke
2023-05-04Remove unused chainparams from BlockManager methodsMarcoFalke
Also, replace pointer with reference while touching the signature.
2023-05-04Add BlockManagerOpts::chainparams referenceMarcoFalke
and use it in blockstorage.cpp
2023-05-02test: Add util to mine invalid blocksMarcoFalke
With the current utils it is only possible to mine valid blocks. This commit adds new util methods to mine invalid blocks.
2023-04-26Merge bitcoin/bitcoin#26933: mempool: disallow txns under min relay fee, ↵glozow
even in packages bf77fc9cb45209b9c560208c65abc94209cd7919 [test] mempool full in package accept (glozow) b51ebccc28e66c1822ab22d2d178be55c6618196 [validation] set PackageValidationState when mempool full (glozow) 563a2ee4f564c8ea5f8313d711b196e260568c04 [policy] disallow transactions under min relay fee, even in packages (glozow) c4554fe894d7af8e666f5d424deccddf516713ef [test] package cpfp bumps parents <mempoolminfee but >=minrelaytxfee (glozow) ac463e87df728689701810c3961155c49fdc5b31 [test util] mock mempool minimum feerate (glozow) Pull request description: Part of package relay, see #27463. Note that this still allows packages to bump transactions that are below the dynamic mempool minimum feerate, which means this still solves the "mempool is congested and my presigned 1sat/vB tx is screwed" problem for all transactions. On master, the package policy (only accessible through regtest-only RPC submitpackage) allows 0-fee (or otherwise below min relay feerate) transactions if they are bumped by a child. However, with default package limits, we don't yet have a DoS-resistant way of ensuring these transactions remain bumped throughout their time in the mempool. Primarily, the fee-bumping child may later be replaced by another transaction that doesn't bump the parent(s). The parent(s) could potentially stay bumped by other transactions, but not enough to ever be selected by the `BlockAssembler` (due to `blockmintxfee`). For example, (tested [here](https://github.com/glozow/bitcoin/commits/26933-motivation)): - The mempool accepts 24 below-minrelayfeerate transactions ("0-fee parents"), all bumped by a single high-fee transaction ("the fee-bumping child"). The fee-bumping child also spends a confirmed UTXO. - Two additional children are added to each 0-fee parent. These children each pay a feerate slightly above the minimum relay feerate (e.g. 1.9sat/vB) such that, for each 0-fee parent, the total fees of its two children divided by the total size of the children and parent is above the minimum relay feerate. - If a block template is built now, all transactions would be selected. - A transaction replaces the the fee-bumping child, spending only the confirmed UTXO and not any of the outputs from the 0-fee parents. - The 0-fee parents now each have 2 children. Their descendant feerates are above minrelayfeerate, which means that they remain in the mempool, even if the mempool evicts all below-minrelayfeerate packages. - If a block template is built now, none of the 0-fee parents or their children would be selected. - Even more low-feerate descendants can be added to these below-minrelayfeerate packages and they will not be evicted until they expire or the mempool reaches capacity. Unless we have a DoS-resistant way of ensuring package CPFP-bumped transactions are always bumped, allowing package CPFP to bump below-minrelayfeerate transactions can result in these problematic situations. See #27018 which proposes a partial solution with some limitations, and contains discussion about potential improvements to eviction strategy. While no adequate solution exists, for now, avoid these situations by requiring all transactions to meet min relay feerate. ACKs for top commit: ajtowns: reACK bf77fc9cb45209b9c560208c65abc94209cd7919 instagibbs: re-ACK https://github.com/bitcoin/bitcoin/pull/26933/commits/bf77fc9cb45209b9c560208c65abc94209cd7919 Tree-SHA512: 28940f41493a9e280b010284316fb8caf1ed7b2090ba9a4ef8a3b2eafc5933601074b142f4f7d4e3c6c4cce99d3146f5c8e1393d9406c6f2070dd41c817985c9
2023-04-21Merge bitcoin/bitcoin#27506: test: prevent intermittent failuresfanquake
10a354f1740a5c1b913d0b6951e80fb5401ab43a test: prevent intermittent failures (Amiti Uttarwar) Pull request description: Follow up to #27214 - add an address to the tried table before the new table to make sure a new table collision is not possible. ACKs for top commit: mzumsande: Code review ACK 10a354f1740a5c1b913d0b6951e80fb5401ab43a - the fix is what I suggested [here](https://github.com/bitcoin/bitcoin/pull/27214#discussion_r1169169601) and should make these intermittent failures impossible. Tree-SHA512: 24099f02e1915395130065af0ef6a2a1893955d222517d156d928765541d9c427da00172a9b5a540163f4d6aae93ca3882e8267eeb35ecc595d42178abc6191c
2023-04-21Merge bitcoin/bitcoin#27464: fuzz: re-enable prioritisetransaction & ↵fanquake
analyzepsbt RPC faa7144d3cf41e6410d942a3c485982ee65b3c6e fuzz: re-enable prioritisetransaction & analyzepsbt RPC (MarcoFalke) Pull request description: The linked issue seems fixed, so it should be fine to re-enable ACKs for top commit: dergoegge: utACK faa7144d3cf41e6410d942a3c485982ee65b3c6e Tree-SHA512: a681c726fceacc27ab5a03d455c7808d33f3cb11fe7d253d455526568af840b29f0c3c1d97c54785ef9277e7891a3aa742ac73ccd3cf115b7606eba50864aaa9
2023-04-21Merge bitcoin/bitcoin#27419: move-only: Extract common/args from util/systemfanquake
be55f545d53d44fdcf2d8ae802e9eae551d120c6 move-only: Extract common/args and common/config.cpp from util/system (TheCharlatan) Pull request description: This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". It is part of a series of patches splitting up the `util/system` files. Its preceding pull request is https://github.com/bitcoin/bitcoin/pull/27254. The pull request contains an extraction of ArgsManager related functions from util/system into their own common/ file. The background of this commit is an ongoing effort to decouple the libbitcoinkernel library from the ArgsManager. The ArgsManager belongs into the common library, since the kernel library should not depend on it. See [doc/design/libraries.md](https://github.com/bitcoin/bitcoin/blob/master/doc/design/libraries.md) for more information on this rationale. ACKs for top commit: MarcoFalke: re-ACK be55f545d53d44fdcf2d8ae802e9eae551d120c6 🚲 ryanofsky: Code review ACK be55f545d53d44fdcf2d8ae802e9eae551d120c6. Just small cleanups since the last review. hebasto: ACK be55f545d53d44fdcf2d8ae802e9eae551d120c6, I have reviewed the code and it looks OK, I agree it can be merged. Tree-SHA512: 90eb03334af0155b823030b4f2ecf286d35058d700ee2ddbbaa445be19e31eb0fe982656f35bd14ecee3ad2c3d0db3746855cb8f3777eff7253713e42873e111
2023-04-20test: prevent intermittent failuresAmiti Uttarwar
Add to the tried table before the new table to make sure a new table collision is not possible Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-04-20Merge bitcoin/bitcoin#25325: Add pool based memory resourceAndrew Chow
9f947fc3d4b779f017332135323b34e8f216f613 Use PoolAllocator for CCoinsMap (Martin Leitner-Ankerl) 5e4ac5abf54f8e6d6330df0c73119aa0cca4c103 Call ReallocateCache() on each Flush() (Martin Leitner-Ankerl) 1afca6b663bb54022afff193fd9d83856606b189 Add PoolResource fuzzer (Martin Leitner-Ankerl) e19943f049ed8aa4f32a1d8440a9fbf160367f0f Calculate memory usage correctly for unordered_maps that use PoolAllocator (Martin Leitner-Ankerl) b8401c3281978beed6198b2f9782b6a8dd35cbd7 Add pool based memory resource & allocator (Martin Leitner-Ankerl) Pull request description: A memory resource similar to `std::pmr::unsynchronized_pool_resource`, but optimized for node-based containers. The goal is to be able to cache more coins with the same memory usage, and allocate/deallocate faster. This is a reimplementation of #22702. The goal was to implement it in a way that is simpler to review & test * There is now a generic `PoolResource` for allocating/deallocating memory. This has practically the same API as `std::pmr::memory_resource`. (Unfortunately I cannot use std::pmr because libc++ simply doesn't implement that API). * Thanks to sipa there is now a fuzzer for PoolResource! On a fast machine I ran it for ~770 million executions without finding any issue. * The estimation of the correct node size is now gone, PoolResource now has multiple pools and just needs to be created large enough to have space for the unordered_map nodes. I ran benchmarks with #22702, mergebase, and this PR. Frequency locked Intel i7-8700, clang++ 13.0.1 to reindex up to block 690000. ```sh bitcoind -dbcache=5000 -assumevalid=00000000000000000002a23d6df20eecec15b21d32c75833cce28f113de888b7 -reindex-chainstate -printtoconsole=0 -stopatheight=690000 ``` The performance is practically identical with #22702, just 0.4% slower. It's ~21% faster than master: ![Progress in Million Transactions over Time(2)](https://user-images.githubusercontent.com/14386/173288685-91952ade-f304-4825-8bfb-0725a71ca17b.png) ![Size of Cache in MiB over Time](https://user-images.githubusercontent.com/14386/173291421-e6b410be-ac77-479b-ad24-5fafcebf81eb.png) Note that on cache drops mergebase's memory doesnt go so far down because it does not free the `CCoinsMap` bucket array. ![Size of Cache in Million tx over Time(1)](https://user-images.githubusercontent.com/14386/173288703-a80c9c9e-93c8-4a16-9df8-610c89c61cc4.png) ACKs for top commit: LarryRuane: ACK 9f947fc3d4b779f017332135323b34e8f216f613 achow101: re-ACK 9f947fc3d4b779f017332135323b34e8f216f613 john-moffett: ACK 9f947fc3d4b779f017332135323b34e8f216f613 jonatack: re-ACK 9f947fc3d4b779f017332135323b34e8f216f613 Tree-SHA512: 48caf57d1775875a612b54388ef64c53952cd48741cacfe20d89049f2fb35301b5c28e69264b7d659a3ca33d4c714d47bafad6fd547c4075f08b45acc87c0f45
2023-04-20Merge bitcoin/bitcoin#27214: addrman: Enable selecting addresses by networkAndrew Chow
17e705428ddf80c7a7f31fe5430d966cf08a37d6 doc: clarify new_only param for Select function (Amiti Uttarwar) b0010c83a1b4a3d21719cb68e37faf9b1172522a bench: test select for a new table with only one address (Amiti Uttarwar) 9b91aae08579c77d2fd5506804c8e2e0cda0d274 bench: add coverage for addrman select with network parameter (Amiti Uttarwar) 22a4d1489c0678a90c00318203cfce61672f20b7 test: increase coverage of addrman select (without network) (Amiti Uttarwar) a98e542e0c18f7cb2340179631806f14b07430c3 test: add addrman test for special case (Amiti Uttarwar) 5c8b4baff27e0ccd27fda6e915b956d1e8dd7ce2 tests: add addrman_select_by_network test (Amiti Uttarwar) 6b229284fd2209938ee8fdffed4d080395b3aa05 addrman: add functionality to select by network (Amiti Uttarwar) 26c3bf11e2487ed0ac578fb92619c148336003cb scripted-diff: rename local variables to match modern conventions (Amiti Uttarwar) 48806412e2bcd023b78fc05f6c9ce092360d1db1 refactor: consolidate select logic for new and tried tables (Amiti Uttarwar) ca2a9c5f8f14b792a14e81f73b1910a4c8799b93 refactor: generalize select logic (Amiti Uttarwar) 052fbcd5a791855406141e85d32e42e297220fe9 addrman: Introduce helper to generalize looking up an addrman entry (Amiti Uttarwar) 9bf078f66c8f286e1ab5e34b8eeed7d80290a897 refactor: update Select_ function (Amiti Uttarwar) Pull request description: For the full context & motivation of this patch, see #27213 This is joint work with mzumsande. This PR adds functionality to `AddrMan::Select` to enable callers to specify a network they are interested in. Along the way, it refactors the function to deduplicate the logic, updates the local variables to match modern conventions, adds test coverage for both the new and existing `Select` logic, and adds bench tests for the worst case performance of both the new and existing `Select` logic. This functionality is used in the parent PR. ACKs for top commit: vasild: ACK 17e705428ddf80c7a7f31fe5430d966cf08a37d6 brunoerg: re-ACK 17e705428ddf80c7a7f31fe5430d966cf08a37d6 ajtowns: ACK 17e705428ddf80c7a7f31fe5430d966cf08a37d6 mzumsande: Code Review ACK 17e705428ddf80c7a7f31fe5430d966cf08a37d6 Tree-SHA512: e99d1ce0c44a15601a3daa37deeadfc9d26208a92969ecffbea358d57ca951102d759734ccf77eacd38db368da0bf5b6fede3cd900d8a77b3061f4adc54e52d8
2023-04-19move-only: Extract common/args and common/config.cpp from util/systemTheCharlatan
This is an extraction of ArgsManager related functions from util/system into their own common file. Config file related functions are moved to common/config.cpp. The background of this commit is an ongoing effort to decouple the libbitcoinkernel library from the ArgsManager. The ArgsManager belongs into the common library, since the kernel library should not depend on it. See doc/design/libraries.md for more information on this rationale.
2023-04-18doc: update references to kernel/chainparams.cppfanquake
2023-04-17bugfix: rest: avoid segfault for invalid URIpablomartin4btc
`evhttp_uri_parse` can return a nullptr, for example when the URI contains invalid characters (e.g. "%"). `GetQueryParameterFromUri` passes the output of `evhttp_uri_parse` straight into `evhttp_uri_get_query`, which means that anyone calling a REST endpoint in which query parameters are used (e.g. `rest_headers`) can cause a segfault. This bugfix is designed to be minimal and without additional behaviour change. Follow-up work should be done to resolve this in a more general and robust way, so not every endpoint has to handle it individually.
2023-04-17[policy] disallow transactions under min relay fee, even in packagesglozow
Avoid adding transactions below min relay feerate because, even if they were bumped through CPFP when entering the mempool, we do not have a DoS-resistant way of ensuring they always remain bumped. In the future, this rule can be relaxed (e.g. to allow packages to bump 0-fee transactions) if we find a way to do so.
2023-04-17[test] package cpfp bumps parents <mempoolminfee but >=minrelaytxfeeglozow
2023-04-17[test util] mock mempool minimum feerateglozow
2023-04-14fuzz: re-enable prioritisetransaction & analyzepsbt RPCMarcoFalke
2023-04-11Merge bitcoin/bitcoin#26662: fuzz: Add HeadersSyncState targetfanquake
3153e7d779ac284f86e433af033d63f13f361b6f [fuzz] Add HeadersSyncState target (dergoegge) 53552affca381cdb5103ecdbcc7f3fb562e66ac4 [headerssync] Make m_commit_offset protected (dergoegge) Pull request description: This adds a fuzz target for the `HeadersSyncState` class. I am unsure how well this is able to cover the logic since it is just processing unserialized CBlockHeaders straight from the fuzz input (headers are sometimes made continuous). However, it does manage to get to the redownload phase so i thought it is better then not having fuzzing at all. It would also be nice to fuzz the p2p logic that is using `HeadersSyncState` (e.g. `TryLowWorkHeadersSync`, `IsContinuationOfLowWorkHeadersSync`) but that likely requires some more work (refactoring👻). ACKs for top commit: mzumsande: ACK 3153e7d779ac284f86e433af033d63f13f361b6f Tree-SHA512: 8a4630ceeeb30e4eeabaa8eb5491d98f0bf900efe7cda07384eaac9f2afaccfbcaa979cc1cc7f0b6ca297a8f5c17a7759f94809dd87eb87d35348d847c83e8ab
2023-04-06[fuzz] Add MiniMiner target + diff fuzz against BlockAssemblerglozow
Co-authored-by: dergoegge <n.goeggi@gmail.com> Co-authored-by: mzumsande <mzumsande@gmail.com> Co-authored-by: Murch <murch@murch.one>
2023-04-06[unit test] GatherClusters and MiniMiner unit testsglozow
Co-authored-by: Murch <murch@murch.one> Co-authored-by: theStack <sebastian.falbesoner@gmail.com> Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
2023-04-05ci: use clang-16 in tidy taskfanquake
2023-04-04Switch hardened derivation marker to h in descriptorsSjors Provoost
This makes it easier to handle descriptor strings manually. E.g. an RPC call that takes an array of descriptors can now use '["desc": ".../0h/..."]'. Both markers can still be parsed. The default for new descriptors is changed to h. In normalized form h is also used. For private keys the chosen marker is preserved in a round trip. The hdkeypath field in getaddressinfo is also impacted by this change.
2023-04-04test: Use steady clock in index testsMarcoFalke
2023-04-04net: Use steady clock in InterruptibleRecvMarcoFalke
2023-04-03refactor: don't avoid sys/types.h on when building for Windowsfanquake
We've already used it unguarded in `httpserver.cpp` for years, with no build issues.
2023-04-03Merge bitcoin/bitcoin#27254: refactor: Extract util/fs from util/systemfanquake
00e9b97f37e0bdf4c647236838c10b68b7ad5be3 refactor: Move fs.* to util/fs.* (TheCharlatan) 106b46d9d25b5228ef009fbbe6f9a7ae35090d15 Add missing fs.h includes (TheCharlatan) b202b3dd6393b415fa68e18dc49c9431dc6b58b2 Add missing cstddef include in assumptions.h (TheCharlatan) 18fb36367a28819bd5ab402344802796a1248979 refactor: Extract util/fs_helpers from util/system (Ben Woosley) Pull request description: This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". This commit was originally authored by empact and is taken from its parent PR #25152. #### Context There is an ongoing effort to decouple the `ArgsManager` used for command line parsing user-provided arguments from the libbitcoinkernel library (https://github.com/bitcoin/bitcoin/pull/25290, https://github.com/bitcoin/bitcoin/pull/25487, https://github.com/bitcoin/bitcoin/pull/25527, https://github.com/bitcoin/bitcoin/pull/25862, https://github.com/bitcoin/bitcoin/pull/26177, and https://github.com/bitcoin/bitcoin/pull/27125). The `ArgsManager` is defined in `system.h`. A similar pull request extracting functionality from `system.h` has been merged in https://github.com/bitcoin/bitcoin/pull/27238. #### Changes Next to providing better code organization, this PR removes some reliance of the tree of libbitcoinkernel header includes on `system.h` (and thus the `ArgsManager` definition) by moving filesystem related functions out of the `system.*` files. There is already a pair of `fs.h` / `fs.cpp` in the top-level `src/` directory. They were not combined with the files introduced here, to keep the patch cleaner and more importantly because they are often included without the utility functions. The new files are therefore named `fs_helpers` and the existing `fs` files are moved into the util directory. Further commits splitting more functionality out of `system.h` are still in #25152 and will be submitted in separate PRs once this PR has been processed. ACKs for top commit: hebasto: ACK 00e9b97f37e0bdf4c647236838c10b68b7ad5be3 Tree-SHA512: 31422f148d14ba3c843b99b1550a6fd77c77f350905ca324f93d4f97b652246bc58fa9696c64d1201979cf88733e40be02d262739bb7d417cf22bf506fdb7666
2023-03-28Merge bitcoin/bitcoin#27344: fuzz: Remove legacy int parse fuzz testsfanquake
faf8dc496e761a15956f8226d727f4bbab8dff82 fuzz: Remove legacy int parse fuzz tests (MarcoFalke) Pull request description: The fuzz tests checked that the result of the new function was equal to the legacy function. (Side note: The checks were incomplete, as evident by the follow-up fix in commit b5c9bb5cb9f4a8db57b33ef7399310c7d6de5822). Given that they haven't found any issues in years (beside missing the above issue, that they couldn't catch), it seems time to remove them. They may come in handy in the rare case that someone would want to modify `LocaleIndependentAtoi()` or `Parse*Int*()`, however that seems unlikely. Also, appropriate checks can be added then. ACKs for top commit: fanquake: ACK faf8dc496e761a15956f8226d727f4bbab8dff82 dergoegge: ACK faf8dc496e761a15956f8226d727f4bbab8dff82 Tree-SHA512: 4ec88b9fa8ba49a923b0604016f0f471b3c9b9e0ba6c5c3dc4e20503c6994789921e7221d9ec467a2a37a73f21a70ba51ba3370ed5ad311dee989e218290b29a
2023-03-28Merge bitcoin/bitcoin#27324: net: #27257 follow-upsfanquake
cd0c8eeb0940790b6ba83786d1c9e362d4dc4829 [net] Pass nRecvFloodSize to CNode (dergoegge) 860402ef2ed728ef096dda4e65e77d566782209f [net] Remove trivial GetConnectionType() getter (dergoegge) b5a85b365a4abd98176b0935015dbb502cc3e6f6 [net] Delete CNetMessage copy constructor/assignment op (dergoegge) Pull request description: Follow-up PR for #27257 * Deletes the copy constructor/assignment operator of `CNetMessage` * Removes trivial getter for the connection type * Avoids passing `nRecvFloodSize` to CNode methods by passing it to `CNode` on creation ACKs for top commit: jnewbery: utACK cd0c8eeb0940790b6ba83786d1c9e362d4dc4829 theStack: ACK cd0c8eeb0940790b6ba83786d1c9e362d4dc4829 Tree-SHA512: 673a758668617f69fba77e61f0eaa1538da27a4849c82c98742436692baa2d7f001129af3e7a66b160e599d12109dac08137a146f10ff9b9ebdc5c2237311d41
2023-03-27test: Fix intermittent failure in ChainStateManager testsMartin Zumsande
Before wiping the ChainStateManager, the validationinterface queue must be drained to avoid accessing deleted memory.
2023-03-27fuzz: Remove legacy int parse fuzz testsMarcoFalke
generated by cgit v1.2.3 (git 2.26.2) at 2024-12-15 18:35:51 +0000