aboutsummaryrefslogtreecommitdiff
path: root/src/test/util
AgeCommit message (Collapse)Author
2022-09-13Merge bitcoin/bitcoin#24513: CChainState -> Chainstateglozow
00eeb31c7660e2c28f189f77a6905dee946ef408 scripted-diff: rename CChainState -> Chainstate (James O'Beirne) Pull request description: Alright alright alright, I know: we hate refactors. We especially hate cosmetic refactors. Nobody knows better than I that changing broad swaths of code out from under our already-abused collaborators, only to send a cascade of rebase bankruptcies, is annoying at best and sadistic at worst. And for a rename! The indignation! But just for a second, imagine yourself. Programming `bitcoin/bitcoin`, on a sandy beach beneath a lapis lazuli sky. You go to type the name of what is probably the most commonly used data structure in the codebase, and you *only hit shift once*. What could you do in such a world? You could do anything. [The only limit is yourself.](https://zombo.com/) --- So maybe you like the idea of this patch but really don't want to deal with rebasing. You're in luck! Here're the commands that will bail you out of rebase bankruptcy: ```sh git rebase -i $(git merge-base HEAD master) \ -x 'sed -i "s/CChainState/Chainstate/g" $(git ls-files | grep -E ".*\.(py|cpp|h)$") && git commit --amend --no-edit' # <commit changed?> git add -u && git rebase --continue ``` --- ~~Anyway I'm not sure how serious I am about this, but I figured it was worth proposing.~~ I have decided I am very serious about this. Maybe we can have nice things every once in a while? ACKs for top commit: MarcoFalke: cr ACK 00eeb31c7660e2c28f189f77a6905dee946ef408 hebasto: ACK 00eeb31c7660e2c28f189f77a6905dee946ef408 glozow: ACK 00eeb31c7660e2c28f189f77a6905dee946ef408, thanks for being the one to propose this w0xlt: ACK https://github.com/bitcoin/bitcoin/pull/24513/commits/00eeb31c7660e2c28f189f77a6905dee946ef408 Tree-SHA512: b828a99780614a9b74f7a9c347ce0687de6f8d75232840f5ffc26e02bbb25a3b1f5f9deabbe44f82ada01459586ee8452a3ee2da05d1b3c48558c8df6f49e1b1
2022-09-09scripted-diff: rename CChainState -> ChainstateJames O'Beirne
-BEGIN VERIFY SCRIPT- sed -i 's/CChainState/Chainstate/g' $(git grep -l CChainState ':(exclude)doc/release-notes*') -END VERIFY SCRIPT- Co-authored-by: MacroFake <falke.marco@gmail.com>
2022-09-05test: remove Boost Test from libtest utilfanquake
Context is the discussion here: https://github.com/bitcoin/bitcoin/pull/25974/files#r961541457.
2022-09-02Merge bitcoin/bitcoin#25962: net: Add CNodeOptions and increase constnessMacroFake
377e9ccda469731d535829f184b70c73ed46b6ef scripted-diff: net: rename permissionFlags to permission_flags (Anthony Towns) 0a7fc428978c4db416fdcf9bf0b79de17d0558d7 net: make CNode::m_prefer_evict const (Anthony Towns) d394156b99d6b9a99aedee78658310d169ca188d net: make CNode::m_permissionFlags const (Anthony Towns) 9dccc3328eeaf9cd66518d812c878def5d014e36 net: add CNodeOptions for optional CNode constructor params (Anthony Towns) Pull request description: Adds CNodeOptions to make it easier to add optional parameters to the CNode constructor, and makes prefer_evict and m_permissionFlags actually const. ACKs for top commit: naumenkogs: ACK 377e9ccda469731d535829f184b70c73ed46b6ef jonatack: ACK 377e9ccda469731d535829f184b70c73ed46b6ef per `git range-diff 52dcb1d 2f3602b 377e9cc` vasild: ACK 377e9ccda469731d535829f184b70c73ed46b6ef ryanofsky: Code review ACK 377e9ccda469731d535829f184b70c73ed46b6ef. Looks good and feel free to ignore suggestions! Tree-SHA512: 06fd6748770bad75ec8c966fdb73b7534c10bd61838f6f1b36b3f3d6a438e58f6a7d0edb011977e5c118ed7ea85325fac35e10dde520fef249f7a780cf500a85
2022-09-01Merge bitcoin/bitcoin#25614: Severity-based logging, step 2Andrew Chow
958048057087e6562b474f9028316c00ec03c2e4 Update debug logging section in the developer notes (Jon Atack) 1abaa31aa3d833caf2290d6c90f57f7f79d146c0 Update -debug and -debugexclude help docs for severity level logging (Jon Atack) 45f92821621a60891044f57c7a7bc4ab4c7d8a01 Create BCLog::Level::Trace log severity level (Jon Atack) 2a8712db4fb5d06f1a525a79bb0f793cb733aaa6 Unit test coverage for -loglevel configuration option (klementtan) eb7bee5f84d41e35cb4296e01bea2aa5ac80a856 Create -loglevel configuration option (klementtan) 98a1f9c68744074f29fa5fa67514218b5ee9edc4 Unit test coverage for log severity levels (klementtan) 9c7507bf76e79da99766a69df939520ea0a125d1 Create BCLog::Logger::LogLevelsString() helper function (klementtan) 8fe3457dbb4146952b92fb9509bbe4e97dc1f05b Update LogAcceptCategory() and unit tests with log severity levels (klementtan) c2797cfc602c5cdd899a7c11b37bb5711cebff38 Add BCLog::Logger::SetLogLevel()/SetCategoryLogLevel() for string inputs (klementtan) f6c0cc03509255ffa4dfd6e2822fce840dd0b181 Add BCLog::Logger::m_category_log_levels data member and getter/setter (Jon Atack) 2978b387bffc226fb1eaca4d30f24a0deedb2a36 Add BCLog::Logger::m_log_level data member and getter/setter (Jon Atack) f1379aeca9d3a8c4d3528de4d0af8298cb42fee4 Simplify BCLog::Level enum class and LogLevelToStr() function (Jon Atack) Pull request description: This is an updated version of https://github.com/bitcoin/bitcoin/pull/25287 and the next steps in parent PR #25203 implementing, with Klement Tan, user-configurable, per-category severity log levels based on an idea by John Newbery and refined in GitHub discussions by Wladimir Van der Laan and Marco Falke. - simplify the `BCLog::Level` enum class and the `LogLevelToStr()` function and add documentation - update the logging logic to filter logs by log level both globally and per-category - add a hidden `-loglevel` help-debug config option to allow testing setting the global or per-category severity level on startup for logging categories enabled with the `-debug` configuration option or the logging RPC (Klement Tan) - add a `trace` log severity level selectable by the user; the plan is for the current debug messages to become trace, LogPrint ones to become debug, and LogPrintf ones to become info, warning, or error ``` $ ./src/bitcoind -help-debug | grep -A10 loglevel -loglevel=<level>|<category>:<level> Set the global or per-category severity level for logging categories enabled with the -debug configuration option or the logging RPC: info, debug, trace (default=info); warning and error levels are always logged. If <category>:<level> is supplied, the setting will override the global one and may be specified multiple times to set multiple category-specific levels. <category> can be: addrman, bench, blockstorage, cmpctblock, coindb, estimatefee, http, i2p, ipc, leveldb, libevent, lock, mempool, mempoolrej, net, proxy, prune, qt, rand, reindex, rpc, selectcoins, tor, util, validation, walletdb, zmq. ``` See the individual commit messages for details. ACKs for top commit: jonatack: One final push per `git range-diff a5d5569 ce3c4c9 9580480` (should be trivial to re-ACK) to ensure this pull changes no default behavior in any way for users or the tests/CI in order to be completely v24 compatible, to update the unit test setup in general, and to update the debug logging section in the developer notes. klementtan: reACK https://github.com/bitcoin/bitcoin/commit/958048057087e6562b474f9028316c00ec03c2e4 1440000bytes: reACK https://github.com/bitcoin/bitcoin/pull/25614/commits/958048057087e6562b474f9028316c00ec03c2e4 vasild: ACK 958048057087e6562b474f9028316c00ec03c2e4 dunxen: reACK 9580480 brunoerg: reACK 958048057087e6562b474f9028316c00ec03c2e4 Tree-SHA512: 476a638e0581f40b5d058a9992691722e8b546471ec85e07cbc990798d1197fbffbd02e1b3d081b4978404e07a428378cdc8e159c0004b81f58be7fb01b7cba0
2022-09-01net: make CNode::m_permissionFlags constAnthony Towns
2022-08-29Require callers of AcceptBlockHeader() to perform anti-dos checksSuhas Daftuar
In order to prevent memory DoS, we must ensure that we don't accept a new header into memory until we've performed anti-DoS checks, such as verifying that the header is part of a sufficiently high work chain. This commit adds a new argument to AcceptBlockHeader() so that we can ensure that all call-sites which might cause a new header to be accepted into memory have to grapple with the question of whether the header is safe to accept, or needs further validation. This patch also fixes two places where low-difficulty-headers could have been processed without such validation (processing an unrequested block from the network, and processing a compact block). Credit to Niklas GΓΆgge for noticing this issue, and thanks to Sjors Provoost for test code.
2022-08-20Create BCLog::Level::Trace log severity levelJon Atack
for verbose log messages for development or debugging only, as bitcoind may run more slowly, that are more granular/frequent than the Debug log level, i.e. for very high-frequency, low-level messages to be logged distinctly from higher-level, less-frequent debug logging that could still be usable in production. An example would be to log higher-level peer events (connection, disconnection, misbehavior, eviction) as Debug, versus Trace for low-level, high-volume p2p messages in the BCLog::NET category. This will enable the user to log only the former without the latter, in order to focus on high-level peer management events. With respect to the name, "trace" is suggested as the most granular level in resources like the following: - https://sematext.com/blog/logging-levels - https://howtodoinjava.com/log4j2/logging-levels Update the test framework and add test coverage.
2022-08-20Create -loglevel configuration optionklementtan
- add a -loglevel=<level>|<category:level> config option to allow users to set a global -loglevel and category-specific log levels. LogPrintLevel messages with a higher severity level than -loglevel will not be printed in the debug log. - for now, this config option is debug-only during the migration to severity-based logging - update unit and functional tests Co-authored-by: "Jon Atack <jon@atack.com>"
2022-08-05Merge bitcoin/bitcoin#25721: refactor: Replace BResult with util::ResultMacroFake
a23cca56c0a7f4a267915b4beba3af3454c51603 refactor: Replace BResult with util::Result (Ryan Ofsky) Pull request description: Rename `BResult` class to `util::Result` and update the class interface to be more compatible with `std::optional` and with a full-featured result class implemented in https://github.com/bitcoin/bitcoin/pull/25665. Motivation for this change is to update existing `BResult` usages now so they don't have to change later when more features are added in https://github.com/bitcoin/bitcoin/pull/25665. This change makes the following improvements originally implemented in https://github.com/bitcoin/bitcoin/pull/25665: - More explicit API. Drops potentially misleading `BResult` constructor that treats any bilingual string argument as an error. Adds `util::Error` constructor so it is never ambiguous when a result is being assigned an error or non-error value. - Better type compatibility. Supports `util::Result<bilingual_str>` return values to hold translated messages which are not errors. - More standard and consistent API. `util::Result` supports most of the same operators and methods as `std::optional`. `BResult` had a less familiar interface with `HasRes`/`GetObj`/`ReleaseObj` methods. The Result/Res/Obj naming was also not internally consistent. - Better code organization. Puts `src/util/` code in the `util::` namespace so naming reflects code organization and it is obvious where the class is coming from. Drops "B" from name because it is undocumented what it stands for (bilingual?) - Has unit tests. ACKs for top commit: MarcoFalke: ACK a23cca56c0a7f4a267915b4beba3af3454c51603 🏡 jonatack: ACK a23cca56c0a7f4a267915b4beba3af3454c51603 Tree-SHA512: 2769791e08cd62f21d850aa13fa7afce4fb6875a9cedc39ad5025150dbc611c2ecfd7b3aba8b980a79fde7fbda13babdfa37340633c69b501b6e89727bad5b31
2022-08-03validationcaches: Add and use ValidationCacheSizesCarl Dong
Also: - Make DEFAULT_MAX_SIG_CACHE_SIZE into constexpr DEFAULT_MAX_SIG_CACHE_BYTES to utilize the compile-time integer arithmetic overflow checking available to constexpr. - Fix comment (MiB instead of MB) for DEFAULT_MAX_SIG_CACHE_BYTES. - Pass in max_size_bytes parameter to InitS*Cache(), modify log line to no longer allude to maxsigcachesize being split evenly between the two validation caches. - Fix possible integer truncation and add a comment. [META] I've kept the integer types as int64_t in order to not introduce unintended behaviour changes, in the next commit we will make them size_t.
2022-08-03cuckoocache: Return approximate memory sizeCarl Dong
Returning the approximate total size eliminates the need for InitS*Cache() to do nElems*sizeof(uint256). The cuckoocache has a better idea of this information.
2022-08-03refactor: Replace BResult with util::ResultRyan Ofsky
Rename `BResult` class to `util::Result` and update the class interface to be more compatible with `std::optional` and with a full-featured result class implemented in https://github.com/bitcoin/bitcoin/pull/25665. Motivation for this change is to update existing `BResult` usages now so they don't have to change later when more features are added in #25665. This change makes the following improvements originally implemented in #25665: - More explicit API. Drops potentially misleading `BResult` constructor that treats any bilingual string argument as an error. Adds `util::Error` constructor so it is never ambiguous when a result is being assigned an error or non-error value. - Better type compatibility. Supports `util::Result<bilingual_str>` return values to hold translated messages which are not errors. - More standard and consistent API. `util::Result` supports most of the same operators and methods as `std::optional`. `BResult` had a less familiar interface with `HasRes`/`GetObj`/`ReleaseObj` methods. The Result/Res/Obj naming was also not internally consistent. - Better code organization. Puts `src/util/` code in the `util::` namespace so naming reflects code organization and it is obvious where the class is coming from. Drops "B" from name because it is undocumented what it stands for (bilingual?) - Has unit tests.
2022-08-02sort after scripted-diffMacroFake
2022-08-02scripted-diff: Move mempool_args to src/nodeMacroFake
It is part of the node library. Also, it won't be moved to the kernel lib, as it will be pruned of ArgsManager. -BEGIN VERIFY SCRIPT- # Move module git mv src/mempool_args.cpp src/node/ git mv src/mempool_args.h src/node/ # Replacements sed -i 's:mempool_args\.h:node/mempool_args.h:g' $(git grep -l mempool_args) sed -i 's:mempool_args\.cpp:node/mempool_args.cpp:g' $(git grep -l mempool_args) sed -i 's:MEMPOOL_ARGS_H:NODE_MEMPOOL_ARGS_H:g' $(git grep -l MEMPOOL_ARGS_H) -END VERIFY SCRIPT-
2022-08-02Return optional error from ApplyArgsManOptionsMacroFake
Also pass in a (for now unused) reference to the params. Both changes are needed for the next commit.
2022-07-20refactor: move compat.h into compat/fanquake
2022-07-20Merge bitcoin/bitcoin#25285: Add AutoFile without ser-type and ser-version ↡fanquake
and use it where possible facc2fa7b8a218a0df6a19772e1641ea68dda2e3 Use AutoFile where possible (MacroFake) 6666803c897e4ad27b45cb74e3a9aa74a335f1bf streams: Add AutoFile without ser-type and ser-version (MacroFake) Pull request description: This was done in the context of https://github.com/bitcoin/bitcoin/pull/25284 , but I think it also makes sense standalone. The basic idea is that serialization type should not be initialized when it is not needed. Same for the serialization version. So do this here for `AutoFile`. `CAutoFile` remains in places where it is not yet possible. ACKs for top commit: laanwj: Code review ACK facc2fa7b8a218a0df6a19772e1641ea68dda2e3 fanquake: ACK facc2fa7b8a218a0df6a19772e1641ea68dda2e3 Tree-SHA512: d82d024d55af57565ac53d9d1517afafc12b46964effba0332de62a6c77869356fa77f89e6d4834438fff44c45b64fccdf5a1358bfea03e28dfe55013b3c099d
2022-07-19refactor: Reduce number of LoadChainstate return valuesRussell Yanofsky
2022-07-19refactor: Reduce number of LoadChainstate parametersRussell Yanofsky
2022-07-19Merge bitcoin/bitcoin#25514: net processing: Move CNode::nServices and ↡MacroFake
CNode::nLocalServices to Peer 8d8eeb422e64ccd08bce92eed2ee9cbbf55ffd67 [net processing] Remove CNode::nLocalServices (John Newbery) 5961f8eea1ad5be1a4bf8da63651e197a20359b2 [net] Return CService from GetLocalAddrForPeer and GetLocalAddress (dergoegge) d9079fe18dc5d81ce290876353555b51125127d1 [net processing] Remove CNode::nServices (John Newbery) 7d1c0369340cb752f0d78e24f4251595534bf5e9 [net processing] Replace fHaveWitness with CanServeWitnesses() (John Newbery) f65e83d51bfb6a34f1d5efccfb3d8786a51a4534 [net processing] Remove fClient and m_limited_node (John Newbery) fc5eb528f7d7b33e2f2e443c5610a1551c7f099b [tests] Connect peer in outbound_slow_chain_eviction by sending p2p messages (John Newbery) 1f52c47d5c09b59fd3153700751c74e63edc7d7e [net processing] Add m_our_services and m_their_services to Peer (John Newbery) Pull request description: Another step in #19398. Which services we offer to a peer and which services they offer to us is application layer data and should not be stored on `CNode`. This is also a prerequisite for adding `PeerManager` unit tests (See #25515). ACKs for top commit: MarcoFalke: ACK 8d8eeb422e64ccd08bce92eed2ee9cbbf55ffd67 πŸ”‘ jnewbery: utACK 8d8eeb422e64ccd08bce92eed2ee9cbbf55ffd67 mzumsande: Code Review ACK 8d8eeb422e64ccd08bce92eed2ee9cbbf55ffd67 Tree-SHA512: e772eb2a0a85db346dd7b453a41011a12756fc7cbfda6a9ef6daa9633b9a47b9770ab3dc02377690f9d02127301c3905ff22905977f758bf90b17a9a35b37523
2022-07-14[net processing] Remove CNode::nLocalServicesJohn Newbery
2022-07-14[net processing] Remove CNode::nServicesJohn Newbery
Use Peer::m_their_services instead
2022-07-14[net processing] Add m_our_services and m_their_services to PeerJohn Newbery
Track services offered by us and the peer in the Peer object.
2022-07-14Use designated initializers for ChainstateManager::OptionsCarl Dong
This wasn't available at the time when ChainstateManager::Options was introduced but is helpful to be explicit and ensure correctness.
2022-07-13Merge bitcoin/bitcoin#25472: build: Increase MS Visual Studio minimum versionfanquake
630c1711b47ce50805f4dd2883777a100f7e5339 refactor: Drop no longer needed `util/designator.h` (Hennadii Stepanov) 88ec5d40dcf5d9f95217b123b48203b2f334c0a1 build: Increase MS Visual Studio minimum version (Hennadii Stepanov) 555f9dd5d39b316bf404017401b5aedc23ec6226 rpc, refactor: Add `decodepsbt_outputs` (Hennadii Stepanov) 0c432cbbfa9e83a68e061b388745326e278369fb rpc, refactor: Add `decodepsbt_inputs` (Hennadii Stepanov) 01d95a3964267d243df00d9bcc93b28adcfe16d7 rpc, refactor: Add `getblock_prevout` (Hennadii Stepanov) Pull request description: Visual Studio 2022 with `/std:c++20` supports [designated initializers](https://github.com/bitcoin/bitcoin/pull/24531). ACKs for top commit: sipsorcery: reACK 630c1711b47ce50805f4dd2883777a100f7e5339. Tree-SHA512: 5b8933940dd69061c6b077512168bebb6fea05d429b63ffbab191950798b4c825e8484b1a651db0ae13f97eae481097d3c16395659c0f3b9f847af2aaf44b65d
2022-07-12Merge bitcoin/bitcoin#25591: move-only: Version handshake to libtest_utilMacroFake
fa4be8e7c324835d3b9eecb5a3825a4c8f77fb2f move-only: InitializeNode to handshake helper (MacroFake) fa7098947ceff1964812d430e769af6d1ad561bd move-only: Version handshake to libtest_util (MacroFake) Pull request description: The version handshake after setting up a peer is an integral part of (unit) testing net processing logic. Thus, make the helper accessible in libtest_util. Also, remove the peerman argument from `FillNode`, as it must be equal to connman's peerman, which can then be used instead. ACKs for top commit: dergoegge: ACK fa4be8e7c324835d3b9eecb5a3825a4c8f77fb2f Tree-SHA512: 8296399dc2c29196bd56584c9b61f1c5a088f96dd3438b07b84e1acf525d867f1e37fdfdeede8a831add25848cda0c221ce3fb873e5ae5ca805a1765aa08eb12
2022-07-12move-only: InitializeNode to handshake helperMacroFake
2022-07-12move-only: Version handshake to libtest_utilMacroFake
2022-07-08wallet: refactor GetNewDestination, use BResultfurszy
2022-07-07refactor: Drop no longer needed `util/designator.h`Hennadii Stepanov
2022-07-07Merge bitcoin/bitcoin#25500: refactor: Move inbound eviction logic to its ↡fanquake
own translation unit 0101d2bc3c3bcf698d6cc2a237a680fc52395987 [net] Move eviction logic to its own file (dergoegge) c741d748d4d9836940b99091cc7be09c65efcb79 [net] Move ConnectionType to its own file (Cory Fields) a3c27070396ab8c2941c437e8099547e8fc9c110 [net] Add connection type to NodeEvictionCandidate (dergoegge) 42aa5d5b6269d27af525d5001907558442e96023 [net] Add NoBan status to NodeEvictionCandidate (dergoegge) Pull request description: This PR splits of the first couple commits from #25268 that move the inbound eviction logic from `net.{h,cpp}` to `eviction.{h,cpp}`. Please look at #25268 for motivation and conceptual review. ACKs for top commit: jnewbery: utACK 0101d2bc3c3bcf698d6cc2a237a680fc52395987 theuni: utACK 0101d2bc3c3bcf698d6cc2a237a680fc52395987. I quickly verified with `git --color-moved` that the move-only changes are indeed move-only. Tree-SHA512: e0c345a698030e049cb22fe281b44503c04403c5be5a3750ca14bfcc603a162ac6bac9a39552472feb57c460102b7ca91430b8ad6268f2efccc49b5e8959331b
2022-07-06[net] Move eviction logic to its own filedergoegge
2022-07-04[net] Add connection type to NodeEvictionCandidatedergoegge
2022-07-04[net] Add NoBan status to NodeEvictionCandidatedergoegge
2022-06-29Use AutoFile where possibleMacroFake
2022-06-29Merge bitcoin/bitcoin#25290: [kernel 3a/n] Decouple `CTxMemPool` from ↡MacroFake
`ArgsManager` d1684beabe5b738c2cc83de83e1aaef11a761b69 fees: Pass in a filepath instead of referencing gArgs (Carl Dong) 9a3d825c30e8e6118d74a4e568744cb9d03f7f5d init: Remove redundant -*mempool*, -limit* queries (Carl Dong) 6c5c60c4124293d948735756f84efc85262ea66f mempool: Use m_limit for UpdateTransactionsFromBlock (Carl Dong) 9e93b1030182eff92ef91181e17c7dd498c7e164 node/ifaces: Use existing MemPoolLimits (Carl Dong) 38af2bcf358a72b9457d370282e57f4be1c5c849 mempoolaccept: Use limits from mempool in constructor (Carl Dong) 9333427014695ac235c96d48791098168dfdc9db mempool: Introduce (still-unused) MemPoolLimits (Carl Dong) 716bb5fbd31077bbe99d11a54d6c2c250afc8085 scripted-diff: Rename anc/desc size limit vars to indicate SI unit (Carl Dong) 1ecc77321deb61b9f6888e4e10752b9d972fd26e scripted-diff: Rename DEFAULT_MEMPOOL_EXPIRY to indicate time unit (Carl Dong) aa9141cd8185cb7ad532bc16feb9d302b05d9697 mempool: Pass in -mempoolexpiry instead of referencing gArgs (Carl Dong) 51c7a41a5eb6fcb60333812c770d80227cf7b64d init: Only determine maxmempool once (Carl Dong) 386c9472c8764738282e6d163b42e15a8feda7ea mempool: Make GetMinFee() with custom size protected (Carl Dong) 82f00de7a6a60cbc9ad0c6e1d0ffb1bc70c49af5 mempool: Pass in -maxmempool instead of referencing gArgs (Carl Dong) f1941e8bfd2eecc478c7660434b1ebf6a64095a0 pool: Add and use MemPoolOptions, ApplyArgsManOptions (Carl Dong) 0199bd35bb44e32ee0db9b51c9d1bd7518c26f19 fuzz/rbf: Add missing TestingSetup (Carl Dong) ccbaf546a68d6cda8ed3efd0598c0e4121b366bb scripted-diff: Rename DEFAULT_MAX_MEMPOOL_SIZE to indicate SI unit (Carl Dong) fc02f77ca604f0221171bfde3059b34f5d0fb1cd ArgsMan: Add Get*Arg functions returning optional (Carl Dong) Pull request description: This is part of the `libbitcoinkernel` project: #24303, https://github.com/bitcoin/bitcoin/projects/18 ----- As mentioned in the Stage 1 Step 2 description of [the `libbitcoinkernel` project](https://github.com/bitcoin/bitcoin/issues/24303), `ArgsManager` will not be part of `libbitcoinkernel`. Therefore, it is important that we remove any dependence on `ArgsManager` by code that will be part of `libbitcoinkernel`. This is the first in a series of PRs aiming to achieve this. This PR removes `CTxMemPool+MempoolAccept`'s dependency on `ArgsManager` by introducing a `CTxMemPool::Options` struct, which is used to specify `CTxMemPool`'s various options at construction time. These options are: - `-maxmempool` -> `CTxMemPool::Options::max_size` - `-mempoolexpiry` -> `CTxMemPool::Options::expiry` - `-limitancestorcount` -> `CTxMemPool::Options::limits::ancestor_count` - `-limitancestorsize` -> `CTxMemPool::Options::limits::ancestor_size` - `-limitdescendantcount` -> `CTxMemPool::Options::limits::descendant_count` - `-limitdescendantsize` -> `CTxMemPool::Options::limits::descendant_size` More context can be gleaned from the commit messages. The important commits are: - 56eb479ded8bfb2ef635bb6f3b484f9d5952c70d "pool: Add and use MemPoolOptions, ApplyArgsManOptions" - a1e08b70f3068f4e8def1c630d8f50cd54da7832 "mempool: Pass in -maxmempool instead of referencing gArgs" - 6f4bf3ede5812b374828f08fc728ceded2f10024 "mempool: Pass in -mempoolexpiry instead of referencing gArgs" - 5958a7fe4806599fc620ee8c1a881ca10fa2dd16 "mempool: Introduce (still-unused) MemPoolLimits" Reviewers: Help needed in the following commits (see commit messages): - a1e08b70f3068f4e8def1c630d8f50cd54da7832 "mempool: Pass in -maxmempool instead of referencing gArgs" - 0695081a797e9a5d7787b78b0f8289dafcc6bff7 "node/ifaces: Use existing MemPoolLimits" Note to Reviewers: There are perhaps an infinite number of ways to architect `CTxMemPool::Options`, the current one tries to keep it simple, usable, and flexible. I hope we don't spend too much time arguing over the design here since that's not the point. In the case that you're 100% certain that a different design is strictly better than this one in every regard, please show us a fully-implemented branch. ----- TODO: - [x] Use the more ergonomic `CTxMemPool::Options` where appropriate - [x] Doxygen comments for `ApplyArgsManOptions`, `MemPoolOptions` ----- Questions for Reviewers: 1. Should we use `std::chrono::seconds` for `CTxMemPool::Options::expiry` and `CTxMemPool::m_expiry` instead of an `int64_t`? Something else? (`std::chrono::hours`?) 2. Should I merge `CTxMemPool::Limits` inside `CTxMemPool::Options`? ACKs for top commit: MarcoFalke: ACK d1684beabe5b738c2cc83de83e1aaef11a761b69 🍜 ryanofsky: Code review ACK d1684beabe5b738c2cc83de83e1aaef11a761b69. Just minor cleanups since last review, mostly switching to brace initialization Tree-SHA512: 2c138e52d69f61c263f1c3648f01c801338a8f576762c815f478ef5148b8b2f51e91ded5c1be915e678c0b14f6cfba894b82afec58d999d39a7bb7c914736e0b
2022-06-28fees: Pass in a filepath instead of referencing gArgsCarl Dong
2022-06-28pool: Add and use MemPoolOptions, ApplyArgsManOptionsCarl Dong
Reviewers: Note that CTxMemPool now requires a non-defaulted CTxMemPool::Options for its constructor. Meaning that there's no need to worry about a stray CTxMemPool constructor somewhere defaulting to something incorrect. All instances of CTxMemPool construction are addressed here in this commit. We set options for CTxMemPool and construct it in many different ways. A good example can be seen in how we determine CTxMemPool's check_ratio in AppInitMain(...). 1. We first set the default based on chainparams's DefaultConsistencyChecks() 2. Then, we apply the ArgsManager option on top of that default 3. Finally, we clamp the result of that between 0 and 1 Million With this patch, most CTxMemPool construction are along the lines of: MemPoolOptions mempool_opts{...default overrides...}; ApplyArgsManOptions(argsman, mempool_opts); ...hard overrides... CTxMemPool pool{mempool_opts}; This "compositional" style of building options means that we can omit unnecessary/irrelevant steps wherever we want but also maintain full customizability. For example: - For users of libbitcoinkernel, where we eventually want to remove ArgsManager, they simply won't call (or even know about) ApplyArgsManOptions. - See src/init.cpp to see how the check_ratio CTxMemPool option works after this change. A MemPoolOptionsForTest helper was also added and used by tests/fuzz tests where a local CTxMemPool needed to be created. The change in src/test/fuzz/tx_pool.cpp seemingly changes behaviour by applying ArgsManager options on top of the CTxMemPool::Options defaults. However, in future commits where we introduce flags like -maxmempool, the call to ApplyArgsManOptions is actually what preserves the existing behaviour. Previously, although it wasn't obvious, our CTxMemPool would consult gArgs for flags like -maxmempool when it needed it, so it already relied on ArgsManager information. This patchset just laid bare the obfuscatory perils of globals. [META] As this patchset progresses, we will move more and more CTxMemPool-relevant options into MemPoolOptions and add their ArgsMan-related logic to ApplyArgsManOptions.
2022-06-28Merge bitcoin/bitcoin#24378: refactor: make bind() and listen() ↡laanwj
mockable/testable b2733ab6a85b234a88b83bdc77a0d043e18385b3 net: add new method Sock::Listen() that wraps listen() (Vasil Dimov) 3ad7de225efce3e76530f56bee8a8f7a75ea0f3c net: add new method Sock::Bind() that wraps bind() (Vasil Dimov) Pull request description: _This is a piece of #21878, chopped off to ease review._ Add new methods `Sock::Bind()` and `Sock::Listen()` that wrap `bind()` and `listen()`. This will help to increase `Sock` usage and make more code mockable. ACKs for top commit: pk-b2: ACK b2733ab6a85b234a88b83bdc77a0d043e18385b3 laanwj: Code review ACK b2733ab6a85b234a88b83bdc77a0d043e18385b3 Tree-SHA512: c6e737606703e2106fe60cc000cfbbae3a7f43deadb25f70531e2cac0457e0b0581440279d14c76c492eb85c12af4adde52c30baf74542c41597e419817488e8
2022-06-28Merge bitcoin/bitcoin#25426: net: add new method Sock::GetSockName() that ↡laanwj
wraps getsockname() and use it in GetBindAddress() a8d6abba5ec4ae2a3375e9be0b739f298899eca2 net: change GetBindAddress() to take Sock argument (Vasil Dimov) 748dbcd9f29dbe4110da8a06f08e3eefa95f5321 net: add new method Sock::GetSockName() that wraps getsockname() (Vasil Dimov) Pull request description: _This is a piece of #21878, chopped off to ease review._ Wrap the syscall `getsockname()` in `Sock::GetSockName()` and change `GetBindAddress()` to take a `Sock` argument so that it can use the wrapper. This further encapsulates syscalls inside the `Sock` class and makes the callers mockable. ACKs for top commit: laanwj: Code review ACK a8d6abba5ec4ae2a3375e9be0b739f298899eca2 Tree-SHA512: 3a73463258c0057487fb3fd67215816b03a1c5160f45e45930eaeef86bb3611ec385794cdb08339aa074feba8ad67cd2bfd3836f6cbd40834e15d933214a05dc
2022-06-22net: rename Sock::Reset() to Sock::Close() and make it privateVasil Dimov
Outside of `Sock`, `Sock::Reset()` was used in just one place (in `i2p.cpp`) which can use the assignment operator instead. This simplifies the public `Sock` API by having one method less.
2022-06-20net: add new method Sock::GetSockName() that wraps getsockname()Vasil Dimov
This will help to increase `Sock` usage and make more code mockable.
2022-06-16Merge bitcoin/bitcoin#24356: refactor: replace CConnman::SocketEvents() with ↡laanwj
mockable Sock::WaitMany() 6e68ccbefea6509c61fc4405a391a517c6057bb0 net: use Sock::WaitMany() instead of CConnman::SocketEvents() (Vasil Dimov) ae263460bab9e6aa112dc99790c8ef06a56ec838 net: introduce Sock::WaitMany() (Vasil Dimov) cc74459768063a923fb6220a4f420eaf211aee7b net: also wait for exceptional events in Sock::Wait() (Vasil Dimov) Pull request description: _This is a piece of #21878, chopped off to ease review._ `Sock::Wait()` waits for IO events on one socket. Introduce a similar `virtual` method `WaitMany()` that waits simultaneously for IO events on more than one socket. Use `WaitMany()` instead of `CConnman::SocketEvents()` (and ditch the latter). Given that the former is a `virtual` method, it can be mocked by unit and fuzz tests. This will help to make bigger parts of `CConnman` testable (unit and fuzz). ACKs for top commit: laanwj: Code review ACK 6e68ccbefea6509c61fc4405a391a517c6057bb0 jonatack: re-ACK 6e68ccbefea6509c61fc4405a391a517c6057bb0 per `git range-diff e18fd47 6747729 6e68ccb`, and verified rebase to master and debug build Tree-SHA512: 917fb6ad880d64d3af1ebb301c06fbd01afd8ff043f49e4055a088ebed6affb7ffe1dcf59292d822f10de5f323b6d52d557cb081dd7434634995f9148efcf08f
2022-06-15Merge bitcoin/bitcoin#25223: [kernel 2e/n] miner: Make `mempool` optional, ↡fanquake
stop constructing temporary empty mempools 0f1a259657280afc727db97689512aef5ca928fc miner: Make mempool optional for BlockAssembler (Carl Dong) cc5739b27df830d138119eaa13f2286d91d0dadd miner: Make UpdatePackagesForAdded static (Carl Dong) f024578b3a5c40e275e23d1c8e82530e235fdbf9 miner: Absorb SkipMapTxEntry into addPackageTxs (Carl Dong) Pull request description: This is part of the libbitcoinkernel project: #24303, https://github.com/bitcoin/bitcoin/projects/18 This is **_NOT_** dependent on, but is a "companion-PR" to #25215. ### Abstract This PR removes the need to construct `BlockAssembler` with temporary, empty mempools in cases where we don't want to source transactions from the mempool (e.g. in `TestChain100Setup::CreateBlock` and `generateblock`). After this PR, `BlockAssembler` will accept a `CTxMemPool` pointer and handle the `nullptr` case instead of requiring a `CTxMemPool` reference. An overview of the changes is best seen in the changes in the header file: ```diff diff --git a/src/node/miner.h b/src/node/miner.h index 7cf8e3fb9e..7e9f503602 100644 --- a/src/node/miner.h +++ b/src/node/miner.h @@ -147,7 +147,7 @@ private: int64_t m_lock_time_cutoff; const CChainParams& chainparams; - const CTxMemPool& m_mempool; + const CTxMemPool* m_mempool; CChainState& m_chainstate; public: @@ -157,8 +157,8 @@ public: CFeeRate blockMinFeeRate; }; - explicit BlockAssembler(CChainState& chainstate, const CTxMemPool& mempool); - explicit BlockAssembler(CChainState& chainstate, const CTxMemPool& mempool, const Options& options); + explicit BlockAssembler(CChainState& chainstate, const CTxMemPool* mempool); + explicit BlockAssembler(CChainState& chainstate, const CTxMemPool* mempool, const Options& options); /** Construct a new block template with coinbase to scriptPubKeyIn */ std::unique_ptr<CBlockTemplate> CreateNewBlock(const CScript& scriptPubKeyIn); @@ -177,7 +177,7 @@ private: /** Add transactions based on feerate including unconfirmed ancestors * Increments nPackagesSelected / nDescendantsUpdated with corresponding * statistics from the package selection (for logging statistics). */ - void addPackageTxs(int& nPackagesSelected, int& nDescendantsUpdated) EXCLUSIVE_LOCKS_REQUIRED(m_mempool.cs); + void addPackageTxs(const CTxMemPool& mempool, int& nPackagesSelected, int& nDescendantsUpdated) EXCLUSIVE_LOCKS_REQUIRED(mempool.cs); // helper functions for addPackageTxs() /** Remove confirmed (inBlock) entries from given set */ @@ -189,15 +189,8 @@ private: * These checks should always succeed, and they're here * only as an extra check in case of suboptimal node configuration */ bool TestPackageTransactions(const CTxMemPool::setEntries& package) const; - /** Return true if given transaction from mapTx has already been evaluated, - * or if the transaction's cached data in mapTx is incorrect. */ - bool SkipMapTxEntry(CTxMemPool::txiter it, indexed_modified_transaction_set& mapModifiedTx, CTxMemPool::setEntries& failedTx) EXCLUSIVE_LOCKS_REQUIRED(m_mempool.cs); /** Sort the package in an order that is valid to appear in a block */ void SortForBlock(const CTxMemPool::setEntries& package, std::vector<CTxMemPool::txiter>& sortedEntries); - /** Add descendants of given transactions to mapModifiedTx with ancestor - * state updated assuming given transactions are inBlock. Returns number - * of updated descendants. */ - int UpdatePackagesForAdded(const CTxMemPool::setEntries& alreadyAdded, indexed_modified_transaction_set& mapModifiedTx) EXCLUSIVE_LOCKS_REQUIRED(m_mempool.cs); }; int64_t UpdateTime(CBlockHeader* pblock, const Consensus::Params& consensusParams, const CBlockIndex* pindexPrev); ``` ### Alternatives Aside from approach in this current PR, we can also take the approach of moving the `CTxMemPool*` argument from the `BlockAssembler` constructor to `BlockAssembler::CreateNewBlock`, since that's where it's needed anyway. I did not push this approach because it requires quite a lot of call sites to be changed. However, I do have it coded up and can do that if people express a strong preference. This would look something like: ``` BlockAssembler::BlockAssembler(CChainState& chainstate, const Options& options); BlockAssembler::CreateNewBlock(const CScript& scriptPubKeyIn, const CTxMemPool* maybe_mempool); ``` ### Future work Although wholly out of scope for this PR, we could potentially refine the `BlockAssembler` interface further, so that we have: ``` BlockAssembler::BlockAssembler(CChainState& chainstate, const Options& options); BlockAssembler::CreateNewBlock(const CScript& scriptPubKeyIn, std::vector<CTransaction>& txs); BlockAssembler::CreateNewBlock(const CScript& scriptPubKeyIn, const CTxMemPool& mempool); ``` Whereby `TestChain100Setup::CreateBlock` and `generateblock` would call the `BlockAssembler::CreateNewBlock` that takes in `CTransaction`s and we can potentially remove `RegenerateCommitments` altogether. All other callers can use the `CTxMemPool` version. ACKs for top commit: glozow: ACK 0f1a259657280afc727db97689512aef5ca928fc laanwj: Code review ACK 0f1a259657280afc727db97689512aef5ca928fc MarcoFalke: ACK 0f1a259657280afc727db97689512aef5ca928fc 🐊 Tree-SHA512: 2b4b1dbb43d85719f241ad1f19ceb7fc50cf764721da425a3d1ff71bd16328c4f86acff22e565bc9abee770d3ac8827a6676b66daa93dbf42dd817ad929e9448
2022-06-09net: introduce Sock::WaitMany()Vasil Dimov
It allows waiting concurrently on more than one socket. Being a `virtual` `Sock` method it can be overriden by tests. Will be used to replace `CConnman::SocketEvents()`.
2022-06-06miner: Make mempool optional for BlockAssemblerCarl Dong
...also adjust callers Changes: - In BlockAssembler::CreateNewBlock, we now only lock m_mempool->cs and call addPackageTxs if m_mempool is not nullptr - BlockAssembler::addPackageTxs now takes in a mempool reference, and is annotated to require that mempool's lock. - In TestChain100Setup::CreateBlock and generateblock, don't construct an empty mempool, just pass in a nullptr for mempool
2022-06-04Merge bitcoin/bitcoin#25065: [kernel 2c/n] Introduce `kernel::Context`, ↡fanquake
encapsulate global init/teardown d87784ac87364fc977bbf9769c8bdb72dea8cbf9 kernel: SanityChecks: Return an error struct (Carl Dong) 265d6393bf9ef52e7ef7de97ca9c031da82a5ad1 Move init::SanityCheck to kernel::SanityCheck (Carl Dong) fed085a1a4cd2787202752b6a0d98e42dce97f09 init: Initialize globals with kernel::Context's life (Carl Dong) 7d03feef8156ef37a4efa01dc591467bc7d957bf kernel: Introduce empty and unused kernel::Context (Carl Dong) eeb4fc20c578b1e428a92d64cc9f8f903a677580 test: Use Set/UnsetGlobals in BasicTestingSetup (Carl Dong) Pull request description: The full `init/common.cpp` is dependent on things like ArgsManager (which we wish to remove from libbitcoinkernel in the future) and sanity checks. These aren't necessary for libbitcoinkernel so we only extract the portion that is necessary (namely `init::{Set,Unset}Globals()`. ACKs for top commit: theuni: ACK d87784ac87364fc977bbf9769c8bdb72dea8cbf9 vasild: ACK d87784ac87364fc977bbf9769c8bdb72dea8cbf9 Tree-SHA512: cd6b4923ea1865001b5f0caed9a4ff99c198d22bf74154d935dc09a47fda22ebe585ec912398cea69f722454ed1dbb4898faab5a2d02fb4c5e719c5c8d71a3f9
2022-06-02init: Initialize globals with kernel::Context's lifeCarl Dong
...instead of explicitly calling init::{Set,Unset}Globals. Cool thing about this is that in both the testing and bitcoin-chainstate codepaths, we no longer need to explicitly unset globals. The kernel::Context goes out of scope and the globals are unset "automatically". Also construct kernel::Context outside of AppInitSanityChecks()
2022-05-31test: Use Set/UnsetGlobals in BasicTestingSetupCarl Dong
...instead of calling initialization functions directly and having to keep around a ECCVerifyHandle member variable. This makes the initialization codepath of our tests more closely resemble those of AppInitMain and potentially eases the review of subsequent commit removing init::{Set,Unset}Globals. [META] In a future commit, we will introduce a kernel::Context which calls init::{Set,Unset}Globals in its ctor and dtor. It will be owned by node::NodeContext, so in the end, this patchset won't have made the previously local ECCVerifyHandle global.
generated by cgit v1.2.3 (git 2.26.2) at 2024-09-02 13:36:27 +0000