diff options
author | fanquake <fanquake@gmail.com> | 2022-07-29 15:43:21 +0100 |
---|---|---|
committer | fanquake <fanquake@gmail.com> | 2022-07-29 15:47:23 +0100 |
commit | 5871b5b5ab57a0caf9b7514eb162c491c83281d5 (patch) | |
tree | b2ca982303e301ae389a0bef181432b9377e5500 /src/validation.cpp | |
parent | b1c8ea45c9e0b658c86ed0bbbf6e0fe44cd40f1c (diff) | |
parent | dd065dae9fcebd6806ff67703ffa8128e80b97cc (diff) | |
download | bitcoin-5871b5b5ab57a0caf9b7514eb162c491c83281d5.tar.xz |
Merge bitcoin/bitcoin#25571: refactor: Make mapBlocksUnknownParent local, and rename it
dd065dae9fcebd6806ff67703ffa8128e80b97cc refactor: Make mapBlocksUnknownParent local, and rename it (Hennadii Stepanov)
Pull request description:
This PR is a second attempt at #19594. This PR has two motivations:
- Improve code hygiene by eliminating a global variable, `mapBlocksUnknownParent`
- Fix fuzz test OOM when running too long ([see #19594 comment](https://github.com/bitcoin/bitcoin/pull/19594#issuecomment-958801638))
A minor added advantage is to release `mapBlocksUnknownParent` memory when the reindexing phase is done. The current situation is somewhat similar to a memory leak because this map exists unused for the remaining lifetime of the process. It's true that this map should be empty of data elements after use, but its internal metadata (indexing structures, etc.) can have non-trivial size because there can be many thousands of simultaneous elements in this map.
This PR helps our efforts to reduce the use of global variables. This variable isn't just global, it's hidden inside a function (it looks like a local variable but has the `static` attribute).
This global variable exists because the `-reindex` processing code calls `LoadExternalBlockFile()` multiple times (once for each block file), but that function must preserve some state between calls (the `mapBlocksUnknownParent` map). This PR fixes this by allocating this map as a local variable in the caller's scope and passing it in on each call. When reindexing completes, the map goes out of scope and is deallocated.
I tested this manually by reindexing on mainnet and signet. Also, the existing `feature_reindex.py` functional test passes.
ACKs for top commit:
mzumsande:
re-ACK dd065dae9fcebd6806ff67703ffa8128e80b97cc
theStack:
re-ACK dd065dae9fcebd6806ff67703ffa8128e80b97cc
shaavan:
reACK dd065dae9fcebd6806ff67703ffa8128e80b97cc
Tree-SHA512: 9cd20e44d2fa1096dd405bc107bc065ea8f904f5b3f63080341b08d8cf57b790df565f58815c2f331377d044d5306708b4bf6bdfc5ef8d0ed85d8e97d744732c
Diffstat (limited to 'src/validation.cpp')
-rw-r--r-- | src/validation.cpp | 23 |
1 files changed, 16 insertions, 7 deletions
diff --git a/src/validation.cpp b/src/validation.cpp index d64ef4df0b..17211956f5 100644 --- a/src/validation.cpp +++ b/src/validation.cpp @@ -57,6 +57,7 @@ #include <warnings.h> #include <algorithm> +#include <cassert> #include <chrono> #include <deque> #include <numeric> @@ -4256,11 +4257,16 @@ bool CChainState::LoadGenesisBlock() return true; } -void CChainState::LoadExternalBlockFile(FILE* fileIn, FlatFilePos* dbp) +void CChainState::LoadExternalBlockFile( + FILE* fileIn, + FlatFilePos* dbp, + std::multimap<uint256, FlatFilePos>* blocks_with_unknown_parent) { AssertLockNotHeld(m_chainstate_mutex); - // Map of disk positions for blocks with unknown parent (only used for reindex) - static std::multimap<uint256, FlatFilePos> mapBlocksUnknownParent; + + // Either both should be specified (-reindex), or neither (-loadblock). + assert(!dbp == !blocks_with_unknown_parent); + int64_t nStart = GetTimeMillis(); int nLoaded = 0; @@ -4310,8 +4316,9 @@ void CChainState::LoadExternalBlockFile(FILE* fileIn, FlatFilePos* dbp) if (hash != m_params.GetConsensus().hashGenesisBlock && !m_blockman.LookupBlockIndex(block.hashPrevBlock)) { LogPrint(BCLog::REINDEX, "%s: Out of order block %s, parent %s not known\n", __func__, hash.ToString(), block.hashPrevBlock.ToString()); - if (dbp) - mapBlocksUnknownParent.insert(std::make_pair(block.hashPrevBlock, *dbp)); + if (dbp && blocks_with_unknown_parent) { + blocks_with_unknown_parent->emplace(block.hashPrevBlock, *dbp); + } continue; } @@ -4340,13 +4347,15 @@ void CChainState::LoadExternalBlockFile(FILE* fileIn, FlatFilePos* dbp) NotifyHeaderTip(*this); + if (!blocks_with_unknown_parent) continue; + // Recursively process earlier encountered successors of this block std::deque<uint256> queue; queue.push_back(hash); while (!queue.empty()) { uint256 head = queue.front(); queue.pop_front(); - std::pair<std::multimap<uint256, FlatFilePos>::iterator, std::multimap<uint256, FlatFilePos>::iterator> range = mapBlocksUnknownParent.equal_range(head); + auto range = blocks_with_unknown_parent->equal_range(head); while (range.first != range.second) { std::multimap<uint256, FlatFilePos>::iterator it = range.first; std::shared_ptr<CBlock> pblockrecursive = std::make_shared<CBlock>(); @@ -4361,7 +4370,7 @@ void CChainState::LoadExternalBlockFile(FILE* fileIn, FlatFilePos* dbp) } } range.first++; - mapBlocksUnknownParent.erase(it); + blocks_with_unknown_parent->erase(it); NotifyHeaderTip(*this); } } |