Merge bitcoin/bitcoin#21526: validation: UpdateTip/CheckBlockIndex assumeutxo support

673a5bd3377929a0a6a62eda8b560e47bc2cca0c test: validation: add unittest for UpdateTip behavior (James O'Beirne)
2705570109a2a90ecfd3f4180944498626fc2707 test: refactor: separate CreateBlock in TestChain100Setup (James O'Beirne)
298bf5d563cc740c6ae71750d86942e0278b22d6 test: refactor: declare NoMalleation const auto (James O'Beirne)
071200993f3a9412821ce5387851d659baf85327 move-only: unittest: add test/util/chainstate.h (James O'Beirne)
8f5710fd0ac5173b577e5d00708485170b321bcc validation: fix CheckBlockIndex for multiple chainstates (James O'Beirne)
5a807736dacfc3e6fa57231219336acf08be38fb validation: insert assumed-valid block index entries into candidates (James O'Beirne)
01a9b8fe719efab2c268dc738bc93cfbdf92edb7 validation: set BLOCK_ASSUMED_VALID during snapshot load (James O'Beirne)
42b2520db93fd9feb3df4101654391fa7d3e2140 chain: add BLOCK_ASSUMED_VALID for use with assumeutxo (James O'Beirne)
b217020df78bc981d221fe04497c831120ef969f validation: change UpdateTip for multiple chainstates (James O'Beirne)
665072a36df2e4c88705fedd4ac7c955d7f6a488 doc: add comment for g_best_block (James O'Beirne)
ac4051d891e2d5c8ac130da16b85b9d880b44720 refactor: remove unused assumeutxo methods (James O'Beirne)
9f6bb539359b98d5b39482ab8a28a68608f0c645 validation: add chainman ref to CChainState (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11) (parent PR: #15606)

  ---

  Modify UpdateTip and CheckBlockIndex for use with multiple chainstates. Includes a new unittest verifying `g_best_block` behavior (previously untested at the unit level) and various changes necessary for running and testing `ProcessNewBlock()`-like behavior on the background validation chainstate.

  This changeset introduces a new block index `nStatus` flag called `BLOCK_ASSUMED_VALID`, and it is applied to block index entries that are beneath the UTXO snapshot base block upon snapshot load. Once each block is validated (during async background validation), the flag is removed. This allows us to avoid (ab)using `BLOCK_VALID_*` flags for snapshot chain block entries, and preserves the original meaning of those flags.

  Note: this PR previously incorporated changes to `LoadBlockIndex()` and `RewindBlockIndex()` as noted in Russ' comments below, but once I generated the changes necessary to test the UpdateTip change, I decided to split this changes out into another PR due to the size of this one.

ACKs for top commit:
  achow101:
    ACK 673a5bd3377929a0a6a62eda8b560e47bc2cca0c
  jonatack:
    Code-review re-ACK 673a5bd3377929a0a6a62eda8b560e47bc2cca0c reviewed diff, rebased to master/debug build/ran unit+functional tests
  naumenkogs:
    ACK 673a5bd3377929a0a6a62eda8b560e47bc2cca0c
  fjahr:
    Code review ACK 673a5bd3377929a0a6a62eda8b560e47bc2cca0c
  ariard:
    utACK 673a5bd3
  ryanofsky:
    Code review ACK 673a5bd3377929a0a6a62eda8b560e47bc2cca0c. Just linker fix and split commit changes mentioned https://github.com/bitcoin/bitcoin/pull/21526#issuecomment-921064563 since last review
  benthecarman:
    ACK 673a5bd3377929a0a6a62eda8b560e47bc2cca0c

Tree-SHA512: 0a6dc23d041b27ed9fd0ee1f3e5971b92fb1d2df2fc9b655d5dc48594235321ab1798d06de2ec55482ac3966a9ed56de8d56e9e29cae75bbe8690bafc2dda383

1 files changed, 76 insertions, 0 deletions

diff --git a/src/test/validation_chainstate_tests.cpp b/src/test/validation_chainstate_tests.cpp
index 315ef22599..726c9ebbb8 100644
--- a/src/test/validation_chainstate_tests.cpp
+++ b/src/test/validation_chainstate_tests.cpp
@@ -2,10 +2,13 @@
 // Distributed under the MIT software license, see the accompanying
 // file COPYING or http://www.opensource.org/licenses/mit-license.php.
 //
+#include <chainparams.h>
 #include <random.h>
 #include <uint256.h>
 #include <consensus/validation.h>
 #include <sync.h>
+#include <rpc/blockchain.h>
+#include <test/util/chainstate.h>
 #include <test/util/setup_common.h>
 #include <validation.h>
 
@@ -73,4 +76,77 @@ BOOST_AUTO_TEST_CASE(validation_chainstate_resize_caches)
     WITH_LOCK(::cs_main, manager.Unload());
 }
 
+//! Test UpdateTip behavior for both active and background chainstates.
+//!
+//! When run on the background chainstate, UpdateTip should do a subset
+//! of what it does for the active chainstate.
+BOOST_FIXTURE_TEST_CASE(chainstate_update_tip, TestChain100Setup)
+{
+    ChainstateManager& chainman = *Assert(m_node.chainman);
+    uint256 curr_tip = ::g_best_block;
+
+    // Mine 10 more blocks, putting at us height 110 where a valid assumeutxo value can
+    // be found.
+    mineBlocks(10);
+
+    // After adding some blocks to the tip, best block should have changed.
+    BOOST_CHECK(::g_best_block != curr_tip);
+
+    BOOST_REQUIRE(CreateAndActivateUTXOSnapshot(m_node, m_path_root));
+
+    // Ensure our active chain is the snapshot chainstate.
+    BOOST_CHECK(chainman.IsSnapshotActive());
+
+    curr_tip = ::g_best_block;
+
+    // Mine a new block on top of the activated snapshot chainstate.
+    mineBlocks(1);  // Defined in TestChain100Setup.
+
+    // After adding some blocks to the snapshot tip, best block should have changed.
+    BOOST_CHECK(::g_best_block != curr_tip);
+
+    curr_tip = ::g_best_block;
+
+    CChainState* background_cs;
+
+    BOOST_CHECK_EQUAL(chainman.GetAll().size(), 2);
+    for (CChainState* cs : chainman.GetAll()) {
+        if (cs != &chainman.ActiveChainstate()) {
+            background_cs = cs;
+        }
+    }
+    BOOST_CHECK(background_cs);
+
+    // Create a block to append to the validation chain.
+    std::vector<CMutableTransaction> noTxns;
+    CScript scriptPubKey = CScript() << ToByteVector(coinbaseKey.GetPubKey()) << OP_CHECKSIG;
+    CBlock validation_block = this->CreateBlock(noTxns, scriptPubKey, *background_cs);
+    auto pblock = std::make_shared<const CBlock>(validation_block);
+    BlockValidationState state;
+    CBlockIndex* pindex = nullptr;
+    const CChainParams& chainparams = Params();
+    bool newblock = false;
+
+    // TODO: much of this is inlined from ProcessNewBlock(); just reuse PNB()
+    // once it is changed to support multiple chainstates.
+    {
+        LOCK(::cs_main);
+        bool checked = CheckBlock(*pblock, state, chainparams.GetConsensus());
+        BOOST_CHECK(checked);
+        bool accepted = background_cs->AcceptBlock(
+            pblock, state, &pindex, true, nullptr, &newblock);
+        BOOST_CHECK(accepted);
+    }
+    // UpdateTip is called here
+    bool block_added = background_cs->ActivateBestChain(state, pblock);
+
+    // Ensure tip is as expected
+    BOOST_CHECK_EQUAL(background_cs->m_chain.Tip()->GetBlockHash(), validation_block.GetHash());
+
+    // g_best_block should be unchanged after adding a block to the background
+    // validation chain.
+    BOOST_CHECK(block_added);
+    BOOST_CHECK_EQUAL(curr_tip, ::g_best_block);
+}
+
 BOOST_AUTO_TEST_SUITE_END()