Merge #16899: UTXO snapshot creation (dumptxoutset)

92b2f5306ba0b3f031293cb8f415b67cb002c2f1 test: add dumptxoutset RPC test (James O'Beirne)
c1ccbc3ddef931896a7e9dcfa6704e305a69fbff devtools: add utxo_snapshot.sh (James O'Beirne)
57cf74c9918d10c69a46e6ceb3cb1a5e04edf5bc rpc: add dumptxoutset (James O'Beirne)
92fafb3a7da66f737e960e541fcfbcadedf6043a coinstats: add coins_count (James O'Beirne)
707fde7b9ba522c22179e2db0ed7b462c65138d9 add unused SnapshotMetadata class (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal

  ---

  This changeset defines the serialization format for UTXO snapshots and adds an RPC command for creating them, `dumptxoutset`. It also adds a convenience script for generating and verifying snapshots at a certain height, since that requires doing a hacky rewind of the chain via `invalidateblock`.

  All of this is unused at the moment.

ACKs for top commit:
  laanwj:
    ACK 92b2f5306ba0b3f031293cb8f415b67cb002c2f1

Tree-SHA512: 200dff87767f157d627e99506ec543465d9329860a6cd49363081619c437163a640a46d008faa92b1f44fd403bfc7a7c9e851c658b5a4849efa9a34ca976bf31

1 files changed, 44 insertions, 0 deletions

diff --git a/contrib/devtools/utxo_snapshot.sh b/contrib/devtools/utxo_snapshot.sh
new file mode 100755
index 0000000000..dee25ff67b
--- /dev/null
+++ b/contrib/devtools/utxo_snapshot.sh
@@ -0,0 +1,44 @@
+#!/usr/bin/env bash
+#
+# Copyright (c) 2019 The Bitcoin Core developers
+# Distributed under the MIT software license, see the accompanying
+# file COPYING or http://www.opensource.org/licenses/mit-license.php.
+#
+export LC_ALL=C
+
+set -ueo pipefail
+
+if (( $# < 3 )); then
+  echo 'Usage: utxo_snapshot.sh <generate-at-height> <snapshot-out-path> <bitcoin-cli-call ...>'
+  echo
+  echo "  if <snapshot-out-path> is '-', don't produce a snapshot file but instead print the "
+  echo "  expected assumeutxo hash"
+  echo
+  echo 'Examples:'
+  echo
+  echo "  ./contrib/devtools/utxo_snapshot.sh 570000 utxo.dat ./src/bitcoin-cli -datadir=\$(pwd)/testdata"
+  echo '  ./contrib/devtools/utxo_snapshot.sh 570000 - ./src/bitcoin-cli'
+  exit 1
+fi
+
+GENERATE_AT_HEIGHT="${1}"; shift;
+OUTPUT_PATH="${1}"; shift;
+# Most of the calls we make take a while to run, so pad with a lengthy timeout.
+BITCOIN_CLI_CALL="${*} -rpcclienttimeout=9999999"
+
+# Block we'll invalidate/reconsider to rewind/fast-forward the chain.
+PIVOT_BLOCKHASH=$($BITCOIN_CLI_CALL getblockhash $(( GENERATE_AT_HEIGHT + 1 )) )
+
+(>&2 echo "Rewinding chain back to height ${GENERATE_AT_HEIGHT} (by invalidating ${PIVOT_BLOCKHASH}); this may take a while")
+${BITCOIN_CLI_CALL} invalidateblock "${PIVOT_BLOCKHASH}"
+
+if [[ "${OUTPUT_PATH}" = "-" ]]; then
+  (>&2 echo "Generating txoutset info...")
+  ${BITCOIN_CLI_CALL} gettxoutsetinfo | grep hash_serialized_2 | sed 's/^.*: "\(.\+\)\+",/\1/g'
+else
+  (>&2 echo "Generating UTXO snapshot...")
+  ${BITCOIN_CLI_CALL} dumptxoutset "${OUTPUT_PATH}"
+fi
+
+(>&2 echo "Restoring chain to original height; this may take a while")
+${BITCOIN_CLI_CALL} reconsiderblock "${PIVOT_BLOCKHASH}"