diff options
author | W. J. van der Laan <laanwj@protonmail.com> | 2021-10-19 15:47:28 +0200 |
---|---|---|
committer | W. J. van der Laan <laanwj@protonmail.com> | 2021-10-19 15:47:53 +0200 |
commit | 986003aff93c099c400c9285b4a2ed63f4b3f180 (patch) | |
tree | 7090f4c628e07cd9fd4202956e77f02b3a697f1d | |
parent | 2d2516196bd2d9d9a60117df72518620151214f4 (diff) | |
parent | 5c34507ecbbdc29c086276d1c62835b461823507 (diff) | |
download | bitcoin-986003aff93c099c400c9285b4a2ed63f4b3f180.tar.xz |
Merge bitcoin/bitcoin#22918: rpc: Add level 3 verbosity to getblock RPC call (#21245 modified)
5c34507ecbbdc29c086276d1c62835b461823507 core_write: Rename calculate_fee to have_undo for clarity (fyquah)
8edf6204a87057a451160d1e61e79d8be112e81f release-notes: Add release note about getblock verbosity level 3. (fyquah)
459104b2aae6eeaadfa5a7e47944f1a34780dacd rest: Add test for prevout fields in getblock (fyquah)
4330af6f72172848f5971a052a8f325ed50eb576 rpc: Add test for level 3 verbosity getblock rpc call. (fyquah)
51dbc167e98daab317baa80cf80bfda337672dab rpc: Add level 3 verbosity to getblock RPC call. (fyquah)
3cc95345ca49b87e8caca9a0e6418c63ae1e463a rpc: Replace boolean argument for tx details with enum class. (fyquah)
Pull request description:
Author of #21245 expressed [time issues](https://github.com/bitcoin/bitcoin/pull/21245#issuecomment-902332088) in the original PR. Given that #21245 has received a lot of review*, I have decided to open this new pull request with [modifications required to get ACK from luke-jr ](https://github.com/bitcoin/bitcoin/pull/21245#issuecomment-905150806) and a few nits of mine.
### Original PR description
> Display the prevout in transaction inputs when calling getblock level 3 verbosity. This PR affects the existing `/rest/block` API by adding a `prevout` fields to tx inputs. This is mentioned in the change to the release notes.
>
> I added some functional tests that
>
> * checks that the RPC call still works when TxUndo can't be found
>
> * Doesn't display the "value" or "scriptPubKey" of the previous output when at a lower verbosity level
>
>
> This "completes" the issue #18771
### Possible improvements
* https://github.com/kiminuo/bitcoin/commit/b0bf4f255f86aeaddce68889087c22f9068f4d97 - I can include even this commit to this PR if deemed useful or I can leave it for a follow-up PR. See https://github.com/bitcoin/bitcoin/pull/21245#issuecomment-894853784 for more context.
### Examples
Examples of the `getblock` output with various verbose levels. Note that `000000000000001f682b188971cc1a121546be4e9d5baf22934fdc7f538288d5` contains only 2 transactions.
#### Verbose level 0
```bash
./bitcoin-cli -testnet getblock 000000000000001f682b188971cc1a121546be4e9d5baf22934fdc7f538288d5 0
```
##### Verbose level 1
```bash
./bitcoin-cli -testnet getblock 000000000000001f682b188971cc1a121546be4e9d5baf22934fdc7f538288d5 1
```
##### Verbose level 2
```bash
./bitcoin-cli -testnet getblock 000000000000001f682b188971cc1a121546be4e9d5baf22934fdc7f538288d5 2
```
##### Verbose level 3
```bash
./bitcoin-cli -testnet getblock 000000000000001f682b188971cc1a121546be4e9d5baf22934fdc7f538288d5 3
```
#### REST
```bash
curl -H "content-type:text/plain;" http://127.0.0.1:18332/rest/block/000000000000001f682b188971cc1a121546be4e9d5baf22934fdc7f538288d5.json
```
<sub>* ... and my everyday obsessive checking of my email inbox whether the PR moves forward.</sub>
Edit laanwj: Removed at symbol from message, and large example output to prevent it from all ending up in the commit message.
ACKs for top commit:
0xB10C:
ACK 5c34507ecbbdc29c086276d1c62835b461823507
meshcollider:
utACK 5c34507ecbbdc29c086276d1c62835b461823507
theStack:
ACK 5c34507ecbbdc29c086276d1c62835b461823507 👘
promag:
Concept ACK 5c34507ecbbdc29c086276d1c62835b461823507
Tree-SHA512: bbff120d8fd76e617b723b102b0c606e0d8eb27f21c631d5f4cdab0892137c4bc7c65b1df144993405f942c91be47a26e80480102af55bff22621c19f518aea3
-rw-r--r-- | doc/release-notes.md | 8 | ||||
-rw-r--r-- | src/bench/rpc_blockchain.cpp | 4 | ||||
-rw-r--r-- | src/core_io.h | 11 | ||||
-rw-r--r-- | src/core_write.cpp | 31 | ||||
-rw-r--r-- | src/rest.cpp | 8 | ||||
-rw-r--r-- | src/rpc/blockchain.cpp | 53 | ||||
-rw-r--r-- | src/rpc/blockchain.h | 3 | ||||
-rwxr-xr-x | test/functional/interface_rest.py | 9 | ||||
-rwxr-xr-x | test/functional/rpc_blockchain.py | 66 |
9 files changed, 148 insertions, 45 deletions
diff --git a/doc/release-notes.md b/doc/release-notes.md index 35ef546dc1..b460cd3eb2 100644 --- a/doc/release-notes.md +++ b/doc/release-notes.md @@ -82,6 +82,14 @@ Updated RPCs `gettransaction verbose=true` and REST endpoints `/rest/tx`, `/rest/getutxos`, `/rest/block` no longer return the `addresses` and `reqSigs` fields, which were previously deprecated in 22.0. (#22650) +- The `getblock` RPC command now supports verbose level 3 containing transaction inputs + `prevout` information. The existing `/rest/block/` REST endpoint is modified to contain + this information too. Every `vin` field will contain an additional `prevout` subfield + describing the spent output. `prevout` contains the following keys: + - `generated` - true if the spent coins was a coinbase. + - `height` + - `value` + - `scriptPubKey` - `listunspent` now includes `ancestorcount`, `ancestorsize`, and `ancestorfees` for each transaction output that is still in the mempool. diff --git a/src/bench/rpc_blockchain.cpp b/src/bench/rpc_blockchain.cpp index c8886a4c23..3bef64f720 100644 --- a/src/bench/rpc_blockchain.cpp +++ b/src/bench/rpc_blockchain.cpp @@ -40,7 +40,7 @@ static void BlockToJsonVerbose(benchmark::Bench& bench) { TestBlockAndIndex data; bench.run([&] { - auto univalue = blockToJSON(data.block, &data.blockindex, &data.blockindex, /*verbose*/ true); + auto univalue = blockToJSON(data.block, &data.blockindex, &data.blockindex, TxVerbosity::SHOW_DETAILS_AND_PREVOUT); ankerl::nanobench::doNotOptimizeAway(univalue); }); } @@ -50,7 +50,7 @@ BENCHMARK(BlockToJsonVerbose); static void BlockToJsonVerboseWrite(benchmark::Bench& bench) { TestBlockAndIndex data; - auto univalue = blockToJSON(data.block, &data.blockindex, &data.blockindex, /*verbose*/ true); + auto univalue = blockToJSON(data.block, &data.blockindex, &data.blockindex, TxVerbosity::SHOW_DETAILS_AND_PREVOUT); bench.run([&] { auto str = univalue.write(); ankerl::nanobench::doNotOptimizeAway(str); diff --git a/src/core_io.h b/src/core_io.h index be93a17efe..4d7199ab12 100644 --- a/src/core_io.h +++ b/src/core_io.h @@ -20,6 +20,15 @@ class uint256; class UniValue; class CTxUndo; +/** + * Verbose level for block's transaction + */ +enum class TxVerbosity { + SHOW_TXID, //!< Only TXID for each block's transaction + SHOW_DETAILS, //!< Include TXID, inputs, outputs, and other common block's transaction information + SHOW_DETAILS_AND_PREVOUT //!< The same as previous option with information about prevouts if available +}; + // core_read.cpp CScript ParseScript(const std::string& s); std::string ScriptToAsmStr(const CScript& script, const bool fAttemptSighashDecode = false); @@ -46,6 +55,6 @@ std::string EncodeHexTx(const CTransaction& tx, const int serializeFlags = 0); std::string SighashToStr(unsigned char sighash_type); void ScriptPubKeyToUniv(const CScript& scriptPubKey, UniValue& out, bool include_hex, bool include_address = true); void ScriptToUniv(const CScript& script, UniValue& out); -void TxToUniv(const CTransaction& tx, const uint256& hashBlock, UniValue& entry, bool include_hex = true, int serialize_flags = 0, const CTxUndo* txundo = nullptr); +void TxToUniv(const CTransaction& tx, const uint256& hashBlock, UniValue& entry, bool include_hex = true, int serialize_flags = 0, const CTxUndo* txundo = nullptr, TxVerbosity verbosity = TxVerbosity::SHOW_DETAILS); #endif // BITCOIN_CORE_IO_H diff --git a/src/core_write.cpp b/src/core_write.cpp index 6b13e4c586..468694b011 100644 --- a/src/core_write.cpp +++ b/src/core_write.cpp @@ -163,7 +163,7 @@ void ScriptPubKeyToUniv(const CScript& scriptPubKey, UniValue& out, bool include out.pushKV("type", GetTxnOutputType(type)); } -void TxToUniv(const CTransaction& tx, const uint256& hashBlock, UniValue& entry, bool include_hex, int serialize_flags, const CTxUndo* txundo) +void TxToUniv(const CTransaction& tx, const uint256& hashBlock, UniValue& entry, bool include_hex, int serialize_flags, const CTxUndo* txundo, TxVerbosity verbosity) { entry.pushKV("txid", tx.GetHash().GetHex()); entry.pushKV("hash", tx.GetWitnessHash().GetHex()); @@ -179,7 +179,7 @@ void TxToUniv(const CTransaction& tx, const uint256& hashBlock, UniValue& entry, // If available, use Undo data to calculate the fee. Note that txundo == nullptr // for coinbase transactions and for transactions where undo data is unavailable. - const bool calculate_fee = txundo != nullptr; + const bool have_undo = txundo != nullptr; CAmount amt_total_in = 0; CAmount amt_total_out = 0; @@ -203,9 +203,28 @@ void TxToUniv(const CTransaction& tx, const uint256& hashBlock, UniValue& entry, } in.pushKV("txinwitness", txinwitness); } - if (calculate_fee) { - const CTxOut& prev_txout = txundo->vprevout[i].out; + if (have_undo) { + const Coin& prev_coin = txundo->vprevout[i]; + const CTxOut& prev_txout = prev_coin.out; + amt_total_in += prev_txout.nValue; + switch (verbosity) { + case TxVerbosity::SHOW_TXID: + case TxVerbosity::SHOW_DETAILS: + break; + + case TxVerbosity::SHOW_DETAILS_AND_PREVOUT: + UniValue o_script_pub_key(UniValue::VOBJ); + ScriptPubKeyToUniv(prev_txout.scriptPubKey, o_script_pub_key, /* includeHex */ true); + + UniValue p(UniValue::VOBJ); + p.pushKV("generated", bool(prev_coin.fCoinBase)); + p.pushKV("height", uint64_t(prev_coin.nHeight)); + p.pushKV("value", ValueFromAmount(prev_txout.nValue)); + p.pushKV("scriptPubKey", o_script_pub_key); + in.pushKV("prevout", p); + break; + } } in.pushKV("sequence", (int64_t)txin.nSequence); vin.push_back(in); @@ -226,13 +245,13 @@ void TxToUniv(const CTransaction& tx, const uint256& hashBlock, UniValue& entry, out.pushKV("scriptPubKey", o); vout.push_back(out); - if (calculate_fee) { + if (have_undo) { amt_total_out += txout.nValue; } } entry.pushKV("vout", vout); - if (calculate_fee) { + if (have_undo) { const CAmount fee = amt_total_in - amt_total_out; CHECK_NONFATAL(MoneyRange(fee)); entry.pushKV("fee", ValueFromAmount(fee)); diff --git a/src/rest.cpp b/src/rest.cpp index f6e34c2d81..3746fd752a 100644 --- a/src/rest.cpp +++ b/src/rest.cpp @@ -262,7 +262,7 @@ static bool rest_headers(const std::any& context, static bool rest_block(const std::any& context, HTTPRequest* req, const std::string& strURIPart, - bool showTxDetails) + TxVerbosity tx_verbosity) { if (!CheckWarmup(req)) return false; @@ -314,7 +314,7 @@ static bool rest_block(const std::any& context, } case RetFormat::JSON: { - UniValue objBlock = blockToJSON(block, tip, pblockindex, showTxDetails); + UniValue objBlock = blockToJSON(block, tip, pblockindex, tx_verbosity); std::string strJSON = objBlock.write() + "\n"; req->WriteHeader("Content-Type", "application/json"); req->WriteReply(HTTP_OK, strJSON); @@ -329,12 +329,12 @@ static bool rest_block(const std::any& context, static bool rest_block_extended(const std::any& context, HTTPRequest* req, const std::string& strURIPart) { - return rest_block(context, req, strURIPart, true); + return rest_block(context, req, strURIPart, TxVerbosity::SHOW_DETAILS_AND_PREVOUT); } static bool rest_block_notxdetails(const std::any& context, HTTPRequest* req, const std::string& strURIPart) { - return rest_block(context, req, strURIPart, false); + return rest_block(context, req, strURIPart, TxVerbosity::SHOW_TXID); } // A bit of a hack - dependency on a function defined in rpc/blockchain.cpp diff --git a/src/rpc/blockchain.cpp b/src/rpc/blockchain.cpp index fc99c8eed9..dadd82e03f 100644 --- a/src/rpc/blockchain.cpp +++ b/src/rpc/blockchain.cpp @@ -200,7 +200,7 @@ UniValue blockheaderToJSON(const CBlockIndex* tip, const CBlockIndex* blockindex return result; } -UniValue blockToJSON(const CBlock& block, const CBlockIndex* tip, const CBlockIndex* blockindex, bool txDetails) +UniValue blockToJSON(const CBlock& block, const CBlockIndex* tip, const CBlockIndex* blockindex, TxVerbosity verbosity) { UniValue result = blockheaderToJSON(tip, blockindex); @@ -208,22 +208,29 @@ UniValue blockToJSON(const CBlock& block, const CBlockIndex* tip, const CBlockIn result.pushKV("size", (int)::GetSerializeSize(block, PROTOCOL_VERSION)); result.pushKV("weight", (int)::GetBlockWeight(block)); UniValue txs(UniValue::VARR); - if (txDetails) { - CBlockUndo blockUndo; - const bool have_undo = !IsBlockPruned(blockindex) && UndoReadFromDisk(blockUndo, blockindex); - for (size_t i = 0; i < block.vtx.size(); ++i) { - const CTransactionRef& tx = block.vtx.at(i); - // coinbase transaction (i == 0) doesn't have undo data - const CTxUndo* txundo = (have_undo && i) ? &blockUndo.vtxundo.at(i - 1) : nullptr; - UniValue objTx(UniValue::VOBJ); - TxToUniv(*tx, uint256(), objTx, true, RPCSerializationFlags(), txundo); - txs.push_back(objTx); - } - } else { - for (const CTransactionRef& tx : block.vtx) { - txs.push_back(tx->GetHash().GetHex()); - } + + switch (verbosity) { + case TxVerbosity::SHOW_TXID: + for (const CTransactionRef& tx : block.vtx) { + txs.push_back(tx->GetHash().GetHex()); + } + break; + + case TxVerbosity::SHOW_DETAILS: + case TxVerbosity::SHOW_DETAILS_AND_PREVOUT: + CBlockUndo blockUndo; + const bool have_undo = !IsBlockPruned(blockindex) && UndoReadFromDisk(blockUndo, blockindex); + + for (size_t i = 0; i < block.vtx.size(); ++i) { + const CTransactionRef& tx = block.vtx.at(i); + // coinbase transaction (i.e. i == 0) doesn't have undo data + const CTxUndo* txundo = (have_undo && i > 0) ? &blockUndo.vtxundo.at(i - 1) : nullptr; + UniValue objTx(UniValue::VOBJ); + TxToUniv(*tx, uint256(), objTx, true, RPCSerializationFlags(), txundo, verbosity); + txs.push_back(objTx); + } } + result.pushKV("tx", txs); return result; @@ -931,7 +938,8 @@ static RPCHelpMan getblock() return RPCHelpMan{"getblock", "\nIf verbosity is 0, returns a string that is serialized, hex-encoded data for block 'hash'.\n" "If verbosity is 1, returns an Object with information about block <hash>.\n" - "If verbosity is 2, returns an Object with information about block <hash> and information about each transaction. \n", + "If verbosity is 2, returns an Object with information about block <hash> and information about each transaction.\n" + "If verbosity is 3, returns an Object with information about block <hash> and information about each transaction, including prevout information for inputs (only for unpruned blocks in the current best chain).\n", { {"blockhash", RPCArg::Type::STR_HEX, RPCArg::Optional::NO, "The block hash"}, {"verbosity|verbose", RPCArg::Type::NUM, RPCArg::Default{1}, "0 for hex-encoded data, 1 for a json object, and 2 for json object with transaction data"}, @@ -1018,7 +1026,16 @@ static RPCHelpMan getblock() return strHex; } - return blockToJSON(block, tip, pblockindex, verbosity >= 2); + TxVerbosity tx_verbosity; + if (verbosity == 1) { + tx_verbosity = TxVerbosity::SHOW_TXID; + } else if (verbosity == 2) { + tx_verbosity = TxVerbosity::SHOW_DETAILS; + } else { + tx_verbosity = TxVerbosity::SHOW_DETAILS_AND_PREVOUT; + } + + return blockToJSON(block, tip, pblockindex, tx_verbosity); }, }; } diff --git a/src/rpc/blockchain.h b/src/rpc/blockchain.h index 09e471afdd..d9c6761f47 100644 --- a/src/rpc/blockchain.h +++ b/src/rpc/blockchain.h @@ -6,6 +6,7 @@ #define BITCOIN_RPC_BLOCKCHAIN_H #include <consensus/amount.h> +#include <core_io.h> #include <streams.h> #include <sync.h> @@ -38,7 +39,7 @@ double GetDifficulty(const CBlockIndex* blockindex); void RPCNotifyBlockChange(const CBlockIndex*); /** Block description to JSON */ -UniValue blockToJSON(const CBlock& block, const CBlockIndex* tip, const CBlockIndex* blockindex, bool txDetails = false) LOCKS_EXCLUDED(cs_main); +UniValue blockToJSON(const CBlock& block, const CBlockIndex* tip, const CBlockIndex* blockindex, TxVerbosity verbosity) LOCKS_EXCLUDED(cs_main); /** Mempool information to JSON */ UniValue MempoolInfoToJSON(const CTxMemPool& pool); diff --git a/test/functional/interface_rest.py b/test/functional/interface_rest.py index adc33bd9df..868bb42604 100755 --- a/test/functional/interface_rest.py +++ b/test/functional/interface_rest.py @@ -318,6 +318,15 @@ class RESTTest (BitcoinTestFramework): if 'coinbase' not in tx['vin'][0]} assert_equal(non_coinbase_txs, set(txs)) + # Verify that the non-coinbase tx has "prevout" key set + for tx_obj in json_obj["tx"]: + for vin in tx_obj["vin"]: + if "coinbase" not in vin: + assert "prevout" in vin + assert_equal(vin["prevout"]["generated"], False) + else: + assert "prevout" not in vin + # Check the same but without tx details json_obj = self.test_rest_request(f"/block/notxdetails/{newblockhash[0]}") for tx in txs: diff --git a/test/functional/rpc_blockchain.py b/test/functional/rpc_blockchain.py index c3c6ade684..eea9ee26cb 100755 --- a/test/functional/rpc_blockchain.py +++ b/test/functional/rpc_blockchain.py @@ -434,17 +434,55 @@ class BlockchainTest(BitcoinTestFramework): miniwallet.send_self_transfer(fee_rate=fee_per_kb, from_node=node) blockhash = self.generate(node, 1)[0] - self.log.info("Test getblock with verbosity 1 doesn't include fee") - block = node.getblock(blockhash, 1) - assert 'fee' not in block['tx'][1] - - self.log.info('Test getblock with verbosity 2 includes expected fee') - block = node.getblock(blockhash, 2) - tx = block['tx'][1] - assert 'fee' in tx - assert_equal(tx['fee'], tx['vsize'] * fee_per_byte) - - self.log.info("Test getblock with verbosity 2 still works with pruned Undo data") + def assert_fee_not_in_block(verbosity): + block = node.getblock(blockhash, verbosity) + assert 'fee' not in block['tx'][1] + + def assert_fee_in_block(verbosity): + block = node.getblock(blockhash, verbosity) + tx = block['tx'][1] + assert 'fee' in tx + assert_equal(tx['fee'], tx['vsize'] * fee_per_byte) + + def assert_vin_contains_prevout(verbosity): + block = node.getblock(blockhash, verbosity) + tx = block["tx"][1] + total_vin = Decimal("0.00000000") + total_vout = Decimal("0.00000000") + for vin in tx["vin"]: + assert "prevout" in vin + assert_equal(set(vin["prevout"].keys()), set(("value", "height", "generated", "scriptPubKey"))) + assert_equal(vin["prevout"]["generated"], True) + total_vin += vin["prevout"]["value"] + for vout in tx["vout"]: + total_vout += vout["value"] + assert_equal(total_vin, total_vout + tx["fee"]) + + def assert_vin_does_not_contain_prevout(verbosity): + block = node.getblock(blockhash, verbosity) + tx = block["tx"][1] + if isinstance(tx, str): + # In verbosity level 1, only the transaction hashes are written + pass + else: + for vin in tx["vin"]: + assert "prevout" not in vin + + self.log.info("Test that getblock with verbosity 1 doesn't include fee") + assert_fee_not_in_block(1) + + self.log.info('Test that getblock with verbosity 2 and 3 includes expected fee') + assert_fee_in_block(2) + assert_fee_in_block(3) + + self.log.info("Test that getblock with verbosity 1 and 2 does not include prevout") + assert_vin_does_not_contain_prevout(1) + assert_vin_does_not_contain_prevout(2) + + self.log.info("Test that getblock with verbosity 3 includes prevout") + assert_vin_contains_prevout(3) + + self.log.info("Test that getblock with verbosity 2 and 3 still works with pruned Undo data") datadir = get_datadir_path(self.options.tmpdir, 0) self.log.info("Test getblock with invalid verbosity type returns proper error message") @@ -458,8 +496,10 @@ class BlockchainTest(BitcoinTestFramework): # Move instead of deleting so we can restore chain state afterwards move_block_file('rev00000.dat', 'rev_wrong') - block = node.getblock(blockhash, 2) - assert 'fee' not in block['tx'][1] + assert_fee_not_in_block(2) + assert_fee_not_in_block(3) + assert_vin_does_not_contain_prevout(2) + assert_vin_does_not_contain_prevout(3) # Restore chain state move_block_file('rev_wrong', 'rev00000.dat') |