diff options
Diffstat (limited to 'test')
| -rw-r--r-- | test/README.md | 17 | ||||
| -rw-r--r-- | test/functional/README.md | 33 | ||||
| -rwxr-xr-x | test/functional/combine_logs.py | 10 | ||||
| -rwxr-xr-x | test/functional/feature_bip68_sequence.py | 21 | ||||
| -rwxr-xr-x | test/functional/feature_block.py | 6 | ||||
| -rwxr-xr-x | test/functional/feature_loadblock.py | 84 | ||||
| -rwxr-xr-x | test/functional/feature_rbf.py | 49 | ||||
| -rwxr-xr-x | test/functional/p2p_segwit.py | 12 | ||||
| -rw-r--r-- | test/functional/test_framework/script.py | 6 | ||||
| -rwxr-xr-x | test/functional/test_framework/script_util.py | 25 | ||||
| -rwxr-xr-x | test/functional/test_runner.py | 1 | ||||
| -rwxr-xr-x | test/functional/wallet_address_types.py | 3 | ||||
| -rwxr-xr-x | test/functional/wallet_backup.py | 8 | ||||
| -rwxr-xr-x | test/functional/wallet_multiwallet.py | 8 | ||||
| -rw-r--r-- | test/lint/README.md | 2 | ||||
| -rwxr-xr-x | test/lint/lint-filenames.sh | 2 | ||||
| -rwxr-xr-x | test/lint/lint-include-guards.sh | 2 | ||||
| -rw-r--r-- | test/sanitizer_suppressions/ubsan | 13 |
18 files changed, 219 insertions, 83 deletions
diff --git a/test/README.md b/test/README.md index 26fd525064..11adc11278 100644 --- a/test/README.md +++ b/test/README.md @@ -88,7 +88,7 @@ By default, up to 4 tests will be run in parallel by test_runner. To specify how many jobs to run, append `--jobs=n` The individual tests and the test_runner harness have many command-line -options. Run `test_runner.py -h` to see them all. +options. Run `test/functional/test_runner.py -h` to see them all. #### Troubleshooting and debugging test failures @@ -101,7 +101,7 @@ killed all its bitcoind nodes), then there may be a port conflict which will cause the test to fail. It is recommended that you run the tests on a system where no other bitcoind processes are running. -On linux, the test_framework will warn if there is another +On linux, the test framework will warn if there is another bitcoind process running when the tests are started. If there are zombie bitcoind processes after test failure, you can kill them @@ -130,7 +130,7 @@ tests will fail. If this happens, remove the cache directory (and make sure bitcoind processes are stopped as above): ```bash -rm -rf cache +rm -rf test/cache killall bitcoind ``` @@ -149,6 +149,15 @@ levels using the logger included in the test_framework, e.g. fails, the `test_framework.log` and bitcoind `debug.log`s will all be dumped to the console to help troubleshooting. +These log files can be located under the test data directory (which is always +printed in the first line of test output): + - `<test data directory>/test_framework.log` + - `<test data directory>/node<node number>/regtest/debug.log`. + +The node number identifies the relevant test node, starting from `node0`, which +corresponds to its position in the nodes list of the specific test, +e.g. `self.nodes[0]`. + To change the level of logs output to the console, use the `-l` command line argument. @@ -157,7 +166,7 @@ aggregate log by running the `combine_logs.py` script. The output can be plain text, colorized text or html. For example: ``` -combine_logs.py -c <test data directory> | less -r +test/functional/combine_logs.py -c <test data directory> | less -r ``` will pipe the colorized logs from the test into less. diff --git a/test/functional/README.md b/test/functional/README.md index 197c2afbe4..a9b83076eb 100644 --- a/test/functional/README.md +++ b/test/functional/README.md @@ -4,13 +4,13 @@ #### Example test -The [example_test.py](example_test.py) is a heavily commented example of a test case that uses both -the RPC and P2P interfaces. If you are writing your first test, copy that file -and modify to fit your needs. +The file [test/functional/example_test.py](example_test.py) is a heavily commented example +of a test case that uses both the RPC and P2P interfaces. If you are writing your first test, copy +that file and modify to fit your needs. #### Coverage -Running `test_runner.py` with the `--coverage` argument tracks which RPCs are +Running `test/functional/test_runner.py` with the `--coverage` argument tracks which RPCs are called by the tests and prints a report of uncovered RPCs in the summary. This can be used (along with the `--extended` argument) to find out which RPCs we don't have test cases for. @@ -82,7 +82,7 @@ P2P messages. These can be found in the following source files: #### Using the P2P interface -- `messages.py` contains all the definitions for objects that pass +- [messages.py](test_framework/messages.py) contains all the definitions for objects that pass over the network (`CBlock`, `CTransaction`, etc, along with the network-level wrappers for them, `msg_block`, `msg_tx`, etc). @@ -96,32 +96,35 @@ the Bitcoin Core node application logic. For custom behaviour, subclass the P2PInterface object and override the callback methods. - Can be used to write tests where specific P2P protocol behavior is tested. -Examples tests are `p2p_unrequested_blocks.py`, `p2p_compactblocks.py`. +Examples tests are [p2p_unrequested_blocks.py](p2p_unrequested_blocks.py), +[p2p_compactblocks.py](p2p_compactblocks.py). -### test-framework modules +### Test framework modules +The following are useful modules for test developers. They are located in +[test/functional/test_framework/](test_framework). -#### [test_framework/authproxy.py](test_framework/authproxy.py) +#### [authproxy.py](test_framework/authproxy.py) Taken from the [python-bitcoinrpc repository](https://github.com/jgarzik/python-bitcoinrpc). -#### [test_framework/test_framework.py](test_framework/test_framework.py) +#### [test_framework.py](test_framework/test_framework.py) Base class for functional tests. -#### [test_framework/util.py](test_framework/util.py) +#### [util.py](test_framework/util.py) Generally useful functions. -#### [test_framework/mininode.py](test_framework/mininode.py) +#### [mininode.py](test_framework/mininode.py) Basic code to support P2P connectivity to a bitcoind. -#### [test_framework/script.py](test_framework/script.py) +#### [script.py](test_framework/script.py) Utilities for manipulating transaction scripts (originally from python-bitcoinlib) -#### [test_framework/key.py](test_framework/key.py) +#### [key.py](test_framework/key.py) Test-only secp256k1 elliptic curve implementation -#### [test_framework/bignum.py](test_framework/bignum.py) +#### [bignum.py](test_framework/bignum.py) Helpers for script.py -#### [test_framework/blocktools.py](test_framework/blocktools.py) +#### [blocktools.py](test_framework/blocktools.py) Helper functions for creating blocks and transactions. ### Benchmarking with perf diff --git a/test/functional/combine_logs.py b/test/functional/combine_logs.py index 367d0f6916..a70d9c4ac1 100755 --- a/test/functional/combine_logs.py +++ b/test/functional/combine_logs.py @@ -8,7 +8,6 @@ If no argument is provided, the most recent test directory will be used.""" import argparse from collections import defaultdict, namedtuple -import glob import heapq import itertools import os @@ -78,10 +77,11 @@ def read_logs(tmp_dir): for each of the input log files.""" # Find out what the folder is called that holds the debug.log file - chain = glob.glob("{}/node0/*/debug.log".format(tmp_dir)) - if chain: - chain = chain[0] # pick the first one if more than one chain was found (should never happen) - chain = re.search(r'node0/(.+?)/debug\.log$', chain).group(1) # extract the chain name + glob = pathlib.Path(tmp_dir).glob('node0/**/debug.log') + path = next(glob, None) + if path: + assert next(glob, None) is None # more than one debug.log, should never happen + chain = re.search(r'node0/(.+?)/debug\.log$', path.as_posix()).group(1) # extract the chain name else: chain = 'regtest' # fallback to regtest (should only happen when none exists) diff --git a/test/functional/feature_bip68_sequence.py b/test/functional/feature_bip68_sequence.py index dd495ba41e..fe45ed34b5 100755 --- a/test/functional/feature_bip68_sequence.py +++ b/test/functional/feature_bip68_sequence.py @@ -8,7 +8,6 @@ import time from test_framework.blocktools import create_block, create_coinbase, add_witness_commitment from test_framework.messages import COIN, COutPoint, CTransaction, CTxIn, CTxOut, FromHex, ToHex -from test_framework.script import CScript from test_framework.test_framework import BitcoinTestFramework from test_framework.util import ( assert_equal, @@ -17,6 +16,7 @@ from test_framework.util import ( satoshi_round, softfork_active, ) +from test_framework.script_util import DUMMY_P2WPKH_SCRIPT SEQUENCE_LOCKTIME_DISABLE_FLAG = (1<<31) SEQUENCE_LOCKTIME_TYPE_FLAG = (1<<22) # this means use time (0 means height) @@ -29,10 +29,9 @@ NOT_FINAL_ERROR = "non-BIP68-final" class BIP68Test(BitcoinTestFramework): def set_test_params(self): self.num_nodes = 2 - # TODO remove output type argument and fix resulting "tx-size-small" errors self.extra_args = [ - ["-acceptnonstdtxn=1", "-addresstype=p2sh-segwit"], - ["-acceptnonstdtxn=0", "-addresstype=p2sh-segwit"], + ["-acceptnonstdtxn=1"], + ["-acceptnonstdtxn=0"], ] def skip_test_if_missing_module(self): @@ -85,7 +84,7 @@ class BIP68Test(BitcoinTestFramework): # input to mature. sequence_value = SEQUENCE_LOCKTIME_DISABLE_FLAG | 1 tx1.vin = [CTxIn(COutPoint(int(utxo["txid"], 16), utxo["vout"]), nSequence=sequence_value)] - tx1.vout = [CTxOut(value, CScript([b'a']))] + tx1.vout = [CTxOut(value, DUMMY_P2WPKH_SCRIPT)] tx1_signed = self.nodes[0].signrawtransactionwithwallet(ToHex(tx1))["hex"] tx1_id = self.nodes[0].sendrawtransaction(tx1_signed) @@ -97,7 +96,7 @@ class BIP68Test(BitcoinTestFramework): tx2.nVersion = 2 sequence_value = sequence_value & 0x7fffffff tx2.vin = [CTxIn(COutPoint(tx1_id, 0), nSequence=sequence_value)] - tx2.vout = [CTxOut(int(value - self.relayfee * COIN), CScript([b'a' * 35]))] + tx2.vout = [CTxOut(int(value - self.relayfee * COIN), DUMMY_P2WPKH_SCRIPT)] tx2.rehash() assert_raises_rpc_error(-26, NOT_FINAL_ERROR, self.nodes[0].sendrawtransaction, ToHex(tx2)) @@ -192,7 +191,7 @@ class BIP68Test(BitcoinTestFramework): value += utxos[j]["amount"]*COIN # Overestimate the size of the tx - signatures should be less than 120 bytes, and leave 50 for the output tx_size = len(ToHex(tx))//2 + 120*num_inputs + 50 - tx.vout.append(CTxOut(int(value-self.relayfee*tx_size*COIN/1000), CScript([b'a']))) + tx.vout.append(CTxOut(int(value-self.relayfee*tx_size*COIN/1000), DUMMY_P2WPKH_SCRIPT)) rawtx = self.nodes[0].signrawtransactionwithwallet(ToHex(tx))["hex"] if (using_sequence_locks and not should_pass): @@ -221,7 +220,7 @@ class BIP68Test(BitcoinTestFramework): tx2 = CTransaction() tx2.nVersion = 2 tx2.vin = [CTxIn(COutPoint(tx1.sha256, 0), nSequence=0)] - tx2.vout = [CTxOut(int(tx1.vout[0].nValue - self.relayfee*COIN), CScript([b'a']))] + tx2.vout = [CTxOut(int(tx1.vout[0].nValue - self.relayfee*COIN), DUMMY_P2WPKH_SCRIPT)] tx2_raw = self.nodes[0].signrawtransactionwithwallet(ToHex(tx2))["hex"] tx2 = FromHex(tx2, tx2_raw) tx2.rehash() @@ -239,7 +238,7 @@ class BIP68Test(BitcoinTestFramework): tx = CTransaction() tx.nVersion = 2 tx.vin = [CTxIn(COutPoint(orig_tx.sha256, 0), nSequence=sequence_value)] - tx.vout = [CTxOut(int(orig_tx.vout[0].nValue - relayfee * COIN), CScript([b'a' * 35]))] + tx.vout = [CTxOut(int(orig_tx.vout[0].nValue - relayfee * COIN), DUMMY_P2WPKH_SCRIPT)] tx.rehash() if (orig_tx.hash in node.getrawmempool()): @@ -352,7 +351,7 @@ class BIP68Test(BitcoinTestFramework): tx2 = CTransaction() tx2.nVersion = 1 tx2.vin = [CTxIn(COutPoint(tx1.sha256, 0), nSequence=0)] - tx2.vout = [CTxOut(int(tx1.vout[0].nValue - self.relayfee*COIN), CScript([b'a']))] + tx2.vout = [CTxOut(int(tx1.vout[0].nValue - self.relayfee*COIN), DUMMY_P2WPKH_SCRIPT)] # sign tx2 tx2_raw = self.nodes[0].signrawtransactionwithwallet(ToHex(tx2))["hex"] @@ -367,7 +366,7 @@ class BIP68Test(BitcoinTestFramework): tx3 = CTransaction() tx3.nVersion = 2 tx3.vin = [CTxIn(COutPoint(tx2.sha256, 0), nSequence=sequence_value)] - tx3.vout = [CTxOut(int(tx2.vout[0].nValue - self.relayfee * COIN), CScript([b'a' * 35]))] + tx3.vout = [CTxOut(int(tx2.vout[0].nValue - self.relayfee * COIN), DUMMY_P2WPKH_SCRIPT)] tx3.rehash() assert_raises_rpc_error(-26, NOT_FINAL_ERROR, self.nodes[0].sendrawtransaction, ToHex(tx3)) diff --git a/test/functional/feature_block.py b/test/functional/feature_block.py index c74270febc..c7e98bd4db 100755 --- a/test/functional/feature_block.py +++ b/test/functional/feature_block.py @@ -46,7 +46,7 @@ from test_framework.script import ( OP_RETURN, OP_TRUE, SIGHASH_ALL, - SignatureHash, + LegacySignatureHash, hash160, ) from test_framework.test_framework import BitcoinTestFramework @@ -532,7 +532,7 @@ class FullBlockTest(BitcoinTestFramework): # second input is corresponding P2SH output from b39 tx.vin.append(CTxIn(COutPoint(b39.vtx[i].sha256, 0), b'')) # Note: must pass the redeem_script (not p2sh_script) to the signature hash function - (sighash, err) = SignatureHash(redeem_script, tx, 1, SIGHASH_ALL) + (sighash, err) = LegacySignatureHash(redeem_script, tx, 1, SIGHASH_ALL) sig = self.coinbase_key.sign_ecdsa(sighash) + bytes(bytearray([SIGHASH_ALL])) scriptSig = CScript([sig, redeem_script]) @@ -1312,7 +1312,7 @@ class FullBlockTest(BitcoinTestFramework): if (scriptPubKey[0] == OP_TRUE): # an anyone-can-spend tx.vin[0].scriptSig = CScript() return - (sighash, err) = SignatureHash(spend_tx.vout[0].scriptPubKey, tx, 0, SIGHASH_ALL) + (sighash, err) = LegacySignatureHash(spend_tx.vout[0].scriptPubKey, tx, 0, SIGHASH_ALL) tx.vin[0].scriptSig = CScript([self.coinbase_key.sign_ecdsa(sighash) + bytes(bytearray([SIGHASH_ALL]))]) def create_and_sign_transaction(self, spend_tx, value, script=CScript([OP_TRUE])): diff --git a/test/functional/feature_loadblock.py b/test/functional/feature_loadblock.py new file mode 100755 index 0000000000..bf2a4ff61f --- /dev/null +++ b/test/functional/feature_loadblock.py @@ -0,0 +1,84 @@ +#!/usr/bin/env python3 +# Copyright (c) 2017-2019 The Bitcoin Core developers +# Distributed under the MIT software license, see the accompanying +# file COPYING or http://www.opensource.org/licenses/mit-license.php. +"""Test loadblock option + +Test the option to start a node with the option loadblock which loads +a serialized blockchain from a file (usually called bootstrap.dat). +To generate that file this test uses the helper scripts available +in contrib/linearize. +""" + +import os +import subprocess +import sys +import tempfile +import urllib + +from test_framework.test_framework import ( + BitcoinTestFramework, +) +from test_framework.util import assert_equal, wait_until + + +class LoadblockTest(BitcoinTestFramework): + def set_test_params(self): + self.setup_clean_chain = True + self.num_nodes = 2 + + def run_test(self): + self.nodes[1].setnetworkactive(state=False) + self.nodes[0].generate(100) + + # Parsing the url of our node to get settings for config file + data_dir = self.nodes[0].datadir + node_url = urllib.parse.urlparse(self.nodes[0].url) + cfg_file = os.path.join(data_dir, "linearize.cfg") + bootstrap_file = os.path.join(self.options.tmpdir, "bootstrap.dat") + genesis_block = self.nodes[0].getblockhash(0) + blocks_dir = os.path.join(data_dir, "regtest", "blocks") + hash_list = tempfile.NamedTemporaryFile(dir=data_dir, + mode='w', + delete=False, + encoding="utf-8") + + self.log.info("Create linearization config file") + with open(cfg_file, "a", encoding="utf-8") as cfg: + cfg.write("datadir={}\n".format(data_dir)) + cfg.write("rpcuser={}\n".format(node_url.username)) + cfg.write("rpcpassword={}\n".format(node_url.password)) + cfg.write("port={}\n".format(node_url.port)) + cfg.write("host={}\n".format(node_url.hostname)) + cfg.write("output_file={}\n".format(bootstrap_file)) + cfg.write("max_height=100\n") + cfg.write("netmagic=fabfb5da\n") + cfg.write("input={}\n".format(blocks_dir)) + cfg.write("genesis={}\n".format(genesis_block)) + cfg.write("hashlist={}\n".format(hash_list.name)) + + base_dir = self.config["environment"]["SRCDIR"] + linearize_dir = os.path.join(base_dir, "contrib", "linearize") + + self.log.info("Run linearization of block hashes") + linearize_hashes_file = os.path.join(linearize_dir, "linearize-hashes.py") + subprocess.run([sys.executable, linearize_hashes_file, cfg_file], + stdout=hash_list, + check=True) + + self.log.info("Run linearization of block data") + linearize_data_file = os.path.join(linearize_dir, "linearize-data.py") + subprocess.run([sys.executable, linearize_data_file, cfg_file], + check=True) + + self.log.info("Restart second, unsynced node with bootstrap file") + self.stop_node(1) + self.start_node(1, ["-loadblock=" + bootstrap_file]) + wait_until(lambda: self.nodes[1].getblockcount() == 100) + + assert_equal(self.nodes[1].getblockchaininfo()['blocks'], 100) + assert_equal(self.nodes[0].getbestblockhash(), self.nodes[1].getbestblockhash()) + + +if __name__ == '__main__': + LoadblockTest().main() diff --git a/test/functional/feature_rbf.py b/test/functional/feature_rbf.py index a1d4ce4c73..e7afbd0272 100755 --- a/test/functional/feature_rbf.py +++ b/test/functional/feature_rbf.py @@ -10,13 +10,14 @@ from test_framework.messages import COIN, COutPoint, CTransaction, CTxIn, CTxOut from test_framework.script import CScript, OP_DROP from test_framework.test_framework import BitcoinTestFramework from test_framework.util import assert_equal, assert_raises_rpc_error, satoshi_round +from test_framework.script_util import DUMMY_P2WPKH_SCRIPT MAX_REPLACEMENT_LIMIT = 100 def txToHex(tx): return tx.serialize().hex() -def make_utxo(node, amount, confirmed=True, scriptPubKey=CScript([1])): +def make_utxo(node, amount, confirmed=True, scriptPubKey=DUMMY_P2WPKH_SCRIPT): """Create a txout with a given amount and scriptPubKey Mines coins as needed. @@ -65,7 +66,6 @@ def make_utxo(node, amount, confirmed=True, scriptPubKey=CScript([1])): class ReplaceByFeeTest(BitcoinTestFramework): def set_test_params(self): self.num_nodes = 1 - # TODO remove output type argument and fix resulting "tx-size-small" errors self.extra_args = [ [ "-acceptnonstdtxn=1", @@ -74,7 +74,6 @@ class ReplaceByFeeTest(BitcoinTestFramework): "-limitancestorsize=101", "-limitdescendantcount=200", "-limitdescendantsize=101", - "-addresstype=p2sh-segwit", ], ] @@ -133,7 +132,7 @@ class ReplaceByFeeTest(BitcoinTestFramework): tx1a = CTransaction() tx1a.vin = [CTxIn(tx0_outpoint, nSequence=0)] - tx1a.vout = [CTxOut(1 * COIN, CScript([b'a' * 35]))] + tx1a.vout = [CTxOut(1 * COIN, DUMMY_P2WPKH_SCRIPT)] tx1a_hex = txToHex(tx1a) tx1a_txid = self.nodes[0].sendrawtransaction(tx1a_hex, 0) @@ -142,7 +141,7 @@ class ReplaceByFeeTest(BitcoinTestFramework): # Should fail because we haven't changed the fee tx1b = CTransaction() tx1b.vin = [CTxIn(tx0_outpoint, nSequence=0)] - tx1b.vout = [CTxOut(1 * COIN, CScript([b'b' * 35]))] + tx1b.vout = [CTxOut(1 * COIN, DUMMY_P2WPKH_SCRIPT + b'a')] tx1b_hex = txToHex(tx1b) # This will raise an exception due to insufficient fee @@ -151,7 +150,7 @@ class ReplaceByFeeTest(BitcoinTestFramework): # Extra 0.1 BTC fee tx1b = CTransaction() tx1b.vin = [CTxIn(tx0_outpoint, nSequence=0)] - tx1b.vout = [CTxOut(int(0.9 * COIN), CScript([b'b' * 35]))] + tx1b.vout = [CTxOut(int(0.9 * COIN), DUMMY_P2WPKH_SCRIPT)] tx1b_hex = txToHex(tx1b) # Works when enabled tx1b_txid = self.nodes[0].sendrawtransaction(tx1b_hex, 0) @@ -186,7 +185,7 @@ class ReplaceByFeeTest(BitcoinTestFramework): # child fees - 40 BTC - so this attempt is rejected. dbl_tx = CTransaction() dbl_tx.vin = [CTxIn(tx0_outpoint, nSequence=0)] - dbl_tx.vout = [CTxOut(initial_nValue - 30 * COIN, CScript([1] * 35))] + dbl_tx.vout = [CTxOut(initial_nValue - 30 * COIN, DUMMY_P2WPKH_SCRIPT)] dbl_tx_hex = txToHex(dbl_tx) # This will raise an exception due to insufficient fee @@ -195,7 +194,7 @@ class ReplaceByFeeTest(BitcoinTestFramework): # Accepted with sufficient fee dbl_tx = CTransaction() dbl_tx.vin = [CTxIn(tx0_outpoint, nSequence=0)] - dbl_tx.vout = [CTxOut(1 * COIN, CScript([1] * 35))] + dbl_tx.vout = [CTxOut(1 * COIN, DUMMY_P2WPKH_SCRIPT)] dbl_tx_hex = txToHex(dbl_tx) self.nodes[0].sendrawtransaction(dbl_tx_hex, 0) @@ -248,7 +247,7 @@ class ReplaceByFeeTest(BitcoinTestFramework): # Attempt double-spend, will fail because too little fee paid dbl_tx = CTransaction() dbl_tx.vin = [CTxIn(tx0_outpoint, nSequence=0)] - dbl_tx.vout = [CTxOut(initial_nValue - fee * n, CScript([1] * 35))] + dbl_tx.vout = [CTxOut(initial_nValue - fee * n, DUMMY_P2WPKH_SCRIPT)] dbl_tx_hex = txToHex(dbl_tx) # This will raise an exception due to insufficient fee assert_raises_rpc_error(-26, "insufficient fee", self.nodes[0].sendrawtransaction, dbl_tx_hex, 0) @@ -256,7 +255,7 @@ class ReplaceByFeeTest(BitcoinTestFramework): # 1 BTC fee is enough dbl_tx = CTransaction() dbl_tx.vin = [CTxIn(tx0_outpoint, nSequence=0)] - dbl_tx.vout = [CTxOut(initial_nValue - fee * n - 1 * COIN, CScript([1] * 35))] + dbl_tx.vout = [CTxOut(initial_nValue - fee * n - 1 * COIN, DUMMY_P2WPKH_SCRIPT)] dbl_tx_hex = txToHex(dbl_tx) self.nodes[0].sendrawtransaction(dbl_tx_hex, 0) @@ -276,7 +275,7 @@ class ReplaceByFeeTest(BitcoinTestFramework): dbl_tx = CTransaction() dbl_tx.vin = [CTxIn(tx0_outpoint, nSequence=0)] - dbl_tx.vout = [CTxOut(initial_nValue - 2 * fee * n, CScript([1] * 35))] + dbl_tx.vout = [CTxOut(initial_nValue - 2 * fee * n, DUMMY_P2WPKH_SCRIPT)] dbl_tx_hex = txToHex(dbl_tx) # This will raise an exception assert_raises_rpc_error(-26, "too many potential replacements", self.nodes[0].sendrawtransaction, dbl_tx_hex, 0) @@ -291,7 +290,7 @@ class ReplaceByFeeTest(BitcoinTestFramework): tx1a = CTransaction() tx1a.vin = [CTxIn(tx0_outpoint, nSequence=0)] - tx1a.vout = [CTxOut(1 * COIN, CScript([b'a' * 35]))] + tx1a.vout = [CTxOut(1 * COIN, DUMMY_P2WPKH_SCRIPT)] tx1a_hex = txToHex(tx1a) self.nodes[0].sendrawtransaction(tx1a_hex, 0) @@ -312,7 +311,7 @@ class ReplaceByFeeTest(BitcoinTestFramework): tx1a = CTransaction() tx1a.vin = [CTxIn(utxo1, nSequence=0)] - tx1a.vout = [CTxOut(int(1.1 * COIN), CScript([b'a' * 35]))] + tx1a.vout = [CTxOut(int(1.1 * COIN), DUMMY_P2WPKH_SCRIPT)] tx1a_hex = txToHex(tx1a) tx1a_txid = self.nodes[0].sendrawtransaction(tx1a_hex, 0) @@ -331,7 +330,7 @@ class ReplaceByFeeTest(BitcoinTestFramework): # Spend tx1a's output to test the indirect case. tx1b = CTransaction() tx1b.vin = [CTxIn(COutPoint(tx1a_txid, 0), nSequence=0)] - tx1b.vout = [CTxOut(1 * COIN, CScript([b'a' * 35]))] + tx1b.vout = [CTxOut(1 * COIN, DUMMY_P2WPKH_SCRIPT)] tx1b_hex = txToHex(tx1b) tx1b_txid = self.nodes[0].sendrawtransaction(tx1b_hex, 0) tx1b_txid = int(tx1b_txid, 16) @@ -352,7 +351,7 @@ class ReplaceByFeeTest(BitcoinTestFramework): tx1 = CTransaction() tx1.vin = [CTxIn(confirmed_utxo)] - tx1.vout = [CTxOut(1 * COIN, CScript([b'a' * 35]))] + tx1.vout = [CTxOut(1 * COIN, DUMMY_P2WPKH_SCRIPT)] tx1_hex = txToHex(tx1) self.nodes[0].sendrawtransaction(tx1_hex, 0) @@ -391,7 +390,7 @@ class ReplaceByFeeTest(BitcoinTestFramework): for i in range(MAX_REPLACEMENT_LIMIT+1): tx_i = CTransaction() tx_i.vin = [CTxIn(COutPoint(txid, i), nSequence=0)] - tx_i.vout = [CTxOut(split_value - fee, CScript([b'a' * 35]))] + tx_i.vout = [CTxOut(split_value - fee, DUMMY_P2WPKH_SCRIPT)] tx_i_hex = txToHex(tx_i) self.nodes[0].sendrawtransaction(tx_i_hex, 0) @@ -424,7 +423,7 @@ class ReplaceByFeeTest(BitcoinTestFramework): # Create a non-opting in transaction tx1a = CTransaction() tx1a.vin = [CTxIn(tx0_outpoint, nSequence=0xffffffff)] - tx1a.vout = [CTxOut(1 * COIN, CScript([b'a' * 35]))] + tx1a.vout = [CTxOut(1 * COIN, DUMMY_P2WPKH_SCRIPT)] tx1a_hex = txToHex(tx1a) tx1a_txid = self.nodes[0].sendrawtransaction(tx1a_hex, 0) @@ -434,7 +433,7 @@ class ReplaceByFeeTest(BitcoinTestFramework): # Shouldn't be able to double-spend tx1b = CTransaction() tx1b.vin = [CTxIn(tx0_outpoint, nSequence=0)] - tx1b.vout = [CTxOut(int(0.9 * COIN), CScript([b'b' * 35]))] + tx1b.vout = [CTxOut(int(0.9 * COIN), DUMMY_P2WPKH_SCRIPT)] tx1b_hex = txToHex(tx1b) # This will raise an exception @@ -445,14 +444,14 @@ class ReplaceByFeeTest(BitcoinTestFramework): # Create a different non-opting in transaction tx2a = CTransaction() tx2a.vin = [CTxIn(tx1_outpoint, nSequence=0xfffffffe)] - tx2a.vout = [CTxOut(1 * COIN, CScript([b'a' * 35]))] + tx2a.vout = [CTxOut(1 * COIN, DUMMY_P2WPKH_SCRIPT)] tx2a_hex = txToHex(tx2a) tx2a_txid = self.nodes[0].sendrawtransaction(tx2a_hex, 0) # Still shouldn't be able to double-spend tx2b = CTransaction() tx2b.vin = [CTxIn(tx1_outpoint, nSequence=0)] - tx2b.vout = [CTxOut(int(0.9 * COIN), CScript([b'b' * 35]))] + tx2b.vout = [CTxOut(int(0.9 * COIN), DUMMY_P2WPKH_SCRIPT)] tx2b_hex = txToHex(tx2b) # This will raise an exception @@ -478,12 +477,12 @@ class ReplaceByFeeTest(BitcoinTestFramework): tx3b = CTransaction() tx3b.vin = [CTxIn(COutPoint(tx1a_txid, 0), nSequence=0)] - tx3b.vout = [CTxOut(int(0.5 * COIN), CScript([b'e' * 35]))] + tx3b.vout = [CTxOut(int(0.5 * COIN), DUMMY_P2WPKH_SCRIPT)] tx3b_hex = txToHex(tx3b) tx3c = CTransaction() tx3c.vin = [CTxIn(COutPoint(tx2a_txid, 0), nSequence=0)] - tx3c.vout = [CTxOut(int(0.5 * COIN), CScript([b'f' * 35]))] + tx3c.vout = [CTxOut(int(0.5 * COIN), DUMMY_P2WPKH_SCRIPT)] tx3c_hex = txToHex(tx3c) self.nodes[0].sendrawtransaction(tx3b_hex, 0) @@ -500,7 +499,7 @@ class ReplaceByFeeTest(BitcoinTestFramework): tx1a = CTransaction() tx1a.vin = [CTxIn(tx0_outpoint, nSequence=0)] - tx1a.vout = [CTxOut(1 * COIN, CScript([b'a' * 35]))] + tx1a.vout = [CTxOut(1 * COIN, DUMMY_P2WPKH_SCRIPT)] tx1a_hex = txToHex(tx1a) tx1a_txid = self.nodes[0].sendrawtransaction(tx1a_hex, 0) @@ -526,14 +525,14 @@ class ReplaceByFeeTest(BitcoinTestFramework): tx2a = CTransaction() tx2a.vin = [CTxIn(tx1_outpoint, nSequence=0)] - tx2a.vout = [CTxOut(1 * COIN, CScript([b'a' * 35]))] + tx2a.vout = [CTxOut(1 * COIN, DUMMY_P2WPKH_SCRIPT)] tx2a_hex = txToHex(tx2a) self.nodes[0].sendrawtransaction(tx2a_hex, 0) # Lower fee, but we'll prioritise it tx2b = CTransaction() tx2b.vin = [CTxIn(tx1_outpoint, nSequence=0)] - tx2b.vout = [CTxOut(int(1.01 * COIN), CScript([b'a' * 35]))] + tx2b.vout = [CTxOut(int(1.01 * COIN), DUMMY_P2WPKH_SCRIPT)] tx2b.rehash() tx2b_hex = txToHex(tx2b) diff --git a/test/functional/p2p_segwit.py b/test/functional/p2p_segwit.py index 98f6b1d71d..0c7edbf434 100755 --- a/test/functional/p2p_segwit.py +++ b/test/functional/p2p_segwit.py @@ -67,8 +67,8 @@ from test_framework.script import ( SIGHASH_ANYONECANPAY, SIGHASH_NONE, SIGHASH_SINGLE, - SegwitVersion1SignatureHash, - SignatureHash, + SegwitV0SignatureHash, + LegacySignatureHash, hash160, ) from test_framework.test_framework import BitcoinTestFramework @@ -103,7 +103,7 @@ def get_p2pkh_script(pubkeyhash): def sign_p2pk_witness_input(script, tx_to, in_idx, hashtype, value, key): """Add signature for a P2PK witness program.""" - tx_hash = SegwitVersion1SignatureHash(script, tx_to, in_idx, hashtype, value) + tx_hash = SegwitV0SignatureHash(script, tx_to, in_idx, hashtype, value) signature = key.sign_ecdsa(tx_hash) + chr(hashtype).encode('latin-1') tx_to.wit.vtxinwit[in_idx].scriptWitness.stack = [signature, script] tx_to.rehash() @@ -1489,7 +1489,7 @@ class SegWitTest(BitcoinTestFramework): tx2.vin.append(CTxIn(COutPoint(tx.sha256, 0), b"")) tx2.vout.append(CTxOut(tx.vout[0].nValue - 1000, script_wsh)) script = get_p2pkh_script(pubkeyhash) - sig_hash = SegwitVersion1SignatureHash(script, tx2, 0, SIGHASH_ALL, tx.vout[0].nValue) + sig_hash = SegwitV0SignatureHash(script, tx2, 0, SIGHASH_ALL, tx.vout[0].nValue) signature = key.sign_ecdsa(sig_hash) + b'\x01' # 0x1 is SIGHASH_ALL tx2.wit.vtxinwit.append(CTxInWitness()) tx2.wit.vtxinwit[0].scriptWitness.stack = [signature, pubkey] @@ -1543,7 +1543,7 @@ class SegWitTest(BitcoinTestFramework): tx5 = CTransaction() tx5.vin.append(CTxIn(COutPoint(tx4.sha256, 0), b"")) tx5.vout.append(CTxOut(tx4.vout[0].nValue - 1000, CScript([OP_TRUE]))) - (sig_hash, err) = SignatureHash(script_pubkey, tx5, 0, SIGHASH_ALL) + (sig_hash, err) = LegacySignatureHash(script_pubkey, tx5, 0, SIGHASH_ALL) signature = key.sign_ecdsa(sig_hash) + b'\x01' # 0x1 is SIGHASH_ALL tx5.vin[0].scriptSig = CScript([signature, pubkey]) tx5.rehash() @@ -1693,7 +1693,7 @@ class SegWitTest(BitcoinTestFramework): tx2.vout.append(CTxOut(tx.vout[0].nValue, CScript([OP_TRUE]))) script = get_p2pkh_script(pubkeyhash) - sig_hash = SegwitVersion1SignatureHash(script, tx2, 0, SIGHASH_ALL, tx.vout[0].nValue) + sig_hash = SegwitV0SignatureHash(script, tx2, 0, SIGHASH_ALL, tx.vout[0].nValue) signature = key.sign_ecdsa(sig_hash) + b'\x01' # 0x1 is SIGHASH_ALL # Check that we can't have a scriptSig diff --git a/test/functional/test_framework/script.py b/test/functional/test_framework/script.py index 384062b808..51aa9057f7 100644 --- a/test/functional/test_framework/script.py +++ b/test/functional/test_framework/script.py @@ -2,7 +2,7 @@ # Copyright (c) 2015-2019 The Bitcoin Core developers # Distributed under the MIT software license, see the accompanying # file COPYING or http://www.opensource.org/licenses/mit-license.php. -"""Functionality to build scripts, as well as SignatureHash(). +"""Functionality to build scripts, as well as signature hash functions. This file is modified from python-bitcoinlib. """ @@ -608,7 +608,7 @@ def FindAndDelete(script, sig): return CScript(r) -def SignatureHash(script, txTo, inIdx, hashtype): +def LegacySignatureHash(script, txTo, inIdx, hashtype): """Consensus-correct SignatureHash Returns (hash, err) to precisely match the consensus-critical behavior of @@ -662,7 +662,7 @@ def SignatureHash(script, txTo, inIdx, hashtype): # Performance optimization probably not necessary for python tests, however. # Note that this corresponds to sigversion == 1 in EvalScript, which is used # for version 0 witnesses. -def SegwitVersion1SignatureHash(script, txTo, inIdx, hashtype, amount): +def SegwitV0SignatureHash(script, txTo, inIdx, hashtype, amount): hashPrevouts = 0 hashSequence = 0 diff --git a/test/functional/test_framework/script_util.py b/test/functional/test_framework/script_util.py new file mode 100755 index 0000000000..5ef67226c4 --- /dev/null +++ b/test/functional/test_framework/script_util.py @@ -0,0 +1,25 @@ +#!/usr/bin/env python3 +# Copyright (c) 2019 The Bitcoin Core developers +# Distributed under the MIT software license, see the accompanying +# file COPYING or http://www.opensource.org/licenses/mit-license.php. +"""Useful Script constants and utils.""" +from test_framework.script import CScript + +# To prevent a "tx-size-small" policy rule error, a transaction has to have a +# non-witness size of at least 82 bytes (MIN_STANDARD_TX_NONWITNESS_SIZE in +# src/policy/policy.h). Considering a Tx with the smallest possible single +# input (blank, empty scriptSig), and with an output omitting the scriptPubKey, +# we get to a minimum size of 60 bytes: +# +# Tx Skeleton: 4 [Version] + 1 [InCount] + 1 [OutCount] + 4 [LockTime] = 10 bytes +# Blank Input: 32 [PrevTxHash] + 4 [Index] + 1 [scriptSigLen] + 4 [SeqNo] = 41 bytes +# Output: 8 [Amount] + 1 [scriptPubKeyLen] = 9 bytes +# +# Hence, the scriptPubKey of the single output has to have a size of at +# least 22 bytes, which corresponds to the size of a P2WPKH scriptPubKey. +# The following script constant consists of a single push of 21 bytes of 'a': +# <PUSH_21> <21-bytes of 'a'> +# resulting in a 22-byte size. It should be used whenever (small) fake +# scriptPubKeys are needed, to guarantee that the minimum transaction size is +# met. +DUMMY_P2WPKH_SCRIPT = CScript([b'a' * 21]) diff --git a/test/functional/test_runner.py b/test/functional/test_runner.py index 9c92091f1d..e0b523b718 100755 --- a/test/functional/test_runner.py +++ b/test/functional/test_runner.py @@ -197,6 +197,7 @@ BASE_SCRIPTS = [ 'feature_uacomment.py', 'wallet_coinbase_category.py', 'feature_filelock.py', + 'feature_loadblock.py', 'p2p_dos_header_tree.py', 'p2p_unrequested_blocks.py', 'feature_includeconf.py', diff --git a/test/functional/wallet_address_types.py b/test/functional/wallet_address_types.py index b72f5c6008..479d67fb66 100755 --- a/test/functional/wallet_address_types.py +++ b/test/functional/wallet_address_types.py @@ -80,6 +80,9 @@ class AddressTypeTest(BitcoinTestFramework): ["-changetype=p2sh-segwit"], [], ] + # whitelist all peers to speed up tx relay / mempool sync + for args in self.extra_args: + args.append("-whitelist=127.0.0.1") def skip_test_if_missing_module(self): self.skip_if_no_wallet() diff --git a/test/functional/wallet_backup.py b/test/functional/wallet_backup.py index 93178c5ab2..bb835dc811 100755 --- a/test/functional/wallet_backup.py +++ b/test/functional/wallet_backup.py @@ -48,7 +48,13 @@ class WalletBackupTest(BitcoinTestFramework): self.num_nodes = 4 self.setup_clean_chain = True # nodes 1, 2,3 are spenders, let's give them a keypool=100 - self.extra_args = [["-keypool=100"], ["-keypool=100"], ["-keypool=100"], []] + # whitelist all peers to speed up tx relay / mempool sync + self.extra_args = [ + ["-keypool=100", "-whitelist=127.0.0.1"], + ["-keypool=100", "-whitelist=127.0.0.1"], + ["-keypool=100", "-whitelist=127.0.0.1"], + ["-whitelist=127.0.0.1"] + ] self.rpc_timeout = 120 def skip_test_if_missing_module(self): diff --git a/test/functional/wallet_multiwallet.py b/test/functional/wallet_multiwallet.py index 68bc45f986..ce0b7e8782 100755 --- a/test/functional/wallet_multiwallet.py +++ b/test/functional/wallet_multiwallet.py @@ -339,14 +339,10 @@ class MultiWalletTest(BitcoinTestFramework): self.log.info("Fail -upgradewallet that results in downgrade") assert_raises_rpc_error( -4, - "Wallet loading failed.", + 'Wallet loading failed: Error loading {}: Wallet requires newer version of {}'.format( + wallet_dir('high_minversion', 'wallet.dat'), self.config['environment']['PACKAGE_NAME']), lambda: self.nodes[0].loadwallet(filename='high_minversion'), ) - self.stop_node( - i=0, - expected_stderr='Error: Error loading {}: Wallet requires newer version of Bitcoin Core'.format( - wallet_dir('high_minversion', 'wallet.dat')), - ) if __name__ == '__main__': diff --git a/test/lint/README.md b/test/lint/README.md index 15974a3598..f415d619ee 100644 --- a/test/lint/README.md +++ b/test/lint/README.md @@ -7,6 +7,8 @@ Check for missing documentation of command line options. commit-script-check.sh ====================== Verification of [scripted diffs](/doc/developer-notes.md#scripted-diffs). +Scripted diffs are only assumed to run on the latest LTS release of Ubuntu. Running them on other operating systems +might require installing GNU tools, such as GNU sed. git-subtree-check.sh ==================== diff --git a/test/lint/lint-filenames.sh b/test/lint/lint-filenames.sh index 5391e43d91..6716cac0fe 100755 --- a/test/lint/lint-filenames.sh +++ b/test/lint/lint-filenames.sh @@ -12,7 +12,7 @@ export LC_ALL=C EXIT_CODE=0 OUTPUT=$(git ls-files --full-name -- "*.[cC][pP][pP]" "*.[hH]" "*.[pP][yY]" "*.[sS][hH]" | \ grep -vE '^[a-z0-9_./-]+$' | \ - grep -vE '^src/(secp256k1|univalue)/') + grep -vE '^src/(secp256k1/|univalue/|test/fuzz/FuzzedDataProvider.h)') if [[ ${OUTPUT} != "" ]]; then echo "Use only lowercase alphanumerics (a-z0-9), underscores (_), hyphens (-) and dots (.)" diff --git a/test/lint/lint-include-guards.sh b/test/lint/lint-include-guards.sh index 464969794b..0d654e796e 100755 --- a/test/lint/lint-include-guards.sh +++ b/test/lint/lint-include-guards.sh @@ -10,7 +10,7 @@ export LC_ALL=C HEADER_ID_PREFIX="BITCOIN_" HEADER_ID_SUFFIX="_H" -REGEXP_EXCLUDE_FILES_WITH_PREFIX="src/(crypto/ctaes/|leveldb/|secp256k1/|tinyformat.h|univalue/)" +REGEXP_EXCLUDE_FILES_WITH_PREFIX="src/(crypto/ctaes/|leveldb/|secp256k1/|test/fuzz/FuzzedDataProvider.h|tinyformat.h|univalue/)" EXIT_CODE=0 for HEADER_FILE in $(git ls-files -- "*.h" | grep -vE "^${REGEXP_EXCLUDE_FILES_WITH_PREFIX}") diff --git a/test/sanitizer_suppressions/ubsan b/test/sanitizer_suppressions/ubsan index 643272de52..e7c690fabe 100644 --- a/test/sanitizer_suppressions/ubsan +++ b/test/sanitizer_suppressions/ubsan @@ -1,9 +1,19 @@ +# -fsanitize=undefined suppressions +# ================================= alignment:move.h alignment:prevector.h -bool:wallet/wallet.cpp float-divide-by-zero:policy/fees.cpp float-divide-by-zero:validation.cpp float-divide-by-zero:wallet/wallet.cpp + +# -fsanitize=integer suppressions +# =============================== +# Unsigned integer overflow occurs when the result of an unsigned integer +# computation cannot be represented in its type. Unlike signed integer overflow, +# this is not undefined behavior, but it is often unintentional. The list below +# contains files in which we expect unsigned integer overflows to occur. The +# list is used to suppress -fsanitize=integer warnings when running our CI UBSan +# job. unsigned-integer-overflow:arith_uint256.h unsigned-integer-overflow:basic_string.h unsigned-integer-overflow:bench/bench.h @@ -32,4 +42,3 @@ unsigned-integer-overflow:stl_bvector.h unsigned-integer-overflow:txmempool.cpp unsigned-integer-overflow:util/strencodings.cpp unsigned-integer-overflow:validation.cpp -vptr:fs.cpp |