diff options
37 files changed, 770 insertions, 436 deletions
diff --git a/.github/ISSUE_TEMPLATE.md b/.github/ISSUE_TEMPLATE.md index a16cedd7fd..acb54dd354 100644 --- a/.github/ISSUE_TEMPLATE.md +++ b/.github/ISSUE_TEMPLATE.md @@ -1,23 +1,35 @@ <!--- Remove sections that do not apply --> + +This issue tracker is only for technical issues related to bitcoin-core. + +General bitcoin questions and/or support requests and are best directed to the [Bitcoin StackExchange](https://bitcoin.stackexchange.com). + ### Describe the issue -### Is the issue reproducible? -#### List steps to reproduce below: -1. -2. -3. +### Can you reliably reproduce the issue? +#### If so, please list the steps to reproduce below: +1. +2. +3. -### Expected behavior +### Expected behaviour Tell us what should happen -### Actual behavior +### Actual behaviour Tell us what happens instead -### Any information in the debug.log file related to this issue? - -### Screenshots (if available) +### Screenshots. +If the issue is related to the GUI, screenshots can be added to this issue via drag & drop. -### What binary version was used (official or self compiled) +### What version of bitcoin-core are you using? +List the version number/commit ID, and if it is an official binary, self compiled or a distribution package such as PPA. -### Machine specs: CPU, RAM, Disk space & OS (Windows, OS X, Linux) +### Machine specs: +- OS: +- CPU: +- RAM: +- Disk size: +- Disk Type (HD/SDD): +### Any extra information that might be useful in the debugging process. +This is normally the contents of a `debug.log` or `config.log` file. Raw text or a link to a pastebin type site are preferred. diff --git a/INSTALL b/INSTALL deleted file mode 100644 index 07ee48427c..0000000000 --- a/INSTALL +++ /dev/null @@ -1,5 +0,0 @@ -Building Bitcoin - -See doc/build-*.md for instructions on building bitcoind, -the intended-for-services, no-graphical-interface, reference -implementation of Bitcoin.
\ No newline at end of file diff --git a/INSTALL.md b/INSTALL.md new file mode 100644 index 0000000000..520a47d960 --- /dev/null +++ b/INSTALL.md @@ -0,0 +1,5 @@ +Building Bitcoin +================ + +See doc/build-*.md for instructions on building the various +elements of the Bitcoin Core reference implementation of Bitcoin. diff --git a/doc/README.md b/doc/README.md index 09a507c9ce..e4fa49614a 100644 --- a/doc/README.md +++ b/doc/README.md @@ -3,7 +3,9 @@ Bitcoin Core 0.13.99 Setup --------------------- -[Bitcoin Core](http://bitcoin.org/en/download) is the original Bitcoin client and it builds the backbone of the network. However, it downloads and stores the entire history of Bitcoin transactions (which is currently several GBs); depending on the speed of your computer and network connection, the synchronization process can take anywhere from a few hours to a day or more. +Bitcoin Core is the original Bitcoin client and it builds the backbone of the network. However, it downloads and stores the entire history of Bitcoin transactions (which is currently several GBs); depending on the speed of your computer and network connection, the synchronization process can take anywhere from a few hours to a day or more. + +To download Bitcoin Core, visit [bitcoincore.org](https://bitcoincore.org/en/releases/). Running --------------------- @@ -46,13 +48,13 @@ Development The Bitcoin repo's [root README](/README.md) contains relevant information on the development process and automated testing. - [Developer Notes](developer-notes.md) -- [Multiwallet Qt Development](multiwallet-qt.md) - [Release Notes](release-notes.md) - [Release Process](release-process.md) - [Source Code Documentation (External Link)](https://dev.visucore.com/bitcoin/doxygen/) - [Translation Process](translation_process.md) - [Translation Strings Policy](translation_strings_policy.md) - [Unit Tests](unit-tests.md) +- [Travis CI](travis-ci.md) - [Unauthenticated REST Interface](REST-interface.md) - [Shared Libraries](shared-libraries.md) - [BIPS](bips.md) @@ -67,11 +69,13 @@ The Bitcoin repo's [root README](/README.md) contains relevant information on th ### Miscellaneous - [Assets Attribution](assets-attribution.md) - [Files](files.md) +- [Reduce Traffic](reduce-traffic.md) - [Tor Support](tor.md) - [Init Scripts (systemd/upstart/openrc)](init.md) +- [ZMQ](zmq.md) License --------------------- -Distributed under the [MIT software license](http://www.opensource.org/licenses/mit-license.php). +Distributed under the [MIT software license](/COPYING). This product includes software developed by the OpenSSL Project for use in the [OpenSSL Toolkit](https://www.openssl.org/). This product includes cryptographic software written by Eric Young ([eay@cryptsoft.com](mailto:eay@cryptsoft.com)), and UPnP software written by Thomas Bernard. diff --git a/doc/bips.md b/doc/bips.md index 039d5114fd..040439fbcb 100644 --- a/doc/bips.md +++ b/doc/bips.md @@ -26,4 +26,9 @@ BIPs that are implemented by Bitcoin Core (up-to-date up to **v0.13.0**): * [`BIP 125`](https://github.com/bitcoin/bips/blob/master/bip-0125.mediawiki): Opt-in full replace-by-fee signaling honoured in mempool and mining as of **v0.12.0** ([PR 6871](https://github.com/bitcoin/bitcoin/pull/6871)). * [`BIP 130`](https://github.com/bitcoin/bips/blob/master/bip-0130.mediawiki): direct headers announcement is negotiated with peer versions `>=70012` as of **v0.12.0** ([PR 6494](https://github.com/bitcoin/bitcoin/pull/6494)). * [`BIP 133`](https://github.com/bitcoin/bips/blob/master/bip-0133.mediawiki): feefilter messages are respected and sent for peer versions `>=70013` as of **v0.13.0** ([PR 7542](https://github.com/bitcoin/bitcoin/pull/7542)). +* [`BIP 141`](https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki): Segregated Witness (Consensus Layer) as of **v0.13.0** ([PR 8149](https://github.com/bitcoin/bitcoin/pull/8149)). +* [`BIP 143`](https://github.com/bitcoin/bips/blob/master/bip-0143.mediawiki): Transaction Signature Verification for Version 0 Witness Program as of **v0.13.0** ([PR 8149](https://github.com/bitcoin/bitcoin/pull/8149)). +* [`BIP 144`](https://github.com/bitcoin/bips/blob/master/bip-0144.mediawiki): Segregated Witness as of **0.13.0** ([PR 8149](https://github.com/bitcoin/bitcoin/pull/8149)). +* [`BIP 145`](https://github.com/bitcoin/bips/blob/master/bip-0145.mediawiki): getblocktemplate updates for Segregated Witness as of **v0.13.0** ([PR 8149](https://github.com/bitcoin/bitcoin/pull/8149)). +* [`BIP 147`](https://github.com/bitcoin/bips/blob/master/bip-0147.mediawiki): NULLDUMMY softfork as of **v0.13.1** ([PR 8636](https://github.com/bitcoin/bitcoin/pull/8636)). * [`BIP 152`](https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki): Compact block transfer and related optimizations are used as of **v0.13.0** ([PR 8068](https://github.com/bitcoin/bitcoin/pull/8068)). diff --git a/doc/build-unix.md b/doc/build-unix.md index 62e3e793e9..5202072f8b 100644 --- a/doc/build-unix.md +++ b/doc/build-unix.md @@ -308,3 +308,35 @@ To build executables for ARM: For further documentation on the depends system see [README.md](../depends/README.md) in the depends directory. + +Building on FreeBSD +-------------------- + +(Updated as of FreeBSD 10.3) + +Clang is installed by default as `cc` compiler, this makes it easier to get +started than on [OpenBSD](build-openbsd.md). Installing dependencies: + + pkg install autoconf automake libtool pkgconf + pkg install boost-libs openssl libevent2 + +(`libressl` instead of `openssl` will also work) + +For the wallet (optional): + + pkg install db5 + +This will give a warning "configure: WARNING: Found Berkeley DB other +than 4.8; wallets opened by this build will not be portable!", but as FreeBSD never +had a binary release, this may not matter. If backwards compatibility +with 4.8-built Bitcoin Core is needed follow the steps under "Berkeley DB" above. + +Then build using: + + ./autogen.sh + ./configure --with-incompatible-bdb CPPFLAGS=-I/usr/local/include/db5 LDFLAGS=-L/usr/local/lib/db5 + make + +*Note on debugging*: The version of `gdb` installed by default is [ancient and considered harmful](https://wiki.freebsd.org/GdbRetirement). +It is not suitable for debugging a multi-threaded C++ program, not even for getting backtraces. Please install the package `gdb` and +use the versioned gdb command e.g. `gdb7111`. diff --git a/doc/multiwallet-qt.md b/doc/multiwallet-qt.md deleted file mode 100644 index 3caab81807..0000000000 --- a/doc/multiwallet-qt.md +++ /dev/null @@ -1,48 +0,0 @@ -Multiwallet Qt Development and Integration Strategy -=================================================== - -In order to support loading of multiple wallets in bitcoin-qt, a few changes in the UI architecture will be needed. -Fortunately, only four of the files in the existing project are affected by this change. - -Two new classes have been implemented in two new .h/.cpp file pairs, with much of the functionality that was previously -implemented in the BitcoinGUI class moved over to these new classes. - -The two existing files most affected, by far, are bitcoingui.h and bitcoingui.cpp, as the BitcoinGUI class will require -some major retrofitting. - -Only requiring some minor changes is bitcoin.cpp. - -Finally, two new headers and source files will have to be added to bitcoin-qt.pro. - -Changes to class BitcoinGUI ---------------------------- -The principal change to the BitcoinGUI class concerns the QStackedWidget instance called centralWidget. -This widget owns five page views: overviewPage, transactionsPage, addressBookPage, receiveCoinsPage, and sendCoinsPage. - -A new class called *WalletView* inheriting from QStackedWidget has been written to handle all renderings and updates of -these page views. In addition to owning these five page views, a WalletView also has a pointer to a WalletModel instance. -This allows the construction of multiple WalletView objects, each rendering a distinct wallet. - -A second class called *WalletFrame* inheriting from QFrame has been written as a container for embedding all wallet-related -controls into BitcoinGUI. At present it contains the WalletView instances for the wallets and does little more than passing on messages -from BitcoinGUI to the currently selected WalletView. It is a WalletFrame instance -that takes the place of what used to be centralWidget in BitcoinGUI. The purpose of this class is to allow future -refinements of the wallet controls with minimal need for further modifications to BitcoinGUI, thus greatly simplifying -merges while reducing the risk of breaking top-level stuff. - -Changes to bitcoin.cpp ----------------------- -bitcoin.cpp is the entry point into bitcoin-qt, and as such, will require some minor modifications to provide hooks for -multiple wallet support. Most importantly will be the way it instantiates WalletModels and passes them to the -singleton BitcoinGUI instance called window. Formerly, BitcoinGUI kept a pointer to a single instance of a WalletModel. -The initial change required is very simple: rather than calling `window.setWalletModel(&walletModel);` we perform the -following two steps: - - window.addWallet("~Default", &walletModel); - window.setCurrentWallet("~Default"); - -The string parameter is just an arbitrary name given to the default wallet. It's been prepended with a tilde to avoid name collisions in the future with additional wallets. - -The shutdown call `window.setWalletModel(0)` has also been removed. In its place is now: - -window.removeAllWallets(); diff --git a/doc/release-notes/release-notes-0.13.0.md b/doc/release-notes/release-notes-0.13.0.md index 5dd3f5a651..f9bf3d75de 100644 --- a/doc/release-notes/release-notes-0.13.0.md +++ b/doc/release-notes/release-notes-0.13.0.md @@ -643,7 +643,7 @@ git merge commit are mentioned. - #8041 `5b736dd` Fix bip9-softforks blockstore issue (MarcoFalke) - #7994 `1f01443` Add op csv tests to script_tests.json (Christewart) - #8038 `e2bf830` Various minor fixes (MarcoFalke) -- #8072 `1b87e5b` Travis: 'make check' in parallel and verbose (MarcoFalke) +- #8072 `1b87e5b` Travis: 'make check' in parallel and verbose (theuni) - #8056 `8844ef1` Remove hardcoded "4 nodes" from test_framework (MarcoFalke) - #8047 `37f9a1f` Test_framework: Set wait-timeout for bitcoind procs (MarcoFalke) - #8095 `6700cc9` Test framework: only cleanup on successful test runs (sdaftuar) diff --git a/doc/release-process.md b/doc/release-process.md index d07f9a89ad..63f75fb399 100644 --- a/doc/release-process.md +++ b/doc/release-process.md @@ -16,6 +16,7 @@ Before every minor and major release: Before every major release: * Update hardcoded [seeds](/contrib/seeds/README.md), see [this pull request](https://github.com/bitcoin/bitcoin/pull/7415) for an example. +* Update [`BLOCK_CHAIN_SIZE`](/src/qt/intro.cpp) to the current size plus some overhead. ### First time / New builders diff --git a/doc/shared-libraries.md b/doc/shared-libraries.md index ec6f16c8aa..dc363582cc 100644 --- a/doc/shared-libraries.md +++ b/doc/shared-libraries.md @@ -30,12 +30,17 @@ The interface is defined in the C header `bitcoinconsensus.h` located in `src/s - `bitcoinconsensus_SCRIPT_FLAGS_VERIFY_NONE` - `bitcoinconsensus_SCRIPT_FLAGS_VERIFY_P2SH` - Evaluate P2SH ([BIP16](https://github.com/bitcoin/bips/blob/master/bip-0016.mediawiki)) subscripts - `bitcoinconsensus_SCRIPT_FLAGS_VERIFY_DERSIG` - Enforce strict DER ([BIP66](https://github.com/bitcoin/bips/blob/master/bip-0066.mediawiki)) compliance +- `bitcoinconsensus_SCRIPT_FLAGS_VERIFY_NULLDUMMY` - Enforce NULLDUMMY ([BIP147](https://github.com/bitcoin/bips/blob/master/bip-0147.mediawiki)) +- `bitcoinconsensus_SCRIPT_FLAGS_VERIFY_CHECKLOCKTIMEVERIFY` - Enable CHECKLOCKTIMEVERIFY ([BIP65](https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki)) +- `bitcoinconsensus_SCRIPT_FLAGS_VERIFY_CHECKSEQUENCEVERIFY` - Enable CHECKSEQUENCEVERIFY ([BIP112](https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki)) +- `bitcoinconsensus_SCRIPT_FLAGS_VERIFY_WITNESS` - Enable WITNESS ([BIP141](https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki)) ##### Errors - `bitcoinconsensus_ERR_OK` - No errors with input parameters *(see the return value of `bitcoinconsensus_verify_script` for the verification status)* - `bitcoinconsensus_ERR_TX_INDEX` - An invalid index for `txTo` - `bitcoinconsensus_ERR_TX_SIZE_MISMATCH` - `txToLen` did not match with the size of `txTo` - `bitcoinconsensus_ERR_DESERIALIZE` - An error deserializing `txTo` +- `bitcoinconsensus_ERR_AMOUNT_REQUIRED` - Input amount is required if WITNESS is used ### Example Implementations - [NBitcoin](https://github.com/NicolasDorier/NBitcoin/blob/master/NBitcoin/Script.cs#L814) (.NET Bindings) diff --git a/doc/travis-ci.txt b/doc/travis-ci.md index 06410405d6..38085cec35 100644 --- a/doc/travis-ci.txt +++ b/doc/travis-ci.md @@ -1,5 +1,8 @@ +Travis CI +========= + Support for using travis-ci has been added in order to automate pull-testing. -See https://travis-ci.org/ for more info +See [travis-ci.org](https://travis-ci.org/) for more info This procedure is different than the pull-tester that came before it in a few ways. @@ -7,7 +10,7 @@ ways. There is nothing to administer. This is a major feature as it means that builds have no local state. Because there is no ability to login to the builders to install packages (tools, dependencies, etc), the entire build -procedure must instead be controlled by a declarative script (.travis.yml). +procedure must instead be controlled by a declarative script `.travis.yml`. This script declares each build configuration, creates virtual machines as necessary, builds, then discards the virtual machines. @@ -16,7 +19,7 @@ than a single pass/fail. This helps to catch build failures and logic errors that present on platforms other than the ones the author has tested. This matrix is defined in the build script and can be changed at any time. -All builders use the dependency-generator in the depends dir, rather than +All builders use the dependency-generator in the [depends dir](/depends), rather than using apt-get to install build dependencies. This guarantees that the tester is using the same versions as Gitian, so the build results are nearly identical to what would be found in a final release. However, this also means that builds diff --git a/qa/rpc-tests/p2p-compactblocks.py b/qa/rpc-tests/p2p-compactblocks.py index ac4655a841..e7f5a1c9c6 100755 --- a/qa/rpc-tests/p2p-compactblocks.py +++ b/qa/rpc-tests/p2p-compactblocks.py @@ -12,14 +12,16 @@ from test_framework.script import CScript, OP_TRUE ''' CompactBlocksTest -- test compact blocks (BIP 152) -''' +Version 1 compact blocks are pre-segwit (txids) +Version 2 compact blocks are post-segwit (wtxids) +''' # TestNode: A peer we use to send messages to bitcoind, and store responses. class TestNode(SingleNodeConnCB): def __init__(self): SingleNodeConnCB.__init__(self) - self.last_sendcmpct = None + self.last_sendcmpct = [] self.last_headers = None self.last_inv = None self.last_cmpctblock = None @@ -28,9 +30,13 @@ class TestNode(SingleNodeConnCB): self.last_getblocktxn = None self.last_block = None self.last_blocktxn = None + # Store the hashes of blocks we've seen announced. + # This is for synchronizing the p2p message traffic, + # so we can eg wait until a particular block is announced. + self.set_announced_blockhashes = set() def on_sendcmpct(self, conn, message): - self.last_sendcmpct = message + self.last_sendcmpct.append(message) def on_block(self, conn, message): self.last_block = message @@ -38,14 +44,22 @@ class TestNode(SingleNodeConnCB): def on_cmpctblock(self, conn, message): self.last_cmpctblock = message self.block_announced = True + self.last_cmpctblock.header_and_shortids.header.calc_sha256() + self.set_announced_blockhashes.add(self.last_cmpctblock.header_and_shortids.header.sha256) def on_headers(self, conn, message): self.last_headers = message self.block_announced = True + for x in self.last_headers.headers: + x.calc_sha256() + self.set_announced_blockhashes.add(x.sha256) def on_inv(self, conn, message): self.last_inv = message - self.block_announced = True + for x in self.last_inv.inv: + if x.type == 2: + self.block_announced = True + self.set_announced_blockhashes.add(x.hash) def on_getdata(self, conn, message): self.last_getdata = message @@ -85,34 +99,42 @@ class TestNode(SingleNodeConnCB): assert(self.received_block_announcement()) self.clear_block_announcement() + # Block until a block announcement for a particular block hash is + # received. + def wait_for_block_announcement(self, block_hash, timeout=30): + def received_hash(): + return (block_hash in self.set_announced_blockhashes) + return wait_until(received_hash, timeout=timeout) class CompactBlocksTest(BitcoinTestFramework): def __init__(self): super().__init__() self.setup_clean_chain = True - self.num_nodes = 1 + # Node0 = pre-segwit, node1 = segwit-aware + self.num_nodes = 2 self.utxos = [] def setup_network(self): self.nodes = [] - # Turn off segwit in this test, as compact blocks don't currently work - # with segwit. (After BIP 152 is updated to support segwit, we can - # test behavior with and without segwit enabled by adding a second node - # to the test.) - self.nodes = start_nodes(self.num_nodes, self.options.tmpdir, [["-debug", "-logtimemicros=1", "-bip9params=segwit:0:0"]]) + # Start up node0 to be a version 1, pre-segwit node. + self.nodes = start_nodes(self.num_nodes, self.options.tmpdir, + [["-debug", "-logtimemicros=1", "-bip9params=segwit:0:0"], + ["-debug", "-logtimemicros", "-txindex"]]) + connect_nodes(self.nodes[0], 1) - def build_block_on_tip(self): - height = self.nodes[0].getblockcount() - tip = self.nodes[0].getbestblockhash() - mtp = self.nodes[0].getblockheader(tip)['mediantime'] + def build_block_on_tip(self, node): + height = node.getblockcount() + tip = node.getbestblockhash() + mtp = node.getblockheader(tip)['mediantime'] block = create_block(int(tip, 16), create_coinbase(height + 1), mtp + 1) block.solve() return block # Create 10 more anyone-can-spend utxo's for testing. def make_utxos(self): - block = self.build_block_on_tip() + # Doesn't matter which node we use, just use node0. + block = self.build_block_on_tip(self.nodes[0]) self.test_node.send_and_ping(msg_block(block)) assert(int(self.nodes[0].getbestblockhash(), 16) == block.sha256) self.nodes[0].generate(100) @@ -125,7 +147,7 @@ class CompactBlocksTest(BitcoinTestFramework): tx.vout.append(CTxOut(out_value, CScript([OP_TRUE]))) tx.rehash() - block2 = self.build_block_on_tip() + block2 = self.build_block_on_tip(self.nodes[0]) block2.vtx.append(tx) block2.hashMerkleRoot = block2.calc_merkle_root() block2.solve() @@ -134,26 +156,30 @@ class CompactBlocksTest(BitcoinTestFramework): self.utxos.extend([[tx.sha256, i, out_value] for i in range(10)]) return - # Test "sendcmpct": - # - No compact block announcements or getdata(MSG_CMPCT_BLOCK) unless - # sendcmpct is sent. - # - If sendcmpct is sent with version > 1, the message is ignored. + # Test "sendcmpct" (between peers preferring the same version): + # - No compact block announcements unless sendcmpct is sent. + # - If sendcmpct is sent with version > preferred_version, the message is ignored. # - If sendcmpct is sent with boolean 0, then block announcements are not # made with compact blocks. # - If sendcmpct is then sent with boolean 1, then new block announcements # are made with compact blocks. - def test_sendcmpct(self): - print("Testing SENDCMPCT p2p message... ") - - # Make sure we get a version 0 SENDCMPCT message from our peer + # If old_node is passed in, request compact blocks with version=preferred-1 + # and verify that it receives block announcements via compact block. + def test_sendcmpct(self, node, test_node, preferred_version, old_node=None): + # Make sure we get a SENDCMPCT message from our peer def received_sendcmpct(): - return (self.test_node.last_sendcmpct is not None) + return (len(test_node.last_sendcmpct) > 0) got_message = wait_until(received_sendcmpct, timeout=30) assert(received_sendcmpct()) assert(got_message) - assert_equal(self.test_node.last_sendcmpct.version, 1) + with mininode_lock: + # Check that the first version received is the preferred one + assert_equal(test_node.last_sendcmpct[0].version, preferred_version) + # And that we receive versions down to 1. + assert_equal(test_node.last_sendcmpct[-1].version, 1) + test_node.last_sendcmpct = [] - tip = int(self.nodes[0].getbestblockhash(), 16) + tip = int(node.getbestblockhash(), 16) def check_announcement_of_new_block(node, peer, predicate): peer.clear_block_announcement() @@ -165,56 +191,75 @@ class CompactBlocksTest(BitcoinTestFramework): assert(predicate(peer)) # We shouldn't get any block announcements via cmpctblock yet. - check_announcement_of_new_block(self.nodes[0], self.test_node, lambda p: p.last_cmpctblock is None) + check_announcement_of_new_block(node, test_node, lambda p: p.last_cmpctblock is None) # Try one more time, this time after requesting headers. - self.test_node.request_headers_and_sync(locator=[tip]) - check_announcement_of_new_block(self.nodes[0], self.test_node, lambda p: p.last_cmpctblock is None and p.last_inv is not None) + test_node.request_headers_and_sync(locator=[tip]) + check_announcement_of_new_block(node, test_node, lambda p: p.last_cmpctblock is None and p.last_inv is not None) # Test a few ways of using sendcmpct that should NOT # result in compact block announcements. # Before each test, sync the headers chain. - self.test_node.request_headers_and_sync(locator=[tip]) + test_node.request_headers_and_sync(locator=[tip]) # Now try a SENDCMPCT message with too-high version sendcmpct = msg_sendcmpct() - sendcmpct.version = 2 - self.test_node.send_and_ping(sendcmpct) - check_announcement_of_new_block(self.nodes[0], self.test_node, lambda p: p.last_cmpctblock is None) + sendcmpct.version = preferred_version+1 + sendcmpct.announce = True + test_node.send_and_ping(sendcmpct) + check_announcement_of_new_block(node, test_node, lambda p: p.last_cmpctblock is None) # Headers sync before next test. - self.test_node.request_headers_and_sync(locator=[tip]) + test_node.request_headers_and_sync(locator=[tip]) # Now try a SENDCMPCT message with valid version, but announce=False - self.test_node.send_and_ping(msg_sendcmpct()) - check_announcement_of_new_block(self.nodes[0], self.test_node, lambda p: p.last_cmpctblock is None) + sendcmpct.version = preferred_version + sendcmpct.announce = False + test_node.send_and_ping(sendcmpct) + check_announcement_of_new_block(node, test_node, lambda p: p.last_cmpctblock is None) # Headers sync before next test. - self.test_node.request_headers_and_sync(locator=[tip]) + test_node.request_headers_and_sync(locator=[tip]) # Finally, try a SENDCMPCT message with announce=True - sendcmpct.version = 1 + sendcmpct.version = preferred_version sendcmpct.announce = True - self.test_node.send_and_ping(sendcmpct) - check_announcement_of_new_block(self.nodes[0], self.test_node, lambda p: p.last_cmpctblock is not None) + test_node.send_and_ping(sendcmpct) + check_announcement_of_new_block(node, test_node, lambda p: p.last_cmpctblock is not None) # Try one more time (no headers sync should be needed!) - check_announcement_of_new_block(self.nodes[0], self.test_node, lambda p: p.last_cmpctblock is not None) + check_announcement_of_new_block(node, test_node, lambda p: p.last_cmpctblock is not None) # Try one more time, after turning on sendheaders - self.test_node.send_and_ping(msg_sendheaders()) - check_announcement_of_new_block(self.nodes[0], self.test_node, lambda p: p.last_cmpctblock is not None) + test_node.send_and_ping(msg_sendheaders()) + check_announcement_of_new_block(node, test_node, lambda p: p.last_cmpctblock is not None) + + # Try one more time, after sending a version-1, announce=false message. + sendcmpct.version = preferred_version-1 + sendcmpct.announce = False + test_node.send_and_ping(sendcmpct) + check_announcement_of_new_block(node, test_node, lambda p: p.last_cmpctblock is not None) # Now turn off announcements + sendcmpct.version = preferred_version sendcmpct.announce = False - self.test_node.send_and_ping(sendcmpct) - check_announcement_of_new_block(self.nodes[0], self.test_node, lambda p: p.last_cmpctblock is None and p.last_headers is not None) + test_node.send_and_ping(sendcmpct) + check_announcement_of_new_block(node, test_node, lambda p: p.last_cmpctblock is None and p.last_headers is not None) + + if old_node is not None: + # Verify that a peer using an older protocol version can receive + # announcements from this node. + sendcmpct.version = preferred_version-1 + sendcmpct.announce = True + old_node.send_and_ping(sendcmpct) + # Header sync + old_node.request_headers_and_sync(locator=[tip]) + check_announcement_of_new_block(node, old_node, lambda p: p.last_cmpctblock is not None) # This test actually causes bitcoind to (reasonably!) disconnect us, so do this last. def test_invalid_cmpctblock_message(self): - print("Testing invalid index in cmpctblock message...") self.nodes[0].generate(101) - block = self.build_block_on_tip() + block = self.build_block_on_tip(self.nodes[0]) cmpct_block = P2PHeaderAndShortIDs() cmpct_block.header = CBlockHeader(block) @@ -227,45 +272,63 @@ class CompactBlocksTest(BitcoinTestFramework): # Compare the generated shortids to what we expect based on BIP 152, given # bitcoind's choice of nonce. - def test_compactblock_construction(self): - print("Testing compactblock headers and shortIDs are correct...") - + def test_compactblock_construction(self, node, test_node, version, use_witness_address): # Generate a bunch of transactions. - self.nodes[0].generate(101) + node.generate(101) num_transactions = 25 - address = self.nodes[0].getnewaddress() + address = node.getnewaddress() + if use_witness_address: + # Want at least one segwit spend, so move all funds to + # a witness address. + address = node.addwitnessaddress(address) + value_to_send = node.getbalance() + node.sendtoaddress(address, satoshi_round(value_to_send-Decimal(0.1))) + node.generate(1) + + segwit_tx_generated = False for i in range(num_transactions): - self.nodes[0].sendtoaddress(address, 0.1) + txid = node.sendtoaddress(address, 0.1) + hex_tx = node.gettransaction(txid)["hex"] + tx = FromHex(CTransaction(), hex_tx) + if not tx.wit.is_null(): + segwit_tx_generated = True - self.test_node.sync_with_ping() + if use_witness_address: + assert(segwit_tx_generated) # check that our test is not broken + + # Wait until we've seen the block announcement for the resulting tip + tip = int(self.nodes[0].getbestblockhash(), 16) + assert(self.test_node.wait_for_block_announcement(tip)) # Now mine a block, and look at the resulting compact block. - self.test_node.clear_block_announcement() - block_hash = int(self.nodes[0].generate(1)[0], 16) + test_node.clear_block_announcement() + block_hash = int(node.generate(1)[0], 16) # Store the raw block in our internal format. - block = FromHex(CBlock(), self.nodes[0].getblock("%02x" % block_hash, False)) + block = FromHex(CBlock(), node.getblock("%02x" % block_hash, False)) [tx.calc_sha256() for tx in block.vtx] block.rehash() # Don't care which type of announcement came back for this test; just # request the compact block if we didn't get one yet. - wait_until(self.test_node.received_block_announcement, timeout=30) + wait_until(test_node.received_block_announcement, timeout=30) + assert(test_node.received_block_announcement()) with mininode_lock: - if self.test_node.last_cmpctblock is None: - self.test_node.clear_block_announcement() + if test_node.last_cmpctblock is None: + test_node.clear_block_announcement() inv = CInv(4, block_hash) # 4 == "CompactBlock" - self.test_node.send_message(msg_getdata([inv])) + test_node.send_message(msg_getdata([inv])) - wait_until(self.test_node.received_block_announcement, timeout=30) + wait_until(test_node.received_block_announcement, timeout=30) + assert(test_node.received_block_announcement()) # Now we should have the compactblock header_and_shortids = None with mininode_lock: - assert(self.test_node.last_cmpctblock is not None) + assert(test_node.last_cmpctblock is not None) # Convert the on-the-wire representation to absolute indexes - header_and_shortids = HeaderAndShortIDs(self.test_node.last_cmpctblock.header_and_shortids) + header_and_shortids = HeaderAndShortIDs(test_node.last_cmpctblock.header_and_shortids) # Check that we got the right block! header_and_shortids.header.calc_sha256() @@ -278,8 +341,17 @@ class CompactBlocksTest(BitcoinTestFramework): # Check that all prefilled_txn entries match what's in the block. for entry in header_and_shortids.prefilled_txn: entry.tx.calc_sha256() + # This checks the non-witness parts of the tx agree assert_equal(entry.tx.sha256, block.vtx[entry.index].sha256) + # And this checks the witness + wtxid = entry.tx.calc_sha256(True) + if version == 2: + assert_equal(wtxid, block.vtx[entry.index].calc_sha256(True)) + else: + # Shouldn't have received a witness + assert(entry.tx.wit.is_null()) + # Check that the cmpctblock message announced all the transactions. assert_equal(len(header_and_shortids.prefilled_txn) + len(header_and_shortids.shortids), len(block.vtx)) @@ -294,7 +366,10 @@ class CompactBlocksTest(BitcoinTestFramework): # Already checked prefilled transactions above header_and_shortids.prefilled_txn.pop(0) else: - shortid = calculate_shortid(k0, k1, block.vtx[index].sha256) + tx_hash = block.vtx[index].sha256 + if version == 2: + tx_hash = block.vtx[index].calc_sha256(True) + shortid = calculate_shortid(k0, k1, tx_hash) assert_equal(shortid, header_and_shortids.shortids[0]) header_and_shortids.shortids.pop(0) index += 1 @@ -302,49 +377,52 @@ class CompactBlocksTest(BitcoinTestFramework): # Test that bitcoind requests compact blocks when we announce new blocks # via header or inv, and that responding to getblocktxn causes the block # to be successfully reconstructed. - def test_compactblock_requests(self): - print("Testing compactblock requests... ") - + # Post-segwit: upgraded nodes would only make this request of cb-version-2, + # NODE_WITNESS peers. Unupgraded nodes would still make this request of + # any cb-version-1-supporting peer. + def test_compactblock_requests(self, node, test_node): # Try announcing a block with an inv or header, expect a compactblock # request for announce in ["inv", "header"]: - block = self.build_block_on_tip() + block = self.build_block_on_tip(node) with mininode_lock: - self.test_node.last_getdata = None + test_node.last_getdata = None if announce == "inv": - self.test_node.send_message(msg_inv([CInv(2, block.sha256)])) + test_node.send_message(msg_inv([CInv(2, block.sha256)])) else: - self.test_node.send_header_for_blocks([block]) - success = wait_until(lambda: self.test_node.last_getdata is not None, timeout=30) + test_node.send_header_for_blocks([block]) + success = wait_until(lambda: test_node.last_getdata is not None, timeout=30) assert(success) - assert_equal(len(self.test_node.last_getdata.inv), 1) - assert_equal(self.test_node.last_getdata.inv[0].type, 4) - assert_equal(self.test_node.last_getdata.inv[0].hash, block.sha256) + assert_equal(len(test_node.last_getdata.inv), 1) + assert_equal(test_node.last_getdata.inv[0].type, 4) + assert_equal(test_node.last_getdata.inv[0].hash, block.sha256) # Send back a compactblock message that omits the coinbase comp_block = HeaderAndShortIDs() comp_block.header = CBlockHeader(block) comp_block.nonce = 0 - comp_block.shortids = [1] # this is useless, and wrong - self.test_node.send_and_ping(msg_cmpctblock(comp_block.to_p2p())) - assert_equal(int(self.nodes[0].getbestblockhash(), 16), block.hashPrevBlock) + [k0, k1] = comp_block.get_siphash_keys() + comp_block.shortids = [ + calculate_shortid(k0, k1, block.vtx[0].sha256) ] + test_node.send_and_ping(msg_cmpctblock(comp_block.to_p2p())) + assert_equal(int(node.getbestblockhash(), 16), block.hashPrevBlock) # Expect a getblocktxn message. with mininode_lock: - assert(self.test_node.last_getblocktxn is not None) - absolute_indexes = self.test_node.last_getblocktxn.block_txn_request.to_absolute() + assert(test_node.last_getblocktxn is not None) + absolute_indexes = test_node.last_getblocktxn.block_txn_request.to_absolute() assert_equal(absolute_indexes, [0]) # should be a coinbase request # Send the coinbase, and verify that the tip advances. msg = msg_blocktxn() msg.block_transactions.blockhash = block.sha256 msg.block_transactions.transactions = [block.vtx[0]] - self.test_node.send_and_ping(msg) - assert_equal(int(self.nodes[0].getbestblockhash(), 16), block.sha256) + test_node.send_and_ping(msg) + assert_equal(int(node.getbestblockhash(), 16), block.sha256) # Create a chain of transactions from given utxo, and add to a new block. - def build_block_with_transactions(self, utxo, num_transactions): - block = self.build_block_on_tip() + def build_block_with_transactions(self, node, utxo, num_transactions): + block = self.build_block_on_tip(node) for i in range(num_transactions): tx = CTransaction() @@ -361,118 +439,113 @@ class CompactBlocksTest(BitcoinTestFramework): # Test that we only receive getblocktxn requests for transactions that the # node needs, and that responding to them causes the block to be # reconstructed. - def test_getblocktxn_requests(self): - print("Testing getblocktxn requests...") + def test_getblocktxn_requests(self, node, test_node, version): + with_witness = (version==2) + + def test_getblocktxn_response(compact_block, peer, expected_result): + msg = msg_cmpctblock(compact_block.to_p2p()) + peer.send_and_ping(msg) + with mininode_lock: + assert(peer.last_getblocktxn is not None) + absolute_indexes = peer.last_getblocktxn.block_txn_request.to_absolute() + assert_equal(absolute_indexes, expected_result) + + def test_tip_after_message(node, peer, msg, tip): + peer.send_and_ping(msg) + assert_equal(int(node.getbestblockhash(), 16), tip) # First try announcing compactblocks that won't reconstruct, and verify # that we receive getblocktxn messages back. utxo = self.utxos.pop(0) - block = self.build_block_with_transactions(utxo, 5) + block = self.build_block_with_transactions(node, utxo, 5) self.utxos.append([block.vtx[-1].sha256, 0, block.vtx[-1].vout[0].nValue]) - comp_block = HeaderAndShortIDs() - comp_block.initialize_from_block(block) + comp_block.initialize_from_block(block, use_witness=with_witness) - self.test_node.send_and_ping(msg_cmpctblock(comp_block.to_p2p())) - with mininode_lock: - assert(self.test_node.last_getblocktxn is not None) - absolute_indexes = self.test_node.last_getblocktxn.block_txn_request.to_absolute() - assert_equal(absolute_indexes, [1, 2, 3, 4, 5]) - msg = msg_blocktxn() - msg.block_transactions = BlockTransactions(block.sha256, block.vtx[1:]) - self.test_node.send_and_ping(msg) - assert_equal(int(self.nodes[0].getbestblockhash(), 16), block.sha256) + test_getblocktxn_response(comp_block, test_node, [1, 2, 3, 4, 5]) + + msg_bt = msg_blocktxn() + if with_witness: + msg_bt = msg_witness_blocktxn() # serialize with witnesses + msg_bt.block_transactions = BlockTransactions(block.sha256, block.vtx[1:]) + test_tip_after_message(node, test_node, msg_bt, block.sha256) utxo = self.utxos.pop(0) - block = self.build_block_with_transactions(utxo, 5) + block = self.build_block_with_transactions(node, utxo, 5) self.utxos.append([block.vtx[-1].sha256, 0, block.vtx[-1].vout[0].nValue]) # Now try interspersing the prefilled transactions - comp_block.initialize_from_block(block, prefill_list=[0, 1, 5]) - self.test_node.send_and_ping(msg_cmpctblock(comp_block.to_p2p())) - with mininode_lock: - assert(self.test_node.last_getblocktxn is not None) - absolute_indexes = self.test_node.last_getblocktxn.block_txn_request.to_absolute() - assert_equal(absolute_indexes, [2, 3, 4]) - msg.block_transactions = BlockTransactions(block.sha256, block.vtx[2:5]) - self.test_node.send_and_ping(msg) - assert_equal(int(self.nodes[0].getbestblockhash(), 16), block.sha256) + comp_block.initialize_from_block(block, prefill_list=[0, 1, 5], use_witness=with_witness) + test_getblocktxn_response(comp_block, test_node, [2, 3, 4]) + msg_bt.block_transactions = BlockTransactions(block.sha256, block.vtx[2:5]) + test_tip_after_message(node, test_node, msg_bt, block.sha256) # Now try giving one transaction ahead of time. utxo = self.utxos.pop(0) - block = self.build_block_with_transactions(utxo, 5) + block = self.build_block_with_transactions(node, utxo, 5) self.utxos.append([block.vtx[-1].sha256, 0, block.vtx[-1].vout[0].nValue]) - self.test_node.send_and_ping(msg_tx(block.vtx[1])) - assert(block.vtx[1].hash in self.nodes[0].getrawmempool()) + test_node.send_and_ping(msg_tx(block.vtx[1])) + assert(block.vtx[1].hash in node.getrawmempool()) # Prefill 4 out of the 6 transactions, and verify that only the one # that was not in the mempool is requested. - comp_block.initialize_from_block(block, prefill_list=[0, 2, 3, 4]) - self.test_node.send_and_ping(msg_cmpctblock(comp_block.to_p2p())) - with mininode_lock: - assert(self.test_node.last_getblocktxn is not None) - absolute_indexes = self.test_node.last_getblocktxn.block_txn_request.to_absolute() - assert_equal(absolute_indexes, [5]) + comp_block.initialize_from_block(block, prefill_list=[0, 2, 3, 4], use_witness=with_witness) + test_getblocktxn_response(comp_block, test_node, [5]) - msg.block_transactions = BlockTransactions(block.sha256, [block.vtx[5]]) - self.test_node.send_and_ping(msg) - assert_equal(int(self.nodes[0].getbestblockhash(), 16), block.sha256) + msg_bt.block_transactions = BlockTransactions(block.sha256, [block.vtx[5]]) + test_tip_after_message(node, test_node, msg_bt, block.sha256) # Now provide all transactions to the node before the block is # announced and verify reconstruction happens immediately. utxo = self.utxos.pop(0) - block = self.build_block_with_transactions(utxo, 10) + block = self.build_block_with_transactions(node, utxo, 10) self.utxos.append([block.vtx[-1].sha256, 0, block.vtx[-1].vout[0].nValue]) for tx in block.vtx[1:]: - self.test_node.send_message(msg_tx(tx)) - self.test_node.sync_with_ping() + test_node.send_message(msg_tx(tx)) + test_node.sync_with_ping() # Make sure all transactions were accepted. - mempool = self.nodes[0].getrawmempool() + mempool = node.getrawmempool() for tx in block.vtx[1:]: assert(tx.hash in mempool) # Clear out last request. with mininode_lock: - self.test_node.last_getblocktxn = None + test_node.last_getblocktxn = None # Send compact block - comp_block.initialize_from_block(block, prefill_list=[0]) - self.test_node.send_and_ping(msg_cmpctblock(comp_block.to_p2p())) + comp_block.initialize_from_block(block, prefill_list=[0], use_witness=with_witness) + test_tip_after_message(node, test_node, msg_cmpctblock(comp_block.to_p2p()), block.sha256) with mininode_lock: # Shouldn't have gotten a request for any transaction - assert(self.test_node.last_getblocktxn is None) - # Tip should have updated - assert_equal(int(self.nodes[0].getbestblockhash(), 16), block.sha256) + assert(test_node.last_getblocktxn is None) # Incorrectly responding to a getblocktxn shouldn't cause the block to be # permanently failed. - def test_incorrect_blocktxn_response(self): - print("Testing handling of incorrect blocktxn responses...") - + def test_incorrect_blocktxn_response(self, node, test_node, version): if (len(self.utxos) == 0): self.make_utxos() utxo = self.utxos.pop(0) - block = self.build_block_with_transactions(utxo, 10) + block = self.build_block_with_transactions(node, utxo, 10) self.utxos.append([block.vtx[-1].sha256, 0, block.vtx[-1].vout[0].nValue]) # Relay the first 5 transactions from the block in advance for tx in block.vtx[1:6]: - self.test_node.send_message(msg_tx(tx)) - self.test_node.sync_with_ping() + test_node.send_message(msg_tx(tx)) + test_node.sync_with_ping() # Make sure all transactions were accepted. - mempool = self.nodes[0].getrawmempool() + mempool = node.getrawmempool() for tx in block.vtx[1:6]: assert(tx.hash in mempool) # Send compact block comp_block = HeaderAndShortIDs() - comp_block.initialize_from_block(block, prefill_list=[0]) - self.test_node.send_and_ping(msg_cmpctblock(comp_block.to_p2p())) + comp_block.initialize_from_block(block, prefill_list=[0], use_witness=(version == 2)) + test_node.send_and_ping(msg_cmpctblock(comp_block.to_p2p())) absolute_indexes = [] with mininode_lock: - assert(self.test_node.last_getblocktxn is not None) - absolute_indexes = self.test_node.last_getblocktxn.block_txn_request.to_absolute() + assert(test_node.last_getblocktxn is not None) + absolute_indexes = test_node.last_getblocktxn.block_txn_request.to_absolute() assert_equal(absolute_indexes, [6, 7, 8, 9, 10]) # Now give an incorrect response. @@ -484,100 +557,107 @@ class CompactBlocksTest(BitcoinTestFramework): # verifying that the block isn't marked bad permanently. This is good # enough for now. msg = msg_blocktxn() + if version==2: + msg = msg_witness_blocktxn() msg.block_transactions = BlockTransactions(block.sha256, [block.vtx[5]] + block.vtx[7:]) - self.test_node.send_and_ping(msg) + test_node.send_and_ping(msg) # Tip should not have updated - assert_equal(int(self.nodes[0].getbestblockhash(), 16), block.hashPrevBlock) + assert_equal(int(node.getbestblockhash(), 16), block.hashPrevBlock) # We should receive a getdata request - success = wait_until(lambda: self.test_node.last_getdata is not None, timeout=10) + success = wait_until(lambda: test_node.last_getdata is not None, timeout=10) assert(success) - assert_equal(len(self.test_node.last_getdata.inv), 1) - assert_equal(self.test_node.last_getdata.inv[0].type, 2) - assert_equal(self.test_node.last_getdata.inv[0].hash, block.sha256) + assert_equal(len(test_node.last_getdata.inv), 1) + assert(test_node.last_getdata.inv[0].type == 2 or test_node.last_getdata.inv[0].type == 2|MSG_WITNESS_FLAG) + assert_equal(test_node.last_getdata.inv[0].hash, block.sha256) # Deliver the block - self.test_node.send_and_ping(msg_block(block)) - assert_equal(int(self.nodes[0].getbestblockhash(), 16), block.sha256) - - def test_getblocktxn_handler(self): - print("Testing getblocktxn handler...") + if version==2: + test_node.send_and_ping(msg_witness_block(block)) + else: + test_node.send_and_ping(msg_block(block)) + assert_equal(int(node.getbestblockhash(), 16), block.sha256) + def test_getblocktxn_handler(self, node, test_node, version): # bitcoind won't respond for blocks whose height is more than 15 blocks # deep. MAX_GETBLOCKTXN_DEPTH = 15 - chain_height = self.nodes[0].getblockcount() + chain_height = node.getblockcount() current_height = chain_height while (current_height >= chain_height - MAX_GETBLOCKTXN_DEPTH): - block_hash = self.nodes[0].getblockhash(current_height) - block = FromHex(CBlock(), self.nodes[0].getblock(block_hash, False)) + block_hash = node.getblockhash(current_height) + block = FromHex(CBlock(), node.getblock(block_hash, False)) msg = msg_getblocktxn() msg.block_txn_request = BlockTransactionsRequest(int(block_hash, 16), []) num_to_request = random.randint(1, len(block.vtx)) msg.block_txn_request.from_absolute(sorted(random.sample(range(len(block.vtx)), num_to_request))) - self.test_node.send_message(msg) - success = wait_until(lambda: self.test_node.last_blocktxn is not None, timeout=10) + test_node.send_message(msg) + success = wait_until(lambda: test_node.last_blocktxn is not None, timeout=10) assert(success) [tx.calc_sha256() for tx in block.vtx] with mininode_lock: - assert_equal(self.test_node.last_blocktxn.block_transactions.blockhash, int(block_hash, 16)) + assert_equal(test_node.last_blocktxn.block_transactions.blockhash, int(block_hash, 16)) all_indices = msg.block_txn_request.to_absolute() for index in all_indices: - tx = self.test_node.last_blocktxn.block_transactions.transactions.pop(0) + tx = test_node.last_blocktxn.block_transactions.transactions.pop(0) tx.calc_sha256() assert_equal(tx.sha256, block.vtx[index].sha256) - self.test_node.last_blocktxn = None + if version == 1: + # Witnesses should have been stripped + assert(tx.wit.is_null()) + else: + # Check that the witness matches + assert_equal(tx.calc_sha256(True), block.vtx[index].calc_sha256(True)) + test_node.last_blocktxn = None current_height -= 1 # Next request should be ignored, as we're past the allowed depth. - block_hash = self.nodes[0].getblockhash(current_height) + block_hash = node.getblockhash(current_height) msg.block_txn_request = BlockTransactionsRequest(int(block_hash, 16), [0]) - self.test_node.send_and_ping(msg) + test_node.send_and_ping(msg) with mininode_lock: - assert_equal(self.test_node.last_blocktxn, None) - - def test_compactblocks_not_at_tip(self): - print("Testing compactblock requests/announcements not at chain tip...") + assert_equal(test_node.last_blocktxn, None) + def test_compactblocks_not_at_tip(self, node, test_node): # Test that requesting old compactblocks doesn't work. MAX_CMPCTBLOCK_DEPTH = 11 new_blocks = [] for i in range(MAX_CMPCTBLOCK_DEPTH): - self.test_node.clear_block_announcement() - new_blocks.append(self.nodes[0].generate(1)[0]) - wait_until(self.test_node.received_block_announcement, timeout=30) + test_node.clear_block_announcement() + new_blocks.append(node.generate(1)[0]) + wait_until(test_node.received_block_announcement, timeout=30) - self.test_node.clear_block_announcement() - self.test_node.send_message(msg_getdata([CInv(4, int(new_blocks[0], 16))])) - success = wait_until(lambda: self.test_node.last_cmpctblock is not None, timeout=30) + test_node.clear_block_announcement() + test_node.send_message(msg_getdata([CInv(4, int(new_blocks[0], 16))])) + success = wait_until(lambda: test_node.last_cmpctblock is not None, timeout=30) assert(success) - self.test_node.clear_block_announcement() - self.nodes[0].generate(1) - wait_until(self.test_node.received_block_announcement, timeout=30) - self.test_node.clear_block_announcement() - self.test_node.send_message(msg_getdata([CInv(4, int(new_blocks[0], 16))])) - success = wait_until(lambda: self.test_node.last_block is not None, timeout=30) + test_node.clear_block_announcement() + node.generate(1) + wait_until(test_node.received_block_announcement, timeout=30) + test_node.clear_block_announcement() + test_node.send_message(msg_getdata([CInv(4, int(new_blocks[0], 16))])) + success = wait_until(lambda: test_node.last_block is not None, timeout=30) assert(success) with mininode_lock: - self.test_node.last_block.block.calc_sha256() - assert_equal(self.test_node.last_block.block.sha256, int(new_blocks[0], 16)) + test_node.last_block.block.calc_sha256() + assert_equal(test_node.last_block.block.sha256, int(new_blocks[0], 16)) # Generate an old compactblock, and verify that it's not accepted. - cur_height = self.nodes[0].getblockcount() - hashPrevBlock = int(self.nodes[0].getblockhash(cur_height-5), 16) - block = self.build_block_on_tip() + cur_height = node.getblockcount() + hashPrevBlock = int(node.getblockhash(cur_height-5), 16) + block = self.build_block_on_tip(node) block.hashPrevBlock = hashPrevBlock block.solve() comp_block = HeaderAndShortIDs() comp_block.initialize_from_block(block) - self.test_node.send_and_ping(msg_cmpctblock(comp_block.to_p2p())) + test_node.send_and_ping(msg_cmpctblock(comp_block.to_p2p())) - tips = self.nodes[0].getchaintips() + tips = node.getchaintips() found = False for x in tips: if x["hash"] == block.hash: @@ -591,18 +671,61 @@ class CompactBlocksTest(BitcoinTestFramework): msg = msg_getblocktxn() msg.block_txn_request = BlockTransactionsRequest(block.sha256, [0]) with mininode_lock: - self.test_node.last_blocktxn = None - self.test_node.send_and_ping(msg) + test_node.last_blocktxn = None + test_node.send_and_ping(msg) with mininode_lock: - assert(self.test_node.last_blocktxn is None) + assert(test_node.last_blocktxn is None) + + def activate_segwit(self, node): + node.generate(144*3) + assert_equal(get_bip9_status(node, "segwit")["status"], 'active') + + def test_end_to_end_block_relay(self, node, listeners): + utxo = self.utxos.pop(0) + + block = self.build_block_with_transactions(node, utxo, 10) + + [l.clear_block_announcement() for l in listeners] + + # ToHex() won't serialize with witness, but this block has no witnesses + # anyway. TODO: repeat this test with witness tx's to a segwit node. + node.submitblock(ToHex(block)) + + for l in listeners: + wait_until(lambda: l.received_block_announcement(), timeout=30) + with mininode_lock: + for l in listeners: + assert(l.last_cmpctblock is not None) + l.last_cmpctblock.header_and_shortids.header.calc_sha256() + assert_equal(l.last_cmpctblock.header_and_shortids.header.sha256, block.sha256) + + # Helper for enabling cb announcements + # Send the sendcmpct request and sync headers + def request_cb_announcements(self, peer, node, version): + tip = node.getbestblockhash() + peer.get_headers(locator=[int(tip, 16)], hashstop=0) + + msg = msg_sendcmpct() + msg.version = version + msg.announce = True + peer.send_and_ping(msg) + def run_test(self): # Setup the p2p connections and start up the network thread. self.test_node = TestNode() + self.segwit_node = TestNode() + self.old_node = TestNode() # version 1 peer <--> segwit node connections = [] connections.append(NodeConn('127.0.0.1', p2p_port(0), self.nodes[0], self.test_node)) + connections.append(NodeConn('127.0.0.1', p2p_port(1), self.nodes[1], + self.segwit_node, services=NODE_NETWORK|NODE_WITNESS)) + connections.append(NodeConn('127.0.0.1', p2p_port(1), self.nodes[1], + self.old_node, services=NODE_NETWORK)) self.test_node.add_connection(connections[0]) + self.segwit_node.add_connection(connections[1]) + self.old_node.add_connection(connections[2]) NetworkThread().start() # Start up network handling in another thread @@ -612,13 +735,107 @@ class CompactBlocksTest(BitcoinTestFramework): # We will need UTXOs to construct transactions in later tests. self.make_utxos() - self.test_sendcmpct() - self.test_compactblock_construction() - self.test_compactblock_requests() - self.test_getblocktxn_requests() - self.test_getblocktxn_handler() - self.test_compactblocks_not_at_tip() - self.test_incorrect_blocktxn_response() + print("Running tests, pre-segwit activation:") + + print("\tTesting SENDCMPCT p2p message... ") + self.test_sendcmpct(self.nodes[0], self.test_node, 1) + sync_blocks(self.nodes) + self.test_sendcmpct(self.nodes[1], self.segwit_node, 2, old_node=self.old_node) + sync_blocks(self.nodes) + + print("\tTesting compactblock construction...") + self.test_compactblock_construction(self.nodes[0], self.test_node, 1, False) + sync_blocks(self.nodes) + self.test_compactblock_construction(self.nodes[1], self.segwit_node, 2, False) + sync_blocks(self.nodes) + + print("\tTesting compactblock requests... ") + self.test_compactblock_requests(self.nodes[0], self.test_node) + sync_blocks(self.nodes) + self.test_compactblock_requests(self.nodes[1], self.segwit_node) + sync_blocks(self.nodes) + + print("\tTesting getblocktxn requests...") + self.test_getblocktxn_requests(self.nodes[0], self.test_node, 1) + sync_blocks(self.nodes) + self.test_getblocktxn_requests(self.nodes[1], self.segwit_node, 2) + sync_blocks(self.nodes) + + print("\tTesting getblocktxn handler...") + self.test_getblocktxn_handler(self.nodes[0], self.test_node, 1) + sync_blocks(self.nodes) + self.test_getblocktxn_handler(self.nodes[1], self.segwit_node, 2) + self.test_getblocktxn_handler(self.nodes[1], self.old_node, 1) + sync_blocks(self.nodes) + + print("\tTesting compactblock requests/announcements not at chain tip...") + self.test_compactblocks_not_at_tip(self.nodes[0], self.test_node) + sync_blocks(self.nodes) + self.test_compactblocks_not_at_tip(self.nodes[1], self.segwit_node) + self.test_compactblocks_not_at_tip(self.nodes[1], self.old_node) + sync_blocks(self.nodes) + + print("\tTesting handling of incorrect blocktxn responses...") + self.test_incorrect_blocktxn_response(self.nodes[0], self.test_node, 1) + sync_blocks(self.nodes) + self.test_incorrect_blocktxn_response(self.nodes[1], self.segwit_node, 2) + sync_blocks(self.nodes) + + # End-to-end block relay tests + print("\tTesting end-to-end block relay...") + self.request_cb_announcements(self.test_node, self.nodes[0], 1) + self.request_cb_announcements(self.old_node, self.nodes[1], 1) + self.request_cb_announcements(self.segwit_node, self.nodes[1], 2) + self.test_end_to_end_block_relay(self.nodes[0], [self.segwit_node, self.test_node, self.old_node]) + self.test_end_to_end_block_relay(self.nodes[1], [self.segwit_node, self.test_node, self.old_node]) + + # Advance to segwit activation + print ("\nAdvancing to segwit activation\n") + self.activate_segwit(self.nodes[1]) + print ("Running tests, post-segwit activation...") + + print("\tTesting compactblock construction...") + self.test_compactblock_construction(self.nodes[1], self.old_node, 1, True) + self.test_compactblock_construction(self.nodes[1], self.segwit_node, 2, True) + sync_blocks(self.nodes) + + print("\tTesting compactblock requests (unupgraded node)... ") + self.test_compactblock_requests(self.nodes[0], self.test_node) + + print("\tTesting getblocktxn requests (unupgraded node)...") + self.test_getblocktxn_requests(self.nodes[0], self.test_node, 1) + + # Need to manually sync node0 and node1, because post-segwit activation, + # node1 will not download blocks from node0. + print("\tSyncing nodes...") + assert(self.nodes[0].getbestblockhash() != self.nodes[1].getbestblockhash()) + while (self.nodes[0].getblockcount() > self.nodes[1].getblockcount()): + block_hash = self.nodes[0].getblockhash(self.nodes[1].getblockcount()+1) + self.nodes[1].submitblock(self.nodes[0].getblock(block_hash, False)) + assert_equal(self.nodes[0].getbestblockhash(), self.nodes[1].getbestblockhash()) + + print("\tTesting compactblock requests (segwit node)... ") + self.test_compactblock_requests(self.nodes[1], self.segwit_node) + + print("\tTesting getblocktxn requests (segwit node)...") + self.test_getblocktxn_requests(self.nodes[1], self.segwit_node, 2) + sync_blocks(self.nodes) + + print("\tTesting getblocktxn handler (segwit node should return witnesses)...") + self.test_getblocktxn_handler(self.nodes[1], self.segwit_node, 2) + self.test_getblocktxn_handler(self.nodes[1], self.old_node, 1) + + # Test that if we submitblock to node1, we'll get a compact block + # announcement to all peers. + # (Post-segwit activation, blocks won't propagate from node0 to node1 + # automatically, so don't bother testing a block announced to node0.) + print("\tTesting end-to-end block relay...") + self.request_cb_announcements(self.test_node, self.nodes[0], 1) + self.request_cb_announcements(self.old_node, self.nodes[1], 1) + self.request_cb_announcements(self.segwit_node, self.nodes[1], 2) + self.test_end_to_end_block_relay(self.nodes[1], [self.segwit_node, self.test_node, self.old_node]) + + print("\tTesting invalid index in cmpctblock message...") self.test_invalid_cmpctblock_message() diff --git a/qa/rpc-tests/p2p-segwit.py b/qa/rpc-tests/p2p-segwit.py index 5c1eb21b1f..c2ea20bb84 100755 --- a/qa/rpc-tests/p2p-segwit.py +++ b/qa/rpc-tests/p2p-segwit.py @@ -912,14 +912,6 @@ class SegWitTest(BitcoinTestFramework): # But eliminating the witness should fix it self.test_node.test_transaction_acceptance(tx, with_witness=False, accepted=True) - # Verify that inv's to test_node come with getdata's for non-witness tx's - # Just tweak the transaction, announce it, and verify we get a getdata - # for a normal tx - tx.vout[0].scriptPubKey = CScript([OP_TRUE, OP_TRUE]) - tx.rehash() - self.test_node.announce_tx_and_wait_for_getdata(tx) - assert(self.test_node.last_getdata.inv[0].type == 1) - # Cleanup: mine the first transaction and update utxo self.nodes[0].generate(1) assert_equal(len(self.nodes[0].getrawmempool()), 0) @@ -1025,7 +1017,7 @@ class SegWitTest(BitcoinTestFramework): def test_block_relay(self, segwit_activated): print("\tTesting block relay") - blocktype = 2|MSG_WITNESS_FLAG if segwit_activated else 2 + blocktype = 2|MSG_WITNESS_FLAG # test_node has set NODE_WITNESS, so all getdata requests should be for # witness blocks. diff --git a/qa/rpc-tests/sendheaders.py b/qa/rpc-tests/sendheaders.py index c3f3180b6b..81b2442e6a 100755 --- a/qa/rpc-tests/sendheaders.py +++ b/qa/rpc-tests/sendheaders.py @@ -80,20 +80,19 @@ e. Announce one more that doesn't connect. Expect: disconnect. ''' -class BaseNode(NodeConnCB): +direct_fetch_response_time = 0.05 + +class BaseNode(SingleNodeConnCB): def __init__(self): - NodeConnCB.__init__(self) - self.connection = None + SingleNodeConnCB.__init__(self) self.last_inv = None self.last_headers = None self.last_block = None - self.ping_counter = 1 - self.last_pong = msg_pong(0) self.last_getdata = None - self.sleep_time = 0.05 self.block_announced = False self.last_getheaders = None self.disconnected = False + self.last_blockhash_announced = None def clear_last_announcement(self): with mininode_lock: @@ -101,9 +100,6 @@ class BaseNode(NodeConnCB): self.last_inv = None self.last_headers = None - def add_connection(self, conn): - self.connection = conn - # Request data for a list of block hashes def get_data(self, block_hashes): msg = msg_getdata() @@ -122,17 +118,17 @@ class BaseNode(NodeConnCB): msg.inv = [CInv(2, blockhash)] self.connection.send_message(msg) - # Wrapper for the NodeConn's send_message function - def send_message(self, message): - self.connection.send_message(message) - def on_inv(self, conn, message): self.last_inv = message self.block_announced = True + self.last_blockhash_announced = message.inv[-1].hash def on_headers(self, conn, message): self.last_headers = message - self.block_announced = True + if len(message.headers): + self.block_announced = True + message.headers[-1].calc_sha256() + self.last_blockhash_announced = message.headers[-1].sha256 def on_block(self, conn, message): self.last_block = message.block @@ -141,9 +137,6 @@ class BaseNode(NodeConnCB): def on_getdata(self, conn, message): self.last_getdata = message - def on_pong(self, conn, message): - self.last_pong = message - def on_getheaders(self, conn, message): self.last_getheaders = message @@ -157,7 +150,7 @@ class BaseNode(NodeConnCB): expect_headers = headers if headers != None else [] expect_inv = inv if inv != None else [] test_function = lambda: self.block_announced - self.sync(test_function) + assert(wait_until(test_function, timeout=60)) with mininode_lock: self.block_announced = False @@ -180,30 +173,14 @@ class BaseNode(NodeConnCB): return success # Syncing helpers - def sync(self, test_function, timeout=60): - while timeout > 0: - with mininode_lock: - if test_function(): - return - time.sleep(self.sleep_time) - timeout -= self.sleep_time - raise AssertionError("Sync failed to complete") - - def sync_with_ping(self, timeout=60): - self.send_message(msg_ping(nonce=self.ping_counter)) - test_function = lambda: self.last_pong.nonce == self.ping_counter - self.sync(test_function, timeout) - self.ping_counter += 1 - return - def wait_for_block(self, blockhash, timeout=60): test_function = lambda: self.last_block != None and self.last_block.sha256 == blockhash - self.sync(test_function, timeout) + assert(wait_until(test_function, timeout=timeout)) return def wait_for_getheaders(self, timeout=60): test_function = lambda: self.last_getheaders != None - self.sync(test_function, timeout) + assert(wait_until(test_function, timeout=timeout)) return def wait_for_getdata(self, hash_list, timeout=60): @@ -211,12 +188,17 @@ class BaseNode(NodeConnCB): return test_function = lambda: self.last_getdata != None and [x.hash for x in self.last_getdata.inv] == hash_list - self.sync(test_function, timeout) + assert(wait_until(test_function, timeout=timeout)) return def wait_for_disconnect(self, timeout=60): test_function = lambda: self.disconnected - self.sync(test_function, timeout) + assert(wait_until(test_function, timeout=timeout)) + return + + def wait_for_block_announcement(self, block_hash, timeout=60): + test_function = lambda: self.last_blockhash_announced == block_hash + assert(wait_until(test_function, timeout=timeout)) return def send_header_for_blocks(self, new_blocks): @@ -266,7 +248,9 @@ class SendHeadersTest(BitcoinTestFramework): def mine_reorg(self, length): self.nodes[0].generate(length) # make sure all invalidated blocks are node0's sync_blocks(self.nodes, wait=0.1) - [x.clear_last_announcement() for x in self.p2p_connections] + for x in self.p2p_connections: + x.wait_for_block_announcement(int(self.nodes[0].getbestblockhash(), 16)) + x.clear_last_announcement() tip_height = self.nodes[1].getblockcount() hash_to_invalidate = self.nodes[1].getblockhash(tip_height-(length-1)) @@ -495,7 +479,7 @@ class SendHeadersTest(BitcoinTestFramework): test_node.send_header_for_blocks(blocks) test_node.sync_with_ping() - test_node.wait_for_getdata([x.sha256 for x in blocks], timeout=test_node.sleep_time) + test_node.wait_for_getdata([x.sha256 for x in blocks], timeout=direct_fetch_response_time) [ test_node.send_message(msg_block(x)) for x in blocks ] @@ -526,13 +510,13 @@ class SendHeadersTest(BitcoinTestFramework): # both blocks (same work as tip) test_node.send_header_for_blocks(blocks[1:2]) test_node.sync_with_ping() - test_node.wait_for_getdata([x.sha256 for x in blocks[0:2]], timeout=test_node.sleep_time) + test_node.wait_for_getdata([x.sha256 for x in blocks[0:2]], timeout=direct_fetch_response_time) # Announcing 16 more headers should trigger direct fetch for 14 more # blocks test_node.send_header_for_blocks(blocks[2:18]) test_node.sync_with_ping() - test_node.wait_for_getdata([x.sha256 for x in blocks[2:16]], timeout=test_node.sleep_time) + test_node.wait_for_getdata([x.sha256 for x in blocks[2:16]], timeout=direct_fetch_response_time) # Announcing 1 more header should not trigger any response test_node.last_getdata = None diff --git a/qa/rpc-tests/test_framework/mininode.py b/qa/rpc-tests/test_framework/mininode.py index 88a3b7e0f7..4d238c08d9 100755 --- a/qa/rpc-tests/test_framework/mininode.py +++ b/qa/rpc-tests/test_framework/mininode.py @@ -452,7 +452,7 @@ class CTransaction(object): else: self.vout = deser_vector(f, CTxOut) if flags != 0: - self.wit.vtxinwit = [CTxInWitness()]*len(self.vin) + self.wit.vtxinwit = [CTxInWitness() for i in range(len(self.vin))] self.wit.deserialize(f) self.nLockTime = struct.unpack("<I", f.read(4))[0] self.sha256 = None @@ -518,8 +518,8 @@ class CTransaction(object): return True def __repr__(self): - return "CTransaction(nVersion=%i vin=%s vout=%s nLockTime=%i)" \ - % (self.nVersion, repr(self.vin), repr(self.vout), self.nLockTime) + return "CTransaction(nVersion=%i vin=%s vout=%s wit=%s nLockTime=%i)" \ + % (self.nVersion, repr(self.vin), repr(self.vout), repr(self.wit), self.nLockTime) class CBlockHeader(object): @@ -755,6 +755,9 @@ class PrefilledTransaction(object): r += self.tx.serialize_without_witness() return r + def serialize_with_witness(self): + return self.serialize(with_witness=True) + def __repr__(self): return "PrefilledTransaction(index=%d, tx=%s)" % (self.index, repr(self.tx)) @@ -779,6 +782,7 @@ class P2PHeaderAndShortIDs(object): self.prefilled_txn = deser_vector(f, PrefilledTransaction) self.prefilled_txn_length = len(self.prefilled_txn) + # When using version 2 compact blocks, we must serialize with_witness. def serialize(self, with_witness=False): r = b"" r += self.header.serialize() @@ -787,12 +791,20 @@ class P2PHeaderAndShortIDs(object): for x in self.shortids: # We only want the first 6 bytes r += struct.pack("<Q", x)[0:6] - r += ser_vector(self.prefilled_txn) + if with_witness: + r += ser_vector(self.prefilled_txn, "serialize_with_witness") + else: + r += ser_vector(self.prefilled_txn) return r def __repr__(self): return "P2PHeaderAndShortIDs(header=%s, nonce=%d, shortids_length=%d, shortids=%s, prefilled_txn_length=%d, prefilledtxn=%s" % (repr(self.header), self.nonce, self.shortids_length, repr(self.shortids), self.prefilled_txn_length, repr(self.prefilled_txn)) +# P2P version of the above that will use witness serialization (for compact +# block version 2) +class P2PHeaderAndShortWitnessIDs(P2PHeaderAndShortIDs): + def serialize(self): + return super(P2PHeaderAndShortWitnessIDs, self).serialize(with_witness=True) # Calculate the BIP 152-compact blocks shortid for a given transaction hash def calculate_shortid(k0, k1, tx_hash): @@ -808,6 +820,7 @@ class HeaderAndShortIDs(object): self.nonce = 0 self.shortids = [] self.prefilled_txn = [] + self.use_witness = False if p2pheaders_and_shortids != None: self.header = p2pheaders_and_shortids.header @@ -819,7 +832,10 @@ class HeaderAndShortIDs(object): last_index = self.prefilled_txn[-1].index def to_p2p(self): - ret = P2PHeaderAndShortIDs() + if self.use_witness: + ret = P2PHeaderAndShortWitnessIDs() + else: + ret = P2PHeaderAndShortIDs() ret.header = self.header ret.nonce = self.nonce ret.shortids_length = len(self.shortids) @@ -840,15 +856,20 @@ class HeaderAndShortIDs(object): key1 = struct.unpack("<Q", hash_header_nonce_as_str[8:16])[0] return [ key0, key1 ] - def initialize_from_block(self, block, nonce=0, prefill_list = [0]): + # Version 2 compact blocks use wtxid in shortids (rather than txid) + def initialize_from_block(self, block, nonce=0, prefill_list = [0], use_witness = False): self.header = CBlockHeader(block) self.nonce = nonce self.prefilled_txn = [ PrefilledTransaction(i, block.vtx[i]) for i in prefill_list ] self.shortids = [] + self.use_witness = use_witness [k0, k1] = self.get_siphash_keys() for i in range(len(block.vtx)): if i not in prefill_list: - self.shortids.append(calculate_shortid(k0, k1, block.vtx[i].sha256)) + tx_hash = block.vtx[i].sha256 + if use_witness: + tx_hash = block.vtx[i].calc_sha256(with_witness=True) + self.shortids.append(calculate_shortid(k0, k1, tx_hash)) def __repr__(self): return "HeaderAndShortIDs(header=%s, nonce=%d, shortids=%s, prefilledtxn=%s" % (repr(self.header), self.nonce, repr(self.shortids), repr(self.prefilled_txn)) @@ -1424,6 +1445,12 @@ class msg_blocktxn(object): def __repr__(self): return "msg_blocktxn(block_transactions=%s)" % (repr(self.block_transactions)) +class msg_witness_blocktxn(msg_blocktxn): + def serialize(self): + r = b"" + r += self.block_transactions.serialize(with_witness=True) + return r + # This is what a callback should look like for NodeConn # Reimplement the on_* functions to provide handling for events class NodeConnCB(object): diff --git a/src/Makefile.qt.include b/src/Makefile.qt.include index a2ed4f47d8..1f9a901d75 100644 --- a/src/Makefile.qt.include +++ b/src/Makefile.qt.include @@ -289,7 +289,7 @@ RES_ICONS = \ qt/res/icons/warning.png \ qt/res/icons/verify.png -BITCOIN_QT_CPP = \ +BITCOIN_QT_BASE_CPP = \ qt/bantablemodel.cpp \ qt/bitcoinaddressvalidator.cpp \ qt/bitcoinamountfield.cpp \ @@ -313,12 +313,9 @@ BITCOIN_QT_CPP = \ qt/trafficgraphwidget.cpp \ qt/utilitydialog.cpp -if TARGET_WINDOWS -BITCOIN_QT_CPP += qt/winshutdownmonitor.cpp -endif +BITCOIN_QT_WINDOWS_CPP = qt/winshutdownmonitor.cpp -if ENABLE_WALLET -BITCOIN_QT_CPP += \ +BITCOIN_QT_WALLET_CPP = \ qt/addressbookpage.cpp \ qt/addresstablemodel.cpp \ qt/askpassphrasedialog.cpp \ @@ -345,6 +342,13 @@ BITCOIN_QT_CPP += \ qt/walletmodel.cpp \ qt/walletmodeltransaction.cpp \ qt/walletview.cpp + +BITCOIN_QT_CPP = $(BITCOIN_QT_BASE_CPP) +if TARGET_WINDOWS +BITCOIN_QT_CPP += $(BITCOIN_QT_WINDOWS_CPP) +endif +if ENABLE_WALLET +BITCOIN_QT_CPP += $(BITCOIN_QT_WALLET_CPP) endif RES_IMAGES = @@ -413,7 +417,7 @@ $(srcdir)/qt/bitcoinstrings.cpp: $(libbitcoin_server_a_SOURCES) $(libbitcoin_wal @test -n $(XGETTEXT) || echo "xgettext is required for updating translations" $(AM_V_GEN) cd $(srcdir); XGETTEXT=$(XGETTEXT) PACKAGE_NAME="$(PACKAGE_NAME)" COPYRIGHT_HOLDERS="$(COPYRIGHT_HOLDERS)" COPYRIGHT_HOLDERS_SUBSTITUTION="$(COPYRIGHT_HOLDERS_SUBSTITUTION)" $(PYTHON) ../share/qt/extract_strings_qt.py $^ -translate: $(srcdir)/qt/bitcoinstrings.cpp $(QT_FORMS_UI) $(QT_FORMS_UI) $(BITCOIN_QT_CPP) $(BITCOIN_QT_H) $(BITCOIN_MM) +translate: $(srcdir)/qt/bitcoinstrings.cpp $(QT_FORMS_UI) $(QT_FORMS_UI) $(BITCOIN_QT_BASE_CPP) $(BITCOIN_QT_WINDOWS_CPP) $(BITCOIN_QT_WALLET_CPP) $(BITCOIN_QT_H) $(BITCOIN_MM) @test -n $(LUPDATE) || echo "lupdate is required for updating translations" $(AM_V_GEN) QT_SELECT=$(QT_SELECT) $(LUPDATE) $^ -locations relative -no-obsolete -ts $(srcdir)/qt/locale/bitcoin_en.ts diff --git a/src/blockencodings.cpp b/src/blockencodings.cpp index df237f8f26..93d3fa372b 100644 --- a/src/blockencodings.cpp +++ b/src/blockencodings.cpp @@ -17,7 +17,7 @@ #define MIN_TRANSACTION_BASE_SIZE (::GetSerializeSize(CTransaction(), SER_NETWORK, PROTOCOL_VERSION | SERIALIZE_TRANSACTION_NO_WITNESS)) -CBlockHeaderAndShortTxIDs::CBlockHeaderAndShortTxIDs(const CBlock& block) : +CBlockHeaderAndShortTxIDs::CBlockHeaderAndShortTxIDs(const CBlock& block, bool fUseWTXID) : nonce(GetRand(std::numeric_limits<uint64_t>::max())), shorttxids(block.vtx.size() - 1), prefilledtxn(1), header(block) { FillShortTxIDSelector(); @@ -25,7 +25,7 @@ CBlockHeaderAndShortTxIDs::CBlockHeaderAndShortTxIDs(const CBlock& block) : prefilledtxn[0] = {0, block.vtx[0]}; for (size_t i = 1; i < block.vtx.size(); i++) { const CTransaction& tx = block.vtx[i]; - shorttxids[i - 1] = GetShortID(tx.GetHash()); + shorttxids[i - 1] = GetShortID(fUseWTXID ? tx.GetWitnessHash() : tx.GetHash()); } } diff --git a/src/blockencodings.h b/src/blockencodings.h index 349fcbd50f..99b1cb140d 100644 --- a/src/blockencodings.h +++ b/src/blockencodings.h @@ -146,7 +146,7 @@ public: // Dummy for deserialization CBlockHeaderAndShortTxIDs() {} - CBlockHeaderAndShortTxIDs(const CBlock& block); + CBlockHeaderAndShortTxIDs(const CBlock& block, bool fUseWTXID); uint64_t GetShortID(const uint256& txhash) const; diff --git a/src/main.cpp b/src/main.cpp index ab67219714..c92a38be98 100644 --- a/src/main.cpp +++ b/src/main.cpp @@ -289,10 +289,21 @@ struct CNodeState { bool fPreferHeaders; //! Whether this peer wants invs or cmpctblocks (when possible) for block announcements. bool fPreferHeaderAndIDs; - //! Whether this peer will send us cmpctblocks if we request them + /** + * Whether this peer will send us cmpctblocks if we request them. + * This is not used to gate request logic, as we really only care about fSupportsDesiredCmpctVersion, + * but is used as a flag to "lock in" the version of compact blocks (fWantsCmpctWitness) we send. + */ bool fProvidesHeaderAndIDs; //! Whether this peer can give us witnesses bool fHaveWitness; + //! Whether this peer wants witnesses in cmpctblocks/blocktxns + bool fWantsCmpctWitness; + /** + * If we've announced NODE_WITNESS to this peer: whether the peer sends witnesses in cmpctblocks/blocktxns, + * otherwise: whether this peer sends non-witnesses in cmpctblocks/blocktxns. + */ + bool fSupportsDesiredCmpctVersion; CNodeState() { fCurrentlyConnected = false; @@ -313,6 +324,8 @@ struct CNodeState { fPreferHeaderAndIDs = false; fProvidesHeaderAndIDs = false; fHaveWitness = false; + fWantsCmpctWitness = false; + fSupportsDesiredCmpctVersion = false; } }; @@ -467,8 +480,8 @@ void UpdateBlockAvailability(NodeId nodeid, const uint256 &hash) { } void MaybeSetPeerAsAnnouncingHeaderAndIDs(const CNodeState* nodestate, CNode* pfrom, CConnman& connman) { - if (pfrom->GetLocalServices() & NODE_WITNESS) { - // Don't ever request compact blocks when segwit is enabled. + if (!nodestate->fSupportsDesiredCmpctVersion) { + // Never ask from peers who can't provide witnesses. return; } if (nodestate->fProvidesHeaderAndIDs) { @@ -476,7 +489,7 @@ void MaybeSetPeerAsAnnouncingHeaderAndIDs(const CNodeState* nodestate, CNode* pf if (nodeid == pfrom->GetId()) return; bool fAnnounceUsingCMPCTBLOCK = false; - uint64_t nCMPCTBLOCKVersion = 1; + uint64_t nCMPCTBLOCKVersion = (pfrom->GetLocalServices() & NODE_WITNESS) ? 2 : 1; if (lNodesAnnouncingHeaderAndIDs.size() >= 3) { // As per BIP152, we only get 3 of our peers to announce // blocks using compact encodings. @@ -4856,11 +4869,12 @@ void static ProcessGetData(CNode* pfrom, const Consensus::Params& consensusParam // they wont have a useful mempool to match against a compact block, // and we don't feel like constructing the object for them, so // instead we respond with the full, non-compact block. + bool fPeerWantsWitness = State(pfrom->GetId())->fWantsCmpctWitness; if (mi->second->nHeight >= chainActive.Height() - 10) { - CBlockHeaderAndShortTxIDs cmpctblock(block); - pfrom->PushMessageWithFlag(SERIALIZE_TRANSACTION_NO_WITNESS, NetMsgType::CMPCTBLOCK, cmpctblock); + CBlockHeaderAndShortTxIDs cmpctblock(block, fPeerWantsWitness); + pfrom->PushMessageWithFlag(fPeerWantsWitness ? 0 : SERIALIZE_TRANSACTION_NO_WITNESS, NetMsgType::CMPCTBLOCK, cmpctblock); } else - pfrom->PushMessageWithFlag(SERIALIZE_TRANSACTION_NO_WITNESS, NetMsgType::BLOCK, block); + pfrom->PushMessageWithFlag(fPeerWantsWitness ? 0 : SERIALIZE_TRANSACTION_NO_WITNESS, NetMsgType::BLOCK, block); } // Trigger the peer node to send a getblocks request for the next batch of inventory @@ -4922,7 +4936,7 @@ void static ProcessGetData(CNode* pfrom, const Consensus::Params& consensusParam uint32_t GetFetchFlags(CNode* pfrom, CBlockIndex* pprev, const Consensus::Params& chainparams) { uint32_t nFetchFlags = 0; - if (IsWitnessEnabled(pprev, chainparams) && State(pfrom->GetId())->fHaveWitness) { + if ((pfrom->GetLocalServices() & NODE_WITNESS) && State(pfrom->GetId())->fHaveWitness) { nFetchFlags |= MSG_WITNESS_FLAG; } return nFetchFlags; @@ -5128,13 +5142,16 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, pfrom->PushMessage(NetMsgType::SENDHEADERS); } if (pfrom->nVersion >= SHORT_IDS_BLOCKS_VERSION) { - // Tell our peer we are willing to provide version-1 cmpctblocks + // Tell our peer we are willing to provide version 1 or 2 cmpctblocks // However, we do not request new block announcements using // cmpctblock messages. // We send this to non-NODE NETWORK peers as well, because // they may wish to request compact blocks from us bool fAnnounceUsingCMPCTBLOCK = false; - uint64_t nCMPCTBLOCKVersion = 1; + uint64_t nCMPCTBLOCKVersion = 2; + if (pfrom->GetLocalServices() & NODE_WITNESS) + pfrom->PushMessage(NetMsgType::SENDCMPCT, fAnnounceUsingCMPCTBLOCK, nCMPCTBLOCKVersion); + nCMPCTBLOCKVersion = 1; pfrom->PushMessage(NetMsgType::SENDCMPCT, fAnnounceUsingCMPCTBLOCK, nCMPCTBLOCKVersion); } } @@ -5195,12 +5212,23 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, else if (strCommand == NetMsgType::SENDCMPCT) { bool fAnnounceUsingCMPCTBLOCK = false; - uint64_t nCMPCTBLOCKVersion = 1; + uint64_t nCMPCTBLOCKVersion = 0; vRecv >> fAnnounceUsingCMPCTBLOCK >> nCMPCTBLOCKVersion; - if (nCMPCTBLOCKVersion == 1) { + if (nCMPCTBLOCKVersion == 1 || ((pfrom->GetLocalServices() & NODE_WITNESS) && nCMPCTBLOCKVersion == 2)) { LOCK(cs_main); - State(pfrom->GetId())->fProvidesHeaderAndIDs = true; - State(pfrom->GetId())->fPreferHeaderAndIDs = fAnnounceUsingCMPCTBLOCK; + // fProvidesHeaderAndIDs is used to "lock in" version of compact blocks we send (fWantsCmpctWitness) + if (!State(pfrom->GetId())->fProvidesHeaderAndIDs) { + State(pfrom->GetId())->fProvidesHeaderAndIDs = true; + State(pfrom->GetId())->fWantsCmpctWitness = nCMPCTBLOCKVersion == 2; + } + if (State(pfrom->GetId())->fWantsCmpctWitness == (nCMPCTBLOCKVersion == 2)) // ignore later version announces + State(pfrom->GetId())->fPreferHeaderAndIDs = fAnnounceUsingCMPCTBLOCK; + if (!State(pfrom->GetId())->fSupportsDesiredCmpctVersion) { + if (pfrom->GetLocalServices() & NODE_WITNESS) + State(pfrom->GetId())->fSupportsDesiredCmpctVersion = (nCMPCTBLOCKVersion == 2); + else + State(pfrom->GetId())->fSupportsDesiredCmpctVersion = (nCMPCTBLOCKVersion == 1); + } } } @@ -5258,7 +5286,7 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, nodestate->nBlocksInFlight < MAX_BLOCKS_IN_TRANSIT_PER_PEER && (!IsWitnessEnabled(chainActive.Tip(), chainparams.GetConsensus()) || State(pfrom->GetId())->fHaveWitness)) { inv.type |= nFetchFlags; - if (nodestate->fProvidesHeaderAndIDs && !(pfrom->GetLocalServices() & NODE_WITNESS)) + if (nodestate->fSupportsDesiredCmpctVersion) vToFetch.push_back(CInv(MSG_CMPCT_BLOCK, inv.hash)); else vToFetch.push_back(inv); @@ -5386,7 +5414,7 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, } resp.txn[i] = block.vtx[req.indexes[i]]; } - pfrom->PushMessageWithFlag(SERIALIZE_TRANSACTION_NO_WITNESS, NetMsgType::BLOCKTXN, resp); + pfrom->PushMessageWithFlag(State(pfrom->GetId())->fWantsCmpctWitness ? 0 : SERIALIZE_TRANSACTION_NO_WITNESS, NetMsgType::BLOCKTXN, resp); } @@ -5650,7 +5678,7 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, // We requested this block for some reason, but our mempool will probably be useless // so we just grab the block via normal getdata std::vector<CInv> vInv(1); - vInv[0] = CInv(MSG_BLOCK, cmpctblock.header.GetHash()); + vInv[0] = CInv(MSG_BLOCK | GetFetchFlags(pfrom, pindex->pprev, chainparams.GetConsensus()), cmpctblock.header.GetHash()); pfrom->PushMessage(NetMsgType::GETDATA, vInv); } return true; @@ -5662,6 +5690,12 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, CNodeState *nodestate = State(pfrom->GetId()); + if (IsWitnessEnabled(pindex->pprev, chainparams.GetConsensus()) && !nodestate->fSupportsDesiredCmpctVersion) { + // Don't bother trying to process compact blocks from v1 peers + // after segwit activates. + return true; + } + // We want to be a bit conservative just to be extra careful about DoS // possibilities in compact block processing... if (pindex->nHeight <= chainActive.Height() + 2) { @@ -5688,7 +5722,7 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, } else if (status == READ_STATUS_FAILED) { // Duplicate txindexes, the block is now in-flight, so just request it std::vector<CInv> vInv(1); - vInv[0] = CInv(MSG_BLOCK, cmpctblock.header.GetHash()); + vInv[0] = CInv(MSG_BLOCK | GetFetchFlags(pfrom, pindex->pprev, chainparams.GetConsensus()), cmpctblock.header.GetHash()); pfrom->PushMessage(NetMsgType::GETDATA, vInv); return true; } @@ -5715,7 +5749,7 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, // We requested this block, but its far into the future, so our // mempool will probably be useless - request the block normally std::vector<CInv> vInv(1); - vInv[0] = CInv(MSG_BLOCK, cmpctblock.header.GetHash()); + vInv[0] = CInv(MSG_BLOCK | GetFetchFlags(pfrom, pindex->pprev, chainparams.GetConsensus()), cmpctblock.header.GetHash()); pfrom->PushMessage(NetMsgType::GETDATA, vInv); return true; } else { @@ -5757,7 +5791,7 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, } else if (status == READ_STATUS_FAILED) { // Might have collided, fall back to getdata now :( std::vector<CInv> invs; - invs.push_back(CInv(MSG_BLOCK, resp.blockhash)); + invs.push_back(CInv(MSG_BLOCK | GetFetchFlags(pfrom, chainActive.Tip(), chainparams.GetConsensus()), resp.blockhash)); pfrom->PushMessage(NetMsgType::GETDATA, invs); } else { CValidationState state; @@ -5906,7 +5940,7 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, pindexLast->GetBlockHash().ToString(), pindexLast->nHeight); } if (vGetData.size() > 0) { - if (nodestate->fProvidesHeaderAndIDs && vGetData.size() == 1 && mapBlocksInFlight.size() == 1 && pindexLast->pprev->IsValid(BLOCK_VALID_CHAIN) && !(pfrom->GetLocalServices() & NODE_WITNESS)) { + if (nodestate->fSupportsDesiredCmpctVersion && vGetData.size() == 1 && mapBlocksInFlight.size() == 1 && pindexLast->pprev->IsValid(BLOCK_VALID_CHAIN)) { // We seem to be rather well-synced, so it appears pfrom was the first to provide us // with this block! Let's get them to announce using compact blocks in the future. MaybeSetPeerAsAnnouncingHeaderAndIDs(nodestate, pfrom, connman); @@ -6351,7 +6385,7 @@ bool SendMessages(CNode* pto, CConnman& connman) // Ping automatically sent as a latency probe & keepalive. pingSend = true; } - if (pingSend) { + if (pingSend && !pto->fDisconnect) { uint64_t nonce = 0; while (nonce == 0) { GetRandBytes((unsigned char*)&nonce, sizeof(nonce)); @@ -6432,7 +6466,7 @@ bool SendMessages(CNode* pto, CConnman& connman) if (pindexBestHeader == NULL) pindexBestHeader = chainActive.Tip(); bool fFetch = state.fPreferredDownload || (nPreferredDownload == 0 && !pto->fClient && !pto->fOneShot); // Download if this is a nice peer, or we have no nice peers and this one might do. - if (!state.fSyncStarted && !pto->fClient && !fImporting && !fReindex) { + if (!state.fSyncStarted && !pto->fClient && !pto->fDisconnect && !fImporting && !fReindex) { // Only actively request headers from a single peer, unless we're close to today. if ((nSyncStarted == 0 && fFetch) || pindexBestHeader->GetBlockTime() > GetAdjustedTime() - 24 * 60 * 60) { state.fSyncStarted = true; @@ -6536,8 +6570,8 @@ bool SendMessages(CNode* pto, CConnman& connman) //TODO: Shouldn't need to reload block from disk, but requires refactor CBlock block; assert(ReadBlockFromDisk(block, pBestIndex, consensusParams)); - CBlockHeaderAndShortTxIDs cmpctblock(block); - pto->PushMessageWithFlag(SERIALIZE_TRANSACTION_NO_WITNESS, NetMsgType::CMPCTBLOCK, cmpctblock); + CBlockHeaderAndShortTxIDs cmpctblock(block, state.fWantsCmpctWitness); + pto->PushMessageWithFlag(state.fWantsCmpctWitness ? 0 : SERIALIZE_TRANSACTION_NO_WITNESS, NetMsgType::CMPCTBLOCK, cmpctblock); state.pindexBestHeaderSent = pBestIndex; } else if (state.fPreferHeaders) { if (vHeaders.size() > 1) { diff --git a/src/net.cpp b/src/net.cpp index 770e2d2959..19dd040997 100644 --- a/src/net.cpp +++ b/src/net.cpp @@ -629,6 +629,7 @@ void CNode::copyStats(CNodeStats &stats) { stats.nodeid = this->GetId(); X(nServices); + X(addr); X(fRelayTxes); X(nLastSend); X(nLastRecv); @@ -505,6 +505,7 @@ public: double dPingWait; double dPingMin; std::string addrLocal; + CAddress addr; }; diff --git a/src/prevector.h b/src/prevector.h index a0e1e140b4..25bce522dc 100644 --- a/src/prevector.h +++ b/src/prevector.h @@ -475,6 +475,14 @@ public: return ((size_t)(sizeof(T))) * _union.capacity; } } + + value_type* data() { + return item_ptr(0); + } + + const value_type* data() const { + return item_ptr(0); + } }; #pragma pack(pop) diff --git a/src/qt/bitcoingui.cpp b/src/qt/bitcoingui.cpp index dd022ee762..af767aa6c6 100644 --- a/src/qt/bitcoingui.cpp +++ b/src/qt/bitcoingui.cpp @@ -74,6 +74,8 @@ const std::string BitcoinGUI::DEFAULT_UIPLATFORM = #endif ; +/** Display name for default wallet name. Uses tilde to avoid name + * collisions in the future with additional wallets */ const QString BitcoinGUI::DEFAULT_WALLET = "~Default"; BitcoinGUI::BitcoinGUI(const PlatformStyle *_platformStyle, const NetworkStyle *networkStyle, QWidget *parent) : diff --git a/src/qt/clientmodel.cpp b/src/qt/clientmodel.cpp index 6d19b3d122..87704c641d 100644 --- a/src/qt/clientmodel.cpp +++ b/src/qt/clientmodel.cpp @@ -256,7 +256,7 @@ static void BlockTipChanged(ClientModel *clientmodel, bool initialSync, const CB int64_t& nLastUpdateNotification = fHeader ? nLastHeaderTipUpdateNotification : nLastBlockTipUpdateNotification; // if we are in-sync, update the UI regardless of last update time - if (fHeader || !initialSync || now - nLastUpdateNotification > MODEL_UPDATE_DELAY) { + if (!initialSync || now - nLastUpdateNotification > MODEL_UPDATE_DELAY) { //pass a async signal to the UI thread QMetaObject::invokeMethod(clientmodel, "numBlocksChanged", Qt::QueuedConnection, Q_ARG(int, pIndex->nHeight), diff --git a/src/qt/modaloverlay.cpp b/src/qt/modaloverlay.cpp index 2de2dde16a..5caade7d38 100644 --- a/src/qt/modaloverlay.cpp +++ b/src/qt/modaloverlay.cpp @@ -13,7 +13,8 @@ ModalOverlay::ModalOverlay(QWidget *parent) : QWidget(parent), ui(new Ui::ModalOverlay), -bestBlockHeight(0), +bestHeaderHeight(0), +bestHeaderDate(QDateTime()), layerIsVisible(false), userClosed(false) { @@ -65,14 +66,9 @@ bool ModalOverlay::event(QEvent* ev) { void ModalOverlay::setKnownBestHeight(int count, const QDateTime& blockDate) { - - /* only update the blockheight if the headerschain-tip is not older then 30 days */ - int64_t now = QDateTime::currentDateTime().toTime_t(); - int64_t btime = blockDate.toTime_t(); - if (btime+3600*24*30 > now) - { - if (count > bestBlockHeight) - bestBlockHeight = count; + if (count > bestHeaderHeight) { + bestHeaderHeight = count; + bestHeaderDate = blockDate; } } @@ -125,11 +121,21 @@ void ModalOverlay::tipUpdate(int count, const QDateTime& blockDate, double nVeri ui->percentageProgress->setText(QString::number(nVerificationProgress*100, 'f', 2)+"%"); ui->progressBar->setValue(nVerificationProgress*100); + if (!bestHeaderDate.isValid()) + // not syncing + return; + + // estimate the number of headers left based on nPowTargetSpacing + // and check if the gui is not aware of the the best header (happens rarely) + int estimateNumHeadersLeft = bestHeaderDate.secsTo(currentDate) / 600; + bool hasBestHeader = bestHeaderHeight >= count; + // show remaining number of blocks - if (bestBlockHeight > 0) - ui->numberOfBlocksLeft->setText(QString::number(bestBlockHeight-count)); - else + if (estimateNumHeadersLeft < 24 && hasBestHeader) { + ui->numberOfBlocksLeft->setText(QString::number(bestHeaderHeight - count)); + } else { ui->expectedTimeLeft->setText(tr("Unknown. Syncing Headers...")); + } } void ModalOverlay::showHide(bool hide, bool userRequested) diff --git a/src/qt/modaloverlay.h b/src/qt/modaloverlay.h index 670c9e58ab..66c0aa78cf 100644 --- a/src/qt/modaloverlay.h +++ b/src/qt/modaloverlay.h @@ -35,7 +35,8 @@ protected: private: Ui::ModalOverlay *ui; - int bestBlockHeight; //best known height (based on the headers) + int bestHeaderHeight; //best known height (based on the headers) + QDateTime bestHeaderDate; QVector<QPair<qint64, double> > blockProcessTime; bool layerIsVisible; bool userClosed; diff --git a/src/qt/rpcconsole.cpp b/src/qt/rpcconsole.cpp index ace9f1ceaa..b6ed75535c 100644 --- a/src/qt/rpcconsole.cpp +++ b/src/qt/rpcconsole.cpp @@ -981,20 +981,21 @@ void RPCConsole::banSelectedNode(int bantime) if (!clientModel || !g_connman) return; - // Get currently selected peer address - QString strNode = GUIUtil::getEntryData(ui->peerWidget, 0, PeerTableModel::Address).toString(); - // Find possible nodes, ban it and clear the selected node - std::string nStr = strNode.toStdString(); - std::string addr; - int port = 0; - SplitHostPort(nStr, port, addr); + if(cachedNodeid == -1) + return; - CNetAddr resolved; - if(!LookupHost(addr.c_str(), resolved, false)) + // Get currently selected peer address + int detailNodeRow = clientModel->getPeerTableModel()->getRowByNodeId(cachedNodeid); + if(detailNodeRow < 0) return; - g_connman->Ban(resolved, BanReasonManuallyAdded, bantime); - clearSelectedNode(); - clientModel->getBanTableModel()->refresh(); + + // Find possible nodes, ban it and clear the selected node + const CNodeCombinedStats *stats = clientModel->getPeerTableModel()->getNodeStats(detailNodeRow); + if(stats) { + g_connman->Ban(stats->nodeStats.addr, BanReasonManuallyAdded, bantime); + clearSelectedNode(); + clientModel->getBanTableModel()->refresh(); + } } void RPCConsole::unbanSelectedNode() diff --git a/src/qt/walletframe.h b/src/qt/walletframe.h index 00c2f56363..7bc6412910 100644 --- a/src/qt/walletframe.h +++ b/src/qt/walletframe.h @@ -19,6 +19,13 @@ QT_BEGIN_NAMESPACE class QStackedWidget; QT_END_NAMESPACE +/** + * A container for embedding all wallet-related + * controls into BitcoinGUI. The purpose of this class is to allow future + * refinements of the wallet controls with minimal need for further + * modifications to BitcoinGUI, thus greatly simplifying merges while + * reducing the risk of breaking top-level stuff. + */ class WalletFrame : public QFrame { Q_OBJECT diff --git a/src/rpc/blockchain.cpp b/src/rpc/blockchain.cpp index 41d862934a..1e04fcc01c 100644 --- a/src/rpc/blockchain.cpp +++ b/src/rpc/blockchain.cpp @@ -1038,7 +1038,7 @@ UniValue getblockchaininfo(const UniValue& params, bool fHelp) " \"verificationprogress\": xxxx, (numeric) estimate of verification progress [0..1]\n" " \"chainwork\": \"xxxx\" (string) total amount of work in active chain, in hexadecimal\n" " \"pruned\": xx, (boolean) if the blocks are subject to pruning\n" - " \"pruneheight\": xxxxxx, (numeric) heighest block available\n" + " \"pruneheight\": xxxxxx, (numeric) lowest-height complete block stored\n" " \"softforks\": [ (array) status of softforks in progress\n" " {\n" " \"id\": \"xxxx\", (string) name of softfork\n" diff --git a/src/script/bitcoinconsensus.h b/src/script/bitcoinconsensus.h index f73a8e30bc..1d2d5c23e4 100644 --- a/src/script/bitcoinconsensus.h +++ b/src/script/bitcoinconsensus.h @@ -1,5 +1,5 @@ // Copyright (c) 2009-2010 Satoshi Nakamoto -// Copyright (c) 2009-2015 The Bitcoin Core developers +// Copyright (c) 2009-2016 The Bitcoin Core developers // Distributed under the MIT software license, see the accompanying // file COPYING or http://www.opensource.org/licenses/mit-license.php. @@ -50,6 +50,7 @@ enum bitcoinconsensus_SCRIPT_FLAGS_VERIFY_NONE = 0, bitcoinconsensus_SCRIPT_FLAGS_VERIFY_P2SH = (1U << 0), // evaluate P2SH (BIP16) subscripts bitcoinconsensus_SCRIPT_FLAGS_VERIFY_DERSIG = (1U << 2), // enforce strict DER (BIP66) compliance + bitcoinconsensus_SCRIPT_FLAGS_VERIFY_NULLDUMMY = (1U << 4), // enforce NULLDUMMY (BIP147) bitcoinconsensus_SCRIPT_FLAGS_VERIFY_CHECKLOCKTIMEVERIFY = (1U << 9), // enable CHECKLOCKTIMEVERIFY (BIP65) bitcoinconsensus_SCRIPT_FLAGS_VERIFY_CHECKSEQUENCEVERIFY = (1U << 10), // enable CHECKSEQUENCEVERIFY (BIP112) bitcoinconsensus_SCRIPT_FLAGS_VERIFY_WITNESS = (1U << 11), // enable WITNESS (BIP141) diff --git a/src/serialize.h b/src/serialize.h index 04ab9aa2e7..1f51da82ff 100644 --- a/src/serialize.h +++ b/src/serialize.h @@ -44,33 +44,32 @@ inline T* NCONST_PTR(const T* val) return const_cast<T*>(val); } -/** - * Get begin pointer of vector (non-const version). - * @note These functions avoid the undefined case of indexing into an empty - * vector, as well as that of indexing after the end of the vector. +/** + * Important: Do not use the following functions in new code, but use v.data() + * and v.data() + v.size() respectively directly. They were once introduced to + * have a compatible, safe way to get the begin and end pointer of a vector. + * However with C++11 the language has built-in functionality for this and it's + * more readable to just use that. */ template <typename V> inline typename V::value_type* begin_ptr(V& v) { - return v.empty() ? NULL : &v[0]; + return v.data(); } -/** Get begin pointer of vector (const version) */ template <typename V> inline const typename V::value_type* begin_ptr(const V& v) { - return v.empty() ? NULL : &v[0]; + return v.data(); } -/** Get end pointer of vector (non-const version) */ template <typename V> inline typename V::value_type* end_ptr(V& v) { - return v.empty() ? NULL : (&v[0] + v.size()); + return v.data() + v.size(); } -/** Get end pointer of vector (const version) */ template <typename V> inline const typename V::value_type* end_ptr(const V& v) { - return v.empty() ? NULL : (&v[0] + v.size()); + return v.data() + v.size(); } /* diff --git a/src/test/bctest.py b/src/test/bctest.py index 39a27fcd0c..d801415c70 100644 --- a/src/test/bctest.py +++ b/src/test/bctest.py @@ -45,13 +45,17 @@ def bctest(testDir, testObj, exeext): print("Return code mismatch for " + outputFn) sys.exit(1) -def bctester(testDir, input_basename, buildenv): +def bctester(testDir, input_basename, buildenv, verbose = False): input_filename = testDir + "/" + input_basename raw_data = open(input_filename).read() input_data = json.loads(raw_data) for testObj in input_data: + if verbose and "description" in testObj: + print ("Testing: " + testObj["description"]) bctest(testDir, testObj, buildenv.exeext) + if verbose and "description" in testObj: + print ("PASS") sys.exit(0) diff --git a/src/test/bitcoin-util-test.py b/src/test/bitcoin-util-test.py index 225b3324e9..3099506d6d 100755 --- a/src/test/bitcoin-util-test.py +++ b/src/test/bitcoin-util-test.py @@ -19,11 +19,14 @@ test/bitcoin-util-test.py --src=[srcdir] if __name__ == '__main__': + verbose = False try: srcdir = os.environ["srcdir"] except: parser = argparse.ArgumentParser(description=help_text) parser.add_argument('-s', '--srcdir') + parser.add_argument('-v', '--verbose', action='store_true') args = parser.parse_args() srcdir = args.srcdir - bctest.bctester(srcdir + "/test/data", "bitcoin-util-test.json", buildenv) + verbose = args.verbose + bctest.bctester(srcdir + "/test/data", "bitcoin-util-test.json", buildenv, verbose = verbose) diff --git a/src/test/blockencodings_tests.cpp b/src/test/blockencodings_tests.cpp index d2392cfb22..7530b013bd 100644 --- a/src/test/blockencodings_tests.cpp +++ b/src/test/blockencodings_tests.cpp @@ -64,7 +64,7 @@ BOOST_AUTO_TEST_CASE(SimpleRoundTripTest) // Do a simple ShortTxIDs RT { - CBlockHeaderAndShortTxIDs shortIDs(block); + CBlockHeaderAndShortTxIDs shortIDs(block, true); CDataStream stream(SER_NETWORK, PROTOCOL_VERSION); stream << shortIDs; @@ -116,7 +116,7 @@ public: stream >> *this; } TestHeaderAndShortIDs(const CBlock& block) : - TestHeaderAndShortIDs(CBlockHeaderAndShortTxIDs(block)) {} + TestHeaderAndShortIDs(CBlockHeaderAndShortTxIDs(block, true)) {} uint64_t GetShortID(const uint256& txhash) const { CDataStream stream(SER_NETWORK, PROTOCOL_VERSION); @@ -267,7 +267,7 @@ BOOST_AUTO_TEST_CASE(EmptyBlockRoundTripTest) // Test simple header round-trip with only coinbase { - CBlockHeaderAndShortTxIDs shortIDs(block); + CBlockHeaderAndShortTxIDs shortIDs(block, false); CDataStream stream(SER_NETWORK, PROTOCOL_VERSION); stream << shortIDs; diff --git a/src/test/data/bitcoin-util-test.json b/src/test/data/bitcoin-util-test.json index 9df61a7e79..de95044597 100644 --- a/src/test/data/bitcoin-util-test.json +++ b/src/test/data/bitcoin-util-test.json @@ -1,61 +1,73 @@ [ { "exec": "./bitcoin-tx", "args": ["-create"], - "output_cmp": "blanktx.hex" + "output_cmp": "blanktx.hex", + "description": "Creates a blank transaction" }, { "exec": "./bitcoin-tx", "args": ["-json","-create"], - "output_cmp": "blanktx.json" + "output_cmp": "blanktx.json", + "description": "Creates a blank transaction (output in json)" }, { "exec": "./bitcoin-tx", "args": ["-"], "input": "blanktx.hex", - "output_cmp": "blanktx.hex" + "output_cmp": "blanktx.hex", + "description": "Creates a blank transaction when nothing is piped into bitcoin-tx" }, { "exec": "./bitcoin-tx", "args": ["-json","-"], "input": "blanktx.hex", - "output_cmp": "blanktx.json" + "output_cmp": "blanktx.json", + "description": "Creates a blank transaction when nothing is piped into bitcoin-tx (output in json)" }, { "exec": "./bitcoin-tx", "args": ["-", "delin=1"], "input": "tx394b54bb.hex", - "output_cmp": "tt-delin1-out.hex" + "output_cmp": "tt-delin1-out.hex", + "description": "Deletes a single input from a transaction" }, { "exec": "./bitcoin-tx", "args": ["-json", "-", "delin=1"], "input": "tx394b54bb.hex", - "output_cmp": "tt-delin1-out.json" + "output_cmp": "tt-delin1-out.json", + "description": "Deletes a single input from a transaction (output in json)" }, { "exec": "./bitcoin-tx", "args": ["-", "delin=31"], "input": "tx394b54bb.hex", - "return_code": 1 + "return_code": 1, + "description": "Attempts to delete an input with a bad index from a transaction. Expected to fail." }, { "exec": "./bitcoin-tx", "args": ["-", "delout=1"], "input": "tx394b54bb.hex", - "output_cmp": "tt-delout1-out.hex" + "output_cmp": "tt-delout1-out.hex", + "description": "Deletes a single output from a transaction" }, { "exec": "./bitcoin-tx", "args": ["-json", "-", "delout=1"], "input": "tx394b54bb.hex", - "output_cmp": "tt-delout1-out.json" + "output_cmp": "tt-delout1-out.json", + "description": "Deletes a single output from a transaction (output in json)" }, { "exec": "./bitcoin-tx", "args": ["-", "delout=2"], "input": "tx394b54bb.hex", - "return_code": 1 + "return_code": 1, + "description": "Attempts to delete an output with a bad index from a transaction. Expected to fail." }, { "exec": "./bitcoin-tx", "args": ["-", "locktime=317000"], "input": "tx394b54bb.hex", - "output_cmp": "tt-locktime317000-out.hex" + "output_cmp": "tt-locktime317000-out.hex", + "description": "Adds an nlocktime to a transaction" }, { "exec": "./bitcoin-tx", "args": ["-json", "-", "locktime=317000"], "input": "tx394b54bb.hex", - "output_cmp": "tt-locktime317000-out.json" + "output_cmp": "tt-locktime317000-out.json", + "description": "Adds an nlocktime to a transaction (output in json)" }, { "exec": "./bitcoin-tx", "args": @@ -65,7 +77,8 @@ "in=22a6f904655d53ae2ff70e701a0bbd90aa3975c0f40bfc6cc996a9049e31cdfc:1", "outaddr=0.18:13tuJJDR2RgArmgfv6JScSdreahzgc4T6o", "outaddr=4:1P8yWvZW8jVihP1bzHeqfE4aoXNX8AVa46"], - "output_cmp": "txcreate1.hex" + "output_cmp": "txcreate1.hex", + "description": "Creates a new transaction with three inputs and two outputs" }, { "exec": "./bitcoin-tx", "args": @@ -76,15 +89,18 @@ "in=22a6f904655d53ae2ff70e701a0bbd90aa3975c0f40bfc6cc996a9049e31cdfc:1", "outaddr=0.18:13tuJJDR2RgArmgfv6JScSdreahzgc4T6o", "outaddr=4:1P8yWvZW8jVihP1bzHeqfE4aoXNX8AVa46"], - "output_cmp": "txcreate1.json" + "output_cmp": "txcreate1.json", + "description": "Creates a new transaction with three inputs and two outputs (output in json)" }, { "exec": "./bitcoin-tx", "args": ["-create", "outscript=0:"], - "output_cmp": "txcreate2.hex" + "output_cmp": "txcreate2.hex", + "description": "Creates a new transaction with a single empty output script" }, { "exec": "./bitcoin-tx", "args": ["-json", "-create", "outscript=0:"], - "output_cmp": "txcreate2.json" + "output_cmp": "txcreate2.json", + "description": "Creates a new transaction with a single empty output script (output in json)" }, { "exec": "./bitcoin-tx", "args": @@ -94,7 +110,8 @@ "set=prevtxs:[{\"txid\":\"4d49a71ec9da436f71ec4ee231d04f292a29cd316f598bb7068feccabdc59485\",\"vout\":0,\"scriptPubKey\":\"76a91491b24bf9f5288532960ac687abb035127b1d28a588ac\"}]", "sign=ALL", "outaddr=0.001:193P6LtvS4nCnkDvM9uXn1gsSRqh4aDAz7"], - "output_cmp": "txcreatesign.hex" + "output_cmp": "txcreatesign.hex", + "description": "Creates a new transaction with a single input and a single output, and then signs the transaction" }, { "exec": "./bitcoin-tx", "args": @@ -105,21 +122,24 @@ "set=prevtxs:[{\"txid\":\"4d49a71ec9da436f71ec4ee231d04f292a29cd316f598bb7068feccabdc59485\",\"vout\":0,\"scriptPubKey\":\"76a91491b24bf9f5288532960ac687abb035127b1d28a588ac\"}]", "sign=ALL", "outaddr=0.001:193P6LtvS4nCnkDvM9uXn1gsSRqh4aDAz7"], - "output_cmp": "txcreatesign.json" + "output_cmp": "txcreatesign.json", + "description": "Creates a new transaction with a single input and a single output, and then signs the transaction (output in json)" }, { "exec": "./bitcoin-tx", "args": ["-create", "in=5897de6bd6027a475eadd57019d4e6872c396d0716c4875a5f1a6fcfdf385c1f:0", "outdata=4:badhexdata"], - "return_code": 1 + "return_code": 1, + "description": "Attempts to create a new transaction with one input and an output with malformed hex data. Expected to fail" }, { "exec": "./bitcoin-tx", "args": ["-create", "in=5897de6bd6027a475eadd57019d4e6872c396d0716c4875a5f1a6fcfdf385c1f:0", "outdata=badhexdata"], - "return_code": 1 + "return_code": 1, + "description": "Attempts to create a new transaction with one input and an output with no value and malformed hex data. Expected to fail" }, { "exec": "./bitcoin-tx", "args": @@ -127,7 +147,8 @@ "in=5897de6bd6027a475eadd57019d4e6872c396d0716c4875a5f1a6fcfdf385c1f:0", "outaddr=0.18:13tuJJDR2RgArmgfv6JScSdreahzgc4T6o", "outdata=4:54686973204f505f52455455524e207472616e73616374696f6e206f7574707574207761732063726561746564206279206d6f646966696564206372656174657261777472616e73616374696f6e2e"], - "output_cmp": "txcreatedata1.hex" + "output_cmp": "txcreatedata1.hex", + "description": "Creates a new transaction with one input, one address output and one data output" }, { "exec": "./bitcoin-tx", "args": @@ -136,7 +157,8 @@ "in=5897de6bd6027a475eadd57019d4e6872c396d0716c4875a5f1a6fcfdf385c1f:0", "outaddr=0.18:13tuJJDR2RgArmgfv6JScSdreahzgc4T6o", "outdata=4:54686973204f505f52455455524e207472616e73616374696f6e206f7574707574207761732063726561746564206279206d6f646966696564206372656174657261777472616e73616374696f6e2e"], - "output_cmp": "txcreatedata1.json" + "output_cmp": "txcreatedata1.json", + "description": "Creates a new transaction with one input, one address output and one data output (output in json)" }, { "exec": "./bitcoin-tx", "args": @@ -144,7 +166,8 @@ "in=5897de6bd6027a475eadd57019d4e6872c396d0716c4875a5f1a6fcfdf385c1f:0", "outaddr=0.18:13tuJJDR2RgArmgfv6JScSdreahzgc4T6o", "outdata=54686973204f505f52455455524e207472616e73616374696f6e206f7574707574207761732063726561746564206279206d6f646966696564206372656174657261777472616e73616374696f6e2e"], - "output_cmp": "txcreatedata2.hex" + "output_cmp": "txcreatedata2.hex", + "description": "Creates a new transaction with one input, one address output and one data (zero value) output" }, { "exec": "./bitcoin-tx", "args": @@ -153,14 +176,16 @@ "in=5897de6bd6027a475eadd57019d4e6872c396d0716c4875a5f1a6fcfdf385c1f:0", "outaddr=0.18:13tuJJDR2RgArmgfv6JScSdreahzgc4T6o", "outdata=54686973204f505f52455455524e207472616e73616374696f6e206f7574707574207761732063726561746564206279206d6f646966696564206372656174657261777472616e73616374696f6e2e"], - "output_cmp": "txcreatedata2.json" + "output_cmp": "txcreatedata2.json", + "description": "Creates a new transaction with one input, one address output and one data (zero value) output (output in json)" }, { "exec": "./bitcoin-tx", "args": ["-create", "in=5897de6bd6027a475eadd57019d4e6872c396d0716c4875a5f1a6fcfdf385c1f:0:4294967293", "outaddr=0.18:13tuJJDR2RgArmgfv6JScSdreahzgc4T6o"], - "output_cmp": "txcreatedata_seq0.hex" + "output_cmp": "txcreatedata_seq0.hex", + "description": "Creates a new transaction with one input with sequence number and one address output" }, { "exec": "./bitcoin-tx", "args": @@ -168,19 +193,22 @@ "-create", "in=5897de6bd6027a475eadd57019d4e6872c396d0716c4875a5f1a6fcfdf385c1f:0:4294967293", "outaddr=0.18:13tuJJDR2RgArmgfv6JScSdreahzgc4T6o"], - "output_cmp": "txcreatedata_seq0.json" + "output_cmp": "txcreatedata_seq0.json", + "description": "Creates a new transaction with one input with sequence number and one address output (output in json)" }, { "exec": "./bitcoin-tx", "args": ["01000000011f5c38dfcf6f1a5f5a87c416076d392c87e6d41970d5ad5e477a02d66bde97580000000000fdffffff0180a81201000000001976a9141fc11f39be1729bf973a7ab6a615ca4729d6457488ac00000000", "in=5897de6bd6027a475eadd57019d4e6872c396d0716c4875a5f1a6fcfdf385c1f:0:1"], - "output_cmp": "txcreatedata_seq1.hex" + "output_cmp": "txcreatedata_seq1.hex", + "description": "Adds a new input with sequence number to a transaction" }, { "exec": "./bitcoin-tx", "args": ["-json", "01000000011f5c38dfcf6f1a5f5a87c416076d392c87e6d41970d5ad5e477a02d66bde97580000000000fdffffff0180a81201000000001976a9141fc11f39be1729bf973a7ab6a615ca4729d6457488ac00000000", "in=5897de6bd6027a475eadd57019d4e6872c396d0716c4875a5f1a6fcfdf385c1f:0:1"], - "output_cmp": "txcreatedata_seq1.json" + "output_cmp": "txcreatedata_seq1.json", + "description": "Adds a new input with sequence number to a transaction (output in json)" } ] diff --git a/src/txmempool.cpp b/src/txmempool.cpp index 3586123d3e..15fa6fbca3 100644 --- a/src/txmempool.cpp +++ b/src/txmempool.cpp @@ -444,7 +444,7 @@ bool CTxMemPool::addUnchecked(const uint256& hash, const CTxMemPoolEntry &entry, totalTxSize += entry.GetTxSize(); minerPolicyEstimator->processTransaction(entry, fCurrentEstimate); - vTxHashes.emplace_back(hash, newit); + vTxHashes.emplace_back(tx.GetWitnessHash(), newit); newit->vTxHashesIdx = vTxHashes.size() - 1; return true; diff --git a/src/txmempool.h b/src/txmempool.h index 6f67dd91d6..941644b2b2 100644 --- a/src/txmempool.h +++ b/src/txmempool.h @@ -465,7 +465,7 @@ public: indexed_transaction_set mapTx; typedef indexed_transaction_set::nth_index<0>::type::iterator txiter; - std::vector<std::pair<uint256, txiter> > vTxHashes; //!< All tx hashes/entries in mapTx, in random order + std::vector<std::pair<uint256, txiter> > vTxHashes; //!< All tx witness hashes/entries in mapTx, in random order struct CompareIteratorByHash { bool operator()(const txiter &a, const txiter &b) const { |