diff options
66 files changed, 481 insertions, 259 deletions
diff --git a/contrib/gitian-downloader/luke-jr-key.pgp b/contrib/gitian-downloader/luke-jr-key.pgp Binary files differindex 275b041d20..4406e6d5be 100644 --- a/contrib/gitian-downloader/luke-jr-key.pgp +++ b/contrib/gitian-downloader/luke-jr-key.pgp diff --git a/doc/README.md b/doc/README.md index 22b7065f62..e3be27d706 100644 --- a/doc/README.md +++ b/doc/README.md @@ -55,6 +55,7 @@ The Bitcoin repo's [root README](https://github.com/bitcoin/bitcoin/blob/master/ - [Release Process](release-process.md) - [Source Code Documentation (External Link)](https://dev.visucore.com/bitcoin/doxygen/) - [Translation Process](translation_process.md) +- [Translation Strings Policy](translation_strings_policy.md) - [Unit Tests](unit-tests.md) - [Unauthenticated REST Interface](REST-interface.md) - [BIPS](bips.md) diff --git a/doc/release-notes.md b/doc/release-notes.md index 0617f0a257..acff1dc6b6 100644 --- a/doc/release-notes.md +++ b/doc/release-notes.md @@ -1,11 +1,138 @@ (note: this is a temporary file, to be added-to by anybody, and moved to release-notes at release time) +Notable changes +=============== + +Block file pruning +---------------------- + +This release supports running a fully validating node without maintaining a copy +of the raw block and undo data on disk. To recap, there are four types of data +related to the blockchain in the bitcoin system: the raw blocks as received over +the network (blk???.dat), the undo data (rev???.dat), the block index and the +UTXO set (both LevelDB databases). The databases are built from the raw data. + +Block pruning allows Bitcoin Core to delete the raw block and undo data once +it's been validated and used to build the databases. At that point, the raw data +is used only to relay blocks to other nodes, to handle reorganizations, to look +up old transactions (if -txindex is enabled or via the RPC/REST interfaces), or +for rescanning the wallet. The block index continues to hold the metadata about +all blocks in the blockchain. + +The user specifies how much space to allot for block & undo files. The minimum +allowed is 550MB. Note that this is in addition to whatever is required for the +block index and UTXO databases. The minimum was chosen so that Bitcoin Core will +be able to maintain at least 288 blocks on disk (two days worth of blocks at 10 +minutes per block). In rare instances it is possible that the amount of space +used will exceed the pruning target in order to keep the required last 288 +blocks on disk. + +Block pruning works during initial sync in the same way as during steady state, +by deleting block files "as you go" whenever disk space is allocated. Thus, if +the user specifies 550MB, once that level is reached the program will begin +deleting the oldest block and undo files, while continuing to download the +blockchain. + +For now, block pruning disables block relay. In the future, nodes with block +pruning will at a minimum relay "new" blocks, meaning blocks that extend their +active chain. + +Block pruning is currently incompatible with running a wallet due to the fact +that block data is used for rescanning the wallet and importing keys or +addresses (which require a rescan.) However, running the wallet with block +pruning will be supported in the near future, subject to those limitations. + +Block pruning is also incompatible with -txindex and will automatically disable +it. + +Once you have pruned blocks, going back to unpruned state requires +re-downloading the entire blockchain. To do this, re-start the node with +-reindex. Note also that any problem that would cause a user to reindex (e.g., +disk corruption) will cause a pruned node to redownload the entire blockchain. +Finally, note that when a pruned node reindexes, it will delete any blk???.dat +and rev???.dat files in the data directory prior to restarting the download. + +To enable block pruning on the command line: + +- `-prune=N`: where N is the number of MB to allot for raw block & undo data. + +Modified RPC calls: + +- `getblockchaininfo` now includes whether we are in pruned mode or not. +- `getblock` will check if the block's data has been pruned and if so, return an +error. +- `getrawtransaction` will no longer be able to locate a transaction that has a +UTXO but where its block file has been pruned. + +Pruning is disabled by default. + +Big endian support +-------------------- + +Experimental support for big-endian CPU architectures was added in this +release. This has been tested on at least MIPS and PPC hosts. The build +system will automatically detect the endianness of the target. + +Memory usage optimization +-------------------------- + +There have been many changes in this release to reduce the default memory usage +of a node, among which: + +- Accurate UTXO cache size accounting (#6102); this makes the option `-dbcache` + precise, where is did a gross underestimation of memory usage before +- Reduce size of per-peer data structure (#6064 and others); this increases the + number of connections that can be supported with the same amount of memory +- Reduce the number of threads (#5964, #5679); lowers the amount of (esp. + virtual) memory needed + +Privacy: Disable wallet transaction broadcast +---------------------------------------------- + +This release adds an option `-walletbroadcast=0` to prevent automatic +transaction broadcast and rebroadcast (#5951). This option allows separating +transaction submission from the node functionality. + +Making use of this, third-party scripts can be written to take care of +transaction (re)broadcast: + +- Send the transaction as normal, either through RPC or the GUI +- Retrieve the transaction data through RPC using `gettransaction` (NOT + `getrawtransaction`). The `hex` field of the result will contain the raw + hexadecimal representation of the transaction +- The transaction can then be broadcasted through arbitrary mechanisms + supported by the script + +One such application is selective Tor usage, where the node runs on the normal +internet but transactions are broadcasted over Tor. + +Privacy: Stream isolation for Tor +---------------------------------- + +This release adds functionality to create a new circuit for every peer +connection, when the software is used with Tor. The new option, +`-proxyrandomize`, is on by default. + +When enabled, every outgoing connection will (potentially) go through a +different exit node. That significantly reduces the chance to get unlucky and +pick a single exit node that is either malicious, or widely banned from the P2P +network. This improves connection reliability as well as privacy, especially +for the initial connections. + +**Important note:** If a non-Tor SOCKS5 proxy is configured that supports +authentication, but doesn't require it, this change may cause it to reject +connections. A user and password is sent where they weren't before. This setup +is exceedingly rare, but in this case `-proxyrandomize=0` can be passed to +disable the behavior. + 0.11.0 Change log ================= Detailed release notes follow. This overview includes changes that affect -behavior, not code moves, refactors or string updates. +behavior, not code moves, refactors and string updates. For convenience in locating +the code changes and accompanying discussion, both the pull request and +git merge commit are mentioned. ### RPC and REST - #5461 `5f7279a` signrawtransaction: validate private key @@ -23,7 +150,7 @@ behavior, not code moves, refactors or string updates. - #5199 `6364408` Add RPC call `gettxoutproof` to generate and verify merkle blocks - #5418 `16341cc` Report missing inputs in sendrawtransaction - #5937 `40f5e8d` show script verification errors in signrawtransaction result -- #5420 `1fd2d39` [REST] getutxos REST command (based on Bip64) +- #5420 `1fd2d39` getutxos REST command (based on Bip64) ### Configuration and command-line options - #5636 `a353ad4` Add option `-allowselfsignedrootcertificate` to allow self signed root certs (for testing payment requests) @@ -56,7 +183,7 @@ behavior, not code moves, refactors or string updates. - #5507 `844ace9` Prevent DOS attacks on in-flight data structures - #5770 `32a8b6a` Sanitize command strings before logging them - #5859 `dd4ffce` Add correct bool combiner for net signals -- #5876 `8e4fd0c` Add a NODE_GETUTXO service bit and document NODE_NETWORK. +- #5876 `8e4fd0c` Add a NODE_GETUTXO service bit and document NODE_NETWORK - #6028 `b9311fb` Move nLastTry from CAddress to CAddrInfo - #5662 `5048465` Change download logic to allow calling getdata on inbound peers - #5971 `18d2832` replace absolute sleep with conditional wait @@ -85,7 +212,7 @@ behavior, not code moves, refactors or string updates. ### Wallet - #2340 `811c71d` Discourage fee sniping with nLockTime -- #5485 `d01bcc4` Enforce minRelayTxFee on wallet created tx and add a maxtxfee option. +- #5485 `d01bcc4` Enforce minRelayTxFee on wallet created tx and add a maxtxfee option - #5508 `9a5cabf` Add RandAddSeedPerfmon to MakeNewKey - #4805 `8204e19` Do not flush the wallet in AddToWalletIfInvolvingMe(..) - #5319 `93b7544` Clean up wallet encryption code diff --git a/doc/translation_strings_policy.md b/doc/translation_strings_policy.md new file mode 100644 index 0000000000..6824b1d8ef --- /dev/null +++ b/doc/translation_strings_policy.md @@ -0,0 +1,72 @@ +Translation Strings Policy +=========================== + +This document provides guidelines for internationalization of the Bitcoin Core software. + +How to translate? +------------------ + +To mark a message as translatable + +- In GUI source code (under `src/qt`): use `tr("...")` + +- In non-GUI source code (under `src`): use `_("...")` + +No internationalization is used for e.g. developer scripts outside `src`. + +Strings to be translated +------------------------- + +On a high level, these strings are to be translated: + +- GUI strings, anything that appears in a dialog or window + +- Command-line option documentation + +### GUI strings + +Anything that appears to the user in the GUI is to be translated. This includes labels, menu items, button texts, tooltips and window titles. +This includes messages passed to the GUI through the UI interface through `InitMessage`, `ThreadSafeMessageBox` or `ShowProgress`. + +### Command-line options + +Documentation for the command line options in the output of `--help` should be translated as well. + +Make sure that default values do not end up in the string, but use string formatting like `strprintf(_("Threshold for disconnecting misbehaving peers (default: %u)"), 100)`. Putting default values in strings has led to accidental translations in the past, and forces the string to be retranslated every time the value changes. + +Do not translate messages that are only shown to developers, such as those that only appear when `--help-debug` is used. + +General recommendations +------------------------ + +### Avoid unnecessary translation strings + +Try not to burden translators with translating messages that are e.g. slight variations of other messages. +In the GUI, avoid the use of text where an icon or symbol will do. +Make sure that placeholder texts in forms don't end up in the list of strings to be translated (use `<string notr="true">`). + +### Make translated strings understandable + +Try to write translation strings in an understandable way, for both the user and the translator. Avoid overly technical or detailed messages + +### Do not translate internal errors + +Do not translate internal errors, or log messages, or messages that appear on the RPC interface. If an error is to be shown to the user, +use a generic message, then log the detailed message to the log. E.g. "Error: A fatal internal error occurred, see debug.log for details". +This helps troubleshooting; if the error is the same for everyone, the likelihood is increased that it can be found using a search engine. + +### Avoid fragments + +Avoid dividing up a message into fragments. Translators see every string separately, so may misunderstand the context if the messages are not self-contained. + +### Avoid HTML in translation strings + +There have been difficulties with use of HTML in translation strings; translators should not be able to accidentally affect the formatting of messages. +This may sometimes be at conflict with the recommendation in the previous section. + +### String freezes + +During a string freeze (often before a major release), no translation strings are to be added, modified or removed. + +This can be checked by executing `make translate` in the `src` directory, then verifying that `bitcoin_en.ts` remains unchanged. + diff --git a/qa/pull-tester/rpc-tests.sh b/qa/pull-tester/rpc-tests.sh index ecde45059b..88772a9c32 100755 --- a/qa/pull-tester/rpc-tests.sh +++ b/qa/pull-tester/rpc-tests.sh @@ -31,18 +31,46 @@ testScripts=( 'proxy_test.py' 'merkle_blocks.py' 'signrawtransactions.py' + 'walletbackup.py' +); +testScriptsExt=( + 'bipdersig-p2p.py' + 'bipdersig.py' + 'getblocktemplate_longpoll.py' + 'getblocktemplate_proposals.py' + 'pruning.py' + 'forknotify.py' + 'invalidateblock.py' + 'keypool.py' + 'receivedby.py' + 'reindex.py' + 'rpcbind_test.py' +# 'script_test.py' + 'smartfees.py' 'maxblocksinflight.py' 'invalidblockrequest.py' 'rawtransactions.py' # 'forknotify.py' ); + +extArg="-extended" +passOn=${@#$extArg} + if [ "x${ENABLE_BITCOIND}${ENABLE_UTILS}${ENABLE_WALLET}" = "x111" ]; then for (( i = 0; i < ${#testScripts[@]}; i++ )) do - if [ -z "$1" ] || [ "$1" == "${testScripts[$i]}" ] || [ "$1.py" == "${testScripts[$i]}" ] + if [ -z "$1" ] || [ "${1:0:1}" == "-" ] || [ "$1" == "${testScripts[$i]}" ] || [ "$1.py" == "${testScripts[$i]}" ] then echo -e "Running testscript \033[1m${testScripts[$i]}...\033[0m" - ${BUILDDIR}/qa/rpc-tests/${testScripts[$i]} --srcdir "${BUILDDIR}/src" + ${BUILDDIR}/qa/rpc-tests/${testScripts[$i]} --srcdir "${BUILDDIR}/src" ${passOn} + fi + done + for (( i = 0; i < ${#testScriptsExt[@]}; i++ )) + do + if [ "$1" == $extArg ] || [ "$1" == "${testScriptsExt[$i]}" ] || [ "$1.py" == "${testScriptsExt[$i]}" ] + then + echo -e "Running \033[1m2nd level\033[0m testscript \033[1m${testScriptsExt[$i]}...\033[0m" + ${BUILDDIR}/qa/rpc-tests/${testScriptsExt[$i]} --srcdir "${BUILDDIR}/src" ${passOn} fi done else diff --git a/qa/rpc-tests/README.md b/qa/rpc-tests/README.md index d9fbb109e8..6221c93d8b 100644 --- a/qa/rpc-tests/README.md +++ b/qa/rpc-tests/README.md @@ -6,25 +6,37 @@ Git subtree of [https://github.com/jgarzik/python-bitcoinrpc](https://github.com Changes to python-bitcoinrpc should be made upstream, and then pulled here using git subtree. -### [test_framework.py](test_framework.py) +### [test_framework/test_framework.py](test_framework/test_framework.py) Base class for new regression tests. -### [listtransactions.py](listtransactions.py) -Tests for the listtransactions RPC call. - -### [util.py](util.py) +### [test_framework/util.py](test_framework/util.py) Generally useful functions. Bash-based tests, to be ported to Python: ----------------------------------------- -- wallet.sh : Exercise wallet send/receive code. -- walletbackup.sh : Exercise wallet backup / dump / import -- txnmall.sh : Test proper accounting of malleable transactions - conflictedbalance.sh : More testing of malleable transaction handling Notes ===== +You can run a single test by calling `qa/pull-tester/rpc-tests.sh <testname>`. + +Run all possible tests with `qa/pull-tester/rpc-tests.sh -extended`. + +Possible options: + +```` +-h, --help show this help message and exit + --nocleanup Leave bitcoinds and test.* datadir on exit or error + --noshutdown Don't stop bitcoinds after the test execution + --srcdir=SRCDIR Source directory containing bitcoind/bitcoin-cli (default: + ../../src) + --tmpdir=TMPDIR Root directory for datadirs + --tracerpc Print out all RPC calls as they are made +``` + +If you set the environment variable `PYTHON_DEBUG=1` you will get some debug output (example: `PYTHON_DEBUG=1 qa/pull-tester/rpc-tests.sh wallet`). + A 200-block -regtest blockchain and wallets for four nodes is created the first time a regression test is run and is stored in the cache/ directory. Each node has 25 mature diff --git a/qa/rpc-tests/bipdersig-p2p.py b/qa/rpc-tests/bipdersig-p2p.py index ff0c878898..41717377b2 100755 --- a/qa/rpc-tests/bipdersig-p2p.py +++ b/qa/rpc-tests/bipdersig-p2p.py @@ -4,14 +4,14 @@ # file COPYING or http://www.opensource.org/licenses/mit-license.php. # -from test_framework import ComparisonTestFramework -from util import * -from mininode import CTransaction, NetworkThread -from blocktools import create_coinbase, create_block +from test_framework.test_framework import ComparisonTestFramework +from test_framework.util import * +from test_framework.mininode import CTransaction, NetworkThread +from test_framework.blocktools import create_coinbase, create_block +from test_framework.comptool import TestInstance, TestManager +from test_framework.script import CScript from binascii import hexlify, unhexlify import cStringIO -from comptool import TestInstance, TestManager -from script import CScript import time # A canonical signature consists of: diff --git a/qa/rpc-tests/bipdersig.py b/qa/rpc-tests/bipdersig.py index 2c43bba865..243f816f65 100755 --- a/qa/rpc-tests/bipdersig.py +++ b/qa/rpc-tests/bipdersig.py @@ -7,9 +7,8 @@ # Test the BIP66 changeover logic # -from test_framework import BitcoinTestFramework -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * import os import shutil diff --git a/qa/rpc-tests/forknotify.py b/qa/rpc-tests/forknotify.py index af22ffb1a5..0acef8e30b 100755 --- a/qa/rpc-tests/forknotify.py +++ b/qa/rpc-tests/forknotify.py @@ -7,9 +7,8 @@ # Test -alertnotify # -from test_framework import BitcoinTestFramework -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * import os import shutil diff --git a/qa/rpc-tests/getblocktemplate_longpoll.py b/qa/rpc-tests/getblocktemplate_longpoll.py index 64fe49b835..aab4562422 100755 --- a/qa/rpc-tests/getblocktemplate_longpoll.py +++ b/qa/rpc-tests/getblocktemplate_longpoll.py @@ -3,9 +3,8 @@ # Distributed under the MIT software license, see the accompanying # file COPYING or http://www.opensource.org/licenses/mit-license.php. -from test_framework import BitcoinTestFramework -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * def check_array_result(object_array, to_match, expected): diff --git a/qa/rpc-tests/getblocktemplate_proposals.py b/qa/rpc-tests/getblocktemplate_proposals.py index a63f456d6b..aca0cd7495 100755 --- a/qa/rpc-tests/getblocktemplate_proposals.py +++ b/qa/rpc-tests/getblocktemplate_proposals.py @@ -3,9 +3,8 @@ # Distributed under the MIT software license, see the accompanying # file COPYING or http://www.opensource.org/licenses/mit-license.php. -from test_framework import BitcoinTestFramework -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * from binascii import a2b_hex, b2a_hex from hashlib import sha256 diff --git a/qa/rpc-tests/getchaintips.py b/qa/rpc-tests/getchaintips.py index 83a9537285..6a2bcb2969 100755 --- a/qa/rpc-tests/getchaintips.py +++ b/qa/rpc-tests/getchaintips.py @@ -7,8 +7,8 @@ # on chains of different lengths, and join the network together again. # This gives us two tips, verify that it works. -from test_framework import BitcoinTestFramework -from util import assert_equal +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import assert_equal class GetChainTipsTest (BitcoinTestFramework): diff --git a/qa/rpc-tests/httpbasics.py b/qa/rpc-tests/httpbasics.py index 24533741e5..64ba49df64 100755 --- a/qa/rpc-tests/httpbasics.py +++ b/qa/rpc-tests/httpbasics.py @@ -7,8 +7,8 @@ # Test REST interface # -from test_framework import BitcoinTestFramework -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * import base64 try: diff --git a/qa/rpc-tests/invalidateblock.py b/qa/rpc-tests/invalidateblock.py index fd8a8e5785..2b9c8154e0 100755 --- a/qa/rpc-tests/invalidateblock.py +++ b/qa/rpc-tests/invalidateblock.py @@ -7,9 +7,8 @@ # Test InvalidateBlock code # -from test_framework import BitcoinTestFramework -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * class InvalidateTest(BitcoinTestFramework): diff --git a/qa/rpc-tests/invalidblockrequest.py b/qa/rpc-tests/invalidblockrequest.py index 8b685ed9b2..64b8e26395 100755 --- a/qa/rpc-tests/invalidblockrequest.py +++ b/qa/rpc-tests/invalidblockrequest.py @@ -4,11 +4,11 @@ # file COPYING or http://www.opensource.org/licenses/mit-license.php. # -from test_framework import ComparisonTestFramework -from util import * -from comptool import TestManager, TestInstance -from mininode import * -from blocktools import * +from test_framework.test_framework import ComparisonTestFramework +from test_framework.util import * +from test_framework.comptool import TestManager, TestInstance +from test_framework.mininode import * +from test_framework.blocktools import * import logging import copy import time diff --git a/qa/rpc-tests/keypool.py b/qa/rpc-tests/keypool.py index 3840ea39d3..aee29a596a 100755 --- a/qa/rpc-tests/keypool.py +++ b/qa/rpc-tests/keypool.py @@ -8,7 +8,6 @@ # Add python-bitcoinrpc to module search path: import os import sys -sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)), "python-bitcoinrpc")) import json import shutil @@ -16,8 +15,7 @@ import subprocess import tempfile import traceback -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException -from util import * +from test_framework.util import * def check_array_result(object_array, to_match, expected): diff --git a/qa/rpc-tests/listtransactions.py b/qa/rpc-tests/listtransactions.py index 11e3635c04..eeae2d2fa2 100755 --- a/qa/rpc-tests/listtransactions.py +++ b/qa/rpc-tests/listtransactions.py @@ -5,9 +5,8 @@ # Exercise the listtransactions API -from test_framework import BitcoinTestFramework -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * def check_array_result(object_array, to_match, expected): diff --git a/qa/rpc-tests/maxblocksinflight.py b/qa/rpc-tests/maxblocksinflight.py index 87c80cd97e..a601147ce8 100755 --- a/qa/rpc-tests/maxblocksinflight.py +++ b/qa/rpc-tests/maxblocksinflight.py @@ -4,9 +4,9 @@ # file COPYING or http://www.opensource.org/licenses/mit-license.php. # -from mininode import * -from test_framework import BitcoinTestFramework -from util import * +from test_framework.mininode import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * import logging ''' diff --git a/qa/rpc-tests/mempool_coinbase_spends.py b/qa/rpc-tests/mempool_coinbase_spends.py index 853d031de4..c64a15b9f5 100755 --- a/qa/rpc-tests/mempool_coinbase_spends.py +++ b/qa/rpc-tests/mempool_coinbase_spends.py @@ -8,9 +8,8 @@ # that spend (directly or indirectly) coinbase transactions. # -from test_framework import BitcoinTestFramework -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * import os import shutil diff --git a/qa/rpc-tests/mempool_resurrect_test.py b/qa/rpc-tests/mempool_resurrect_test.py index 6f7f577e36..19c74bb751 100755 --- a/qa/rpc-tests/mempool_resurrect_test.py +++ b/qa/rpc-tests/mempool_resurrect_test.py @@ -8,9 +8,8 @@ # the blockchain is re-organized. # -from test_framework import BitcoinTestFramework -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * import os import shutil @@ -34,7 +33,6 @@ class MempoolCoinbaseTest(BitcoinTestFramework): def run_test(self): node0_address = self.nodes[0].getnewaddress() - # Spend block 1/2/3's coinbase transactions # Mine a block. # Create three more transactions, spending the spends diff --git a/qa/rpc-tests/mempool_spendcoinbase.py b/qa/rpc-tests/mempool_spendcoinbase.py index ab5817c869..fc17c50692 100755 --- a/qa/rpc-tests/mempool_spendcoinbase.py +++ b/qa/rpc-tests/mempool_spendcoinbase.py @@ -13,9 +13,8 @@ # but less mature coinbase spends are NOT. # -from test_framework import BitcoinTestFramework -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * import os import shutil diff --git a/qa/rpc-tests/merkle_blocks.py b/qa/rpc-tests/merkle_blocks.py index a143d21a21..72a80ce6ca 100755 --- a/qa/rpc-tests/merkle_blocks.py +++ b/qa/rpc-tests/merkle_blocks.py @@ -7,9 +7,8 @@ # Test merkleblock fetch/validation # -from test_framework import BitcoinTestFramework -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * import os import shutil diff --git a/qa/rpc-tests/proxy_test.py b/qa/rpc-tests/proxy_test.py index d6d9e6725b..9a9b2f5300 100755 --- a/qa/rpc-tests/proxy_test.py +++ b/qa/rpc-tests/proxy_test.py @@ -7,9 +7,9 @@ import traceback, sys from binascii import hexlify import time, os -from socks5 import Socks5Configuration, Socks5Command, Socks5Server, AddressType -from test_framework import BitcoinTestFramework -from util import * +from test_framework.socks5 import Socks5Configuration, Socks5Command, Socks5Server, AddressType +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * ''' Test plan: - Start bitcoind's with different proxy configurations diff --git a/qa/rpc-tests/pruning.py b/qa/rpc-tests/pruning.py index 85fd1c982a..2824c51ce7 100755 --- a/qa/rpc-tests/pruning.py +++ b/qa/rpc-tests/pruning.py @@ -11,9 +11,8 @@ # This test takes 30 mins or more (up to 2 hours) # ******** -from test_framework import BitcoinTestFramework -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * import os.path def calc_usage(blockdir): diff --git a/qa/rpc-tests/python-bitcoinrpc/bitcoinrpc/.gitignore b/qa/rpc-tests/python-bitcoinrpc/bitcoinrpc/.gitignore deleted file mode 100644 index 0d20b6487c..0000000000 --- a/qa/rpc-tests/python-bitcoinrpc/bitcoinrpc/.gitignore +++ /dev/null @@ -1 +0,0 @@ -*.pyc diff --git a/qa/rpc-tests/python-bitcoinrpc/setup.py b/qa/rpc-tests/python-bitcoinrpc/setup.py deleted file mode 100644 index 43cdb1c038..0000000000 --- a/qa/rpc-tests/python-bitcoinrpc/setup.py +++ /dev/null @@ -1,15 +0,0 @@ -#!/usr/bin/env python2 - -from distutils.core import setup - -setup(name='python-bitcoinrpc', - version='0.1', - description='Enhanced version of python-jsonrpc for use with Bitcoin', - long_description=open('README').read(), - author='Jeff Garzik', - author_email='<jgarzik@exmulti.com>', - maintainer='Jeff Garzik', - maintainer_email='<jgarzik@exmulti.com>', - url='http://www.github.com/jgarzik/python-bitcoinrpc', - packages=['bitcoinrpc'], - classifiers=['License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)', 'Operating System :: OS Independent']) diff --git a/qa/rpc-tests/rawtransactions.py b/qa/rpc-tests/rawtransactions.py index 3d80c97d74..1378514c84 100755 --- a/qa/rpc-tests/rawtransactions.py +++ b/qa/rpc-tests/rawtransactions.py @@ -8,8 +8,8 @@ # that spend (directly or indirectly) coinbase transactions. # -from test_framework import BitcoinTestFramework -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * from pprint import pprint from time import sleep diff --git a/qa/rpc-tests/receivedby.py b/qa/rpc-tests/receivedby.py index 1a681e1aae..16d6bd4cf1 100755 --- a/qa/rpc-tests/receivedby.py +++ b/qa/rpc-tests/receivedby.py @@ -5,9 +5,8 @@ # Exercise the listreceivedbyaddress API -from test_framework import BitcoinTestFramework -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * def get_sub_array_from_array(object_array, to_match): diff --git a/qa/rpc-tests/reindex.py b/qa/rpc-tests/reindex.py index fe767586bb..f2e3f248ea 100755 --- a/qa/rpc-tests/reindex.py +++ b/qa/rpc-tests/reindex.py @@ -6,9 +6,8 @@ # # Test -reindex with CheckBlockIndex # -from test_framework import BitcoinTestFramework -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * import os.path class ReindexTest(BitcoinTestFramework): diff --git a/qa/rpc-tests/rest.py b/qa/rpc-tests/rest.py index a6a671ec88..6c51b2fcd9 100755 --- a/qa/rpc-tests/rest.py +++ b/qa/rpc-tests/rest.py @@ -7,8 +7,9 @@ # Test REST interface # -from test_framework import BitcoinTestFramework -from util import * + +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * from struct import * import binascii import json diff --git a/qa/rpc-tests/rpcbind_test.py b/qa/rpc-tests/rpcbind_test.py index 655e00b6e7..04110c2831 100755 --- a/qa/rpc-tests/rpcbind_test.py +++ b/qa/rpc-tests/rpcbind_test.py @@ -8,7 +8,6 @@ # Add python-bitcoinrpc to module search path: import os import sys -sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)), "python-bitcoinrpc")) import json import shutil @@ -16,9 +15,8 @@ import subprocess import tempfile import traceback -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException -from util import * -from netutil import * +from test_framework.util import * +from test_framework.netutil import * def run_bind_test(tmpdir, allow_ips, connect_to, addresses, expected): ''' diff --git a/qa/rpc-tests/script_test.py b/qa/rpc-tests/script_test.py index 1ba3a478a8..860fa56b64 100755 --- a/qa/rpc-tests/script_test.py +++ b/qa/rpc-tests/script_test.py @@ -19,12 +19,12 @@ that flag, we use a block time before the switchover date). NOTE: This test is very slow and may take more than 40 minutes to run. ''' -from test_framework import ComparisonTestFramework -from util import * -from comptool import TestInstance, TestManager -from mininode import * -from blocktools import * -from script import * +from test_framework.test_framework import ComparisonTestFramework +from test_framework.util import * +from test_framework.comptool import TestInstance, TestManager +from test_framework.mininode import * +from test_framework.blocktools import * +from test_framework.script import * import logging import copy import json @@ -42,7 +42,7 @@ class ScriptTestFile(object): def load_files(self): for f in self.files: - self.data.extend(json.loads(open(f).read())) + self.data.extend(json.loads(open(os.path.dirname(os.path.abspath(__file__))+"/"+f).read())) # Skip over records that are not long enough to be tests def get_records(self): diff --git a/qa/rpc-tests/signrawtransactions.py b/qa/rpc-tests/signrawtransactions.py index 943634bd19..d51d6ee610 100755 --- a/qa/rpc-tests/signrawtransactions.py +++ b/qa/rpc-tests/signrawtransactions.py @@ -3,8 +3,8 @@ # Distributed under the MIT software license, see the accompanying # file COPYING or http://www.opensource.org/licenses/mit-license.php. -from test_framework import BitcoinTestFramework -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * class SignRawTransactionsTest(BitcoinTestFramework): diff --git a/qa/rpc-tests/smartfees.py b/qa/rpc-tests/smartfees.py index 69f3c22c17..c15c5fda09 100755 --- a/qa/rpc-tests/smartfees.py +++ b/qa/rpc-tests/smartfees.py @@ -7,9 +7,8 @@ # Test fee estimation code # -from test_framework import BitcoinTestFramework -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * # Construct 2 trivial P2SH's and the ScriptSigs that spend them # So we can create many many transactions without needing to spend diff --git a/qa/rpc-tests/python-bitcoinrpc/bitcoinrpc/__init__.py b/qa/rpc-tests/test_framework/__init__.py index e69de29bb2..e69de29bb2 100644 --- a/qa/rpc-tests/python-bitcoinrpc/bitcoinrpc/__init__.py +++ b/qa/rpc-tests/test_framework/__init__.py diff --git a/qa/rpc-tests/python-bitcoinrpc/bitcoinrpc/authproxy.py b/qa/rpc-tests/test_framework/authproxy.py index bc7d655fdf..bc7d655fdf 100644 --- a/qa/rpc-tests/python-bitcoinrpc/bitcoinrpc/authproxy.py +++ b/qa/rpc-tests/test_framework/authproxy.py diff --git a/qa/rpc-tests/bignum.py b/qa/rpc-tests/test_framework/bignum.py index b0c58ccd47..b0c58ccd47 100644 --- a/qa/rpc-tests/bignum.py +++ b/qa/rpc-tests/test_framework/bignum.py diff --git a/qa/rpc-tests/blockstore.py b/qa/rpc-tests/test_framework/blockstore.py index c57b6df81b..c57b6df81b 100644 --- a/qa/rpc-tests/blockstore.py +++ b/qa/rpc-tests/test_framework/blockstore.py diff --git a/qa/rpc-tests/blocktools.py b/qa/rpc-tests/test_framework/blocktools.py index f397fe7cd6..f397fe7cd6 100644 --- a/qa/rpc-tests/blocktools.py +++ b/qa/rpc-tests/test_framework/blocktools.py diff --git a/qa/rpc-tests/comptool.py b/qa/rpc-tests/test_framework/comptool.py index 23a979250c..23a979250c 100755 --- a/qa/rpc-tests/comptool.py +++ b/qa/rpc-tests/test_framework/comptool.py diff --git a/qa/rpc-tests/mininode.py b/qa/rpc-tests/test_framework/mininode.py index b7d78e74fa..b7d78e74fa 100755 --- a/qa/rpc-tests/mininode.py +++ b/qa/rpc-tests/test_framework/mininode.py diff --git a/qa/rpc-tests/netutil.py b/qa/rpc-tests/test_framework/netutil.py index b30a88a4f7..b30a88a4f7 100644 --- a/qa/rpc-tests/netutil.py +++ b/qa/rpc-tests/test_framework/netutil.py diff --git a/qa/rpc-tests/script.py b/qa/rpc-tests/test_framework/script.py index 03695b8635..e37ab5d45a 100644 --- a/qa/rpc-tests/script.py +++ b/qa/rpc-tests/test_framework/script.py @@ -14,7 +14,7 @@ Functionality to build scripts, as well as SignatureHash(). from __future__ import absolute_import, division, print_function, unicode_literals -from mininode import CTransaction, CTxOut, hash256 +from test_framework.mininode import CTransaction, CTxOut, hash256 import sys bchr = chr @@ -27,7 +27,7 @@ if sys.version > '3': import copy import struct -import bignum +import test_framework.bignum MAX_SCRIPT_SIZE = 10000 MAX_SCRIPT_ELEMENT_SIZE = 520 diff --git a/qa/rpc-tests/socks5.py b/qa/rpc-tests/test_framework/socks5.py index 1dbfb98d5d..1dbfb98d5d 100644 --- a/qa/rpc-tests/socks5.py +++ b/qa/rpc-tests/test_framework/socks5.py diff --git a/qa/rpc-tests/test_framework.py b/qa/rpc-tests/test_framework/test_framework.py index 15a357a340..5671431f6e 100755 --- a/qa/rpc-tests/test_framework.py +++ b/qa/rpc-tests/test_framework/test_framework.py @@ -8,13 +8,12 @@ # Add python-bitcoinrpc to module search path: import os import sys -sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)), "python-bitcoinrpc")) import shutil import tempfile import traceback -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException +from authproxy import AuthServiceProxy, JSONRPCException from util import * diff --git a/qa/rpc-tests/util.py b/qa/rpc-tests/test_framework/util.py index 997bbcc373..c236ec2602 100644 --- a/qa/rpc-tests/util.py +++ b/qa/rpc-tests/test_framework/util.py @@ -8,7 +8,6 @@ # Add python-bitcoinrpc to module search path: import os import sys -sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)), "python-bitcoinrpc")) from decimal import Decimal, ROUND_DOWN import json @@ -18,7 +17,7 @@ import subprocess import time import re -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException +from authproxy import AuthServiceProxy, JSONRPCException from util import * def p2p_port(n): diff --git a/qa/rpc-tests/txn_doublespend.py b/qa/rpc-tests/txn_doublespend.py index fe9168944b..99dcdae552 100755 --- a/qa/rpc-tests/txn_doublespend.py +++ b/qa/rpc-tests/txn_doublespend.py @@ -7,10 +7,9 @@ # Test proper accounting with malleable transactions # -from test_framework import BitcoinTestFramework -from bitcoinrpc.authproxy import AuthServiceProxy, JSONRPCException +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * from decimal import Decimal -from util import * import os import shutil diff --git a/qa/rpc-tests/wallet.py b/qa/rpc-tests/wallet.py index b8965b3662..46dc7765b6 100755 --- a/qa/rpc-tests/wallet.py +++ b/qa/rpc-tests/wallet.py @@ -19,9 +19,8 @@ # k) test ResendWalletTransactions - create transactions, startup fourth node, make sure it syncs # -from test_framework import BitcoinTestFramework -from util import * - +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * class WalletTest (BitcoinTestFramework): diff --git a/qa/rpc-tests/walletbackup.py b/qa/rpc-tests/walletbackup.py index b9fc862234..da100d7fc0 100755 --- a/qa/rpc-tests/walletbackup.py +++ b/qa/rpc-tests/walletbackup.py @@ -33,8 +33,8 @@ Shutdown again, restore using importwallet, and confirm again balances are correct. """ -from test_framework import BitcoinTestFramework -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * from random import randint import logging logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.INFO) diff --git a/qa/rpc-tests/zapwallettxes.py b/qa/rpc-tests/zapwallettxes.py index 045614e94c..0ec8ec5364 100755 --- a/qa/rpc-tests/zapwallettxes.py +++ b/qa/rpc-tests/zapwallettxes.py @@ -3,8 +3,8 @@ # Distributed under the MIT software license, see the accompanying # file COPYING or http://www.opensource.org/licenses/mit-license.php. -from test_framework import BitcoinTestFramework -from util import * +from test_framework.test_framework import BitcoinTestFramework +from test_framework.util import * class ZapWalletTXesTest (BitcoinTestFramework): diff --git a/src/Makefile.am b/src/Makefile.am index c77a262617..1c2f770418 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -92,6 +92,7 @@ BITCOIN_CORE_H = \ compressor.h \ consensus/consensus.h \ consensus/params.h \ + consensus/validation.h \ core_io.h \ eccryptoverify.h \ ecwrapper.h \ diff --git a/src/consensus/validation.h b/src/consensus/validation.h new file mode 100644 index 0000000000..c92bec4fae --- /dev/null +++ b/src/consensus/validation.h @@ -0,0 +1,80 @@ +// Copyright (c) 2009-2010 Satoshi Nakamoto +// Copyright (c) 2009-2014 The Bitcoin Core developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#ifndef BITCOIN_CONSENSUS_VALIDATION_H +#define BITCOIN_CONSENSUS_VALIDATION_H + +#include <string> + +/** "reject" message codes */ +static const unsigned char REJECT_MALFORMED = 0x01; +static const unsigned char REJECT_INVALID = 0x10; +static const unsigned char REJECT_OBSOLETE = 0x11; +static const unsigned char REJECT_DUPLICATE = 0x12; +static const unsigned char REJECT_NONSTANDARD = 0x40; +static const unsigned char REJECT_DUST = 0x41; +static const unsigned char REJECT_INSUFFICIENTFEE = 0x42; +static const unsigned char REJECT_CHECKPOINT = 0x43; + +/** Capture information about block/transaction validation */ +class CValidationState { +private: + enum mode_state { + MODE_VALID, //! everything ok + MODE_INVALID, //! network rule violation (DoS value may be set) + MODE_ERROR, //! run-time error + } mode; + int nDoS; + std::string strRejectReason; + unsigned char chRejectCode; + bool corruptionPossible; +public: + CValidationState() : mode(MODE_VALID), nDoS(0), chRejectCode(0), corruptionPossible(false) {} + bool DoS(int level, bool ret = false, + unsigned char chRejectCodeIn=0, std::string strRejectReasonIn="", + bool corruptionIn=false) { + chRejectCode = chRejectCodeIn; + strRejectReason = strRejectReasonIn; + corruptionPossible = corruptionIn; + if (mode == MODE_ERROR) + return ret; + nDoS += level; + mode = MODE_INVALID; + return ret; + } + bool Invalid(bool ret = false, + unsigned char _chRejectCode=0, std::string _strRejectReason="") { + return DoS(0, ret, _chRejectCode, _strRejectReason); + } + bool Error(std::string strRejectReasonIn="") { + if (mode == MODE_VALID) + strRejectReason = strRejectReasonIn; + mode = MODE_ERROR; + return false; + } + bool IsValid() const { + return mode == MODE_VALID; + } + bool IsInvalid() const { + return mode == MODE_INVALID; + } + bool IsError() const { + return mode == MODE_ERROR; + } + bool IsInvalid(int &nDoSOut) const { + if (IsInvalid()) { + nDoSOut = nDoS; + return true; + } + return false; + } + bool CorruptionPossible() const { + return corruptionPossible; + } + unsigned char GetRejectCode() const { return chRejectCode; } + std::string GetRejectReason() const { return strRejectReason; } +}; + +#endif // BITCOIN_CONSENSUS_VALIDATION_H diff --git a/src/init.cpp b/src/init.cpp index dc9080fec3..f4136f09e9 100644 --- a/src/init.cpp +++ b/src/init.cpp @@ -13,6 +13,7 @@ #include "amount.h" #include "checkpoints.h" #include "compat/sanity.h" +#include "consensus/validation.h" #include "key.h" #include "main.h" #include "miner.h" @@ -667,6 +668,10 @@ bool AppInit2(boost::thread_group& threadGroup, CScheduler& scheduler) // to protect privacy, do not listen by default if a default proxy server is specified if (SoftSetBoolArg("-listen", false)) LogPrintf("%s: parameter interaction: -proxy set -> setting -listen=0\n", __func__); + // to protect privacy, do not use UPNP when a proxy is set. The user may still specify -listen=1 + // to listen locally, so don't rely on this happening through -listen below. + if (SoftSetBoolArg("-upnp", false)) + LogPrintf("%s: parameter interaction: -proxy set -> setting -upnp=0\n", __func__); // to protect privacy, do not discover addresses by default if (SoftSetBoolArg("-discover", false)) LogPrintf("%s: parameter interaction: -proxy set -> setting -discover=0\n", __func__); diff --git a/src/main.cpp b/src/main.cpp index c30f29f824..039092cd19 100644 --- a/src/main.cpp +++ b/src/main.cpp @@ -11,6 +11,7 @@ #include "chainparams.h" #include "checkpoints.h" #include "checkqueue.h" +#include "consensus/validation.h" #include "init.h" #include "merkleblock.h" #include "net.h" @@ -263,9 +264,9 @@ void UpdatePreferredDownload(CNode* node, CNodeState* state) } // Returns time at which to timeout block request (nTime in microseconds) -int64_t GetBlockTimeout(int64_t nTime, int nValidatedQueuedBefore) +int64_t GetBlockTimeout(int64_t nTime, int nValidatedQueuedBefore, const Consensus::Params &consensusParams) { - return nTime + 500000 * Params().GetConsensus().nPowTargetSpacing * (4 + nValidatedQueuedBefore); + return nTime + 500000 * consensusParams.nPowTargetSpacing * (4 + nValidatedQueuedBefore); } void InitializeNode(NodeId nodeid, const CNode *pnode) { @@ -309,7 +310,7 @@ void MarkBlockAsReceived(const uint256& hash) { } // Requires cs_main. -void MarkBlockAsInFlight(NodeId nodeid, const uint256& hash, CBlockIndex *pindex = NULL) { +void MarkBlockAsInFlight(NodeId nodeid, const uint256& hash, const Consensus::Params& consensusParams, CBlockIndex *pindex = NULL) { CNodeState *state = State(nodeid); assert(state != NULL); @@ -317,7 +318,7 @@ void MarkBlockAsInFlight(NodeId nodeid, const uint256& hash, CBlockIndex *pindex MarkBlockAsReceived(hash); int64_t nNow = GetTimeMicros(); - QueuedBlock newentry = {hash, pindex, nNow, pindex != NULL, GetBlockTimeout(nNow, nQueuedValidatedHeaders)}; + QueuedBlock newentry = {hash, pindex, nNow, pindex != NULL, GetBlockTimeout(nNow, nQueuedValidatedHeaders, consensusParams)}; nQueuedValidatedHeaders += newentry.fValidatedHeaders; list<QueuedBlock>::iterator it = state->vBlocksInFlight.insert(state->vBlocksInFlight.end(), newentry); state->nBlocksInFlight++; @@ -1145,7 +1146,7 @@ bool GetTransaction(const uint256 &hash, CTransaction &txOut, uint256 &hashBlock // CBlock and CBlockIndex // -bool WriteBlockToDisk(CBlock& block, CDiskBlockPos& pos) +bool WriteBlockToDisk(CBlock& block, CDiskBlockPos& pos, const CMessageHeader::MessageStartChars& messageStart) { // Open history file to append CAutoFile fileout(OpenBlockFile(pos), SER_DISK, CLIENT_VERSION); @@ -1154,7 +1155,7 @@ bool WriteBlockToDisk(CBlock& block, CDiskBlockPos& pos) // Write index header unsigned int nSize = fileout.GetSerializeSize(block); - fileout << FLATDATA(Params().MessageStart()) << nSize; + fileout << FLATDATA(messageStart) << nSize; // Write block long fileOutPos = ftell(fileout.Get()); @@ -1508,7 +1509,7 @@ bool CheckInputs(const CTransaction& tx, CValidationState &state, const CCoinsVi namespace { -bool UndoWriteToDisk(const CBlockUndo& blockundo, CDiskBlockPos& pos, const uint256& hashBlock) +bool UndoWriteToDisk(const CBlockUndo& blockundo, CDiskBlockPos& pos, const uint256& hashBlock, const CMessageHeader::MessageStartChars& messageStart) { // Open history file to append CAutoFile fileout(OpenUndoFile(pos), SER_DISK, CLIENT_VERSION); @@ -1517,7 +1518,7 @@ bool UndoWriteToDisk(const CBlockUndo& blockundo, CDiskBlockPos& pos, const uint // Write index header unsigned int nSize = fileout.GetSerializeSize(blockundo); - fileout << FLATDATA(Params().MessageStart()) << nSize; + fileout << FLATDATA(messageStart) << nSize; // Write undo data long fileOutPos = ftell(fileout.Get()); @@ -1562,6 +1563,24 @@ bool UndoReadFromDisk(CBlockUndo& blockundo, const CDiskBlockPos& pos, const uin return true; } +/** Abort with a message */ +bool AbortNode(const std::string& strMessage, const std::string& userMessage="") +{ + strMiscWarning = strMessage; + LogPrintf("*** %s\n", strMessage); + uiInterface.ThreadSafeMessageBox( + userMessage.empty() ? _("Error: A fatal internal error occured, see debug.log for details") : userMessage, + "", CClientUIInterface::MSG_ERROR); + StartShutdown(); + return false; +} + +bool AbortNode(CValidationState& state, const std::string& strMessage, const std::string& userMessage="") +{ + AbortNode(strMessage, userMessage); + return state.Error(strMessage); +} + } // anon namespace /** @@ -1899,8 +1918,8 @@ bool ConnectBlock(const CBlock& block, CValidationState& state, CBlockIndex* pin CDiskBlockPos pos; if (!FindUndoPos(state, pindex->nFile, pos, ::GetSerializeSize(blockundo, SER_DISK, CLIENT_VERSION) + 40)) return error("ConnectBlock(): FindUndoPos failed"); - if (!UndoWriteToDisk(blockundo, pos, pindex->pprev->GetBlockHash())) - return state.Abort("Failed to write undo data"); + if (!UndoWriteToDisk(blockundo, pos, pindex->pprev->GetBlockHash(), chainparams.MessageStart())) + return AbortNode(state, "Failed to write undo data"); // update nUndoPos in block index pindex->nUndoPos = pos.nPos; @@ -1913,7 +1932,7 @@ bool ConnectBlock(const CBlock& block, CValidationState& state, CBlockIndex* pin if (fTxIndex) if (!pblocktree->WriteTxIndex(vPos)) - return state.Abort("Failed to write transaction index"); + return AbortNode(state, "Failed to write transaction index"); // add this block to the view's block chain view.SetBestBlock(pindex->GetBlockHash()); @@ -2008,7 +2027,7 @@ bool static FlushStateToDisk(CValidationState &state, FlushStateMode mode) { setDirtyBlockIndex.erase(it++); } if (!pblocktree->WriteBatchSync(vFiles, nLastBlockFile, vBlocks)) { - return state.Abort("Files to write to block index database"); + return AbortNode(state, "Files to write to block index database"); } } // Finally remove any pruned files @@ -2027,7 +2046,7 @@ bool static FlushStateToDisk(CValidationState &state, FlushStateMode mode) { return state.Error("out of disk space"); // Flush the chainstate (which may refer to block index entries). if (!pcoinsTip->Flush()) - return state.Abort("Failed to write to coin database"); + return AbortNode(state, "Failed to write to coin database"); nLastFlush = nNow; } if ((mode == FLUSH_STATE_ALWAYS || mode == FLUSH_STATE_PERIODIC) && nNow > nLastSetChain + (int64_t)DATABASE_WRITE_INTERVAL * 1000000) { @@ -2036,7 +2055,7 @@ bool static FlushStateToDisk(CValidationState &state, FlushStateMode mode) { nLastSetChain = nNow; } } catch (const std::runtime_error& e) { - return state.Abort(std::string("System error while flushing: ") + e.what()); + return AbortNode(state, std::string("System error while flushing: ") + e.what()); } return true; } @@ -2100,7 +2119,7 @@ bool static DisconnectTip(CValidationState &state) { // Read block from disk. CBlock block; if (!ReadBlockFromDisk(block, pindexDelete)) - return state.Abort("Failed to read block"); + return AbortNode(state, "Failed to read block"); // Apply the block atomically to the chain state. int64_t nStart = GetTimeMicros(); { @@ -2151,7 +2170,7 @@ bool static ConnectTip(CValidationState &state, CBlockIndex *pindexNew, CBlock * CBlock block; if (!pblock) { if (!ReadBlockFromDisk(block, pindexNew)) - return state.Abort("Failed to read block"); + return AbortNode(state, "Failed to read block"); pblock = █ } // Apply the block atomically to the chain state. @@ -2822,6 +2841,7 @@ bool AcceptBlockHeader(const CBlockHeader& block, CValidationState& state, CBloc bool AcceptBlock(CBlock& block, CValidationState& state, CBlockIndex** ppindex, CDiskBlockPos* dbp) { + const CChainParams& chainparams = Params(); AssertLockHeld(cs_main); CBlockIndex *&pindex = *ppindex; @@ -2857,12 +2877,12 @@ bool AcceptBlock(CBlock& block, CValidationState& state, CBlockIndex** ppindex, if (!FindBlockPos(state, blockPos, nBlockSize+8, nHeight, block.GetBlockTime(), dbp != NULL)) return error("AcceptBlock(): FindBlockPos failed"); if (dbp == NULL) - if (!WriteBlockToDisk(block, blockPos)) - return state.Abort("Failed to write block"); + if (!WriteBlockToDisk(block, blockPos, chainparams.MessageStart())) + AbortNode(state, "Failed to write block"); if (!ReceivedBlockTransactions(block, state, pindex, blockPos)) return error("AcceptBlock(): ReceivedBlockTransactions failed"); } catch (const std::runtime_error& e) { - return state.Abort(std::string("System error: ") + e.what()); + return AbortNode(state, std::string("System error: ") + e.what()); } if (fCheckForPruning) @@ -2937,24 +2957,6 @@ bool TestBlockValidity(CValidationState &state, const CBlock& block, CBlockIndex return true; } - - - - - - - -bool AbortNode(const std::string &strMessage, const std::string &userMessage) { - strMiscWarning = strMessage; - LogPrintf("*** %s\n", strMessage); - uiInterface.ThreadSafeMessageBox( - userMessage.empty() ? _("Error: A fatal internal error occured, see debug.log for details") : userMessage, - "", CClientUIInterface::MSG_ERROR); - StartShutdown(); - return false; -} - - /** * BLOCK PRUNING CODE */ @@ -3365,6 +3367,7 @@ bool LoadBlockIndex() bool InitBlockIndex() { + const CChainParams& chainparams = Params(); LOCK(cs_main); // Check whether we're already initialized if (chainActive.Genesis() != NULL) @@ -3385,7 +3388,7 @@ bool InitBlockIndex() { CValidationState state; if (!FindBlockPos(state, blockPos, nBlockSize+8, 0, block.GetBlockTime())) return error("LoadBlockIndex(): FindBlockPos failed"); - if (!WriteBlockToDisk(block, blockPos)) + if (!WriteBlockToDisk(block, blockPos, chainparams.MessageStart())) return error("LoadBlockIndex(): writing genesis block to disk failed"); CBlockIndex *pindex = AddToBlockIndex(block); if (!ReceivedBlockTransactions(block, state, pindex, blockPos)) @@ -4180,7 +4183,7 @@ bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, vToFetch.push_back(inv); // Mark block as in flight already, even though the actual "getdata" message only goes out // later (within the same cs_main lock, though). - MarkBlockAsInFlight(pfrom->GetId(), inv.hash); + MarkBlockAsInFlight(pfrom->GetId(), inv.hash, chainparams.GetConsensus()); } LogPrint("net", "getheaders (%d) %s to peer=%d\n", pindexBestHeader->nHeight, inv.hash.ToString(), pfrom->id); } @@ -5033,7 +5036,7 @@ bool SendMessages(CNode* pto, bool fSendTrickle) // more quickly than once every 5 minutes, then we'll shorten the download window for this block). if (!pto->fDisconnect && state.vBlocksInFlight.size() > 0) { QueuedBlock &queuedBlock = state.vBlocksInFlight.front(); - int64_t nTimeoutIfRequestedNow = GetBlockTimeout(nNow, nQueuedValidatedHeaders - state.nBlocksInFlightValidHeaders); + int64_t nTimeoutIfRequestedNow = GetBlockTimeout(nNow, nQueuedValidatedHeaders - state.nBlocksInFlightValidHeaders, consensusParams); if (queuedBlock.nTimeDisconnect > nTimeoutIfRequestedNow) { LogPrint("net", "Reducing block download timeout for peer=%d block=%s, orig=%d new=%d\n", pto->id, queuedBlock.hash.ToString(), queuedBlock.nTimeDisconnect, nTimeoutIfRequestedNow); queuedBlock.nTimeDisconnect = nTimeoutIfRequestedNow; @@ -5054,7 +5057,7 @@ bool SendMessages(CNode* pto, bool fSendTrickle) FindNextBlocksToDownload(pto->GetId(), MAX_BLOCKS_IN_TRANSIT_PER_PEER - state.nBlocksInFlight, vToDownload, staller); BOOST_FOREACH(CBlockIndex *pindex, vToDownload) { vGetData.push_back(CInv(MSG_BLOCK, pindex->GetBlockHash())); - MarkBlockAsInFlight(pto->GetId(), pindex->GetBlockHash(), pindex); + MarkBlockAsInFlight(pto->GetId(), pindex->GetBlockHash(), consensusParams, pindex); LogPrint("net", "Requesting block %s (%d) peer=%d\n", pindex->GetBlockHash().ToString(), pindex->nHeight, pto->id); } diff --git a/src/main.h b/src/main.h index 6c380b104d..9bf7bbb2f8 100644 --- a/src/main.h +++ b/src/main.h @@ -89,16 +89,6 @@ static const unsigned int DATABASE_FLUSH_INTERVAL = 24 * 60 * 60; /** Maximum length of reject messages. */ static const unsigned int MAX_REJECT_MESSAGE_LENGTH = 111; -/** "reject" message codes */ -static const unsigned char REJECT_MALFORMED = 0x01; -static const unsigned char REJECT_INVALID = 0x10; -static const unsigned char REJECT_OBSOLETE = 0x11; -static const unsigned char REJECT_DUPLICATE = 0x12; -static const unsigned char REJECT_NONSTANDARD = 0x40; -static const unsigned char REJECT_DUST = 0x41; -static const unsigned char REJECT_INSUFFICIENTFEE = 0x42; -static const unsigned char REJECT_CHECKPOINT = 0x43; - struct BlockHasher { size_t operator()(const uint256& hash) const { return hash.GetCheapHash(); } @@ -230,8 +220,6 @@ void UnlinkPrunedFiles(std::set<int>& setFilesToPrune); /** Create a new block index entry for a given block hash */ CBlockIndex * InsertBlockIndex(uint256 hash); -/** Abort with a message */ -bool AbortNode(const std::string &msg, const std::string &userMessage=""); /** Get statistics from node state */ bool GetNodeStateStats(NodeId nodeid, CNodeStateStats &stats); /** Increase a node's misbehavior score. */ @@ -374,7 +362,7 @@ public: /** Functions for disk access for blocks */ -bool WriteBlockToDisk(CBlock& block, CDiskBlockPos& pos); +bool WriteBlockToDisk(CBlock& block, CDiskBlockPos& pos, const CMessageHeader::MessageStartChars& messageStart); bool ReadBlockFromDisk(CBlock& block, const CDiskBlockPos& pos); bool ReadBlockFromDisk(CBlock& block, const CBlockIndex* pindex); @@ -461,69 +449,6 @@ public: } }; -/** Capture information about block/transaction validation */ -class CValidationState { -private: - enum mode_state { - MODE_VALID, //! everything ok - MODE_INVALID, //! network rule violation (DoS value may be set) - MODE_ERROR, //! run-time error - } mode; - int nDoS; - std::string strRejectReason; - unsigned char chRejectCode; - bool corruptionPossible; -public: - CValidationState() : mode(MODE_VALID), nDoS(0), chRejectCode(0), corruptionPossible(false) {} - bool DoS(int level, bool ret = false, - unsigned char chRejectCodeIn=0, std::string strRejectReasonIn="", - bool corruptionIn=false) { - chRejectCode = chRejectCodeIn; - strRejectReason = strRejectReasonIn; - corruptionPossible = corruptionIn; - if (mode == MODE_ERROR) - return ret; - nDoS += level; - mode = MODE_INVALID; - return ret; - } - bool Invalid(bool ret = false, - unsigned char _chRejectCode=0, std::string _strRejectReason="") { - return DoS(0, ret, _chRejectCode, _strRejectReason); - } - bool Error(std::string strRejectReasonIn="") { - if (mode == MODE_VALID) - strRejectReason = strRejectReasonIn; - mode = MODE_ERROR; - return false; - } - bool Abort(const std::string &msg) { - AbortNode(msg); - return Error(msg); - } - bool IsValid() const { - return mode == MODE_VALID; - } - bool IsInvalid() const { - return mode == MODE_INVALID; - } - bool IsError() const { - return mode == MODE_ERROR; - } - bool IsInvalid(int &nDoSOut) const { - if (IsInvalid()) { - nDoSOut = nDoS; - return true; - } - return false; - } - bool CorruptionPossible() const { - return corruptionPossible; - } - unsigned char GetRejectCode() const { return chRejectCode; } - std::string GetRejectReason() const { return strRejectReason; } -}; - /** RAII wrapper for VerifyDB: Verify consistency of the block and coin databases */ class CVerifyDB { public: diff --git a/src/miner.cpp b/src/miner.cpp index 804a688523..7a57b42e30 100644 --- a/src/miner.cpp +++ b/src/miner.cpp @@ -8,6 +8,7 @@ #include "amount.h" #include "chainparams.h" #include "consensus/consensus.h" +#include "consensus/validation.h" #include "hash.h" #include "main.h" #include "net.h" diff --git a/src/rpcblockchain.cpp b/src/rpcblockchain.cpp index 4315c5ec18..79528db2fe 100644 --- a/src/rpcblockchain.cpp +++ b/src/rpcblockchain.cpp @@ -4,7 +4,9 @@ // file COPYING or http://www.opensource.org/licenses/mit-license.php. #include "checkpoints.h" +#include "consensus/validation.h" #include "main.h" +#include "primitives/transaction.h" #include "rpcserver.h" #include "sync.h" #include "util.h" diff --git a/src/rpcmining.cpp b/src/rpcmining.cpp index a6a8b9422f..d58d438573 100644 --- a/src/rpcmining.cpp +++ b/src/rpcmining.cpp @@ -6,6 +6,7 @@ #include "amount.h" #include "chainparams.h" #include "consensus/consensus.h" +#include "consensus/validation.h" #include "core_io.h" #include "init.h" #include "main.h" diff --git a/src/rpcrawtransaction.cpp b/src/rpcrawtransaction.cpp index e84d2693a2..3e37b797e8 100644 --- a/src/rpcrawtransaction.cpp +++ b/src/rpcrawtransaction.cpp @@ -4,13 +4,14 @@ // file COPYING or http://www.opensource.org/licenses/mit-license.php. #include "base58.h" -#include "primitives/transaction.h" +#include "consensus/validation.h" #include "core_io.h" #include "init.h" #include "keystore.h" #include "main.h" #include "merkleblock.h" #include "net.h" +#include "primitives/transaction.h" #include "rpcserver.h" #include "script/script.h" #include "script/script_error.h" diff --git a/src/test/checkblock_tests.cpp b/src/test/checkblock_tests.cpp index 7abfad151e..51530c4de5 100644 --- a/src/test/checkblock_tests.cpp +++ b/src/test/checkblock_tests.cpp @@ -2,16 +2,11 @@ // Distributed under the MIT software license, see the accompanying // file COPYING or http://www.opensource.org/licenses/mit-license.php. -// -// Unit tests for block.CheckBlock() -// - - - #include "clientversion.h" +#include "consensus/validation.h" #include "main.h" -#include "utiltime.h" #include "test/test_bitcoin.h" +#include "utiltime.h" #include <cstdio> diff --git a/src/test/miner_tests.cpp b/src/test/miner_tests.cpp index d7ea91607c..b6365b1b3a 100644 --- a/src/test/miner_tests.cpp +++ b/src/test/miner_tests.cpp @@ -2,6 +2,7 @@ // Distributed under the MIT software license, see the accompanying // file COPYING or http://www.opensource.org/licenses/mit-license.php. +#include "consensus/validation.h" #include "main.h" #include "miner.h" #include "pubkey.h" diff --git a/src/test/sighash_tests.cpp b/src/test/sighash_tests.cpp index afb7a41bbd..87be2217c4 100644 --- a/src/test/sighash_tests.cpp +++ b/src/test/sighash_tests.cpp @@ -2,15 +2,16 @@ // Distributed under the MIT software license, see the accompanying // file COPYING or http://www.opensource.org/licenses/mit-license.php. +#include "consensus/validation.h" #include "data/sighash.json.h" #include "main.h" #include "random.h" -#include "serialize.h" -#include "script/script.h" #include "script/interpreter.h" +#include "script/script.h" +#include "serialize.h" +#include "test/test_bitcoin.h" #include "util.h" #include "version.h" -#include "test/test_bitcoin.h" #include <iostream> diff --git a/src/test/transaction_tests.cpp b/src/test/transaction_tests.cpp index 2a3083316e..d12535e438 100644 --- a/src/test/transaction_tests.cpp +++ b/src/test/transaction_tests.cpp @@ -7,12 +7,13 @@ #include "test/test_bitcoin.h" #include "clientversion.h" +#include "consensus/validation.h" +#include "core_io.h" #include "key.h" #include "keystore.h" #include "main.h" #include "script/script.h" #include "script/script_error.h" -#include "core_io.h" #include <map> #include <string> diff --git a/src/txmempool.cpp b/src/txmempool.cpp index 071fa9d52c..1c16e2092e 100644 --- a/src/txmempool.cpp +++ b/src/txmempool.cpp @@ -7,6 +7,7 @@ #include "clientversion.h" #include "consensus/consensus.h" +#include "consensus/validation.h" #include "main.h" #include "policy/fees.h" #include "streams.h" diff --git a/src/wallet/wallet.cpp b/src/wallet/wallet.cpp index 3396a3a188..d892c66eda 100644 --- a/src/wallet/wallet.cpp +++ b/src/wallet/wallet.cpp @@ -9,6 +9,7 @@ #include "checkpoints.h" #include "coincontrol.h" #include "consensus/consensus.h" +#include "consensus/validation.h" #include "main.h" #include "net.h" #include "script/script.h" diff --git a/src/wallet/walletdb.cpp b/src/wallet/walletdb.cpp index 13eaf95aa6..f777926e72 100644 --- a/src/wallet/walletdb.cpp +++ b/src/wallet/walletdb.cpp @@ -6,6 +6,7 @@ #include "wallet/walletdb.h" #include "base58.h" +#include "consensus/validation.h" #include "main.h" #include "protocol.h" #include "serialize.h" |