diff options
78 files changed, 1872 insertions, 700 deletions
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 971c6e427e..7216db0500 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -46,15 +46,15 @@ facilitates social contribution, easy testing and peer review. To contribute a patch, the workflow is as follows: - 1. Fork repository ([only the first time](https://help.github.com/en/articles/fork-a-repo)). + 1. Fork repository ([only for the first time](https://help.github.com/en/articles/fork-a-repo)) 1. Create topic branch 1. Commit patches The project coding conventions in the [developer notes](doc/developer-notes.md) -must be adhered to. +must be followed. -In general [commits should be atomic](https://en.wikipedia.org/wiki/Atomic_commit#Atomic_commit_convention) -and diffs should be easy to read. For this reason do not mix any formatting +In general, [commits should be atomic](https://en.wikipedia.org/wiki/Atomic_commit#Atomic_commit_convention) +and diffs should be easy to read. For this reason, do not mix any formatting fixes or code moves with actual code changes. Commit messages should be verbose by default consisting of a short subject line @@ -100,7 +100,7 @@ Examples: qt: Add feed bump button log: Fix typo in log message -Note that translations should not be submitted as pull requests, please see +Note that translations should not be submitted as pull requests. Please see [Translation Process](https://github.com/bitcoin/bitcoin/blob/master/doc/translation_process.md) for more information on helping with translations. @@ -113,7 +113,7 @@ patch does together with any justification/reasoning. You should include references to any discussions (for example other tickets or mailing list discussions). -At this stage one should expect comments and review from other contributors. You +At this stage, one should expect comments and review from other contributors. You can add more commits to your pull request by committing them locally and pushing to your fork until you have satisfied all feedback. @@ -133,8 +133,8 @@ before it will be merged. The basic squashing workflow is shown below. # Save and quit. git push -f # (force push to GitHub) -Please update the resulting commit message if needed, it should read as a -coherent message. In most cases this means that you should not just list the +Please update the resulting commit message if needed. It should read as a +coherent message. In most cases, this means that you should not just list the interim commits. If you have problems with squashing (or other workflows with `git`), you can @@ -187,9 +187,9 @@ in the future, they may be removed by the Repository Maintainer. Refactoring is a necessary part of any software project's evolution. The following guidelines cover refactoring pull requests for the project. -There are three categories of refactoring, code only moves, code style fixes, -code refactoring. In general refactoring pull requests should not mix these -three kinds of activity in order to make refactoring pull requests easy to +There are three categories of refactoring: code-only moves, code style fixes, and +code refactoring. In general, refactoring pull requests should not mix these +three kinds of activities in order to make refactoring pull requests easy to review and uncontroversial. In all cases, refactoring PRs must not change the behaviour of code within the pull request (bugs must be preserved as is). @@ -323,6 +323,31 @@ about: when someone else is asking for feedback on their code, and universe balances out. +Backporting +----------- + +Security and bug fixes can be backported from `master` to release +branches. +If the backport is non-trivial, it may be appropriate to open an +additional PR, to backport the change, only after the original PR +has been merged. +Otherwise, backports will be done in batches and +the maintainers will use the proper `Needs backport (...)` labels +when needed (the original author does not need to worry). + +A backport should contain the following metadata in the commit body: + +``` +Github-Pull: #<PR number> +Rebased-From: <commit hash of the original commit> +``` + +Have a look at [an example backport PR]( +https://github.com/bitcoin/bitcoin/pull/16189). + +Also see the [backport.py script]( +https://github.com/bitcoin-core/bitcoin-maintainer-tools#backport). + Release Policy -------------- diff --git a/build-aux/m4/ax_boost_chrono.m4 b/build-aux/m4/ax_boost_chrono.m4 deleted file mode 100644 index 4cd3b86041..0000000000 --- a/build-aux/m4/ax_boost_chrono.m4 +++ /dev/null @@ -1,118 +0,0 @@ -# =========================================================================== -# https://www.gnu.org/software/autoconf-archive/ax_boost_chrono.html -# =========================================================================== -# -# SYNOPSIS -# -# AX_BOOST_CHRONO -# -# DESCRIPTION -# -# Test for Chrono library from the Boost C++ libraries. The macro requires -# a preceding call to AX_BOOST_BASE. Further documentation is available at -# <http://randspringer.de/boost/index.html>. -# -# This macro calls: -# -# AC_SUBST(BOOST_CHRONO_LIB) -# -# And sets: -# -# HAVE_BOOST_CHRONO -# -# LICENSE -# -# Copyright (c) 2012 Xiyue Deng <manphiz@gmail.com> -# -# Copying and distribution of this file, with or without modification, are -# permitted in any medium without royalty provided the copyright notice -# and this notice are preserved. This file is offered as-is, without any -# warranty. - -#serial 5 - -AC_DEFUN([AX_BOOST_CHRONO], -[ - AC_ARG_WITH([boost-chrono], - AS_HELP_STRING([--with-boost-chrono@<:@=special-lib@:>@], - [use the Chrono library from boost - it is possible to specify a certain library for the linker - e.g. --with-boost-chrono=boost_chrono-gcc-mt ]), - [ - if test "$withval" = "no"; then - want_boost="no" - elif test "$withval" = "yes"; then - want_boost="yes" - ax_boost_user_chrono_lib="" - else - want_boost="yes" - ax_boost_user_chrono_lib="$withval" - fi - ], - [want_boost="yes"] - ) - - if test "x$want_boost" = "xyes"; then - AC_REQUIRE([AC_PROG_CC]) - AC_REQUIRE([AC_CANONICAL_BUILD]) - CPPFLAGS_SAVED="$CPPFLAGS" - CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS" - export CPPFLAGS - - LDFLAGS_SAVED="$LDFLAGS" - LDFLAGS="$LDFLAGS $BOOST_LDFLAGS" - export LDFLAGS - - AC_CACHE_CHECK(whether the Boost::Chrono library is available, - ax_cv_boost_chrono, - [AC_LANG_PUSH([C++]) - CXXFLAGS_SAVE=$CXXFLAGS - - AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[@%:@include <boost/chrono.hpp>]], - [[boost::chrono::system_clock::time_point* time = new boost::chrono::system_clock::time_point; delete time;]])], - ax_cv_boost_chrono=yes, ax_cv_boost_chrono=no) - CXXFLAGS=$CXXFLAGS_SAVE - AC_LANG_POP([C++]) - ]) - if test "x$ax_cv_boost_chrono" = "xyes"; then - AC_SUBST(BOOST_CPPFLAGS) - - AC_DEFINE(HAVE_BOOST_CHRONO,,[define if the Boost::Chrono library is available]) - BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\/@:>@*//'` - - LDFLAGS_SAVE=$LDFLAGS - if test "x$ax_boost_user_chrono_lib" = "x"; then - for libextension in `ls $BOOSTLIBDIR/libboost_chrono*.so* $BOOSTLIBDIR/libboost_chrono*.dylib* $BOOSTLIBDIR/libboost_chrono*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\(boost_chrono.*\)\.so.*$;\1;' -e 's;^lib\(boost_chrono.*\)\.dylib.*$;\1;' -e 's;^lib\(boost_chrono.*\)\.a.*$;\1;'` ; do - ax_lib=${libextension} - AC_CHECK_LIB($ax_lib, exit, - [BOOST_CHRONO_LIB="-l$ax_lib"; AC_SUBST(BOOST_CHRONO_LIB) link_chrono="yes"; break], - [link_chrono="no"]) - done - if test "x$link_chrono" != "xyes"; then - for libextension in `ls $BOOSTLIBDIR/boost_chrono*.dll* $BOOSTLIBDIR/boost_chrono*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\(boost_chrono.*\)\.dll.*$;\1;' -e 's;^\(boost_chrono.*\)\.a.*$;\1;'` ; do - ax_lib=${libextension} - AC_CHECK_LIB($ax_lib, exit, - [BOOST_CHRONO_LIB="-l$ax_lib"; AC_SUBST(BOOST_CHRONO_LIB) link_chrono="yes"; break], - [link_chrono="no"]) - done - fi - - else - for ax_lib in $ax_boost_user_chrono_lib boost_chrono-$ax_boost_user_chrono_lib; do - AC_CHECK_LIB($ax_lib, exit, - [BOOST_CHRONO_LIB="-l$ax_lib"; AC_SUBST(BOOST_CHRONO_LIB) link_chrono="yes"; break], - [link_chrono="no"]) - done - - fi - if test "x$ax_lib" = "x"; then - AC_MSG_ERROR(Could not find a version of the Boost::Chrono library!) - fi - if test "x$link_chrono" = "xno"; then - AC_MSG_ERROR(Could not link against $ax_lib !) - fi - fi - - CPPFLAGS="$CPPFLAGS_SAVED" - LDFLAGS="$LDFLAGS_SAVED" - fi -]) diff --git a/build_msvc/bitcoin_config.h b/build_msvc/bitcoin_config.h index 47c0a5f407..ea304dc9fc 100644 --- a/build_msvc/bitcoin_config.h +++ b/build_msvc/bitcoin_config.h @@ -47,9 +47,6 @@ /* define if the Boost library is available */ #define HAVE_BOOST /**/ -/* define if the Boost::Chrono library is available */ -#define HAVE_BOOST_CHRONO /**/ - /* define if the Boost::Filesystem library is available */ #define HAVE_BOOST_FILESYSTEM /**/ diff --git a/ci/test/00_setup_env_native_asan.sh b/ci/test/00_setup_env_native_asan.sh index 24c043cb52..d5f39daaf5 100644 --- a/ci/test/00_setup_env_native_asan.sh +++ b/ci/test/00_setup_env_native_asan.sh @@ -7,7 +7,7 @@ export LC_ALL=C.UTF-8 export CONTAINER_NAME=ci_native_asan -export PACKAGES="clang-8 llvm-8 python3-zmq qtbase5-dev qttools5-dev-tools libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libzmq3-dev libqrencode-dev" +export PACKAGES="clang-8 llvm-8 python3-zmq qtbase5-dev qttools5-dev-tools libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libzmq3-dev libqrencode-dev" # Use clang-8 instead of default clang (which is clang-6 on Bionic) to avoid spurious segfaults when running on ppc64le export NO_DEPENDS=1 export GOAL="install" diff --git a/ci/test/00_setup_env_native_fuzz.sh b/ci/test/00_setup_env_native_fuzz.sh index af93dd1d68..a739ad50d2 100644 --- a/ci/test/00_setup_env_native_fuzz.sh +++ b/ci/test/00_setup_env_native_fuzz.sh @@ -7,7 +7,7 @@ export LC_ALL=C.UTF-8 export CONTAINER_NAME=ci_native_fuzz -export PACKAGES="clang-8 llvm-8 python3 libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev" +export PACKAGES="clang-8 llvm-8 python3 libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev libboost-thread-dev" export NO_DEPENDS=1 export RUN_UNIT_TESTS=false export RUN_FUNCTIONAL_TESTS=false diff --git a/ci/test/00_setup_env_native_fuzz_with_valgrind.sh b/ci/test/00_setup_env_native_fuzz_with_valgrind.sh index 45b13a669d..fabb3affa4 100644 --- a/ci/test/00_setup_env_native_fuzz_with_valgrind.sh +++ b/ci/test/00_setup_env_native_fuzz_with_valgrind.sh @@ -7,7 +7,7 @@ export LC_ALL=C.UTF-8 export CONTAINER_NAME=ci_native_fuzz_valgrind -export PACKAGES="clang-8 llvm-8 python3 libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev valgrind" +export PACKAGES="clang-8 llvm-8 python3 libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev libboost-thread-dev valgrind" export NO_DEPENDS=1 export RUN_UNIT_TESTS=false export RUN_FUNCTIONAL_TESTS=false diff --git a/ci/test/00_setup_env_native_tsan.sh b/ci/test/00_setup_env_native_tsan.sh index 37884128fe..4d3f345ca6 100644 --- a/ci/test/00_setup_env_native_tsan.sh +++ b/ci/test/00_setup_env_native_tsan.sh @@ -8,7 +8,7 @@ export LC_ALL=C.UTF-8 export CONTAINER_NAME=ci_native_tsan export DOCKER_NAME_TAG=ubuntu:16.04 -export PACKAGES="clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libzmq3-dev libqrencode-dev" +export PACKAGES="clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libzmq3-dev libqrencode-dev" export NO_DEPENDS=1 export GOAL="install" export BITCOIN_CONFIG="--enable-zmq --disable-wallet --with-gui=qt5 CPPFLAGS='-DARENA_DEBUG -DDEBUG_LOCKORDER' --with-sanitizers=thread --disable-hardening --disable-asm CC=clang CXX=clang++" diff --git a/ci/test/00_setup_env_native_valgrind.sh b/ci/test/00_setup_env_native_valgrind.sh index 156a83590c..f112aa87f9 100644 --- a/ci/test/00_setup_env_native_valgrind.sh +++ b/ci/test/00_setup_env_native_valgrind.sh @@ -7,10 +7,10 @@ export LC_ALL=C.UTF-8 export CONTAINER_NAME=ci_native_valgrind -export PACKAGES="valgrind clang llvm python3-zmq libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libzmq3-dev" +export PACKAGES="valgrind clang llvm python3-zmq libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libzmq3-dev" export USE_VALGRIND=1 export NO_DEPENDS=1 -export TEST_RUNNER_EXTRA="p2p_segwit.py" # Only run one test for now. TODO enable all and bump timeouts +export TEST_RUNNER_EXTRA="--exclude feature_abortnode,feature_block,rpc_bind" # Excluded for now export RUN_FUNCTIONAL_TESTS=true export GOAL="install" export BITCOIN_CONFIG="--enable-zmq --with-incompatible-bdb --with-gui=no CC=clang CXX=clang++" # TODO enable GUI diff --git a/configure.ac b/configure.ac index 8433615bb2..1c0f39d7f7 100644 --- a/configure.ac +++ b/configure.ac @@ -1139,7 +1139,6 @@ fi AX_BOOST_SYSTEM AX_BOOST_FILESYSTEM AX_BOOST_THREAD -AX_BOOST_CHRONO dnl Boost 1.56 through 1.62 allow using std::atomic instead of its own atomic dnl counter implementations. In 1.63 and later the std::atomic approach is default. @@ -1206,7 +1205,7 @@ fi if test x$use_boost = xyes; then -BOOST_LIBS="$BOOST_LDFLAGS $BOOST_SYSTEM_LIB $BOOST_FILESYSTEM_LIB $BOOST_THREAD_LIB $BOOST_CHRONO_LIB" +BOOST_LIBS="$BOOST_LDFLAGS $BOOST_SYSTEM_LIB $BOOST_FILESYSTEM_LIB $BOOST_THREAD_LIB" dnl If boost (prior to 1.57) was built without c++11, it emulated scoped enums diff --git a/depends/packages/boost.mk b/depends/packages/boost.mk index cd0e70fb1c..cbe4fe4d97 100644 --- a/depends/packages/boost.mk +++ b/depends/packages/boost.mk @@ -22,7 +22,7 @@ $(package)_config_opts_armv7a_android=address-model=32 $(package)_toolset_$(host_os)=gcc $(package)_archiver_$(host_os)=$($(package)_ar) $(package)_toolset_darwin=clang-darwin -$(package)_config_libraries=chrono,filesystem,system,thread,test +$(package)_config_libraries=filesystem,system,thread,test $(package)_cxxflags=-std=c++11 -fvisibility=hidden $(package)_cxxflags_linux=-fPIC $(package)_cxxflags_android=-fPIC diff --git a/doc/build-unix.md b/doc/build-unix.md index e799e709fa..6b51db5f55 100644 --- a/doc/build-unix.md +++ b/doc/build-unix.md @@ -80,7 +80,7 @@ Build requirements: Now, you can either build from self-compiled [depends](/depends/README.md) or install the required dependencies: - sudo apt-get install libevent-dev libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev + sudo apt-get install libevent-dev libboost-system-dev libboost-filesystem-dev libboost-test-dev libboost-thread-dev BerkeleyDB is required for the wallet. diff --git a/doc/developer-notes.md b/doc/developer-notes.md index d106aab3e4..207c4ba7c6 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -288,6 +288,7 @@ $ valgrind --suppressions=contrib/valgrind.supp src/test/test_bitcoin $ valgrind --suppressions=contrib/valgrind.supp --leak-check=full \ --show-leak-kinds=all src/test/test_bitcoin --log_level=test_suite $ valgrind -v --leak-check=full src/bitcoind -printtoconsole +$ ./test/functional/test_runner.py --valgrind ``` ### Compiling for test coverage diff --git a/doc/files.md b/doc/files.md index c2296b45fa..cd23d547bb 100644 --- a/doc/files.md +++ b/doc/files.md @@ -78,6 +78,10 @@ Subdirectory | File(s) | Description 3. A wallet database path can be specified by `-wallet` option. +4. `wallet.dat` files must not be shared across different node instances, as that can result in key-reuse and double-spends due the lack of synchronization between instances. + +5. Any copy or backup of the wallet should be done through a `backupwallet` call in order to update and lock the wallet, preventing any file corruption caused by updates during the copy. + ## GUI settings `bitcoin-qt` uses [`QSettings`](https://doc.qt.io/qt-5/qsettings.html) class; this implies platform-specific [locations where application settings are stored](https://doc.qt.io/qt-5/qsettings.html#locations-where-application-settings-are-stored). diff --git a/doc/release-notes.md b/doc/release-notes.md index cb46408d94..22d5767b7b 100644 --- a/doc/release-notes.md +++ b/doc/release-notes.md @@ -139,6 +139,14 @@ New settings - RPC Whitelist system. It can give certain RPC users permissions to only some RPC calls. It can be set with two command line arguments (`rpcwhitelist` and `rpcwhitelistdefault`). (#12763) +- A new `-asmap` configuration option has been added to enable IP-to-ASN mapping + for bucketing of the network peers to diversify the network connections. The + legacy /16 prefix mapping remains the default. See [issue + #16599](https://github.com/bitcoin/bitcoin/issues/16599), [PR + #16702](https://github.com/bitcoin/bitcoin/pull/16702), and the `bitcoind + help` for more information. This option is experimental and subject to changes + or removal in future releases. + Updated settings ---------------- diff --git a/doc/release-notes/release-notes-0.19.1.md b/doc/release-notes/release-notes-0.19.1.md new file mode 100644 index 0000000000..5746bebb0d --- /dev/null +++ b/doc/release-notes/release-notes-0.19.1.md @@ -0,0 +1,115 @@ +0.19.1 Release Notes +=============================== + +Bitcoin Core version 0.19.1 is now available from: + + <https://bitcoincore.org/bin/bitcoin-core-0.19.1/> + +This minor release includes various bug fixes and performance +improvements, as well as updated translations. + +Please report bugs using the issue tracker at GitHub: + + <https://github.com/bitcoin/bitcoin/issues> + +To receive security and update notifications, please subscribe to: + + <https://bitcoincore.org/en/list/announcements/join/> + +How to Upgrade +============== + +If you are running an older version, shut it down. Wait until it has completely +shut down (which might take a few minutes for older versions), then run the +installer (on Windows) or just copy over `/Applications/Bitcoin-Qt` (on Mac) +or `bitcoind`/`bitcoin-qt` (on Linux). + +Upgrading directly from a version of Bitcoin Core that has reached its EOL is +possible, but it might take some time if the datadir needs to be migrated. Old +wallet versions of Bitcoin Core are generally supported. + +Compatibility +============== + +Bitcoin Core is supported and extensively tested on operating systems using +the Linux kernel, macOS 10.10+, and Windows 7 and newer. It is not recommended +to use Bitcoin Core on unsupported systems. + +Bitcoin Core should also work on most other Unix-like systems but is not +as frequently tested on them. + +From Bitcoin Core 0.17.0 onwards, macOS versions earlier than 10.10 are no +longer supported, as Bitcoin Core is now built using Qt 5.9.x which requires +macOS 10.10+. Additionally, Bitcoin Core does not yet change appearance when +macOS "dark mode" is activated. + +In addition to previously supported CPU platforms, this release's pre-compiled +distribution provides binaries for the RISC-V platform. + +0.19.1 change log +================= + +### Wallet +- #17643 Fix origfee return for bumpfee with feerate arg (instagibbs) +- #16963 Fix `unique_ptr` usage in boost::signals2 (promag) +- #17258 Fix issue with conflicted mempool tx in listsinceblock (adamjonas, mchrostowski) +- #17924 Bug: IsUsedDestination shouldn't use key id as script id for ScriptHash (instagibbs) +- #17621 IsUsedDestination should count any known single-key address (instagibbs) +- #17843 Reset reused transactions cache (fjahr) + +### RPC and other APIs +- #17687 cli: Fix fatal leveldb error when specifying -blockfilterindex=basic twice (brakmic) +- #17728 require second argument only for scantxoutset start action (achow101) +- #17445 zmq: Fix due to invalid argument and multiple notifiers (promag) +- #17524 psbt: handle unspendable psbts (achow101) +- #17156 psbt: check that various indexes and amounts are within bounds (achow101) + +### GUI +- #17427 Fix missing qRegisterMetaType for `size_t` (hebasto) +- #17695 disable File-\>CreateWallet during startup (fanquake) +- #17634 Fix comparison function signature (hebasto) +- #18062 Fix unintialized WalletView::progressDialog (promag) + +### Tests and QA +- #17416 Appveyor improvement - text file for vcpkg package list (sipsorcery) +- #17488 fix "bitcoind already running" warnings on macOS (fanquake) +- #17980 add missing #include to fix compiler errors (kallewoof) + +### Platform support +- #17736 Update msvc build for Visual Studio 2019 v16.4 (sipsorcery) +- #17364 Updates to appveyor config for VS2019 and Qt5.9.8 + msvc project fixes (sipsorcery) +- #17887 bug-fix macos: give free bytes to `F_PREALLOCATE` (kallewoof) + +### Miscellaneous +- #17897 init: Stop indexes on shutdown after ChainStateFlushed callback (jimpo) +- #17450 util: Add missing headers to util/fees.cpp (hebasto) +- #17654 Unbreak build with Boost 1.72.0 (jbeich) +- #17857 scripts: Fix symbol-check & security-check argument passing (fanquake) +- #17762 Log to net category for exceptions in ProcessMessages (laanwj) +- #18100 Update univalue subtree (MarcoFalke) + +Credits +======= + +Thanks to everyone who directly contributed to this release: + +- Aaron Clauson +- Adam Jonas +- Andrew Chow +- Fabian Jahr +- fanquake +- Gregory Sanders +- Harris +- Hennadii Stepanov +- Jan Beich +- Jim Posen +- João Barbosa +- Karl-Johan Alm +- Luke Dashjr +- MarcoFalke +- Michael Chrostowski +- Russell Yanofsky +- Wladimir J. van der Laan + +As well as to everyone that helped with translations on +[Transifex](https://www.transifex.com/bitcoin/bitcoin/). diff --git a/src/Makefile.am b/src/Makefile.am index 0a1c45cf8c..6fc6d5b5a3 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -178,7 +178,6 @@ BITCOIN_CORE_H = \ random.h \ randomenv.h \ reverse_iterator.h \ - reverselock.h \ rpc/blockchain.h \ rpc/client.h \ rpc/protocol.h \ @@ -242,7 +241,6 @@ BITCOIN_CORE_H = \ wallet/fees.h \ wallet/ismine.h \ wallet/load.h \ - wallet/psbtwallet.h \ wallet/rpcwallet.h \ wallet/scriptpubkeyman.h \ wallet/wallet.h \ @@ -350,7 +348,6 @@ libbitcoin_wallet_a_SOURCES = \ wallet/feebumper.cpp \ wallet/fees.cpp \ wallet/load.cpp \ - wallet/psbtwallet.cpp \ wallet/rpcdump.cpp \ wallet/rpcwallet.cpp \ wallet/scriptpubkeyman.cpp \ diff --git a/src/Makefile.test.include b/src/Makefile.test.include index 7823bf4c05..5a2ecff651 100644 --- a/src/Makefile.test.include +++ b/src/Makefile.test.include @@ -2,7 +2,6 @@ # Distributed under the MIT software license, see the accompanying # file COPYING or http://www.opensource.org/licenses/mit-license.php. - FUZZ_TARGETS = \ test/fuzz/addr_info_deserialize \ test/fuzz/address_deserialize \ @@ -36,10 +35,13 @@ FUZZ_TARGETS = \ test/fuzz/integer \ test/fuzz/inv_deserialize \ test/fuzz/key \ + test/fuzz/key_io \ test/fuzz/key_origin_info_deserialize \ + test/fuzz/locale \ test/fuzz/merkle_block_deserialize \ test/fuzz/messageheader_deserialize \ test/fuzz/netaddr_deserialize \ + test/fuzz/netaddress \ test/fuzz/out_point_deserialize \ test/fuzz/p2p_transport_deserializer \ test/fuzz/parse_hd_keypath \ @@ -50,6 +52,31 @@ FUZZ_TARGETS = \ test/fuzz/partial_merkle_tree_deserialize \ test/fuzz/partially_signed_transaction_deserialize \ test/fuzz/prefilled_transaction_deserialize \ + test/fuzz/process_message \ + test/fuzz/process_message_addr \ + test/fuzz/process_message_block \ + test/fuzz/process_message_blocktxn \ + test/fuzz/process_message_cmpctblock \ + test/fuzz/process_message_feefilter \ + test/fuzz/process_message_filteradd \ + test/fuzz/process_message_filterclear \ + test/fuzz/process_message_filterload \ + test/fuzz/process_message_getaddr \ + test/fuzz/process_message_getblocks \ + test/fuzz/process_message_getblocktxn \ + test/fuzz/process_message_getdata \ + test/fuzz/process_message_getheaders \ + test/fuzz/process_message_headers \ + test/fuzz/process_message_inv \ + test/fuzz/process_message_mempool \ + test/fuzz/process_message_notfound \ + test/fuzz/process_message_ping \ + test/fuzz/process_message_pong \ + test/fuzz/process_message_sendcmpct \ + test/fuzz/process_message_sendheaders \ + test/fuzz/process_message_tx \ + test/fuzz/process_message_verack \ + test/fuzz/process_message_version \ test/fuzz/psbt \ test/fuzz/psbt_input_deserialize \ test/fuzz/psbt_output_deserialize \ @@ -58,6 +85,8 @@ FUZZ_TARGETS = \ test/fuzz/script \ test/fuzz/script_deserialize \ test/fuzz/script_flags \ + test/fuzz/script_ops \ + test/fuzz/scriptnum_ops \ test/fuzz/service_deserialize \ test/fuzz/spanparsing \ test/fuzz/strprintf \ @@ -437,12 +466,24 @@ test_fuzz_key_LDADD = $(FUZZ_SUITE_LD_COMMON) test_fuzz_key_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) test_fuzz_key_SOURCES = $(FUZZ_SUITE) test/fuzz/key.cpp +test_fuzz_key_io_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) +test_fuzz_key_io_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_key_io_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_key_io_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_key_io_SOURCES = $(FUZZ_SUITE) test/fuzz/key_io.cpp + test_fuzz_key_origin_info_deserialize_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DKEY_ORIGIN_INFO_DESERIALIZE=1 test_fuzz_key_origin_info_deserialize_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) test_fuzz_key_origin_info_deserialize_LDADD = $(FUZZ_SUITE_LD_COMMON) test_fuzz_key_origin_info_deserialize_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) test_fuzz_key_origin_info_deserialize_SOURCES = $(FUZZ_SUITE) test/fuzz/deserialize.cpp +test_fuzz_locale_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) +test_fuzz_locale_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_locale_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_locale_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_locale_SOURCES = $(FUZZ_SUITE) test/fuzz/locale.cpp + test_fuzz_merkle_block_deserialize_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMERKLE_BLOCK_DESERIALIZE=1 test_fuzz_merkle_block_deserialize_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) test_fuzz_merkle_block_deserialize_LDADD = $(FUZZ_SUITE_LD_COMMON) @@ -461,6 +502,12 @@ test_fuzz_netaddr_deserialize_LDADD = $(FUZZ_SUITE_LD_COMMON) test_fuzz_netaddr_deserialize_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) test_fuzz_netaddr_deserialize_SOURCES = $(FUZZ_SUITE) test/fuzz/deserialize.cpp +test_fuzz_netaddress_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) +test_fuzz_netaddress_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_netaddress_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_netaddress_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_netaddress_SOURCES = $(FUZZ_SUITE) test/fuzz/netaddress.cpp + test_fuzz_out_point_deserialize_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DOUT_POINT_DESERIALIZE=1 test_fuzz_out_point_deserialize_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) test_fuzz_out_point_deserialize_LDADD = $(FUZZ_SUITE_LD_COMMON) @@ -521,6 +568,156 @@ test_fuzz_prefilled_transaction_deserialize_LDADD = $(FUZZ_SUITE_LD_COMMON) test_fuzz_prefilled_transaction_deserialize_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) test_fuzz_prefilled_transaction_deserialize_SOURCES = $(FUZZ_SUITE) test/fuzz/deserialize.cpp +test_fuzz_process_message_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) +test_fuzz_process_message_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_addr_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=addr +test_fuzz_process_message_addr_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_addr_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_addr_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_addr_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_block_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=block +test_fuzz_process_message_block_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_block_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_block_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_block_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_blocktxn_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=blocktxn +test_fuzz_process_message_blocktxn_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_blocktxn_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_blocktxn_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_blocktxn_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_cmpctblock_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=cmpctblock +test_fuzz_process_message_cmpctblock_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_cmpctblock_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_cmpctblock_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_cmpctblock_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_feefilter_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=feefilter +test_fuzz_process_message_feefilter_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_feefilter_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_feefilter_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_feefilter_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_filteradd_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=filteradd +test_fuzz_process_message_filteradd_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_filteradd_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_filteradd_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_filteradd_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_filterclear_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=filterclear +test_fuzz_process_message_filterclear_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_filterclear_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_filterclear_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_filterclear_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_filterload_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=filterload +test_fuzz_process_message_filterload_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_filterload_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_filterload_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_filterload_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_getaddr_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=getaddr +test_fuzz_process_message_getaddr_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_getaddr_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_getaddr_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_getaddr_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_getblocks_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=getblocks +test_fuzz_process_message_getblocks_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_getblocks_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_getblocks_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_getblocks_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_getblocktxn_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=getblocktxn +test_fuzz_process_message_getblocktxn_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_getblocktxn_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_getblocktxn_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_getblocktxn_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_getdata_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=getdata +test_fuzz_process_message_getdata_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_getdata_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_getdata_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_getdata_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_getheaders_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=getheaders +test_fuzz_process_message_getheaders_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_getheaders_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_getheaders_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_getheaders_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_headers_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=headers +test_fuzz_process_message_headers_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_headers_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_headers_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_headers_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_inv_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=inv +test_fuzz_process_message_inv_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_inv_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_inv_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_inv_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_mempool_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=mempool +test_fuzz_process_message_mempool_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_mempool_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_mempool_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_mempool_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_notfound_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=notfound +test_fuzz_process_message_notfound_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_notfound_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_notfound_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_notfound_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_ping_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=ping +test_fuzz_process_message_ping_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_ping_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_ping_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_ping_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_pong_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=pong +test_fuzz_process_message_pong_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_pong_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_pong_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_pong_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_sendcmpct_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=sendcmpct +test_fuzz_process_message_sendcmpct_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_sendcmpct_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_sendcmpct_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_sendcmpct_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_sendheaders_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=sendheaders +test_fuzz_process_message_sendheaders_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_sendheaders_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_sendheaders_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_sendheaders_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_tx_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=tx +test_fuzz_process_message_tx_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_tx_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_tx_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_tx_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_verack_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=verack +test_fuzz_process_message_verack_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_verack_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_verack_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_verack_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + +test_fuzz_process_message_version_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DMESSAGE_TYPE=version +test_fuzz_process_message_version_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_process_message_version_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_process_message_version_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_process_message_version_SOURCES = $(FUZZ_SUITE) test/fuzz/process_message.cpp + test_fuzz_psbt_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) test_fuzz_psbt_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) test_fuzz_psbt_LDADD = $(FUZZ_SUITE_LD_COMMON) @@ -569,6 +766,18 @@ test_fuzz_script_flags_LDADD = $(FUZZ_SUITE_LD_COMMON) test_fuzz_script_flags_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) test_fuzz_script_flags_SOURCES = $(FUZZ_SUITE) test/fuzz/script_flags.cpp +test_fuzz_script_ops_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) +test_fuzz_script_ops_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_script_ops_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_script_ops_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_script_ops_SOURCES = $(FUZZ_SUITE) test/fuzz/script_ops.cpp + +test_fuzz_scriptnum_ops_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) +test_fuzz_scriptnum_ops_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) +test_fuzz_scriptnum_ops_LDADD = $(FUZZ_SUITE_LD_COMMON) +test_fuzz_scriptnum_ops_LDFLAGS = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS) +test_fuzz_scriptnum_ops_SOURCES = $(FUZZ_SUITE) test/fuzz/scriptnum_ops.cpp + test_fuzz_service_deserialize_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES) -DSERVICE_DESERIALIZE=1 test_fuzz_service_deserialize_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS) test_fuzz_service_deserialize_LDADD = $(FUZZ_SUITE_LD_COMMON) diff --git a/src/index/base.cpp b/src/index/base.cpp index dcb8e99fc1..3f3b301246 100644 --- a/src/index/base.cpp +++ b/src/index/base.cpp @@ -270,7 +270,7 @@ void BaseIndex::ChainStateFlushed(const CBlockLocator& locator) Commit(); } -bool BaseIndex::BlockUntilSyncedToCurrentChain() +bool BaseIndex::BlockUntilSyncedToCurrentChain() const { AssertLockNotHeld(cs_main); diff --git a/src/index/base.h b/src/index/base.h index d0088d9c9a..fb30aa7a0e 100644 --- a/src/index/base.h +++ b/src/index/base.h @@ -97,7 +97,7 @@ public: /// sync once and only needs to process blocks in the ValidationInterface /// queue. If the index is catching up from far behind, this method does /// not block and immediately returns false. - bool BlockUntilSyncedToCurrentChain(); + bool BlockUntilSyncedToCurrentChain() const; void Interrupt(); diff --git a/src/init.cpp b/src/init.cpp index 14d489617c..a637aac4d2 100644 --- a/src/init.cpp +++ b/src/init.cpp @@ -206,6 +206,7 @@ void Shutdown(NodeContext& node) // After everything has been shut down, but before things get flushed, stop the // CScheduler/checkqueue threadGroup + if (node.scheduler) node.scheduler->stop(); threadGroup.interrupt_all(); threadGroup.join_all(); diff --git a/src/interfaces/wallet.cpp b/src/interfaces/wallet.cpp index baea71d0bb..01ade56b2a 100644 --- a/src/interfaces/wallet.cpp +++ b/src/interfaces/wallet.cpp @@ -19,7 +19,6 @@ #include <wallet/fees.h> #include <wallet/ismine.h> #include <wallet/load.h> -#include <wallet/psbtwallet.h> #include <wallet/rpcwallet.h> #include <wallet/wallet.h> @@ -119,19 +118,15 @@ public: } bool getPubKey(const CScript& script, const CKeyID& address, CPubKey& pub_key) override { - std::unique_ptr<SigningProvider> provider = m_wallet->GetSigningProvider(script); + std::unique_ptr<SigningProvider> provider = m_wallet->GetSolvingProvider(script); if (provider) { return provider->GetPubKey(address, pub_key); } return false; } - bool getPrivKey(const CScript& script, const CKeyID& address, CKey& key) override + SigningResult signMessage(const std::string& message, const PKHash& pkhash, std::string& str_sig) override { - std::unique_ptr<SigningProvider> provider = m_wallet->GetSigningProvider(script); - if (provider) { - return provider->GetKey(address, key); - } - return false; + return m_wallet->SignMessage(message, pkhash, str_sig); } bool isSpendable(const CTxDestination& dest) override { return m_wallet->IsMine(dest) & ISMINE_SPENDABLE; } bool haveWatchOnly() override @@ -361,9 +356,9 @@ public: bool& complete, int sighash_type = 1 /* SIGHASH_ALL */, bool sign = true, - bool bip32derivs = false) override + bool bip32derivs = false) const override { - return FillPSBT(m_wallet.get(), psbtx, complete, sighash_type, sign, bip32derivs); + return m_wallet->FillPSBT(psbtx, complete, sighash_type, sign, bip32derivs); } WalletBalances getBalances() override { @@ -468,7 +463,7 @@ public: } unsigned int getConfirmTarget() override { return m_wallet->m_confirm_target; } bool hdEnabled() override { return m_wallet->IsHDEnabled(); } - bool canGetAddresses() override { return m_wallet->CanGetAddresses(); } + bool canGetAddresses() const override { return m_wallet->CanGetAddresses(); } bool IsWalletFlagSet(uint64_t flag) override { return m_wallet->IsWalletFlagSet(flag); } OutputType getDefaultAddressType() override { return m_wallet->m_default_address_type; } OutputType getDefaultChangeType() override { return m_wallet->m_default_change_type; } diff --git a/src/interfaces/wallet.h b/src/interfaces/wallet.h index d4280e8091..9476c9f77f 100644 --- a/src/interfaces/wallet.h +++ b/src/interfaces/wallet.h @@ -10,6 +10,7 @@ #include <script/standard.h> // For CTxDestination #include <support/allocators/secure.h> // For SecureString #include <ui_interface.h> // For ChangeType +#include <util/message.h> #include <functional> #include <map> @@ -84,8 +85,8 @@ public: //! Get public key. virtual bool getPubKey(const CScript& script, const CKeyID& address, CPubKey& pub_key) = 0; - //! Get private key. - virtual bool getPrivKey(const CScript& script, const CKeyID& address, CKey& key) = 0; + //! Sign message + virtual SigningResult signMessage(const std::string& message, const PKHash& pkhash, std::string& str_sig) = 0; //! Return whether wallet has private key. virtual bool isSpendable(const CTxDestination& dest) = 0; @@ -196,7 +197,7 @@ public: bool& complete, int sighash_type = 1 /* SIGHASH_ALL */, bool sign = true, - bool bip32derivs = false) = 0; + bool bip32derivs = false) const = 0; //! Get balances. virtual WalletBalances getBalances() = 0; @@ -246,7 +247,7 @@ public: virtual bool hdEnabled() = 0; // Return whether the wallet is blank. - virtual bool canGetAddresses() = 0; + virtual bool canGetAddresses() const = 0; // check if a certain wallet flag is set. virtual bool IsWalletFlagSet(uint64_t flag) = 0; diff --git a/src/net_processing.cpp b/src/net_processing.cpp index fa09b60630..e4df9fb90e 100644 --- a/src/net_processing.cpp +++ b/src/net_processing.cpp @@ -1908,7 +1908,7 @@ void static ProcessOrphanTx(CConnman* connman, std::set<uint256>& orphan_work_se } } -bool static ProcessMessage(CNode* pfrom, const std::string& strCommand, CDataStream& vRecv, int64_t nTimeReceived, const CChainParams& chainparams, CConnman* connman, BanMan* banman, const std::atomic<bool>& interruptMsgProc) +bool ProcessMessage(CNode* pfrom, const std::string& strCommand, CDataStream& vRecv, int64_t nTimeReceived, const CChainParams& chainparams, CConnman* connman, BanMan* banman, const std::atomic<bool>& interruptMsgProc) { LogPrint(BCLog::NET, "received: %s (%u bytes) peer=%d\n", SanitizeString(strCommand), vRecv.size(), pfrom->GetId()); if (gArgs.IsArgSet("-dropmessagestest") && GetRand(gArgs.GetArg("-dropmessagestest", 0)) == 0) diff --git a/src/qt/sendcoinsdialog.cpp b/src/qt/sendcoinsdialog.cpp index cc01aafb23..4ddee513a1 100644 --- a/src/qt/sendcoinsdialog.cpp +++ b/src/qt/sendcoinsdialog.cpp @@ -26,7 +26,6 @@ #include <ui_interface.h> #include <wallet/coincontrol.h> #include <wallet/fees.h> -#include <wallet/psbtwallet.h> #include <wallet/wallet.h> #include <QFontMetrics> diff --git a/src/qt/signverifymessagedialog.cpp b/src/qt/signverifymessagedialog.cpp index 883dcecf9a..4552753bf6 100644 --- a/src/qt/signverifymessagedialog.cpp +++ b/src/qt/signverifymessagedialog.cpp @@ -133,20 +133,27 @@ void SignVerifyMessageDialog::on_signMessageButton_SM_clicked() return; } - CKey key; - if (!model->wallet().getPrivKey(GetScriptForDestination(destination), CKeyID(*pkhash), key)) - { - ui->statusLabel_SM->setStyleSheet("QLabel { color: red; }"); - ui->statusLabel_SM->setText(tr("Private key for the entered address is not available.")); - return; - } - const std::string& message = ui->messageIn_SM->document()->toPlainText().toStdString(); std::string signature; + SigningResult res = model->wallet().signMessage(message, *pkhash, signature); + + QString error; + switch (res) { + case SigningResult::OK: + error = tr("No error"); + break; + case SigningResult::PRIVATE_KEY_NOT_AVAILABLE: + error = tr("Private key for the entered address is not available."); + break; + case SigningResult::SIGNING_FAILED: + error = tr("Message signing failed."); + break; + // no default case, so the compiler can warn about missing cases + } - if (!MessageSign(key, message, signature)) { + if (res != SigningResult::OK) { ui->statusLabel_SM->setStyleSheet("QLabel { color: red; }"); - ui->statusLabel_SM->setText(QString("<nobr>") + tr("Message signing failed.") + QString("</nobr>")); + ui->statusLabel_SM->setText(QString("<nobr>") + error + QString("</nobr>")); return; } diff --git a/src/reverselock.h b/src/reverselock.h deleted file mode 100644 index 9d9cc9fd77..0000000000 --- a/src/reverselock.h +++ /dev/null @@ -1,34 +0,0 @@ -// Copyright (c) 2015-2016 The Bitcoin Core developers -// Distributed under the MIT software license, see the accompanying -// file COPYING or http://www.opensource.org/licenses/mit-license.php. - -#ifndef BITCOIN_REVERSELOCK_H -#define BITCOIN_REVERSELOCK_H - -/** - * An RAII-style reverse lock. Unlocks on construction and locks on destruction. - */ -template<typename Lock> -class reverse_lock -{ -public: - - explicit reverse_lock(Lock& _lock) : lock(_lock) { - _lock.unlock(); - _lock.swap(templock); - } - - ~reverse_lock() { - templock.lock(); - templock.swap(lock); - } - -private: - reverse_lock(reverse_lock const&); - reverse_lock& operator=(reverse_lock const&); - - Lock& lock; - Lock templock; -}; - -#endif // BITCOIN_REVERSELOCK_H diff --git a/src/rpc/misc.cpp b/src/rpc/misc.cpp index 60a3a0dda9..8357183934 100644 --- a/src/rpc/misc.cpp +++ b/src/rpc/misc.cpp @@ -393,7 +393,7 @@ static UniValue mockscheduler(const JSONRPCRequest& request) // protect against null pointer dereference CHECK_NONFATAL(g_rpc_node); CHECK_NONFATAL(g_rpc_node->scheduler); - g_rpc_node->scheduler->MockForward(boost::chrono::seconds(delta_seconds)); + g_rpc_node->scheduler->MockForward(std::chrono::seconds(delta_seconds)); return NullUniValue; } diff --git a/src/rpc/rawtransaction_util.cpp b/src/rpc/rawtransaction_util.cpp index 40334883c5..54baec6c6f 100644 --- a/src/rpc/rawtransaction_util.cpp +++ b/src/rpc/rawtransaction_util.cpp @@ -272,55 +272,27 @@ void SignTransaction(CMutableTransaction& mtx, const SigningProvider* keystore, { int nHashType = ParseSighashString(hashType); - bool fHashSingle = ((nHashType & ~SIGHASH_ANYONECANPAY) == SIGHASH_SINGLE); - // Script verification errors - UniValue vErrors(UniValue::VARR); - - // Use CTransaction for the constant parts of the - // transaction to avoid rehashing. - const CTransaction txConst(mtx); - // Sign what we can: - for (unsigned int i = 0; i < mtx.vin.size(); i++) { - CTxIn& txin = mtx.vin[i]; - auto coin = coins.find(txin.prevout); - if (coin == coins.end() || coin->second.IsSpent()) { - TxInErrorToJSON(txin, vErrors, "Input not found or already spent"); - continue; - } - const CScript& prevPubKey = coin->second.out.scriptPubKey; - const CAmount& amount = coin->second.out.nValue; - - SignatureData sigdata = DataFromTransaction(mtx, i, coin->second.out); - // Only sign SIGHASH_SINGLE if there's a corresponding output: - if (!fHashSingle || (i < mtx.vout.size())) { - ProduceSignature(*keystore, MutableTransactionSignatureCreator(&mtx, i, amount, nHashType), prevPubKey, sigdata); - } - - UpdateInput(txin, sigdata); + std::map<int, std::string> input_errors; - // amount must be specified for valid segwit signature - if (amount == MAX_MONEY && !txin.scriptWitness.IsNull()) { - throw JSONRPCError(RPC_TYPE_ERROR, strprintf("Missing amount for %s", coin->second.out.ToString())); - } + bool complete = SignTransaction(mtx, keystore, coins, nHashType, input_errors); + SignTransactionResultToJSON(mtx, complete, coins, input_errors, result); +} - ScriptError serror = SCRIPT_ERR_OK; - if (!VerifyScript(txin.scriptSig, prevPubKey, &txin.scriptWitness, STANDARD_SCRIPT_VERIFY_FLAGS, TransactionSignatureChecker(&txConst, i, amount), &serror)) { - if (serror == SCRIPT_ERR_INVALID_STACK_OPERATION) { - // Unable to sign input and verification failed (possible attempt to partially sign). - TxInErrorToJSON(txin, vErrors, "Unable to sign input, invalid stack size (possibly missing key)"); - } else if (serror == SCRIPT_ERR_SIG_NULLFAIL) { - // Verification failed (possibly due to insufficient signatures). - TxInErrorToJSON(txin, vErrors, "CHECK(MULTI)SIG failing with non-zero signature (possibly need more signatures)"); - } else { - TxInErrorToJSON(txin, vErrors, ScriptErrorString(serror)); - } +void SignTransactionResultToJSON(CMutableTransaction& mtx, bool complete, const std::map<COutPoint, Coin>& coins, std::map<int, std::string>& input_errors, UniValue& result) +{ + // Make errors UniValue + UniValue vErrors(UniValue::VARR); + for (const auto& err_pair : input_errors) { + if (err_pair.second == "Missing amount") { + // This particular error needs to be an exception for some reason + throw JSONRPCError(RPC_TYPE_ERROR, strprintf("Missing amount for %s", coins.at(mtx.vin.at(err_pair.first).prevout).out.ToString())); } + TxInErrorToJSON(mtx.vin.at(err_pair.first), vErrors, err_pair.second); } - bool fComplete = vErrors.empty(); result.pushKV("hex", EncodeHexTx(CTransaction(mtx))); - result.pushKV("complete", fComplete); + result.pushKV("complete", complete); if (!vErrors.empty()) { if (result.exists("errors")) { vErrors.push_backV(result["errors"].getValues()); diff --git a/src/rpc/rawtransaction_util.h b/src/rpc/rawtransaction_util.h index 4750fd64ed..436db5dc60 100644 --- a/src/rpc/rawtransaction_util.h +++ b/src/rpc/rawtransaction_util.h @@ -6,6 +6,7 @@ #define BITCOIN_RPC_RAWTRANSACTION_UTIL_H #include <map> +#include <string> class FillableSigningProvider; class UniValue; @@ -24,6 +25,7 @@ class SigningProvider; * @param result JSON object where signed transaction results accumulate */ void SignTransaction(CMutableTransaction& mtx, const SigningProvider* keystore, const std::map<COutPoint, Coin>& coins, const UniValue& hashType, UniValue& result); +void SignTransactionResultToJSON(CMutableTransaction& mtx, bool complete, const std::map<COutPoint, Coin>& coins, std::map<int, std::string>& input_errors, UniValue& result); /** * Parse a prevtxs UniValue array and get the map of coins from it diff --git a/src/scheduler.cpp b/src/scheduler.cpp index 72cca89d99..7cb7754fde 100644 --- a/src/scheduler.cpp +++ b/src/scheduler.cpp @@ -5,7 +5,6 @@ #include <scheduler.h> #include <random.h> -#include <reverselock.h> #include <assert.h> #include <utility> @@ -20,18 +19,9 @@ CScheduler::~CScheduler() } -#if BOOST_VERSION < 105000 -static boost::system_time toPosixTime(const boost::chrono::system_clock::time_point& t) -{ - // Creating the posix_time using from_time_t loses sub-second precision. So rather than exporting the time_point to time_t, - // start with a posix_time at the epoch (0) and add the milliseconds that have passed since then. - return boost::posix_time::from_time_t(0) + boost::posix_time::milliseconds(boost::chrono::duration_cast<boost::chrono::milliseconds>(t.time_since_epoch()).count()); -} -#endif - void CScheduler::serviceQueue() { - boost::unique_lock<boost::mutex> lock(newTaskMutex); + WAIT_LOCK(newTaskMutex, lock); ++nThreadsServicingQueue; // newTaskMutex is locked throughout this loop EXCEPT @@ -40,7 +30,7 @@ void CScheduler::serviceQueue() while (!shouldStop()) { try { if (!shouldStop() && taskQueue.empty()) { - reverse_lock<boost::unique_lock<boost::mutex> > rlock(lock); + REVERSE_LOCK(lock); } while (!shouldStop() && taskQueue.empty()) { // Wait until there is something to do. @@ -50,21 +40,13 @@ void CScheduler::serviceQueue() // Wait until either there is a new task, or until // the time of the first item on the queue: -// wait_until needs boost 1.50 or later; older versions have timed_wait: -#if BOOST_VERSION < 105000 - while (!shouldStop() && !taskQueue.empty() && - newTaskScheduled.timed_wait(lock, toPosixTime(taskQueue.begin()->first))) { - // Keep waiting until timeout - } -#else - // Some boost versions have a conflicting overload of wait_until that returns void. - // Explicitly use a template here to avoid hitting that overload. while (!shouldStop() && !taskQueue.empty()) { - boost::chrono::system_clock::time_point timeToWaitFor = taskQueue.begin()->first; - if (newTaskScheduled.wait_until<>(lock, timeToWaitFor) == boost::cv_status::timeout) + std::chrono::system_clock::time_point timeToWaitFor = taskQueue.begin()->first; + if (newTaskScheduled.wait_until(lock, timeToWaitFor) == std::cv_status::timeout) { break; // Exit loop after timeout, it means we reached the time of the event + } } -#endif + // If there are multiple threads, the queue can empty while we're waiting (another // thread may service the task we were waiting on). if (shouldStop() || taskQueue.empty()) @@ -76,7 +58,7 @@ void CScheduler::serviceQueue() { // Unlock before calling f, so it can reschedule itself or another task // without deadlocking: - reverse_lock<boost::unique_lock<boost::mutex> > rlock(lock); + REVERSE_LOCK(lock); f(); } } catch (...) { @@ -91,7 +73,7 @@ void CScheduler::serviceQueue() void CScheduler::stop(bool drain) { { - boost::unique_lock<boost::mutex> lock(newTaskMutex); + LOCK(newTaskMutex); if (drain) stopWhenEmpty = true; else @@ -100,10 +82,10 @@ void CScheduler::stop(bool drain) newTaskScheduled.notify_all(); } -void CScheduler::schedule(CScheduler::Function f, boost::chrono::system_clock::time_point t) +void CScheduler::schedule(CScheduler::Function f, std::chrono::system_clock::time_point t) { { - boost::unique_lock<boost::mutex> lock(newTaskMutex); + LOCK(newTaskMutex); taskQueue.insert(std::make_pair(t, f)); } newTaskScheduled.notify_one(); @@ -111,18 +93,18 @@ void CScheduler::schedule(CScheduler::Function f, boost::chrono::system_clock::t void CScheduler::scheduleFromNow(CScheduler::Function f, int64_t deltaMilliSeconds) { - schedule(f, boost::chrono::system_clock::now() + boost::chrono::milliseconds(deltaMilliSeconds)); + schedule(f, std::chrono::system_clock::now() + std::chrono::milliseconds(deltaMilliSeconds)); } -void CScheduler::MockForward(boost::chrono::seconds delta_seconds) +void CScheduler::MockForward(std::chrono::seconds delta_seconds) { - assert(delta_seconds.count() > 0 && delta_seconds < boost::chrono::hours{1}); + assert(delta_seconds.count() > 0 && delta_seconds < std::chrono::hours{1}); { - boost::unique_lock<boost::mutex> lock(newTaskMutex); + LOCK(newTaskMutex); // use temp_queue to maintain updated schedule - std::multimap<boost::chrono::system_clock::time_point, Function> temp_queue; + std::multimap<std::chrono::system_clock::time_point, Function> temp_queue; for (const auto& element : taskQueue) { temp_queue.emplace_hint(temp_queue.cend(), element.first - delta_seconds, element.second); @@ -147,10 +129,10 @@ void CScheduler::scheduleEvery(CScheduler::Function f, int64_t deltaMilliSeconds scheduleFromNow(std::bind(&Repeat, this, f, deltaMilliSeconds), deltaMilliSeconds); } -size_t CScheduler::getQueueInfo(boost::chrono::system_clock::time_point &first, - boost::chrono::system_clock::time_point &last) const +size_t CScheduler::getQueueInfo(std::chrono::system_clock::time_point &first, + std::chrono::system_clock::time_point &last) const { - boost::unique_lock<boost::mutex> lock(newTaskMutex); + LOCK(newTaskMutex); size_t result = taskQueue.size(); if (!taskQueue.empty()) { first = taskQueue.begin()->first; @@ -160,7 +142,7 @@ size_t CScheduler::getQueueInfo(boost::chrono::system_clock::time_point &first, } bool CScheduler::AreThreadsServicingQueue() const { - boost::unique_lock<boost::mutex> lock(newTaskMutex); + LOCK(newTaskMutex); return nThreadsServicingQueue; } @@ -174,7 +156,7 @@ void SingleThreadedSchedulerClient::MaybeScheduleProcessQueue() { if (m_are_callbacks_running) return; if (m_callbacks_pending.empty()) return; } - m_pscheduler->schedule(std::bind(&SingleThreadedSchedulerClient::ProcessQueue, this)); + m_pscheduler->schedule(std::bind(&SingleThreadedSchedulerClient::ProcessQueue, this), std::chrono::system_clock::now()); } void SingleThreadedSchedulerClient::ProcessQueue() { diff --git a/src/scheduler.h b/src/scheduler.h index d18be0ea5e..4d5aa3068e 100644 --- a/src/scheduler.h +++ b/src/scheduler.h @@ -7,11 +7,12 @@ // // NOTE: -// boost::thread / boost::chrono should be ported to std::thread / std::chrono +// boost::thread should be ported to std::thread // when we support C++11. // -#include <boost/chrono/chrono.hpp> -#include <boost/thread.hpp> +#include <condition_variable> +#include <functional> +#include <list> #include <map> #include <sync.h> @@ -27,8 +28,8 @@ // s->scheduleFromNow(std::bind(Class::func, this, argument), 3); // boost::thread* t = new boost::thread(std::bind(CScheduler::serviceQueue, s)); // -// ... then at program shutdown, clean up the thread running serviceQueue: -// t->interrupt(); +// ... then at program shutdown, make sure to call stop() to clean up the thread(s) running serviceQueue: +// s->stop(); // t->join(); // delete t; // delete s; // Must be done after thread is interrupted/joined. @@ -43,7 +44,7 @@ public: typedef std::function<void()> Function; // Call func at/after time t - void schedule(Function f, boost::chrono::system_clock::time_point t=boost::chrono::system_clock::now()); + void schedule(Function f, std::chrono::system_clock::time_point t); // Convenience method: call f once deltaMilliSeconds from now void scheduleFromNow(Function f, int64_t deltaMilliSeconds); @@ -60,7 +61,7 @@ public: * Iterates through items on taskQueue and reschedules them * to be delta_seconds sooner. */ - void MockForward(boost::chrono::seconds delta_seconds); + void MockForward(std::chrono::seconds delta_seconds); // To keep things as simple as possible, there is no unschedule. @@ -75,20 +76,20 @@ public: // Returns number of tasks waiting to be serviced, // and first and last task times - size_t getQueueInfo(boost::chrono::system_clock::time_point &first, - boost::chrono::system_clock::time_point &last) const; + size_t getQueueInfo(std::chrono::system_clock::time_point &first, + std::chrono::system_clock::time_point &last) const; // Returns true if there are threads actively running in serviceQueue() bool AreThreadsServicingQueue() const; private: - std::multimap<boost::chrono::system_clock::time_point, Function> taskQueue; - boost::condition_variable newTaskScheduled; - mutable boost::mutex newTaskMutex; - int nThreadsServicingQueue; - bool stopRequested; - bool stopWhenEmpty; - bool shouldStop() const { return stopRequested || (stopWhenEmpty && taskQueue.empty()); } + mutable Mutex newTaskMutex; + std::condition_variable newTaskScheduled; + std::multimap<std::chrono::system_clock::time_point, Function> taskQueue GUARDED_BY(newTaskMutex); + int nThreadsServicingQueue GUARDED_BY(newTaskMutex); + bool stopRequested GUARDED_BY(newTaskMutex); + bool stopWhenEmpty GUARDED_BY(newTaskMutex); + bool shouldStop() const EXCLUSIVE_LOCKS_REQUIRED(newTaskMutex) { return stopRequested || (stopWhenEmpty && taskQueue.empty()); } }; /** diff --git a/src/script/sign.cpp b/src/script/sign.cpp index 58eae3ce96..fe8292fe57 100644 --- a/src/script/sign.cpp +++ b/src/script/sign.cpp @@ -465,3 +465,54 @@ bool IsSegWitOutput(const SigningProvider& provider, const CScript& script) } return false; } + +bool SignTransaction(CMutableTransaction& mtx, const SigningProvider* keystore, const std::map<COutPoint, Coin>& coins, int nHashType, std::map<int, std::string>& input_errors) +{ + bool fHashSingle = ((nHashType & ~SIGHASH_ANYONECANPAY) == SIGHASH_SINGLE); + + // Use CTransaction for the constant parts of the + // transaction to avoid rehashing. + const CTransaction txConst(mtx); + // Sign what we can: + for (unsigned int i = 0; i < mtx.vin.size(); i++) { + CTxIn& txin = mtx.vin[i]; + auto coin = coins.find(txin.prevout); + if (coin == coins.end() || coin->second.IsSpent()) { + input_errors[i] = "Input not found or already spent"; + continue; + } + const CScript& prevPubKey = coin->second.out.scriptPubKey; + const CAmount& amount = coin->second.out.nValue; + + SignatureData sigdata = DataFromTransaction(mtx, i, coin->second.out); + // Only sign SIGHASH_SINGLE if there's a corresponding output: + if (!fHashSingle || (i < mtx.vout.size())) { + ProduceSignature(*keystore, MutableTransactionSignatureCreator(&mtx, i, amount, nHashType), prevPubKey, sigdata); + } + + UpdateInput(txin, sigdata); + + // amount must be specified for valid segwit signature + if (amount == MAX_MONEY && !txin.scriptWitness.IsNull()) { + input_errors[i] = "Missing amount"; + continue; + } + + ScriptError serror = SCRIPT_ERR_OK; + if (!VerifyScript(txin.scriptSig, prevPubKey, &txin.scriptWitness, STANDARD_SCRIPT_VERIFY_FLAGS, TransactionSignatureChecker(&txConst, i, amount), &serror)) { + if (serror == SCRIPT_ERR_INVALID_STACK_OPERATION) { + // Unable to sign input and verification failed (possible attempt to partially sign). + input_errors[i] = "Unable to sign input, invalid stack size (possibly missing key)"; + } else if (serror == SCRIPT_ERR_SIG_NULLFAIL) { + // Verification failed (possibly due to insufficient signatures). + input_errors[i] = "CHECK(MULTI)SIG failing with non-zero signature (possibly need more signatures)"; + } else { + input_errors[i] = ScriptErrorString(serror); + } + } else { + // If this input succeeds, make sure there is no error set for it + input_errors.erase(i); + } + } + return input_errors.empty(); +} diff --git a/src/script/sign.h b/src/script/sign.h index 033c9ba19e..f03af0713f 100644 --- a/src/script/sign.h +++ b/src/script/sign.h @@ -6,6 +6,7 @@ #ifndef BITCOIN_SCRIPT_SIGN_H #define BITCOIN_SCRIPT_SIGN_H +#include <coins.h> #include <hash.h> #include <pubkey.h> #include <script/interpreter.h> @@ -168,4 +169,7 @@ bool IsSolvable(const SigningProvider& provider, const CScript& script); /** Check whether a scriptPubKey is known to be segwit. */ bool IsSegWitOutput(const SigningProvider& provider, const CScript& script); +/** Sign the CMutableTransaction */ +bool SignTransaction(CMutableTransaction& mtx, const SigningProvider* provider, const std::map<COutPoint, Coin>& coins, int sighash, std::map<int, std::string>& input_errors); + #endif // BITCOIN_SCRIPT_SIGN_H diff --git a/src/sync.cpp b/src/sync.cpp index 924e7b5bb0..71657a7439 100644 --- a/src/sync.cpp +++ b/src/sync.cpp @@ -13,7 +13,7 @@ #include <util/strencodings.h> #include <util/threadnames.h> - +#include <system_error> #include <map> #include <set> @@ -60,6 +60,11 @@ struct CLockLocation { mutexName, sourceFile, itostr(sourceLine), (fTry ? " (TRY)" : ""), m_thread_name); } + std::string Name() const + { + return mutexName; + } + private: bool fTry; std::string mutexName; @@ -155,6 +160,18 @@ void EnterCritical(const char* pszName, const char* pszFile, int nLine, void* cs push_lock(cs, CLockLocation(pszName, pszFile, nLine, fTry, util::ThreadGetInternalName())); } +void CheckLastCritical(void* cs, std::string& lockname, const char* guardname, const char* file, int line) +{ + if (!g_lockstack.empty()) { + const auto& lastlock = g_lockstack.back(); + if (lastlock.first == cs) { + lockname = lastlock.second.Name(); + return; + } + } + throw std::system_error(EPERM, std::generic_category(), strprintf("%s:%s %s was not most recent critical section locked", file, line, guardname)); +} + void LeaveCritical() { pop_lock(); diff --git a/src/sync.h b/src/sync.h index 0cdbb59c70..204734c273 100644 --- a/src/sync.h +++ b/src/sync.h @@ -50,6 +50,7 @@ LEAVE_CRITICAL_SECTION(mutex); // no RAII #ifdef DEBUG_LOCKORDER void EnterCritical(const char* pszName, const char* pszFile, int nLine, void* cs, bool fTry = false); void LeaveCritical(); +void CheckLastCritical(void* cs, std::string& lockname, const char* guardname, const char* file, int line); std::string LocksHeld(); void AssertLockHeldInternal(const char* pszName, const char* pszFile, int nLine, void* cs) ASSERT_EXCLUSIVE_LOCK(cs); void AssertLockNotHeldInternal(const char* pszName, const char* pszFile, int nLine, void* cs); @@ -64,6 +65,7 @@ extern bool g_debug_lockorder_abort; #else void static inline EnterCritical(const char* pszName, const char* pszFile, int nLine, void* cs, bool fTry = false) {} void static inline LeaveCritical() {} +void static inline CheckLastCritical(void* cs, std::string& lockname, const char* guardname, const char* file, int line) {} void static inline AssertLockHeldInternal(const char* pszName, const char* pszFile, int nLine, void* cs) ASSERT_EXCLUSIVE_LOCK(cs) {} void static inline AssertLockNotHeldInternal(const char* pszName, const char* pszFile, int nLine, void* cs) {} void static inline DeleteLock(void* cs) {} @@ -171,8 +173,45 @@ public: { return Base::owns_lock(); } + +protected: + // needed for reverse_lock + UniqueLock() { } + +public: + /** + * An RAII-style reverse lock. Unlocks on construction and locks on destruction. + */ + class reverse_lock { + public: + explicit reverse_lock(UniqueLock& _lock, const char* _guardname, const char* _file, int _line) : lock(_lock), file(_file), line(_line) { + CheckLastCritical((void*)lock.mutex(), lockname, _guardname, _file, _line); + lock.unlock(); + LeaveCritical(); + lock.swap(templock); + } + + ~reverse_lock() { + templock.swap(lock); + EnterCritical(lockname.c_str(), file.c_str(), line, (void*)lock.mutex()); + lock.lock(); + } + + private: + reverse_lock(reverse_lock const&); + reverse_lock& operator=(reverse_lock const&); + + UniqueLock& lock; + UniqueLock templock; + std::string lockname; + const std::string file; + const int line; + }; + friend class reverse_lock; }; +#define REVERSE_LOCK(g) decltype(g)::reverse_lock PASTE2(revlock, __COUNTER__)(g, #g, __FILE__, __LINE__) + template<typename MutexArg> using DebugLock = UniqueLock<typename std::remove_reference<typename std::remove_pointer<MutexArg>::type>::type>; diff --git a/src/test/fuzz/bloom_filter.cpp b/src/test/fuzz/bloom_filter.cpp index b78744d9df..d1112f8e62 100644 --- a/src/test/fuzz/bloom_filter.cpp +++ b/src/test/fuzz/bloom_filter.cpp @@ -27,7 +27,7 @@ void test_one_input(const std::vector<uint8_t>& buffer) while (fuzzed_data_provider.remaining_bytes() > 0) { switch (fuzzed_data_provider.ConsumeIntegralInRange(0, 6)) { case 0: { - const std::vector<unsigned char>& b = ConsumeRandomLengthByteVector(fuzzed_data_provider); + const std::vector<unsigned char> b = ConsumeRandomLengthByteVector(fuzzed_data_provider); (void)bloom_filter.contains(b); bloom_filter.insert(b); const bool present = bloom_filter.contains(b); diff --git a/src/test/fuzz/hex.cpp b/src/test/fuzz/hex.cpp index 54693180be..2de6100d7b 100644 --- a/src/test/fuzz/hex.cpp +++ b/src/test/fuzz/hex.cpp @@ -2,8 +2,12 @@ // Distributed under the MIT software license, see the accompanying // file COPYING or http://www.opensource.org/licenses/mit-license.php. +#include <core_io.h> +#include <primitives/block.h> +#include <rpc/util.h> #include <test/fuzz/fuzz.h> - +#include <uint256.h> +#include <univalue.h> #include <util/strencodings.h> #include <cassert> @@ -19,4 +23,14 @@ void test_one_input(const std::vector<uint8_t>& buffer) if (IsHex(random_hex_string)) { assert(ToLower(random_hex_string) == hex_data); } + (void)IsHexNumber(random_hex_string); + uint256 result; + (void)ParseHashStr(random_hex_string, result); + (void)uint256S(random_hex_string); + try { + (void)HexToPubKey(random_hex_string); + } catch (const UniValue&) { + } + CBlockHeader block_header; + (void)DecodeHexBlockHeader(block_header, random_hex_string); } diff --git a/src/test/fuzz/integer.cpp b/src/test/fuzz/integer.cpp index 2d47c631cb..350d3d3331 100644 --- a/src/test/fuzz/integer.cpp +++ b/src/test/fuzz/integer.cpp @@ -23,6 +23,7 @@ #include <test/fuzz/FuzzedDataProvider.h> #include <test/fuzz/fuzz.h> #include <uint256.h> +#include <util/moneystr.h> #include <util/strencodings.h> #include <util/system.h> #include <util/time.h> @@ -76,11 +77,19 @@ void test_one_input(const std::vector<uint8_t>& buffer) (void)DecompressAmount(u64); (void)FormatISO8601Date(i64); (void)FormatISO8601DateTime(i64); + // FormatMoney(i) not defined when i == std::numeric_limits<int64_t>::min() + if (i64 != std::numeric_limits<int64_t>::min()) { + int64_t parsed_money; + if (ParseMoney(FormatMoney(i64), parsed_money)) { + assert(parsed_money == i64); + } + } (void)GetSizeOfCompactSize(u64); (void)GetSpecialScriptSize(u32); // (void)GetVirtualTransactionSize(i64, i64); // function defined only for a subset of int64_t inputs // (void)GetVirtualTransactionSize(i64, i64, u32); // function defined only for a subset of int64_t/uint32_t inputs (void)HexDigit(ch); + (void)MoneyRange(i64); (void)i64tostr(i64); (void)IsDigit(ch); (void)IsSpace(ch); @@ -106,6 +115,14 @@ void test_one_input(const std::vector<uint8_t>& buffer) (void)SipHashUint256(u64, u64, u256); (void)SipHashUint256Extra(u64, u64, u256, u32); (void)ToLower(ch); + (void)ToUpper(ch); + // ValueFromAmount(i) not defined when i == std::numeric_limits<int64_t>::min() + if (i64 != std::numeric_limits<int64_t>::min()) { + int64_t parsed_money; + if (ParseMoney(ValueFromAmount(i64).getValStr(), parsed_money)) { + assert(parsed_money == i64); + } + } const arith_uint256 au256 = UintToArith256(u256); assert(ArithToUint256(au256) == u256); diff --git a/src/test/fuzz/key_io.cpp b/src/test/fuzz/key_io.cpp new file mode 100644 index 0000000000..f44628b0f8 --- /dev/null +++ b/src/test/fuzz/key_io.cpp @@ -0,0 +1,48 @@ +// Copyright (c) 2020 The Bitcoin Core developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include <chainparams.h> +#include <key_io.h> +#include <rpc/util.h> +#include <script/signingprovider.h> +#include <script/standard.h> +#include <test/fuzz/fuzz.h> + +#include <cassert> +#include <cstdint> +#include <string> +#include <vector> + +void initialize() +{ + SelectParams(CBaseChainParams::REGTEST); +} + +void test_one_input(const std::vector<uint8_t>& buffer) +{ + const std::string random_string(buffer.begin(), buffer.end()); + + const CKey key = DecodeSecret(random_string); + if (key.IsValid()) { + assert(key == DecodeSecret(EncodeSecret(key))); + } + + const CExtKey ext_key = DecodeExtKey(random_string); + if (ext_key.key.size() == 32) { + assert(ext_key == DecodeExtKey(EncodeExtKey(ext_key))); + } + + const CExtPubKey ext_pub_key = DecodeExtPubKey(random_string); + if (ext_pub_key.pubkey.size() == CPubKey::COMPRESSED_SIZE) { + assert(ext_pub_key == DecodeExtPubKey(EncodeExtPubKey(ext_pub_key))); + } + + const CTxDestination tx_destination = DecodeDestination(random_string); + (void)DescribeAddress(tx_destination); + (void)GetKeyForDestination(/* store */ {}, tx_destination); + (void)GetScriptForDestination(tx_destination); + (void)IsValidDestination(tx_destination); + + (void)IsValidDestinationString(random_string); +} diff --git a/src/test/fuzz/locale.cpp b/src/test/fuzz/locale.cpp new file mode 100644 index 0000000000..c8288123e8 --- /dev/null +++ b/src/test/fuzz/locale.cpp @@ -0,0 +1,96 @@ +// Copyright (c) 2020 The Bitcoin Core developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include <test/fuzz/fuzz.h> +#include <test/fuzz/FuzzedDataProvider.h> +#include <tinyformat.h> +#include <util/strencodings.h> + +#include <cassert> +#include <clocale> +#include <cstdint> +#include <locale> +#include <string> +#include <vector> + +namespace { +const std::string locale_identifiers[] = { + "C", "C.UTF-8", "aa_DJ", "aa_DJ.ISO-8859-1", "aa_DJ.UTF-8", "aa_ER", "aa_ER.UTF-8", "aa_ET", "aa_ET.UTF-8", "af_ZA", "af_ZA.ISO-8859-1", "af_ZA.UTF-8", "agr_PE", "agr_PE.UTF-8", "ak_GH", "ak_GH.UTF-8", "am_ET", "am_ET.UTF-8", "an_ES", "an_ES.ISO-8859-15", "an_ES.UTF-8", "anp_IN", "anp_IN.UTF-8", "ar_AE", "ar_AE.ISO-8859-6", "ar_AE.UTF-8", "ar_BH", "ar_BH.ISO-8859-6", "ar_BH.UTF-8", "ar_DZ", "ar_DZ.ISO-8859-6", "ar_DZ.UTF-8", "ar_EG", "ar_EG.ISO-8859-6", "ar_EG.UTF-8", "ar_IN", "ar_IN.UTF-8", "ar_IQ", "ar_IQ.ISO-8859-6", "ar_IQ.UTF-8", "ar_JO", "ar_JO.ISO-8859-6", "ar_JO.UTF-8", "ar_KW", "ar_KW.ISO-8859-6", "ar_KW.UTF-8", "ar_LB", "ar_LB.ISO-8859-6", "ar_LB.UTF-8", "ar_LY", "ar_LY.ISO-8859-6", "ar_LY.UTF-8", "ar_MA", "ar_MA.ISO-8859-6", "ar_MA.UTF-8", "ar_OM", "ar_OM.ISO-8859-6", "ar_OM.UTF-8", "ar_QA", "ar_QA.ISO-8859-6", "ar_QA.UTF-8", "ar_SA", "ar_SA.ISO-8859-6", "ar_SA.UTF-8", "ar_SD", "ar_SD.ISO-8859-6", "ar_SD.UTF-8", "ar_SS", "ar_SS.UTF-8", "ar_SY", "ar_SY.ISO-8859-6", "ar_SY.UTF-8", "ar_TN", "ar_TN.ISO-8859-6", "ar_TN.UTF-8", "ar_YE", "ar_YE.ISO-8859-6", "ar_YE.UTF-8", "as_IN", "as_IN.UTF-8", "ast_ES", "ast_ES.ISO-8859-15", "ast_ES.UTF-8", "ayc_PE", "ayc_PE.UTF-8", "az_AZ", "az_AZ.UTF-8", "az_IR", "az_IR.UTF-8", "be_BY", "be_BY.CP1251", "be_BY.UTF-8", "bem_ZM", "bem_ZM.UTF-8", "ber_DZ", "ber_DZ.UTF-8", "ber_MA", "ber_MA.UTF-8", "bg_BG", "bg_BG.CP1251", "bg_BG.UTF-8", "bho_IN", "bho_IN.UTF-8", "bho_NP", "bho_NP.UTF-8", "bi_VU", "bi_VU.UTF-8", "bn_BD", "bn_BD.UTF-8", "bn_IN", "bn_IN.UTF-8", "bo_CN", "bo_CN.UTF-8", "bo_IN", "bo_IN.UTF-8", "br_FR", "br_FR.ISO-8859-1", "br_FR.UTF-8", "brx_IN", "brx_IN.UTF-8", "bs_BA", "bs_BA.ISO-8859-2", "bs_BA.UTF-8", "byn_ER", "byn_ER.UTF-8", "ca_AD", "ca_AD.ISO-8859-15", "ca_AD.UTF-8", "ca_ES", "ca_ES.ISO-8859-1", "ca_ES.UTF-8", "ca_FR", "ca_FR.ISO-8859-15", "ca_FR.UTF-8", "ca_IT", "ca_IT.ISO-8859-15", "ca_IT.UTF-8", "ce_RU", "ce_RU.UTF-8", "chr_US", "chr_US.UTF-8", "ckb_IQ", "ckb_IQ.UTF-8", "cmn_TW", "cmn_TW.UTF-8", "crh_UA", "crh_UA.UTF-8", "csb_PL", "csb_PL.UTF-8", "cs_CZ", "cs_CZ.ISO-8859-2", "cs_CZ.UTF-8", "cv_RU", "cv_RU.UTF-8", "cy_GB", "cy_GB.ISO-8859-14", "cy_GB.UTF-8", "da_DK", "da_DK.ISO-8859-1", "da_DK.UTF-8", "de_AT", "de_AT.ISO-8859-1", "de_AT.UTF-8", "de_BE", "de_BE.ISO-8859-1", "de_BE.UTF-8", "de_CH", "de_CH.ISO-8859-1", "de_CH.UTF-8", "de_DE", "de_DE.ISO-8859-1", "de_DE.UTF-8", "de_IT", "de_IT.ISO-8859-1", "de_IT.UTF-8", "de_LU", "de_LU.ISO-8859-1", "de_LU.UTF-8", "doi_IN", "doi_IN.UTF-8", "dv_MV", "dv_MV.UTF-8", "dz_BT", "dz_BT.UTF-8", "el_CY", "el_CY.ISO-8859-7", "el_CY.UTF-8", "el_GR", "el_GR.ISO-8859-7", "el_GR.UTF-8", "en_AG", "en_AG.UTF-8", "en_AU", "en_AU.ISO-8859-1", "en_AU.UTF-8", "en_BW", "en_BW.ISO-8859-1", "en_BW.UTF-8", "en_CA", "en_CA.ISO-8859-1", "en_CA.UTF-8", "en_DK", "en_DK.ISO-8859-1", "en_DK.ISO-8859-15", "en_DK.UTF-8", "en_GB", "en_GB.ISO-8859-1", "en_GB.ISO-8859-15", "en_GB.UTF-8", "en_HK", "en_HK.ISO-8859-1", "en_HK.UTF-8", "en_IE", "en_IE.ISO-8859-1", "en_IE.UTF-8", "en_IL", "en_IL.UTF-8", "en_IN", "en_IN.UTF-8", "en_NG", "en_NG.UTF-8", "en_NZ", "en_NZ.ISO-8859-1", "en_NZ.UTF-8", "en_PH", "en_PH.ISO-8859-1", "en_PH.UTF-8", "en_SG", "en_SG.ISO-8859-1", "en_SG.UTF-8", "en_US", "en_US.ISO-8859-1", "en_US.ISO-8859-15", "en_US.UTF-8", "en_ZA", "en_ZA.ISO-8859-1", "en_ZA.UTF-8", "en_ZM", "en_ZM.UTF-8", "en_ZW", "en_ZW.ISO-8859-1", "en_ZW.UTF-8", "es_AR", "es_AR.ISO-8859-1", "es_AR.UTF-8", "es_BO", "es_BO.ISO-8859-1", "es_BO.UTF-8", "es_CL", "es_CL.ISO-8859-1", "es_CL.UTF-8", "es_CO", "es_CO.ISO-8859-1", "es_CO.UTF-8", "es_CR", "es_CR.ISO-8859-1", "es_CR.UTF-8", "es_CU", "es_CU.UTF-8", "es_DO", "es_DO.ISO-8859-1", "es_DO.UTF-8", "es_EC", "es_EC.ISO-8859-1", "es_EC.UTF-8", "es_ES", "es_ES.ISO-8859-1", "es_ES.UTF-8", "es_GT", "es_GT.ISO-8859-1", "es_GT.UTF-8", "es_HN", "es_HN.ISO-8859-1", "es_HN.UTF-8", "es_MX", "es_MX.ISO-8859-1", "es_MX.UTF-8", "es_NI", "es_NI.ISO-8859-1", "es_NI.UTF-8", "es_PA", "es_PA.ISO-8859-1", "es_PA.UTF-8", "es_PE", "es_PE.ISO-8859-1", "es_PE.UTF-8", "es_PR", "es_PR.ISO-8859-1", "es_PR.UTF-8", "es_PY", "es_PY.ISO-8859-1", "es_PY.UTF-8", "es_SV", "es_SV.ISO-8859-1", "es_SV.UTF-8", "es_US", "es_US.ISO-8859-1", "es_US.UTF-8", "es_UY", "es_UY.ISO-8859-1", "es_UY.UTF-8", "es_VE", "es_VE.ISO-8859-1", "es_VE.UTF-8", "et_EE", "et_EE.ISO-8859-1", "et_EE.ISO-8859-15", "et_EE.UTF-8", "eu_ES", "eu_ES.ISO-8859-1", "eu_ES.UTF-8", "eu_FR", "eu_FR.ISO-8859-1", "eu_FR.UTF-8", "fa_IR", "fa_IR.UTF-8", "ff_SN", "ff_SN.UTF-8", "fi_FI", "fi_FI.ISO-8859-1", "fi_FI.UTF-8", "fil_PH", "fil_PH.UTF-8", "fo_FO", "fo_FO.ISO-8859-1", "fo_FO.UTF-8", "fr_BE", "fr_BE.ISO-8859-1", "fr_BE.UTF-8", "fr_CA", "fr_CA.ISO-8859-1", "fr_CA.UTF-8", "fr_CH", "fr_CH.ISO-8859-1", "fr_CH.UTF-8", "fr_FR", "fr_FR.ISO-8859-1", "fr_FR.UTF-8", "fr_LU", "fr_LU.ISO-8859-1", "fr_LU.UTF-8", "fur_IT", "fur_IT.UTF-8", "fy_DE", "fy_DE.UTF-8", "fy_NL", "fy_NL.UTF-8", "ga_IE", "ga_IE.ISO-8859-1", "ga_IE.UTF-8", "gd_GB", "gd_GB.ISO-8859-15", "gd_GB.UTF-8", "gez_ER", "gez_ER.UTF-8", "gez_ET", "gez_ET.UTF-8", "gl_ES", "gl_ES.ISO-8859-1", "gl_ES.UTF-8", "gu_IN", "gu_IN.UTF-8", "gv_GB", "gv_GB.ISO-8859-1", "gv_GB.UTF-8", "hak_TW", "hak_TW.UTF-8", "ha_NG", "ha_NG.UTF-8", "he_IL", "he_IL.ISO-8859-8", "he_IL.UTF-8", "hif_FJ", "hif_FJ.UTF-8", "hi_IN", "hi_IN.UTF-8", "hne_IN", "hne_IN.UTF-8", "hr_HR", "hr_HR.ISO-8859-2", "hr_HR.UTF-8", "hsb_DE", "hsb_DE.ISO-8859-2", "hsb_DE.UTF-8", "ht_HT", "ht_HT.UTF-8", "hu_HU", "hu_HU.ISO-8859-2", "hu_HU.UTF-8", "hy_AM", "hy_AM.ARMSCII-8", "hy_AM.UTF-8", "ia_FR", "ia_FR.UTF-8", "id_ID", "id_ID.ISO-8859-1", "id_ID.UTF-8", "ig_NG", "ig_NG.UTF-8", "ik_CA", "ik_CA.UTF-8", "is_IS", "is_IS.ISO-8859-1", "is_IS.UTF-8", "it_CH", "it_CH.ISO-8859-1", "it_CH.UTF-8", "it_IT", "it_IT.ISO-8859-1", "it_IT.UTF-8", "iu_CA", "iu_CA.UTF-8", "kab_DZ", "kab_DZ.UTF-8", "ka_GE", "ka_GE.GEORGIAN-PS", "ka_GE.UTF-8", "kk_KZ", "kk_KZ.PT154", "kk_KZ.RK1048", "kk_KZ.UTF-8", "kl_GL", "kl_GL.ISO-8859-1", "kl_GL.UTF-8", "km_KH", "km_KH.UTF-8", "kn_IN", "kn_IN.UTF-8", "kok_IN", "kok_IN.UTF-8", "ks_IN", "ks_IN.UTF-8", "ku_TR", "ku_TR.ISO-8859-9", "ku_TR.UTF-8", "kw_GB", "kw_GB.ISO-8859-1", "kw_GB.UTF-8", "ky_KG", "ky_KG.UTF-8", "lb_LU", "lb_LU.UTF-8", "lg_UG", "lg_UG.ISO-8859-10", "lg_UG.UTF-8", "li_BE", "li_BE.UTF-8", "lij_IT", "lij_IT.UTF-8", "li_NL", "li_NL.UTF-8", "ln_CD", "ln_CD.UTF-8", "lo_LA", "lo_LA.UTF-8", "lt_LT", "lt_LT.ISO-8859-13", "lt_LT.UTF-8", "lv_LV", "lv_LV.ISO-8859-13", "lv_LV.UTF-8", "lzh_TW", "lzh_TW.UTF-8", "mag_IN", "mag_IN.UTF-8", "mai_IN", "mai_IN.UTF-8", "mai_NP", "mai_NP.UTF-8", "mfe_MU", "mfe_MU.UTF-8", "mg_MG", "mg_MG.ISO-8859-15", "mg_MG.UTF-8", "mhr_RU", "mhr_RU.UTF-8", "mi_NZ", "mi_NZ.ISO-8859-13", "mi_NZ.UTF-8", "miq_NI", "miq_NI.UTF-8", "mjw_IN", "mjw_IN.UTF-8", "mk_MK", "mk_MK.ISO-8859-5", "mk_MK.UTF-8", "ml_IN", "ml_IN.UTF-8", "mni_IN", "mni_IN.UTF-8", "mn_MN", "mn_MN.UTF-8", "mr_IN", "mr_IN.UTF-8", "ms_MY", "ms_MY.ISO-8859-1", "ms_MY.UTF-8", "mt_MT", "mt_MT.ISO-8859-3", "mt_MT.UTF-8", "my_MM", "my_MM.UTF-8", "nan_TW", "nan_TW.UTF-8", "nb_NO", "nb_NO.ISO-8859-1", "nb_NO.UTF-8", "nds_DE", "nds_DE.UTF-8", "nds_NL", "nds_NL.UTF-8", "ne_NP", "ne_NP.UTF-8", "nhn_MX", "nhn_MX.UTF-8", "niu_NU", "niu_NU.UTF-8", "niu_NZ", "niu_NZ.UTF-8", "nl_AW", "nl_AW.UTF-8", "nl_BE", "nl_BE.ISO-8859-1", "nl_BE.UTF-8", "nl_NL", "nl_NL.ISO-8859-1", "nl_NL.UTF-8", "nn_NO", "nn_NO.ISO-8859-1", "nn_NO.UTF-8", "nr_ZA", "nr_ZA.UTF-8", "nso_ZA", "nso_ZA.UTF-8", "oc_FR", "oc_FR.ISO-8859-1", "oc_FR.UTF-8", "om_ET", "om_ET.UTF-8", "om_KE", "om_KE.ISO-8859-1", "om_KE.UTF-8", "or_IN", "or_IN.UTF-8", "os_RU", "os_RU.UTF-8", "pa_IN", "pa_IN.UTF-8", "pap_AW", "pap_AW.UTF-8", "pap_CW", "pap_CW.UTF-8", "pa_PK", "pa_PK.UTF-8", "pl_PL", "pl_PL.ISO-8859-2", "pl_PL.UTF-8", "ps_AF", "ps_AF.UTF-8", "pt_BR", "pt_BR.ISO-8859-1", "pt_BR.UTF-8", "pt_PT", "pt_PT.ISO-8859-1", "pt_PT.UTF-8", "quz_PE", "quz_PE.UTF-8", "raj_IN", "raj_IN.UTF-8", "ro_RO", "ro_RO.ISO-8859-2", "ro_RO.UTF-8", "ru_RU", "ru_RU.CP1251", "ru_RU.ISO-8859-5", "ru_RU.KOI8-R", "ru_RU.UTF-8", "ru_UA", "ru_UA.KOI8-U", "ru_UA.UTF-8", "rw_RW", "rw_RW.UTF-8", "sa_IN", "sa_IN.UTF-8", "sat_IN", "sat_IN.UTF-8", "sc_IT", "sc_IT.UTF-8", "sd_IN", "sd_IN.UTF-8", "sd_PK", "sd_PK.UTF-8", "se_NO", "se_NO.UTF-8", "sgs_LT", "sgs_LT.UTF-8", "shn_MM", "shn_MM.UTF-8", "shs_CA", "shs_CA.UTF-8", "sid_ET", "sid_ET.UTF-8", "si_LK", "si_LK.UTF-8", "sk_SK", "sk_SK.ISO-8859-2", "sk_SK.UTF-8", "sl_SI", "sl_SI.ISO-8859-2", "sl_SI.UTF-8", "sm_WS", "sm_WS.UTF-8", "so_DJ", "so_DJ.ISO-8859-1", "so_DJ.UTF-8", "so_ET", "so_ET.UTF-8", "so_KE", "so_KE.ISO-8859-1", "so_KE.UTF-8", "so_SO", "so_SO.ISO-8859-1", "so_SO.UTF-8", "sq_AL", "sq_AL.ISO-8859-1", "sq_AL.UTF-8", "sq_MK", "sq_MK.UTF-8", "sr_ME", "sr_ME.UTF-8", "sr_RS", "sr_RS.UTF-8", "ss_ZA", "ss_ZA.UTF-8", "st_ZA", "st_ZA.ISO-8859-1", "st_ZA.UTF-8", "sv_FI", "sv_FI.ISO-8859-1", "sv_FI.UTF-8", "sv_SE", "sv_SE.ISO-8859-1", "sv_SE.ISO-8859-15", "sv_SE.UTF-8", "sw_KE", "sw_KE.UTF-8", "sw_TZ", "sw_TZ.UTF-8", "szl_PL", "szl_PL.UTF-8", "ta_IN", "ta_IN.UTF-8", "ta_LK", "ta_LK.UTF-8", "te_IN", "te_IN.UTF-8", "tg_TJ", "tg_TJ.KOI8-T", "tg_TJ.UTF-8", "the_NP", "the_NP.UTF-8", "th_TH", "th_TH.TIS-620", "th_TH.UTF-8", "ti_ER", "ti_ER.UTF-8", "ti_ET", "ti_ET.UTF-8", "tig_ER", "tig_ER.UTF-8", "tk_TM", "tk_TM.UTF-8", "tl_PH", "tl_PH.ISO-8859-1", "tl_PH.UTF-8", "tn_ZA", "tn_ZA.UTF-8", "to_TO", "to_TO.UTF-8", "tpi_PG", "tpi_PG.UTF-8", "tr_CY", "tr_CY.ISO-8859-9", "tr_CY.UTF-8", "tr_TR", "tr_TR.ISO-8859-9", "tr_TR.UTF-8", "ts_ZA", "ts_ZA.UTF-8", "tt_RU", "tt_RU.UTF-8", "ug_CN", "ug_CN.UTF-8", "uk_UA", "uk_UA.KOI8-U", "uk_UA.UTF-8", "unm_US", "unm_US.UTF-8", "ur_IN", "ur_IN.UTF-8", "ur_PK", "ur_PK.UTF-8", "uz_UZ", "uz_UZ.ISO-8859-1", "uz_UZ.UTF-8", "ve_ZA", "ve_ZA.UTF-8", "vi_VN", "vi_VN.UTF-8", "wa_BE", "wa_BE.ISO-8859-1", "wa_BE.UTF-8", "wae_CH", "wae_CH.UTF-8", "wal_ET", "wal_ET.UTF-8", "wo_SN", "wo_SN.UTF-8", "xh_ZA", "xh_ZA.ISO-8859-1", "xh_ZA.UTF-8", "yi_US", "yi_US.CP1255", "yi_US.UTF-8", "yo_NG", "yo_NG.UTF-8", "yue_HK", "yue_HK.UTF-8", "yuw_PG", "yuw_PG.UTF-8", "zh_CN", "zh_CN.GB18030", "zh_CN.GB2312", "zh_CN.GBK", "zh_CN.UTF-8", "zh_HK", "zh_HK.BIG5-HKSCS", "zh_HK.UTF-8", "zh_SG", "zh_SG.GB2312", "zh_SG.GBK", "zh_SG.UTF-8", "zh_TW", "zh_TW.BIG5", "zh_TW.EUC-TW", "zh_TW.UTF-8", "zu_ZA", "zu_ZA.ISO-8859-1", "zu_ZA.UTF-8" +}; + +std::string ConsumeLocaleIdentifier(FuzzedDataProvider& fuzzed_data_provider) +{ + return fuzzed_data_provider.PickValueInArray<std::string>(locale_identifiers); +} + +bool IsAvailableLocale(const std::string& locale_identifier) +{ + try { + (void)std::locale(locale_identifier); + } catch (const std::runtime_error&) { + return false; + } + return true; +} +} // namespace + +void test_one_input(const std::vector<uint8_t>& buffer) +{ + FuzzedDataProvider fuzzed_data_provider(buffer.data(), buffer.size()); + const std::string locale_identifier = ConsumeLocaleIdentifier(fuzzed_data_provider); + if (!IsAvailableLocale(locale_identifier)) { + return; + } + const char* c_locale = std::setlocale(LC_ALL, "C"); + assert(c_locale != nullptr); + + const std::string random_string = fuzzed_data_provider.ConsumeRandomLengthString(5); + int32_t parseint32_out_without_locale; + const bool parseint32_without_locale = ParseInt32(random_string, &parseint32_out_without_locale); + int64_t parseint64_out_without_locale; + const bool parseint64_without_locale = ParseInt64(random_string, &parseint64_out_without_locale); + const int64_t atoi64_without_locale = atoi64(random_string); + const int atoi_without_locale = atoi(random_string); + const int64_t atoi64c_without_locale = atoi64(random_string.c_str()); + const int64_t random_int64 = fuzzed_data_provider.ConsumeIntegral<int64_t>(); + const std::string i64tostr_without_locale = i64tostr(random_int64); + const int32_t random_int32 = fuzzed_data_provider.ConsumeIntegral<int32_t>(); + const std::string itostr_without_locale = itostr(random_int32); + const std::string strprintf_int_without_locale = strprintf("%d", random_int64); + const double random_double = fuzzed_data_provider.ConsumeFloatingPoint<double>(); + const std::string strprintf_double_without_locale = strprintf("%f", random_double); + + const char* new_locale = std::setlocale(LC_ALL, locale_identifier.c_str()); + assert(new_locale != nullptr); + + int32_t parseint32_out_with_locale; + const bool parseint32_with_locale = ParseInt32(random_string, &parseint32_out_with_locale); + assert(parseint32_without_locale == parseint32_with_locale); + if (parseint32_without_locale) { + assert(parseint32_out_without_locale == parseint32_out_with_locale); + } + int64_t parseint64_out_with_locale; + const bool parseint64_with_locale = ParseInt64(random_string, &parseint64_out_with_locale); + assert(parseint64_without_locale == parseint64_with_locale); + if (parseint64_without_locale) { + assert(parseint64_out_without_locale == parseint64_out_with_locale); + } + const int64_t atoi64_with_locale = atoi64(random_string); + assert(atoi64_without_locale == atoi64_with_locale); + const int64_t atoi64c_with_locale = atoi64(random_string.c_str()); + assert(atoi64c_without_locale == atoi64c_with_locale); + const int atoi_with_locale = atoi(random_string); + assert(atoi_without_locale == atoi_with_locale); + const std::string i64tostr_with_locale = i64tostr(random_int64); + assert(i64tostr_without_locale == i64tostr_with_locale); + const std::string itostr_with_locale = itostr(random_int32); + assert(itostr_without_locale == itostr_with_locale); + const std::string strprintf_int_with_locale = strprintf("%d", random_int64); + assert(strprintf_int_without_locale == strprintf_int_with_locale); + const std::string strprintf_double_with_locale = strprintf("%f", random_double); + assert(strprintf_double_without_locale == strprintf_double_with_locale); + + const std::locale current_cpp_locale; + assert(current_cpp_locale == std::locale::classic()); +} diff --git a/src/test/fuzz/netaddress.cpp b/src/test/fuzz/netaddress.cpp new file mode 100644 index 0000000000..7d87ebc214 --- /dev/null +++ b/src/test/fuzz/netaddress.cpp @@ -0,0 +1,123 @@ +// Copyright (c) 2020 The Bitcoin Core developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include <netaddress.h> +#include <test/fuzz/FuzzedDataProvider.h> +#include <test/fuzz/fuzz.h> + +#include <cassert> +#include <cstdint> +#include <netinet/in.h> +#include <vector> + +namespace { +CNetAddr ConsumeNetAddr(FuzzedDataProvider& fuzzed_data_provider) noexcept +{ + const Network network = fuzzed_data_provider.PickValueInArray({Network::NET_IPV4, Network::NET_IPV6, Network::NET_INTERNAL, Network::NET_ONION}); + if (network == Network::NET_IPV4) { + const in_addr v4_addr = { + .s_addr = fuzzed_data_provider.ConsumeIntegral<uint32_t>()}; + return CNetAddr{v4_addr}; + } else if (network == Network::NET_IPV6) { + if (fuzzed_data_provider.remaining_bytes() < 16) { + return CNetAddr{}; + } + in6_addr v6_addr = {}; + memcpy(v6_addr.s6_addr, fuzzed_data_provider.ConsumeBytes<uint8_t>(16).data(), 16); + return CNetAddr{v6_addr, fuzzed_data_provider.ConsumeIntegral<uint32_t>()}; + } else if (network == Network::NET_INTERNAL) { + CNetAddr net_addr; + net_addr.SetInternal(fuzzed_data_provider.ConsumeBytesAsString(32)); + return net_addr; + } else if (network == Network::NET_ONION) { + CNetAddr net_addr; + net_addr.SetSpecial(fuzzed_data_provider.ConsumeBytesAsString(32)); + return net_addr; + } else { + assert(false); + } +} +}; // namespace + +void test_one_input(const std::vector<uint8_t>& buffer) +{ + FuzzedDataProvider fuzzed_data_provider(buffer.data(), buffer.size()); + + const CNetAddr net_addr = ConsumeNetAddr(fuzzed_data_provider); + for (int i = 0; i < 15; ++i) { + (void)net_addr.GetByte(i); + } + (void)net_addr.GetHash(); + (void)net_addr.GetNetClass(); + if (net_addr.GetNetwork() == Network::NET_IPV4) { + assert(net_addr.IsIPv4()); + } + if (net_addr.GetNetwork() == Network::NET_IPV6) { + assert(net_addr.IsIPv6()); + } + if (net_addr.GetNetwork() == Network::NET_ONION) { + assert(net_addr.IsTor()); + } + if (net_addr.GetNetwork() == Network::NET_INTERNAL) { + assert(net_addr.IsInternal()); + } + if (net_addr.GetNetwork() == Network::NET_UNROUTABLE) { + assert(!net_addr.IsRoutable()); + } + (void)net_addr.IsBindAny(); + if (net_addr.IsInternal()) { + assert(net_addr.GetNetwork() == Network::NET_INTERNAL); + } + if (net_addr.IsIPv4()) { + assert(net_addr.GetNetwork() == Network::NET_IPV4 || net_addr.GetNetwork() == Network::NET_UNROUTABLE); + } + if (net_addr.IsIPv6()) { + assert(net_addr.GetNetwork() == Network::NET_IPV6 || net_addr.GetNetwork() == Network::NET_UNROUTABLE); + } + (void)net_addr.IsLocal(); + if (net_addr.IsRFC1918() || net_addr.IsRFC2544() || net_addr.IsRFC6598() || net_addr.IsRFC5737() || net_addr.IsRFC3927()) { + assert(net_addr.IsIPv4()); + } + (void)net_addr.IsRFC2544(); + if (net_addr.IsRFC3849() || net_addr.IsRFC3964() || net_addr.IsRFC4380() || net_addr.IsRFC4843() || net_addr.IsRFC7343() || net_addr.IsRFC4862() || net_addr.IsRFC6052() || net_addr.IsRFC6145()) { + assert(net_addr.IsIPv6()); + } + (void)net_addr.IsRFC3927(); + (void)net_addr.IsRFC3964(); + if (net_addr.IsRFC4193()) { + assert(net_addr.GetNetwork() == Network::NET_ONION || net_addr.GetNetwork() == Network::NET_INTERNAL || net_addr.GetNetwork() == Network::NET_UNROUTABLE); + } + (void)net_addr.IsRFC4380(); + (void)net_addr.IsRFC4843(); + (void)net_addr.IsRFC4862(); + (void)net_addr.IsRFC5737(); + (void)net_addr.IsRFC6052(); + (void)net_addr.IsRFC6145(); + (void)net_addr.IsRFC6598(); + (void)net_addr.IsRFC7343(); + if (!net_addr.IsRoutable()) { + assert(net_addr.GetNetwork() == Network::NET_UNROUTABLE || net_addr.GetNetwork() == Network::NET_INTERNAL); + } + if (net_addr.IsTor()) { + assert(net_addr.GetNetwork() == Network::NET_ONION); + } + (void)net_addr.IsValid(); + (void)net_addr.ToString(); + (void)net_addr.ToStringIP(); + + const CSubNet sub_net{net_addr, fuzzed_data_provider.ConsumeIntegral<int32_t>()}; + (void)sub_net.IsValid(); + (void)sub_net.ToString(); + + const CService service{net_addr, fuzzed_data_provider.ConsumeIntegral<uint16_t>()}; + (void)service.GetKey(); + (void)service.GetPort(); + (void)service.ToString(); + (void)service.ToStringIPPort(); + (void)service.ToStringPort(); + + const CNetAddr other_net_addr = ConsumeNetAddr(fuzzed_data_provider); + (void)net_addr.GetReachabilityFrom(&other_net_addr); + (void)sub_net.Match(other_net_addr); +} diff --git a/src/test/fuzz/process_message.cpp b/src/test/fuzz/process_message.cpp new file mode 100644 index 0000000000..934f741068 --- /dev/null +++ b/src/test/fuzz/process_message.cpp @@ -0,0 +1,98 @@ +// Copyright (c) 2020 The Bitcoin Core developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include <banman.h> +#include <chainparams.h> +#include <consensus/consensus.h> +#include <net.h> +#include <net_processing.h> +#include <protocol.h> +#include <scheduler.h> +#include <script/script.h> +#include <streams.h> +#include <test/fuzz/FuzzedDataProvider.h> +#include <test/fuzz/fuzz.h> +#include <test/util/mining.h> +#include <test/util/setup_common.h> +#include <util/memory.h> +#include <validationinterface.h> +#include <version.h> + +#include <algorithm> +#include <atomic> +#include <cassert> +#include <chrono> +#include <cstdint> +#include <iosfwd> +#include <iostream> +#include <map> +#include <memory> +#include <set> +#include <string> +#include <vector> + +bool ProcessMessage(CNode* pfrom, const std::string& strCommand, CDataStream& vRecv, int64_t nTimeReceived, const CChainParams& chainparams, CConnman* connman, BanMan* banman, const std::atomic<bool>& interruptMsgProc); + +namespace { + +#ifdef MESSAGE_TYPE +#define TO_STRING_(s) #s +#define TO_STRING(s) TO_STRING_(s) +const std::string LIMIT_TO_MESSAGE_TYPE{TO_STRING(MESSAGE_TYPE)}; +#else +const std::string LIMIT_TO_MESSAGE_TYPE; +#endif + +const std::map<std::string, std::set<std::string>> EXPECTED_DESERIALIZATION_EXCEPTIONS = { + {"CDataStream::read(): end of data: iostream error", {"addr", "block", "blocktxn", "cmpctblock", "feefilter", "filteradd", "filterload", "getblocks", "getblocktxn", "getdata", "getheaders", "headers", "inv", "notfound", "ping", "sendcmpct", "tx"}}, + {"CompactSize exceeds limit of type: iostream error", {"cmpctblock"}}, + {"differential value overflow: iostream error", {"getblocktxn"}}, + {"index overflowed 16 bits: iostream error", {"getblocktxn"}}, + {"index overflowed 16-bits: iostream error", {"cmpctblock"}}, + {"indexes overflowed 16 bits: iostream error", {"getblocktxn"}}, + {"non-canonical ReadCompactSize(): iostream error", {"addr", "block", "blocktxn", "cmpctblock", "filteradd", "filterload", "getblocks", "getblocktxn", "getdata", "getheaders", "headers", "inv", "notfound", "tx"}}, + {"ReadCompactSize(): size too large: iostream error", {"addr", "block", "blocktxn", "cmpctblock", "filteradd", "filterload", "getblocks", "getblocktxn", "getdata", "getheaders", "headers", "inv", "notfound", "tx"}}, + {"Superfluous witness record: iostream error", {"block", "blocktxn", "cmpctblock", "tx"}}, + {"Unknown transaction optional data: iostream error", {"block", "blocktxn", "cmpctblock", "tx"}}, +}; + +const RegTestingSetup* g_setup; +} // namespace + +void initialize() +{ + static RegTestingSetup setup{}; + g_setup = &setup; + + for (int i = 0; i < 2 * COINBASE_MATURITY; i++) { + MineBlock(g_setup->m_node, CScript() << OP_TRUE); + } + SyncWithValidationInterfaceQueue(); +} + +void test_one_input(const std::vector<uint8_t>& buffer) +{ + FuzzedDataProvider fuzzed_data_provider(buffer.data(), buffer.size()); + const std::string random_message_type{fuzzed_data_provider.ConsumeBytesAsString(CMessageHeader::COMMAND_SIZE).c_str()}; + if (!LIMIT_TO_MESSAGE_TYPE.empty() && random_message_type != LIMIT_TO_MESSAGE_TYPE) { + return; + } + CDataStream random_bytes_data_stream{fuzzed_data_provider.ConsumeRemainingBytes<unsigned char>(), SER_NETWORK, PROTOCOL_VERSION}; + CNode p2p_node{0, ServiceFlags(NODE_NETWORK | NODE_WITNESS | NODE_BLOOM), 0, INVALID_SOCKET, CAddress{CService{in_addr{0x0100007f}, 7777}, NODE_NETWORK}, 0, 0, CAddress{}, std::string{}, false}; + p2p_node.fSuccessfullyConnected = true; + p2p_node.nVersion = PROTOCOL_VERSION; + p2p_node.SetSendVersion(PROTOCOL_VERSION); + g_setup->m_node.peer_logic->InitializeNode(&p2p_node); + try { + (void)ProcessMessage(&p2p_node, random_message_type, random_bytes_data_stream, GetTimeMillis(), Params(), g_setup->m_node.connman.get(), g_setup->m_node.banman.get(), std::atomic<bool>{false}); + } catch (const std::ios_base::failure& e) { + const std::string exception_message{e.what()}; + const auto p = EXPECTED_DESERIALIZATION_EXCEPTIONS.find(exception_message); + if (p == EXPECTED_DESERIALIZATION_EXCEPTIONS.cend() || p->second.count(random_message_type) == 0) { + std::cout << "Unexpected exception when processing message type \"" << random_message_type << "\": " << exception_message << std::endl; + assert(false); + } + } + SyncWithValidationInterfaceQueue(); +} diff --git a/src/test/fuzz/rolling_bloom_filter.cpp b/src/test/fuzz/rolling_bloom_filter.cpp index ce69c4e8da..3b37321977 100644 --- a/src/test/fuzz/rolling_bloom_filter.cpp +++ b/src/test/fuzz/rolling_bloom_filter.cpp @@ -24,7 +24,7 @@ void test_one_input(const std::vector<uint8_t>& buffer) while (fuzzed_data_provider.remaining_bytes() > 0) { switch (fuzzed_data_provider.ConsumeIntegralInRange(0, 2)) { case 0: { - const std::vector<unsigned char>& b = ConsumeRandomLengthByteVector(fuzzed_data_provider); + const std::vector<unsigned char> b = ConsumeRandomLengthByteVector(fuzzed_data_provider); (void)rolling_bloom_filter.contains(b); rolling_bloom_filter.insert(b); const bool present = rolling_bloom_filter.contains(b); diff --git a/src/test/fuzz/script.cpp b/src/test/fuzz/script.cpp index 0469e87de6..0d18784302 100644 --- a/src/test/fuzz/script.cpp +++ b/src/test/fuzz/script.cpp @@ -15,12 +15,15 @@ #include <script/standard.h> #include <streams.h> #include <test/fuzz/fuzz.h> +#include <univalue.h> #include <util/memory.h> void initialize() { // Fuzzers using pubkey must hold an ECCVerifyHandle. static const auto verify_handle = MakeUnique<ECCVerifyHandle>(); + + SelectParams(CBaseChainParams::REGTEST); } void test_one_input(const std::vector<uint8_t>& buffer) @@ -28,7 +31,22 @@ void test_one_input(const std::vector<uint8_t>& buffer) const CScript script(buffer.begin(), buffer.end()); std::vector<unsigned char> compressed; - (void)CompressScript(script, compressed); + if (CompressScript(script, compressed)) { + const unsigned int size = compressed[0]; + compressed.erase(compressed.begin()); + assert(size >= 0 && size <= 5); + CScript decompressed_script; + const bool ok = DecompressScript(decompressed_script, size, compressed); + assert(ok); + assert(script == decompressed_script); + } + + for (unsigned int size = 0; size < 6; ++size) { + std::vector<unsigned char> vch(GetSpecialScriptSize(size), 0x00); + vch.insert(vch.end(), buffer.begin(), buffer.end()); + CScript decompressed_script; + (void)DecompressScript(decompressed_script, size, vch); + } CTxDestination address; (void)ExtractDestination(script, address); @@ -61,4 +79,17 @@ void test_one_input(const std::vector<uint8_t>& buffer) (void)script.IsPushOnly(); (void)script.IsUnspendable(); (void)script.GetSigOpCount(/* fAccurate= */ false); + + (void)FormatScript(script); + (void)ScriptToAsmStr(script, false); + (void)ScriptToAsmStr(script, true); + + UniValue o1(UniValue::VOBJ); + ScriptPubKeyToUniv(script, o1, true); + UniValue o2(UniValue::VOBJ); + ScriptPubKeyToUniv(script, o2, false); + UniValue o3(UniValue::VOBJ); + ScriptToUniv(script, o3, true); + UniValue o4(UniValue::VOBJ); + ScriptToUniv(script, o4, false); } diff --git a/src/test/fuzz/script_ops.cpp b/src/test/fuzz/script_ops.cpp new file mode 100644 index 0000000000..0cd129ba7a --- /dev/null +++ b/src/test/fuzz/script_ops.cpp @@ -0,0 +1,67 @@ +// Copyright (c) 2020 The Bitcoin Core developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include <script/script.h> +#include <test/fuzz/FuzzedDataProvider.h> +#include <test/fuzz/fuzz.h> +#include <test/fuzz/util.h> + +#include <cstdint> +#include <string> +#include <vector> + +void test_one_input(const std::vector<uint8_t>& buffer) +{ + FuzzedDataProvider fuzzed_data_provider(buffer.data(), buffer.size()); + CScript script = ConsumeScript(fuzzed_data_provider); + while (fuzzed_data_provider.remaining_bytes() > 0) { + switch (fuzzed_data_provider.ConsumeIntegralInRange(0, 7)) { + case 0: + script += ConsumeScript(fuzzed_data_provider); + break; + case 1: + script = script + ConsumeScript(fuzzed_data_provider); + break; + case 2: + script << fuzzed_data_provider.ConsumeIntegral<int64_t>(); + break; + case 3: + script << ConsumeOpcodeType(fuzzed_data_provider); + break; + case 4: + script << ConsumeScriptNum(fuzzed_data_provider); + break; + case 5: + script << ConsumeRandomLengthByteVector(fuzzed_data_provider); + break; + case 6: + script.clear(); + break; + case 7: { + (void)script.GetSigOpCount(false); + (void)script.GetSigOpCount(true); + (void)script.GetSigOpCount(script); + (void)script.HasValidOps(); + (void)script.IsPayToScriptHash(); + (void)script.IsPayToWitnessScriptHash(); + (void)script.IsPushOnly(); + (void)script.IsUnspendable(); + { + CScript::const_iterator pc = script.begin(); + opcodetype opcode; + (void)script.GetOp(pc, opcode); + std::vector<uint8_t> data; + (void)script.GetOp(pc, opcode, data); + (void)script.IsPushOnly(pc); + } + { + int version; + std::vector<uint8_t> program; + (void)script.IsWitnessProgram(version, program); + } + break; + } + } + } +} diff --git a/src/test/fuzz/scriptnum_ops.cpp b/src/test/fuzz/scriptnum_ops.cpp new file mode 100644 index 0000000000..db44bb9e19 --- /dev/null +++ b/src/test/fuzz/scriptnum_ops.cpp @@ -0,0 +1,137 @@ +// Copyright (c) 2020 The Bitcoin Core developers +// Distributed under the MIT software license, see the accompanying +// file COPYING or http://www.opensource.org/licenses/mit-license.php. + +#include <script/script.h> +#include <test/fuzz/FuzzedDataProvider.h> +#include <test/fuzz/fuzz.h> +#include <test/fuzz/util.h> + +#include <cassert> +#include <cstdint> +#include <limits> +#include <vector> + +namespace { +bool IsValidAddition(const CScriptNum& lhs, const CScriptNum& rhs) +{ + return rhs == 0 || (rhs > 0 && lhs <= CScriptNum{std::numeric_limits<int64_t>::max()} - rhs) || (rhs < 0 && lhs >= CScriptNum{std::numeric_limits<int64_t>::min()} - rhs); +} + +bool IsValidSubtraction(const CScriptNum& lhs, const CScriptNum& rhs) +{ + return rhs == 0 || (rhs > 0 && lhs >= CScriptNum{std::numeric_limits<int64_t>::min()} + rhs) || (rhs < 0 && lhs <= CScriptNum{std::numeric_limits<int64_t>::max()} + rhs); +} +} // namespace + +void test_one_input(const std::vector<uint8_t>& buffer) +{ + FuzzedDataProvider fuzzed_data_provider(buffer.data(), buffer.size()); + CScriptNum script_num = ConsumeScriptNum(fuzzed_data_provider); + while (fuzzed_data_provider.remaining_bytes() > 0) { + switch (fuzzed_data_provider.ConsumeIntegralInRange(0, 11)) { + case 0: { + const int64_t i = fuzzed_data_provider.ConsumeIntegral<int64_t>(); + assert((script_num == i) != (script_num != i)); + assert((script_num <= i) != script_num > i); + assert((script_num >= i) != (script_num < i)); + // Avoid signed integer overflow: + // script/script.h:264:93: runtime error: signed integer overflow: -2261405121394637306 + -9223372036854775802 cannot be represented in type 'long' + if (IsValidAddition(script_num, CScriptNum{i})) { + assert((script_num + i) - i == script_num); + } + // Avoid signed integer overflow: + // script/script.h:265:93: runtime error: signed integer overflow: 9223371895120855039 - -9223372036854710486 cannot be represented in type 'long' + if (IsValidSubtraction(script_num, CScriptNum{i})) { + assert((script_num - i) + i == script_num); + } + break; + } + case 1: { + const CScriptNum random_script_num = ConsumeScriptNum(fuzzed_data_provider); + assert((script_num == random_script_num) != (script_num != random_script_num)); + assert((script_num <= random_script_num) != (script_num > random_script_num)); + assert((script_num >= random_script_num) != (script_num < random_script_num)); + // Avoid signed integer overflow: + // script/script.h:264:93: runtime error: signed integer overflow: -9223126527765971126 + -9223372036854756825 cannot be represented in type 'long' + if (IsValidAddition(script_num, random_script_num)) { + assert((script_num + random_script_num) - random_script_num == script_num); + } + // Avoid signed integer overflow: + // script/script.h:265:93: runtime error: signed integer overflow: 6052837899185946624 - -9223372036854775808 cannot be represented in type 'long' + if (IsValidSubtraction(script_num, random_script_num)) { + assert((script_num - random_script_num) + random_script_num == script_num); + } + break; + } + case 2: { + const CScriptNum random_script_num = ConsumeScriptNum(fuzzed_data_provider); + if (!IsValidAddition(script_num, random_script_num)) { + // Avoid assertion failure: + // ./script/script.h:292: CScriptNum &CScriptNum::operator+=(const int64_t &): Assertion `rhs == 0 || (rhs > 0 && m_value <= std::numeric_limits<int64_t>::max() - rhs) || (rhs < 0 && m_value >= std::numeric_limits<int64_t>::min() - rhs)' failed. + break; + } + script_num += random_script_num; + break; + } + case 3: { + const CScriptNum random_script_num = ConsumeScriptNum(fuzzed_data_provider); + if (!IsValidSubtraction(script_num, random_script_num)) { + // Avoid assertion failure: + // ./script/script.h:300: CScriptNum &CScriptNum::operator-=(const int64_t &): Assertion `rhs == 0 || (rhs > 0 && m_value >= std::numeric_limits<int64_t>::min() + rhs) || (rhs < 0 && m_value <= std::numeric_limits<int64_t>::max() + rhs)' failed. + break; + } + script_num -= random_script_num; + break; + } + case 4: + script_num = script_num & fuzzed_data_provider.ConsumeIntegral<int64_t>(); + break; + case 5: + script_num = script_num & ConsumeScriptNum(fuzzed_data_provider); + break; + case 6: + script_num &= ConsumeScriptNum(fuzzed_data_provider); + break; + case 7: + if (script_num == CScriptNum{std::numeric_limits<int64_t>::min()}) { + // Avoid assertion failure: + // ./script/script.h:279: CScriptNum CScriptNum::operator-() const: Assertion `m_value != std::numeric_limits<int64_t>::min()' failed. + break; + } + script_num = -script_num; + break; + case 8: + script_num = fuzzed_data_provider.ConsumeIntegral<int64_t>(); + break; + case 9: { + const int64_t random_integer = fuzzed_data_provider.ConsumeIntegral<int64_t>(); + if (!IsValidAddition(script_num, CScriptNum{random_integer})) { + // Avoid assertion failure: + // ./script/script.h:292: CScriptNum &CScriptNum::operator+=(const int64_t &): Assertion `rhs == 0 || (rhs > 0 && m_value <= std::numeric_limits<int64_t>::max() - rhs) || (rhs < 0 && m_value >= std::numeric_limits<int64_t>::min() - rhs)' failed. + break; + } + script_num += random_integer; + break; + } + case 10: { + const int64_t random_integer = fuzzed_data_provider.ConsumeIntegral<int64_t>(); + if (!IsValidSubtraction(script_num, CScriptNum{random_integer})) { + // Avoid assertion failure: + // ./script/script.h:300: CScriptNum &CScriptNum::operator-=(const int64_t &): Assertion `rhs == 0 || (rhs > 0 && m_value >= std::numeric_limits<int64_t>::min() + rhs) || (rhs < 0 && m_value <= std::numeric_limits<int64_t>::max() + rhs)' failed. + break; + } + script_num -= random_integer; + break; + } + case 11: + script_num &= fuzzed_data_provider.ConsumeIntegral<int64_t>(); + break; + } + // Avoid negation failure: + // script/script.h:332:35: runtime error: negation of -9223372036854775808 cannot be represented in type 'int64_t' (aka 'long'); cast to an unsigned type to negate this value to itself + if (script_num != CScriptNum{std::numeric_limits<int64_t>::min()}) { + (void)script_num.getvch(); + } + } +} diff --git a/src/test/fuzz/transaction.cpp b/src/test/fuzz/transaction.cpp index fefafda36b..1ec69cc23d 100644 --- a/src/test/fuzz/transaction.cpp +++ b/src/test/fuzz/transaction.cpp @@ -2,6 +2,7 @@ // Distributed under the MIT software license, see the accompanying // file COPYING or http://www.opensource.org/licenses/mit-license.php. +#include <chainparams.h> #include <coins.h> #include <consensus/tx_check.h> #include <consensus/tx_verify.h> @@ -13,12 +14,18 @@ #include <primitives/transaction.h> #include <streams.h> #include <test/fuzz/fuzz.h> +#include <univalue.h> #include <util/rbf.h> #include <validation.h> #include <version.h> #include <cassert> +void initialize() +{ + SelectParams(CBaseChainParams::REGTEST); +} + void test_one_input(const std::vector<uint8_t>& buffer) { CDataStream ds(buffer, SER_NETWORK, INIT_PROTO_VERSION); @@ -85,4 +92,21 @@ void test_one_input(const std::vector<uint8_t>& buffer) (void)IsStandardTx(tx, reason); (void)RecursiveDynamicUsage(tx); (void)SignalsOptInRBF(tx); + + CCoinsView coins_view; + const CCoinsViewCache coins_view_cache(&coins_view); + (void)AreInputsStandard(tx, coins_view_cache); + (void)IsWitnessStandard(tx, coins_view_cache); + + UniValue u(UniValue::VOBJ); + // ValueFromAmount(i) not defined when i == std::numeric_limits<int64_t>::min() + bool skip_tx_to_univ = false; + for (const CTxOut& txout : tx.vout) { + if (txout.nValue == std::numeric_limits<int64_t>::min()) { + skip_tx_to_univ = true; + } + } + if (!skip_tx_to_univ) { + TxToUniv(tx, /* hashBlock */ {}, u); + } } diff --git a/src/test/fuzz/util.h b/src/test/fuzz/util.h index 62907c7e0b..2f4aa9ad2b 100644 --- a/src/test/fuzz/util.h +++ b/src/test/fuzz/util.h @@ -7,9 +7,11 @@ #include <attributes.h> #include <optional.h> +#include <script/script.h> #include <serialize.h> #include <streams.h> #include <test/fuzz/FuzzedDataProvider.h> +#include <test/fuzz/fuzz.h> #include <version.h> #include <cstdint> @@ -25,7 +27,7 @@ NODISCARD inline std::vector<uint8_t> ConsumeRandomLengthByteVector(FuzzedDataPr template <typename T> NODISCARD inline Optional<T> ConsumeDeserializable(FuzzedDataProvider& fuzzed_data_provider, size_t max_length = 4096) noexcept { - const std::vector<uint8_t>& buffer = ConsumeRandomLengthByteVector(fuzzed_data_provider, max_length); + const std::vector<uint8_t> buffer = ConsumeRandomLengthByteVector(fuzzed_data_provider, max_length); CDataStream ds{buffer, SER_NETWORK, INIT_PROTO_VERSION}; T obj; try { @@ -36,4 +38,20 @@ NODISCARD inline Optional<T> ConsumeDeserializable(FuzzedDataProvider& fuzzed_da return obj; } +NODISCARD inline opcodetype ConsumeOpcodeType(FuzzedDataProvider& fuzzed_data_provider) noexcept +{ + return static_cast<opcodetype>(fuzzed_data_provider.ConsumeIntegralInRange<uint32_t>(0, MAX_OPCODE)); +} + +NODISCARD inline CScript ConsumeScript(FuzzedDataProvider& fuzzed_data_provider) noexcept +{ + const std::vector<uint8_t> b = ConsumeRandomLengthByteVector(fuzzed_data_provider); + return {b.begin(), b.end()}; +} + +NODISCARD inline CScriptNum ConsumeScriptNum(FuzzedDataProvider& fuzzed_data_provider) noexcept +{ + return CScriptNum{fuzzed_data_provider.ConsumeIntegral<int64_t>()}; +} + #endif // BITCOIN_TEST_FUZZ_UTIL_H diff --git a/src/test/reverselock_tests.cpp b/src/test/reverselock_tests.cpp index 532fe143ae..4e51b8c02a 100644 --- a/src/test/reverselock_tests.cpp +++ b/src/test/reverselock_tests.cpp @@ -2,7 +2,7 @@ // Distributed under the MIT software license, see the accompanying // file COPYING or http://www.opensource.org/licenses/mit-license.php. -#include <reverselock.h> +#include <sync.h> #include <test/util/setup_common.h> #include <boost/test/unit_test.hpp> @@ -11,21 +11,50 @@ BOOST_FIXTURE_TEST_SUITE(reverselock_tests, BasicTestingSetup) BOOST_AUTO_TEST_CASE(reverselock_basics) { - boost::mutex mutex; - boost::unique_lock<boost::mutex> lock(mutex); + Mutex mutex; + WAIT_LOCK(mutex, lock); BOOST_CHECK(lock.owns_lock()); { - reverse_lock<boost::unique_lock<boost::mutex> > rlock(lock); + REVERSE_LOCK(lock); BOOST_CHECK(!lock.owns_lock()); } BOOST_CHECK(lock.owns_lock()); } +BOOST_AUTO_TEST_CASE(reverselock_multiple) +{ + Mutex mutex2; + Mutex mutex; + WAIT_LOCK(mutex2, lock2); + WAIT_LOCK(mutex, lock); + + // Make sure undoing two locks succeeds + { + REVERSE_LOCK(lock); + BOOST_CHECK(!lock.owns_lock()); + REVERSE_LOCK(lock2); + BOOST_CHECK(!lock2.owns_lock()); + } + BOOST_CHECK(lock.owns_lock()); + BOOST_CHECK(lock2.owns_lock()); +} + BOOST_AUTO_TEST_CASE(reverselock_errors) { - boost::mutex mutex; - boost::unique_lock<boost::mutex> lock(mutex); + Mutex mutex2; + Mutex mutex; + WAIT_LOCK(mutex2, lock2); + WAIT_LOCK(mutex, lock); + +#ifdef DEBUG_LOCKORDER + // Make sure trying to reverse lock a previous lock fails + try { + REVERSE_LOCK(lock2); + BOOST_CHECK(false); // REVERSE_LOCK(lock2) succeeded + } catch(...) { } + BOOST_CHECK(lock2.owns_lock()); +#endif // Make sure trying to reverse lock an unlocked lock fails lock.unlock(); @@ -34,7 +63,7 @@ BOOST_AUTO_TEST_CASE(reverselock_errors) bool failed = false; try { - reverse_lock<boost::unique_lock<boost::mutex> > rlock(lock); + REVERSE_LOCK(lock); } catch(...) { failed = true; } @@ -49,7 +78,7 @@ BOOST_AUTO_TEST_CASE(reverselock_errors) lock.lock(); BOOST_CHECK(lock.owns_lock()); { - reverse_lock<boost::unique_lock<boost::mutex> > rlock(lock); + REVERSE_LOCK(lock); BOOST_CHECK(!lock.owns_lock()); } diff --git a/src/test/scheduler_tests.cpp b/src/test/scheduler_tests.cpp index 00eec58bdf..7d26840b73 100644 --- a/src/test/scheduler_tests.cpp +++ b/src/test/scheduler_tests.cpp @@ -11,13 +11,13 @@ BOOST_AUTO_TEST_SUITE(scheduler_tests) -static void microTask(CScheduler& s, boost::mutex& mutex, int& counter, int delta, boost::chrono::system_clock::time_point rescheduleTime) +static void microTask(CScheduler& s, boost::mutex& mutex, int& counter, int delta, std::chrono::system_clock::time_point rescheduleTime) { { boost::unique_lock<boost::mutex> lock(mutex); counter += delta; } - boost::chrono::system_clock::time_point noTime = boost::chrono::system_clock::time_point::min(); + std::chrono::system_clock::time_point noTime = std::chrono::system_clock::time_point::min(); if (rescheduleTime != noTime) { CScheduler::Function f = std::bind(µTask, std::ref(s), std::ref(mutex), std::ref(counter), -delta + 1, noTime); s.schedule(f, rescheduleTime); @@ -45,15 +45,15 @@ BOOST_AUTO_TEST_CASE(manythreads) auto randomMsec = [](FastRandomContext& rc) -> int { return -11 + (int)rc.randrange(1012); }; // [-11, 1000] auto randomDelta = [](FastRandomContext& rc) -> int { return -1000 + (int)rc.randrange(2001); }; // [-1000, 1000] - boost::chrono::system_clock::time_point start = boost::chrono::system_clock::now(); - boost::chrono::system_clock::time_point now = start; - boost::chrono::system_clock::time_point first, last; + std::chrono::system_clock::time_point start = std::chrono::system_clock::now(); + std::chrono::system_clock::time_point now = start; + std::chrono::system_clock::time_point first, last; size_t nTasks = microTasks.getQueueInfo(first, last); BOOST_CHECK(nTasks == 0); for (int i = 0; i < 100; ++i) { - boost::chrono::system_clock::time_point t = now + boost::chrono::microseconds(randomMsec(rng)); - boost::chrono::system_clock::time_point tReschedule = now + boost::chrono::microseconds(500 + randomMsec(rng)); + std::chrono::system_clock::time_point t = now + std::chrono::microseconds(randomMsec(rng)); + std::chrono::system_clock::time_point tReschedule = now + std::chrono::microseconds(500 + randomMsec(rng)); int whichCounter = zeroToNine(rng); CScheduler::Function f = std::bind(µTask, std::ref(microTasks), std::ref(counterMutex[whichCounter]), std::ref(counter[whichCounter]), @@ -71,14 +71,14 @@ BOOST_AUTO_TEST_CASE(manythreads) microThreads.create_thread(std::bind(&CScheduler::serviceQueue, µTasks)); UninterruptibleSleep(std::chrono::microseconds{600}); - now = boost::chrono::system_clock::now(); + now = std::chrono::system_clock::now(); // More threads and more tasks: for (int i = 0; i < 5; i++) microThreads.create_thread(std::bind(&CScheduler::serviceQueue, µTasks)); for (int i = 0; i < 100; i++) { - boost::chrono::system_clock::time_point t = now + boost::chrono::microseconds(randomMsec(rng)); - boost::chrono::system_clock::time_point tReschedule = now + boost::chrono::microseconds(500 + randomMsec(rng)); + std::chrono::system_clock::time_point t = now + std::chrono::microseconds(randomMsec(rng)); + std::chrono::system_clock::time_point tReschedule = now + std::chrono::microseconds(500 + randomMsec(rng)); int whichCounter = zeroToNine(rng); CScheduler::Function f = std::bind(µTask, std::ref(microTasks), std::ref(counterMutex[whichCounter]), std::ref(counter[whichCounter]), @@ -142,7 +142,6 @@ BOOST_AUTO_TEST_CASE(singlethreadedscheduler_ordered) BOOST_CHECK_EQUAL(counter2, 100); } -/* disabled for now. See discussion in https://github.com/bitcoin/bitcoin/pull/18174 BOOST_AUTO_TEST_CASE(mockforward) { CScheduler scheduler; @@ -157,14 +156,14 @@ BOOST_AUTO_TEST_CASE(mockforward) scheduler.scheduleFromNow(dummy, 8*min_in_milli); // check taskQueue - boost::chrono::system_clock::time_point first, last; + std::chrono::system_clock::time_point first, last; size_t num_tasks = scheduler.getQueueInfo(first, last); BOOST_CHECK_EQUAL(num_tasks, 3ul); std::thread scheduler_thread([&]() { scheduler.serviceQueue(); }); // bump the scheduler forward 5 minutes - scheduler.MockForward(boost::chrono::seconds(5*60)); + scheduler.MockForward(std::chrono::seconds(5*60)); // ensure scheduler has chance to process all tasks queued for before 1 ms from now. scheduler.scheduleFromNow([&scheduler]{ scheduler.stop(false); }, 1); @@ -178,11 +177,10 @@ BOOST_AUTO_TEST_CASE(mockforward) BOOST_CHECK_EQUAL(counter, 2); // check that the time of the remaining job has been updated - boost::chrono::system_clock::time_point now = boost::chrono::system_clock::now(); - int delta = boost::chrono::duration_cast<boost::chrono::seconds>(first - now).count(); + std::chrono::system_clock::time_point now = std::chrono::system_clock::now(); + int delta = std::chrono::duration_cast<std::chrono::seconds>(first - now).count(); // should be between 2 & 3 minutes from now BOOST_CHECK(delta > 2*60 && delta < 3*60); } -*/ BOOST_AUTO_TEST_SUITE_END() diff --git a/src/test/txindex_tests.cpp b/src/test/txindex_tests.cpp index 6474711cad..3550a02316 100644 --- a/src/test/txindex_tests.cpp +++ b/src/test/txindex_tests.cpp @@ -70,6 +70,8 @@ BOOST_FIXTURE_TEST_CASE(txindex_initial_sync, TestChain100Setup) // shutdown sequence (c.f. Shutdown() in init.cpp) txindex.Stop(); + // txindex job may be scheduled, so stop scheduler before destructing + m_node.scheduler->stop(); threadGroup.interrupt_all(); threadGroup.join_all(); diff --git a/src/test/util/setup_common.cpp b/src/test/util/setup_common.cpp index fc736bc3a1..ad3ff48860 100644 --- a/src/test/util/setup_common.cpp +++ b/src/test/util/setup_common.cpp @@ -13,6 +13,7 @@ #include <init.h> #include <miner.h> #include <net.h> +#include <net_processing.h> #include <noui.h> #include <pow.h> #include <rpc/blockchain.h> @@ -62,7 +63,7 @@ std::ostream& operator<<(std::ostream& os, const uint256& num) } BasicTestingSetup::BasicTestingSetup(const std::string& chainName) - : m_path_root{fs::temp_directory_path() / "test_common_" PACKAGE_NAME / std::to_string(g_insecure_rand_ctx_temp_path.rand32())} + : m_path_root{fs::temp_directory_path() / "test_common_" PACKAGE_NAME / g_insecure_rand_ctx_temp_path.rand256().ToString()} { fs::create_directories(m_path_root); gArgs.ForceSetArg("-datadir", m_path_root.string()); @@ -136,10 +137,12 @@ TestingSetup::TestingSetup(const std::string& chainName) : BasicTestingSetup(cha m_node.mempool->setSanityCheck(1.0); m_node.banman = MakeUnique<BanMan>(GetDataDir() / "banlist.dat", nullptr, DEFAULT_MISBEHAVING_BANTIME); m_node.connman = MakeUnique<CConnman>(0x1337, 0x1337); // Deterministic randomness for tests. + m_node.peer_logic = MakeUnique<PeerLogicValidation>(m_node.connman.get(), m_node.banman.get(), *m_node.scheduler); } TestingSetup::~TestingSetup() { + if (m_node.scheduler) m_node.scheduler->stop(); threadGroup.interrupt_all(); threadGroup.join_all(); GetMainSignals().FlushBackgroundCallbacks(); diff --git a/src/util/message.cpp b/src/util/message.cpp index 17603a43d2..1e7128d225 100644 --- a/src/util/message.cpp +++ b/src/util/message.cpp @@ -76,3 +76,17 @@ uint256 MessageHash(const std::string& message) return hasher.GetHash(); } + +std::string SigningResultString(const SigningResult res) +{ + switch (res) { + case SigningResult::OK: + return "No error"; + case SigningResult::PRIVATE_KEY_NOT_AVAILABLE: + return "Private key not available"; + case SigningResult::SIGNING_FAILED: + return "Sign failed"; + // no default case, so the compiler can warn about missing cases + } + assert(false); +} diff --git a/src/util/message.h b/src/util/message.h index 01fd14ce2d..b31c5f5761 100644 --- a/src/util/message.h +++ b/src/util/message.h @@ -39,6 +39,12 @@ enum class MessageVerificationResult { OK }; +enum class SigningResult { + OK, //!< No error + PRIVATE_KEY_NOT_AVAILABLE, + SIGNING_FAILED, +}; + /** Verify a signed message. * @param[in] address Signer's bitcoin address, it must refer to a public key. * @param[in] signature The signature in base64 format. @@ -65,4 +71,6 @@ bool MessageSign( */ uint256 MessageHash(const std::string& message); +std::string SigningResultString(const SigningResult res); + #endif // BITCOIN_UTIL_MESSAGE_H diff --git a/src/wallet/db.cpp b/src/wallet/db.cpp index aa6e91c05e..67be4d85d2 100644 --- a/src/wallet/db.cpp +++ b/src/wallet/db.cpp @@ -850,7 +850,7 @@ bool BerkeleyDatabase::Rewrite(const char* pszSkip) return BerkeleyBatch::Rewrite(*this, pszSkip); } -bool BerkeleyDatabase::Backup(const std::string& strDest) +bool BerkeleyDatabase::Backup(const std::string& strDest) const { if (IsDummy()) { return false; diff --git a/src/wallet/db.h b/src/wallet/db.h index abec3ae4e2..bebaa55d05 100644 --- a/src/wallet/db.h +++ b/src/wallet/db.h @@ -157,7 +157,7 @@ public: /** Back up the entire database to a file. */ - bool Backup(const std::string& strDest); + bool Backup(const std::string& strDest) const; /** Make sure all changes are flushed to disk. */ @@ -193,7 +193,7 @@ private: * Only to be used at a low level, application should ideally not care * about this. */ - bool IsDummy() { return env == nullptr; } + bool IsDummy() const { return env == nullptr; } }; /** RAII class that provides access to a Berkeley database */ diff --git a/src/wallet/psbtwallet.cpp b/src/wallet/psbtwallet.cpp deleted file mode 100644 index d995fb06d4..0000000000 --- a/src/wallet/psbtwallet.cpp +++ /dev/null @@ -1,77 +0,0 @@ -// Copyright (c) 2009-2019 The Bitcoin Core developers -// Distributed under the MIT software license, see the accompanying -// file COPYING or http://www.opensource.org/licenses/mit-license.php. - -#include <wallet/psbtwallet.h> - -TransactionError FillPSBT(const CWallet* pwallet, PartiallySignedTransaction& psbtx, bool& complete, int sighash_type, bool sign, bool bip32derivs) -{ - LOCK(pwallet->cs_wallet); - // Get all of the previous transactions - complete = true; - for (unsigned int i = 0; i < psbtx.tx->vin.size(); ++i) { - const CTxIn& txin = psbtx.tx->vin[i]; - PSBTInput& input = psbtx.inputs.at(i); - - if (PSBTInputSigned(input)) { - continue; - } - - // Verify input looks sane. This will check that we have at most one uxto, witness or non-witness. - if (!input.IsSane()) { - return TransactionError::INVALID_PSBT; - } - - // If we have no utxo, grab it from the wallet. - if (!input.non_witness_utxo && input.witness_utxo.IsNull()) { - const uint256& txhash = txin.prevout.hash; - const auto it = pwallet->mapWallet.find(txhash); - if (it != pwallet->mapWallet.end()) { - const CWalletTx& wtx = it->second; - // We only need the non_witness_utxo, which is a superset of the witness_utxo. - // The signing code will switch to the smaller witness_utxo if this is ok. - input.non_witness_utxo = wtx.tx; - } - } - - // Get the Sighash type - if (sign && input.sighash_type > 0 && input.sighash_type != sighash_type) { - return TransactionError::SIGHASH_MISMATCH; - } - - // Get the scriptPubKey to know which SigningProvider to use - CScript script; - if (!input.witness_utxo.IsNull()) { - script = input.witness_utxo.scriptPubKey; - } else if (input.non_witness_utxo) { - if (txin.prevout.n >= input.non_witness_utxo->vout.size()) { - return TransactionError::MISSING_INPUTS; - } - script = input.non_witness_utxo->vout[txin.prevout.n].scriptPubKey; - } else { - // There's no UTXO so we can just skip this now - complete = false; - continue; - } - SignatureData sigdata; - input.FillSignatureData(sigdata); - std::unique_ptr<SigningProvider> provider = pwallet->GetSigningProvider(script, sigdata); - if (!provider) { - complete = false; - continue; - } - - complete &= SignPSBTInput(HidingSigningProvider(provider.get(), !sign, !bip32derivs), psbtx, i, sighash_type); - } - - // Fill in the bip32 keypaths and redeemscripts for the outputs so that hardware wallets can identify change - for (unsigned int i = 0; i < psbtx.tx->vout.size(); ++i) { - const CTxOut& out = psbtx.tx->vout.at(i); - std::unique_ptr<SigningProvider> provider = pwallet->GetSigningProvider(out.scriptPubKey); - if (provider) { - UpdatePSBTOutput(HidingSigningProvider(provider.get(), true, !bip32derivs), psbtx, i); - } - } - - return TransactionError::OK; -} diff --git a/src/wallet/psbtwallet.h b/src/wallet/psbtwallet.h deleted file mode 100644 index 8285bb901c..0000000000 --- a/src/wallet/psbtwallet.h +++ /dev/null @@ -1,32 +0,0 @@ -// Copyright (c) 2009-2019 The Bitcoin Core developers -// Distributed under the MIT software license, see the accompanying -// file COPYING or http://www.opensource.org/licenses/mit-license.php. - -#ifndef BITCOIN_WALLET_PSBTWALLET_H -#define BITCOIN_WALLET_PSBTWALLET_H - -#include <psbt.h> -#include <wallet/wallet.h> - -/** - * Fills out a PSBT with information from the wallet. Fills in UTXOs if we have - * them. Tries to sign if sign=true. Sets `complete` if the PSBT is now complete - * (i.e. has all required signatures or signature-parts, and is ready to - * finalize.) Sets `error` and returns false if something goes wrong. - * - * @param[in] pwallet pointer to a wallet - * @param[in] psbtx PartiallySignedTransaction to fill in - * @param[out] complete indicates whether the PSBT is now complete - * @param[in] sighash_type the sighash type to use when signing (if PSBT does not specify) - * @param[in] sign whether to sign or not - * @param[in] bip32derivs whether to fill in bip32 derivation information if available - * return error - */ -NODISCARD TransactionError FillPSBT(const CWallet* pwallet, - PartiallySignedTransaction& psbtx, - bool& complete, - int sighash_type = 1 /* SIGHASH_ALL */, - bool sign = true, - bool bip32derivs = true); - -#endif // BITCOIN_WALLET_PSBTWALLET_H diff --git a/src/wallet/rpcdump.cpp b/src/wallet/rpcdump.cpp index 4253038090..8df3491060 100644 --- a/src/wallet/rpcdump.cpp +++ b/src/wallet/rpcdump.cpp @@ -54,7 +54,7 @@ static std::string DecodeDumpString(const std::string &str) { return ret.str(); } -static bool GetWalletAddressesForKey(LegacyScriptPubKeyMan* spk_man, CWallet* const pwallet, const CKeyID& keyid, std::string& strAddr, std::string& strLabel) EXCLUSIVE_LOCKS_REQUIRED(pwallet->cs_wallet) +static bool GetWalletAddressesForKey(LegacyScriptPubKeyMan* spk_man, const CWallet* const pwallet, const CKeyID& keyid, std::string& strAddr, std::string& strLabel) EXCLUSIVE_LOCKS_REQUIRED(pwallet->cs_wallet) { bool fLabelFound = false; CKey key; @@ -65,7 +65,7 @@ static bool GetWalletAddressesForKey(LegacyScriptPubKeyMan* spk_man, CWallet* co strAddr += ","; } strAddr += EncodeDestination(dest); - strLabel = EncodeDumpString(pwallet->mapAddressBook[dest].name); + strLabel = EncodeDumpString(pwallet->mapAddressBook.at(dest).name); fLabelFound = true; } } @@ -676,7 +676,7 @@ UniValue importwallet(const JSONRPCRequest& request) UniValue dumpprivkey(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; } @@ -724,7 +724,7 @@ UniValue dumpprivkey(const JSONRPCRequest& request) UniValue dumpwallet(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; } diff --git a/src/wallet/rpcwallet.cpp b/src/wallet/rpcwallet.cpp index d9d92f8984..44e580a52a 100644 --- a/src/wallet/rpcwallet.cpp +++ b/src/wallet/rpcwallet.cpp @@ -27,7 +27,6 @@ #include <util/vector.h> #include <wallet/coincontrol.h> #include <wallet/feebumper.h> -#include <wallet/psbtwallet.h> #include <wallet/rpcwallet.h> #include <wallet/wallet.h> #include <wallet/walletdb.h> @@ -40,7 +39,7 @@ static const std::string WALLET_ENDPOINT_BASE = "/wallet/"; -static inline bool GetAvoidReuseFlag(CWallet * const pwallet, const UniValue& param) { +static inline bool GetAvoidReuseFlag(const CWallet* const pwallet, const UniValue& param) { bool can_avoid_reuse = pwallet->IsWalletFlagSet(WALLET_FLAG_AVOID_REUSE); bool avoid_reuse = param.isNull() ? can_avoid_reuse : param.get_bool(); @@ -343,7 +342,7 @@ static CTransactionRef SendMoney(interfaces::Chain::Lock& locked_chain, CWallet CScript scriptPubKey = GetScriptForDestination(address); // Create and send the transaction - CAmount nFeeRequired; + CAmount nFeeRequired = 0; std::string strError; std::vector<CRecipient> vecSend; int nChangePosRet = -1; @@ -458,7 +457,7 @@ static UniValue sendtoaddress(const JSONRPCRequest& request) static UniValue listaddressgroupings(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -520,7 +519,7 @@ static UniValue listaddressgroupings(const JSONRPCRequest& request) static UniValue signmessage(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -566,22 +565,12 @@ static UniValue signmessage(const JSONRPCRequest& request) throw JSONRPCError(RPC_TYPE_ERROR, "Address does not refer to key"); } - CScript script_pub_key = GetScriptForDestination(*pkhash); - std::unique_ptr<SigningProvider> provider = pwallet->GetSigningProvider(script_pub_key); - if (!provider) { - throw JSONRPCError(RPC_WALLET_ERROR, "Private key not available"); - } - - CKey key; - CKeyID keyID(*pkhash); - if (!provider->GetKey(keyID, key)) { - throw JSONRPCError(RPC_WALLET_ERROR, "Private key not available"); - } - std::string signature; - - if (!MessageSign(key, strMessage, signature)) { - throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Sign failed"); + SigningResult err = pwallet->SignMessage(strMessage, *pkhash, signature); + if (err == SigningResult::SIGNING_FAILED) { + throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, SigningResultString(err)); + } else if (err != SigningResult::OK){ + throw JSONRPCError(RPC_WALLET_ERROR, SigningResultString(err)); } return signature; @@ -590,7 +579,7 @@ static UniValue signmessage(const JSONRPCRequest& request) static UniValue getreceivedbyaddress(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -660,7 +649,7 @@ static UniValue getreceivedbyaddress(const JSONRPCRequest& request) static UniValue getreceivedbylabel(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -728,7 +717,7 @@ static UniValue getreceivedbylabel(const JSONRPCRequest& request) static UniValue getbalance(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -786,7 +775,7 @@ static UniValue getbalance(const JSONRPCRequest& request) static UniValue getunconfirmedbalance(const JSONRPCRequest &request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -1041,7 +1030,7 @@ struct tallyitem } }; -static UniValue ListReceived(interfaces::Chain::Lock& locked_chain, CWallet * const pwallet, const UniValue& params, bool by_label) EXCLUSIVE_LOCKS_REQUIRED(pwallet->cs_wallet) +static UniValue ListReceived(interfaces::Chain::Lock& locked_chain, const CWallet* const pwallet, const UniValue& params, bool by_label) EXCLUSIVE_LOCKS_REQUIRED(pwallet->cs_wallet) { // Minimum confirmations int nMinDepth = 1; @@ -1190,7 +1179,7 @@ static UniValue ListReceived(interfaces::Chain::Lock& locked_chain, CWallet * co static UniValue listreceivedbyaddress(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -1242,7 +1231,7 @@ static UniValue listreceivedbyaddress(const JSONRPCRequest& request) static UniValue listreceivedbylabel(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -1302,7 +1291,7 @@ static void MaybePushAddress(UniValue & entry, const CTxDestination &dest) * @param filter_ismine The "is mine" filter flags. * @param filter_label Optional label string to filter incoming transactions. */ -static void ListTransactions(interfaces::Chain::Lock& locked_chain, CWallet* const pwallet, const CWalletTx& wtx, int nMinDepth, bool fLong, UniValue& ret, const isminefilter& filter_ismine, const std::string* filter_label) EXCLUSIVE_LOCKS_REQUIRED(pwallet->cs_wallet) +static void ListTransactions(interfaces::Chain::Lock& locked_chain, const CWallet* const pwallet, const CWalletTx& wtx, int nMinDepth, bool fLong, UniValue& ret, const isminefilter& filter_ismine, const std::string* filter_label) EXCLUSIVE_LOCKS_REQUIRED(pwallet->cs_wallet) { CAmount nFee; std::list<COutputEntry> listReceived; @@ -1325,7 +1314,7 @@ static void ListTransactions(interfaces::Chain::Lock& locked_chain, CWallet* con entry.pushKV("category", "send"); entry.pushKV("amount", ValueFromAmount(-s.amount)); if (pwallet->mapAddressBook.count(s.destination)) { - entry.pushKV("label", pwallet->mapAddressBook[s.destination].name); + entry.pushKV("label", pwallet->mapAddressBook.at(s.destination).name); } entry.pushKV("vout", s.vout); entry.pushKV("fee", ValueFromAmount(-nFee)); @@ -1342,7 +1331,7 @@ static void ListTransactions(interfaces::Chain::Lock& locked_chain, CWallet* con { std::string label; if (pwallet->mapAddressBook.count(r.destination)) { - label = pwallet->mapAddressBook[r.destination].name; + label = pwallet->mapAddressBook.at(r.destination).name; } if (filter_label && label != *filter_label) { continue; @@ -1402,7 +1391,7 @@ static const std::vector<RPCResult> TransactionDescriptionString() UniValue listtransactions(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -1516,7 +1505,7 @@ UniValue listtransactions(const JSONRPCRequest& request) static UniValue listsinceblock(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -1658,7 +1647,7 @@ static UniValue listsinceblock(const JSONRPCRequest& request) static UniValue gettransaction(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -1817,7 +1806,7 @@ static UniValue abandontransaction(const JSONRPCRequest& request) static UniValue backupwallet(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -2266,7 +2255,7 @@ static UniValue lockunspent(const JSONRPCRequest& request) static UniValue listlockunspent(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -2432,7 +2421,7 @@ static UniValue getbalances(const JSONRPCRequest& request) static UniValue getwalletinfo(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -2814,7 +2803,7 @@ static UniValue unloadwallet(const JSONRPCRequest& request) static UniValue listunspent(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -2973,7 +2962,7 @@ static UniValue listunspent(const JSONRPCRequest& request) entry.pushKV("label", i->second.name); } - std::unique_ptr<SigningProvider> provider = pwallet->GetSigningProvider(scriptPubKey); + std::unique_ptr<SigningProvider> provider = pwallet->GetSolvingProvider(scriptPubKey); if (provider) { if (scriptPubKey.IsPayToScriptHash()) { const CScriptID& hash = CScriptID(boost::get<ScriptHash>(address)); @@ -3013,7 +3002,7 @@ static UniValue listunspent(const JSONRPCRequest& request) entry.pushKV("spendable", out.fSpendable); entry.pushKV("solvable", out.fSolvable); if (out.fSolvable) { - std::unique_ptr<SigningProvider> provider = pwallet->GetSigningProvider(scriptPubKey); + std::unique_ptr<SigningProvider> provider = pwallet->GetSolvingProvider(scriptPubKey); if (provider) { auto descriptor = InferDescriptor(scriptPubKey, *provider); entry.pushKV("desc", descriptor->ToString()); @@ -3248,7 +3237,7 @@ static UniValue fundrawtransaction(const JSONRPCRequest& request) UniValue signrawtransactionwithwallet(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -3329,23 +3318,15 @@ UniValue signrawtransactionwithwallet(const JSONRPCRequest& request) // Parse the prevtxs array ParsePrevouts(request.params[1], nullptr, coins); - std::set<std::shared_ptr<SigningProvider>> providers; - for (const std::pair<COutPoint, Coin> coin_pair : coins) { - std::unique_ptr<SigningProvider> provider = pwallet->GetSigningProvider(coin_pair.second.out.scriptPubKey); - if (provider) { - providers.insert(std::move(provider)); - } - } - if (providers.size() == 0) { - // When there are no available providers, use a dummy SigningProvider so we can check if the tx is complete - providers.insert(std::make_shared<SigningProvider>()); - } + int nHashType = ParseSighashString(request.params[2]); + + // Script verification errors + std::map<int, std::string> input_errors; + bool complete = pwallet->SignTransaction(mtx, coins, nHashType, input_errors); UniValue result(UniValue::VOBJ); - for (std::shared_ptr<SigningProvider> provider : providers) { - SignTransaction(mtx, provider.get(), coins, request.params[2], result); - } - return result; + SignTransactionResultToJSON(mtx, complete, coins, input_errors, result); + return result; } static UniValue bumpfee(const JSONRPCRequest& request) @@ -3524,7 +3505,7 @@ static UniValue bumpfee(const JSONRPCRequest& request) } else { PartiallySignedTransaction psbtx(mtx); bool complete = false; - const TransactionError err = FillPSBT(pwallet, psbtx, complete, SIGHASH_ALL, false /* sign */, true /* bip32derivs */); + const TransactionError err = pwallet->FillPSBT(psbtx, complete, SIGHASH_ALL, false /* sign */, true /* bip32derivs */); CHECK_NONFATAL(err == TransactionError::OK); CHECK_NONFATAL(!complete); CDataStream ssTx(SER_NETWORK, PROTOCOL_VERSION); @@ -3728,14 +3709,14 @@ public: UniValue operator()(const WitnessUnknown& id) const { return UniValue(UniValue::VOBJ); } }; -static UniValue DescribeWalletAddress(CWallet* pwallet, const CTxDestination& dest) +static UniValue DescribeWalletAddress(const CWallet* const pwallet, const CTxDestination& dest) { UniValue ret(UniValue::VOBJ); UniValue detail = DescribeAddress(dest); CScript script = GetScriptForDestination(dest); std::unique_ptr<SigningProvider> provider = nullptr; if (pwallet) { - provider = pwallet->GetSigningProvider(script); + provider = pwallet->GetSolvingProvider(script); } ret.pushKVs(detail); ret.pushKVs(boost::apply_visitor(DescribeWalletAddressVisitor(provider.get()), dest)); @@ -3756,7 +3737,7 @@ static UniValue AddressBookDataToJSON(const CAddressBookData& data, const bool v UniValue getaddressinfo(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -3837,7 +3818,7 @@ UniValue getaddressinfo(const JSONRPCRequest& request) CScript scriptPubKey = GetScriptForDestination(dest); ret.pushKV("scriptPubKey", HexStr(scriptPubKey.begin(), scriptPubKey.end())); - std::unique_ptr<SigningProvider> provider = pwallet->GetSigningProvider(scriptPubKey); + std::unique_ptr<SigningProvider> provider = pwallet->GetSolvingProvider(scriptPubKey); isminetype mine = pwallet->IsMine(dest); ret.pushKV("ismine", bool(mine & ISMINE_SPENDABLE)); @@ -3858,7 +3839,7 @@ UniValue getaddressinfo(const JSONRPCRequest& request) // be associated with an address, so the label should be equivalent to the // value of the name key/value pair in the labels array below. if ((pwallet->chain().rpcEnableDeprecated("label")) && (pwallet->mapAddressBook.count(dest))) { - ret.pushKV("label", pwallet->mapAddressBook[dest].name); + ret.pushKV("label", pwallet->mapAddressBook.at(dest).name); } ret.pushKV("ischange", pwallet->IsChange(scriptPubKey)); @@ -3881,7 +3862,7 @@ UniValue getaddressinfo(const JSONRPCRequest& request) // stable if we allow multiple labels to be associated with an address in // the future. UniValue labels(UniValue::VARR); - std::map<CTxDestination, CAddressBookData>::iterator mi = pwallet->mapAddressBook.find(dest); + std::map<CTxDestination, CAddressBookData>::const_iterator mi = pwallet->mapAddressBook.find(dest); if (mi != pwallet->mapAddressBook.end()) { // DEPRECATED: The previous behavior of returning an array containing a // JSON object of `name` and `purpose` key/value pairs is deprecated. @@ -3899,7 +3880,7 @@ UniValue getaddressinfo(const JSONRPCRequest& request) static UniValue getaddressesbylabel(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -3958,7 +3939,7 @@ static UniValue getaddressesbylabel(const JSONRPCRequest& request) static UniValue listlabels(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -4091,7 +4072,7 @@ UniValue sethdseed(const JSONRPCRequest& request) UniValue walletprocesspsbt(const JSONRPCRequest& request) { std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request); - CWallet* const pwallet = wallet.get(); + const CWallet* const pwallet = wallet.get(); if (!EnsureWalletIsAvailable(pwallet, request.fHelp)) { return NullUniValue; @@ -4141,7 +4122,7 @@ UniValue walletprocesspsbt(const JSONRPCRequest& request) bool sign = request.params[1].isNull() ? true : request.params[1].get_bool(); bool bip32derivs = request.params[3].isNull() ? true : request.params[3].get_bool(); bool complete = true; - const TransactionError err = FillPSBT(pwallet, psbtx, complete, nHashType, sign, bip32derivs); + const TransactionError err = pwallet->FillPSBT(psbtx, complete, nHashType, sign, bip32derivs); if (err != TransactionError::OK) { throw JSONRPCTransactionError(err); } @@ -4264,7 +4245,7 @@ UniValue walletcreatefundedpsbt(const JSONRPCRequest& request) // Fill transaction with out data but don't sign bool bip32derivs = request.params[4].isNull() ? true : request.params[4].get_bool(); bool complete = true; - const TransactionError err = FillPSBT(pwallet, psbtx, complete, 1, false, bip32derivs); + const TransactionError err = pwallet->FillPSBT(psbtx, complete, 1, false, bip32derivs); if (err != TransactionError::OK) { throw JSONRPCTransactionError(err); } diff --git a/src/wallet/scriptpubkeyman.cpp b/src/wallet/scriptpubkeyman.cpp index 0c95ab29b1..6fe1d84d64 100644 --- a/src/wallet/scriptpubkeyman.cpp +++ b/src/wallet/scriptpubkeyman.cpp @@ -5,6 +5,7 @@ #include <key_io.h> #include <outputtype.h> #include <script/descriptor.h> +#include <script/sign.h> #include <util/bip32.h> #include <util/strencodings.h> #include <util/translation.h> @@ -358,7 +359,7 @@ bool LegacyScriptPubKeyMan::IsHDEnabled() const return !hdChain.seed_id.IsNull(); } -bool LegacyScriptPubKeyMan::CanGetAddresses(bool internal) +bool LegacyScriptPubKeyMan::CanGetAddresses(bool internal) const { LOCK(cs_KeyStore); // Check if the keypool has keys @@ -441,7 +442,7 @@ static int64_t GetOldestKeyTimeInPool(const std::set<int64_t>& setKeyPool, Walle return keypool.nTime; } -int64_t LegacyScriptPubKeyMan::GetOldestKeyPoolTime() +int64_t LegacyScriptPubKeyMan::GetOldestKeyPoolTime() const { LOCK(cs_KeyStore); @@ -459,7 +460,7 @@ int64_t LegacyScriptPubKeyMan::GetOldestKeyPoolTime() return oldestKey; } -size_t LegacyScriptPubKeyMan::KeypoolCountExternalKeys() +size_t LegacyScriptPubKeyMan::KeypoolCountExternalKeys() const { LOCK(cs_KeyStore); return setExternalKeyPool.size() + set_pre_split_keypool.size(); @@ -477,7 +478,7 @@ int64_t LegacyScriptPubKeyMan::GetTimeFirstKey() const return nTimeFirstKey; } -std::unique_ptr<SigningProvider> LegacyScriptPubKeyMan::GetSigningProvider(const CScript& script) const +std::unique_ptr<SigningProvider> LegacyScriptPubKeyMan::GetSolvingProvider(const CScript& script) const { return MakeUnique<LegacySigningProvider>(*this); } @@ -505,6 +506,67 @@ bool LegacyScriptPubKeyMan::CanProvide(const CScript& script, SignatureData& sig } } +bool LegacyScriptPubKeyMan::SignTransaction(CMutableTransaction& tx, const std::map<COutPoint, Coin>& coins, int sighash, std::map<int, std::string>& input_errors) const +{ + return ::SignTransaction(tx, this, coins, sighash, input_errors); +} + +SigningResult LegacyScriptPubKeyMan::SignMessage(const std::string& message, const PKHash& pkhash, std::string& str_sig) const +{ + CKeyID key_id(pkhash); + CKey key; + if (!GetKey(key_id, key)) { + return SigningResult::PRIVATE_KEY_NOT_AVAILABLE; + } + + if (MessageSign(key, message, str_sig)) { + return SigningResult::OK; + } + return SigningResult::SIGNING_FAILED; +} + +TransactionError LegacyScriptPubKeyMan::FillPSBT(PartiallySignedTransaction& psbtx, int sighash_type, bool sign, bool bip32derivs) const +{ + for (unsigned int i = 0; i < psbtx.tx->vin.size(); ++i) { + const CTxIn& txin = psbtx.tx->vin[i]; + PSBTInput& input = psbtx.inputs.at(i); + + if (PSBTInputSigned(input)) { + continue; + } + + // Verify input looks sane. This will check that we have at most one uxto, witness or non-witness. + if (!input.IsSane()) { + return TransactionError::INVALID_PSBT; + } + + // Get the Sighash type + if (sign && input.sighash_type > 0 && input.sighash_type != sighash_type) { + return TransactionError::SIGHASH_MISMATCH; + } + + // Check non_witness_utxo has specified prevout + if (input.non_witness_utxo) { + if (txin.prevout.n >= input.non_witness_utxo->vout.size()) { + return TransactionError::MISSING_INPUTS; + } + } else if (input.witness_utxo.IsNull()) { + // There's no UTXO so we can just skip this now + continue; + } + SignatureData sigdata; + input.FillSignatureData(sigdata); + SignPSBTInput(HidingSigningProvider(this, !sign, !bip32derivs), psbtx, i, sighash_type); + } + + // Fill in the bip32 keypaths and redeemscripts for the outputs so that hardware wallets can identify change + for (unsigned int i = 0; i < psbtx.tx->vout.size(); ++i) { + UpdatePSBTOutput(HidingSigningProvider(this, true, !bip32derivs), psbtx, i); + } + + return TransactionError::OK; +} + const CKeyMetadata* LegacyScriptPubKeyMan::GetMetadata(const CTxDestination& dest) const { LOCK(cs_KeyStore); @@ -973,7 +1035,7 @@ void LegacyScriptPubKeyMan::LoadKeyPool(int64_t nIndex, const CKeyPool &keypool) mapKeyMetadata[keyid] = CKeyMetadata(keypool.nTime); } -bool LegacyScriptPubKeyMan::CanGenerateKeys() +bool LegacyScriptPubKeyMan::CanGenerateKeys() const { // A wallet can generate keys if it has an HD seed (IsHDEnabled) or it is a non-HD wallet (pre FEATURE_HD) LOCK(cs_KeyStore); diff --git a/src/wallet/scriptpubkeyman.h b/src/wallet/scriptpubkeyman.h index 7b1c023bc9..8512eadf31 100644 --- a/src/wallet/scriptpubkeyman.h +++ b/src/wallet/scriptpubkeyman.h @@ -5,8 +5,11 @@ #ifndef BITCOIN_WALLET_SCRIPTPUBKEYMAN_H #define BITCOIN_WALLET_SCRIPTPUBKEYMAN_H +#include <psbt.h> #include <script/signingprovider.h> #include <script/standard.h> +#include <util/error.h> +#include <util/message.h> #include <wallet/crypter.h> #include <wallet/ismine.h> #include <wallet/walletdb.h> @@ -184,7 +187,7 @@ public: virtual bool IsHDEnabled() const { return false; } /* Returns true if the wallet can give out new addresses. This means it has keys in the keypool or can generate new keys */ - virtual bool CanGetAddresses(bool internal = false) { return false; } + virtual bool CanGetAddresses(bool internal = false) const { return false; } /** Upgrades the wallet to the specified version */ virtual bool Upgrade(int prev_version, std::string& error) { return false; } @@ -194,22 +197,29 @@ public: //! The action to do when the DB needs rewrite virtual void RewriteDB() {} - virtual int64_t GetOldestKeyPoolTime() { return GetTime(); } + virtual int64_t GetOldestKeyPoolTime() const { return GetTime(); } - virtual size_t KeypoolCountExternalKeys() { return 0; } + virtual size_t KeypoolCountExternalKeys() const { return 0; } virtual unsigned int GetKeyPoolSize() const { return 0; } virtual int64_t GetTimeFirstKey() const { return 0; } virtual const CKeyMetadata* GetMetadata(const CTxDestination& dest) const { return nullptr; } - virtual std::unique_ptr<SigningProvider> GetSigningProvider(const CScript& script) const { return nullptr; } + virtual std::unique_ptr<SigningProvider> GetSolvingProvider(const CScript& script) const { return nullptr; } - /** Whether this ScriptPubKeyMan can provide a SigningProvider (via GetSigningProvider) that, combined with - * sigdata, can produce a valid signature. + /** Whether this ScriptPubKeyMan can provide a SigningProvider (via GetSolvingProvider) that, combined with + * sigdata, can produce solving data. */ virtual bool CanProvide(const CScript& script, SignatureData& sigdata) { return false; } + /** Creates new signatures and adds them to the transaction. Returns whether all inputs were signed */ + virtual bool SignTransaction(CMutableTransaction& tx, const std::map<COutPoint, Coin>& coins, int sighash, std::map<int, std::string>& input_errors) const { return false; } + /** Sign a message with the given script */ + virtual SigningResult SignMessage(const std::string& message, const PKHash& pkhash, std::string& str_sig) const { return SigningResult::SIGNING_FAILED; }; + /** Adds script and derivation path information to a PSBT, and optionally signs it. */ + virtual TransactionError FillPSBT(PartiallySignedTransaction& psbt, int sighash_type = 1 /* SIGHASH_ALL */, bool sign = true, bool bip32derivs = false) const { return TransactionError::INVALID_PSBT; } + virtual uint256 GetID() const { return uint256(); } /** Prepends the wallet name in logging output to ease debugging in multi-wallet use cases */ @@ -336,20 +346,24 @@ public: void RewriteDB() override; - int64_t GetOldestKeyPoolTime() override; - size_t KeypoolCountExternalKeys() override; + int64_t GetOldestKeyPoolTime() const override; + size_t KeypoolCountExternalKeys() const override; unsigned int GetKeyPoolSize() const override; int64_t GetTimeFirstKey() const override; const CKeyMetadata* GetMetadata(const CTxDestination& dest) const override; - bool CanGetAddresses(bool internal = false) override; + bool CanGetAddresses(bool internal = false) const override; - std::unique_ptr<SigningProvider> GetSigningProvider(const CScript& script) const override; + std::unique_ptr<SigningProvider> GetSolvingProvider(const CScript& script) const override; bool CanProvide(const CScript& script, SignatureData& sigdata) override; + bool SignTransaction(CMutableTransaction& tx, const std::map<COutPoint, Coin>& coins, int sighash, std::map<int, std::string>& input_errors) const override; + SigningResult SignMessage(const std::string& message, const PKHash& pkhash, std::string& str_sig) const override; + TransactionError FillPSBT(PartiallySignedTransaction& psbt, int sighash_type = 1 /* SIGHASH_ALL */, bool sign = true, bool bip32derivs = false) const override; + uint256 GetID() const override; // Map from Key ID to key metadata. @@ -410,7 +424,7 @@ public: bool ImportScriptPubKeys(const std::set<CScript>& script_pub_keys, const bool have_solving_data, const int64_t timestamp) EXCLUSIVE_LOCKS_REQUIRED(cs_KeyStore); /* Returns true if the wallet can generate new keys */ - bool CanGenerateKeys(); + bool CanGenerateKeys() const; /* Generates a new HD seed (will not be activated) */ CPubKey GenerateNewSeed(); @@ -447,7 +461,7 @@ public: std::set<CKeyID> GetKeys() const override; }; -/** Wraps a LegacyScriptPubKeyMan so that it can be returned in a new unique_ptr */ +/** Wraps a LegacyScriptPubKeyMan so that it can be returned in a new unique_ptr. Does not provide privkeys */ class LegacySigningProvider : public SigningProvider { private: @@ -458,8 +472,8 @@ public: bool GetCScript(const CScriptID &scriptid, CScript& script) const override { return m_spk_man.GetCScript(scriptid, script); } bool HaveCScript(const CScriptID &scriptid) const override { return m_spk_man.HaveCScript(scriptid); } bool GetPubKey(const CKeyID &address, CPubKey& pubkey) const override { return m_spk_man.GetPubKey(address, pubkey); } - bool GetKey(const CKeyID &address, CKey& key) const override { return m_spk_man.GetKey(address, key); } - bool HaveKey(const CKeyID &address) const override { return m_spk_man.HaveKey(address); } + bool GetKey(const CKeyID &address, CKey& key) const override { return false; } + bool HaveKey(const CKeyID &address) const override { return false; } bool GetKeyOrigin(const CKeyID& keyid, KeyOriginInfo& info) const override { return m_spk_man.GetKeyOrigin(keyid, info); } }; diff --git a/src/wallet/test/psbt_wallet_tests.cpp b/src/wallet/test/psbt_wallet_tests.cpp index f923de6178..8b7b7af21d 100644 --- a/src/wallet/test/psbt_wallet_tests.cpp +++ b/src/wallet/test/psbt_wallet_tests.cpp @@ -5,7 +5,6 @@ #include <key_io.h> #include <util/bip32.h> #include <util/strencodings.h> -#include <wallet/psbtwallet.h> #include <wallet/wallet.h> #include <boost/test/unit_test.hpp> @@ -61,7 +60,7 @@ BOOST_AUTO_TEST_CASE(psbt_updater_test) // Fill transaction with our data bool complete = true; - BOOST_REQUIRE_EQUAL(TransactionError::OK, FillPSBT(&m_wallet, psbtx, complete, SIGHASH_ALL, false, true)); + BOOST_REQUIRE_EQUAL(TransactionError::OK, m_wallet.FillPSBT(psbtx, complete, SIGHASH_ALL, false, true)); // Get the final tx CDataStream ssTx(SER_NETWORK, PROTOCOL_VERSION); @@ -74,9 +73,7 @@ BOOST_AUTO_TEST_CASE(psbt_updater_test) // Try to sign the mutated input SignatureData sigdata; - psbtx.inputs[0].FillSignatureData(sigdata); - const std::unique_ptr<SigningProvider> provider = m_wallet.GetSigningProvider(ws1, sigdata); - BOOST_CHECK(!SignPSBTInput(*provider, psbtx, 0, SIGHASH_ALL)); + BOOST_CHECK(spk_man->FillPSBT(psbtx, SIGHASH_ALL, true, true) != TransactionError::OK); } BOOST_AUTO_TEST_CASE(parse_hd_keypath) diff --git a/src/wallet/wallet.cpp b/src/wallet/wallet.cpp index 5a97f62b71..79e29d050f 100644 --- a/src/wallet/wallet.cpp +++ b/src/wallet/wallet.cpp @@ -1150,7 +1150,7 @@ void CWallet::UpdatedBlockTip() } -void CWallet::BlockUntilSyncedToCurrentChain() { +void CWallet::BlockUntilSyncedToCurrentChain() const { AssertLockNotHeld(cs_wallet); // Skip the queue-draining stuff if we know we're caught up with // ::ChainActive().Tip(), otherwise put a callback in the validation interface queue and wait @@ -1333,7 +1333,7 @@ bool CWallet::IsHDEnabled() const return result; } -bool CWallet::CanGetAddresses(bool internal) +bool CWallet::CanGetAddresses(bool internal) const { LOCK(cs_wallet); if (m_spk_managers.empty()) return false; @@ -1407,7 +1407,7 @@ bool CWallet::DummySignInput(CTxIn &tx_in, const CTxOut &txout, bool use_max_sig const CScript& scriptPubKey = txout.scriptPubKey; SignatureData sigdata; - std::unique_ptr<SigningProvider> provider = GetSigningProvider(scriptPubKey); + std::unique_ptr<SigningProvider> provider = GetSolvingProvider(scriptPubKey); if (!provider) { // We don't know about this scriptpbuKey; return false; @@ -2171,7 +2171,7 @@ void CWallet::AvailableCoins(interfaces::Chain::Lock& locked_chain, std::vector< continue; } - std::unique_ptr<SigningProvider> provider = GetSigningProvider(wtx.tx->vout[i].scriptPubKey); + std::unique_ptr<SigningProvider> provider = GetSolvingProvider(wtx.tx->vout[i].scriptPubKey); bool solvable = provider ? IsSolvable(*provider, wtx.tx->vout[i].scriptPubKey) : false; bool spendable = ((mine & ISMINE_SPENDABLE) != ISMINE_NO) || (((mine & ISMINE_WATCH_ONLY) != ISMINE_NO) && (coinControl && coinControl->fAllowWatchOnly && solvable)); @@ -2410,34 +2410,172 @@ bool CWallet::SelectCoins(const std::vector<COutput>& vAvailableCoins, const CAm return res; } -bool CWallet::SignTransaction(CMutableTransaction& tx) +bool CWallet::SignTransaction(CMutableTransaction& tx) const { AssertLockHeld(cs_wallet); - // sign the new tx - int nIn = 0; + // Build coins map + std::map<COutPoint, Coin> coins; for (auto& input : tx.vin) { std::map<uint256, CWalletTx>::const_iterator mi = mapWallet.find(input.prevout.hash); if(mi == mapWallet.end() || input.prevout.n >= mi->second.tx->vout.size()) { return false; } - const CScript& scriptPubKey = mi->second.tx->vout[input.prevout.n].scriptPubKey; - const CAmount& amount = mi->second.tx->vout[input.prevout.n].nValue; + const CWalletTx& wtx = mi->second; + coins[input.prevout] = Coin(wtx.tx->vout[input.prevout.n], wtx.m_confirm.block_height, wtx.IsCoinBase()); + } + std::map<int, std::string> input_errors; + return SignTransaction(tx, coins, SIGHASH_ALL, input_errors); +} + +bool CWallet::SignTransaction(CMutableTransaction& tx, const std::map<COutPoint, Coin>& coins, int sighash, std::map<int, std::string>& input_errors) const +{ + // Sign the tx with ScriptPubKeyMans + // Because each ScriptPubKeyMan can sign more than one input, we need to keep track of each ScriptPubKeyMan that has signed this transaction. + // Each iteration, we may sign more txins than the txin that is specified in that iteration. + // We assume that each input is signed by only one ScriptPubKeyMan. + std::set<uint256> visited_spk_mans; + for (unsigned int i = 0; i < tx.vin.size(); i++) { + // Get the prevout + CTxIn& txin = tx.vin[i]; + auto coin = coins.find(txin.prevout); + if (coin == coins.end() || coin->second.IsSpent()) { + input_errors[i] = "Input not found or already spent"; + continue; + } + + // Check if this input is complete + SignatureData sigdata = DataFromTransaction(tx, i, coin->second.out); + if (sigdata.complete) { + continue; + } + + // Input needs to be signed, find the right ScriptPubKeyMan + std::set<ScriptPubKeyMan*> spk_mans = GetScriptPubKeyMans(coin->second.out.scriptPubKey, sigdata); + if (spk_mans.size() == 0) { + input_errors[i] = "Unable to sign input, missing keys"; + continue; + } + + for (auto& spk_man : spk_mans) { + // If we've already been signed by this spk_man, skip it + if (visited_spk_mans.count(spk_man->GetID()) > 0) { + continue; + } + + // Sign the tx. + // spk_man->SignTransaction will return true if the transaction is complete, + // so we can exit early and return true if that happens. + if (spk_man->SignTransaction(tx, coins, sighash, input_errors)) { + return true; + } + + // Add this spk_man to visited_spk_mans so we can skip it later + visited_spk_mans.insert(spk_man->GetID()); + } + } + return false; +} + +TransactionError CWallet::FillPSBT(PartiallySignedTransaction& psbtx, bool& complete, int sighash_type, bool sign, bool bip32derivs) const +{ + LOCK(cs_wallet); + // Get all of the previous transactions + for (unsigned int i = 0; i < psbtx.tx->vin.size(); ++i) { + const CTxIn& txin = psbtx.tx->vin[i]; + PSBTInput& input = psbtx.inputs.at(i); + + if (PSBTInputSigned(input)) { + continue; + } + + // Verify input looks sane. This will check that we have at most one uxto, witness or non-witness. + if (!input.IsSane()) { + return TransactionError::INVALID_PSBT; + } + + // If we have no utxo, grab it from the wallet. + if (!input.non_witness_utxo && input.witness_utxo.IsNull()) { + const uint256& txhash = txin.prevout.hash; + const auto it = mapWallet.find(txhash); + if (it != mapWallet.end()) { + const CWalletTx& wtx = it->second; + // We only need the non_witness_utxo, which is a superset of the witness_utxo. + // The signing code will switch to the smaller witness_utxo if this is ok. + input.non_witness_utxo = wtx.tx; + } + } + } + + // Fill in information from ScriptPubKeyMans + // Because each ScriptPubKeyMan may be able to fill more than one input, we need to keep track of each ScriptPubKeyMan that has filled this psbt. + // Each iteration, we may fill more inputs than the input that is specified in that iteration. + // We assume that each input is filled by only one ScriptPubKeyMan + std::set<uint256> visited_spk_mans; + for (unsigned int i = 0; i < psbtx.tx->vin.size(); ++i) { + const CTxIn& txin = psbtx.tx->vin[i]; + PSBTInput& input = psbtx.inputs.at(i); + + if (PSBTInputSigned(input)) { + continue; + } + + // Get the scriptPubKey to know which ScriptPubKeyMan to use + CScript script; + if (!input.witness_utxo.IsNull()) { + script = input.witness_utxo.scriptPubKey; + } else if (input.non_witness_utxo) { + if (txin.prevout.n >= input.non_witness_utxo->vout.size()) { + return TransactionError::MISSING_INPUTS; + } + script = input.non_witness_utxo->vout[txin.prevout.n].scriptPubKey; + } else { + // There's no UTXO so we can just skip this now + continue; + } SignatureData sigdata; + input.FillSignatureData(sigdata); + std::set<ScriptPubKeyMan*> spk_mans = GetScriptPubKeyMans(script, sigdata); + if (spk_mans.size() == 0) { + continue; + } - std::unique_ptr<SigningProvider> provider = GetSigningProvider(scriptPubKey); - if (!provider) { - // We don't know about this scriptpbuKey; - return false; + for (auto& spk_man : spk_mans) { + // If we've already been signed by this spk_man, skip it + if (visited_spk_mans.count(spk_man->GetID()) > 0) { + continue; + } + + // Fill in the information from the spk_man + TransactionError res = spk_man->FillPSBT(psbtx, sighash_type, sign, bip32derivs); + if (res != TransactionError::OK) { + return res; + } + + // Add this spk_man to visited_spk_mans so we can skip it later + visited_spk_mans.insert(spk_man->GetID()); } + } - if (!ProduceSignature(*provider, MutableTransactionSignatureCreator(&tx, nIn, amount, SIGHASH_ALL), scriptPubKey, sigdata)) { - return false; + // Complete if every input is now signed + complete = true; + for (const auto& input : psbtx.inputs) { + complete &= PSBTInputSigned(input); + } + + return TransactionError::OK; +} + +SigningResult CWallet::SignMessage(const std::string& message, const PKHash& pkhash, std::string& str_sig) const +{ + SignatureData sigdata; + CScript script_pub_key = GetScriptForDestination(pkhash); + for (const auto& spk_man_pair : m_spk_managers) { + if (spk_man_pair.second->CanProvide(script_pub_key, sigdata)) { + return spk_man_pair.second->SignMessage(message, pkhash, str_sig); } - UpdateInput(input, sigdata); - nIn++; } - return true; + return SigningResult::PRIVATE_KEY_NOT_AVAILABLE; } bool CWallet::FundTransaction(CMutableTransaction& tx, CAmount& nFeeRet, int& nChangePosInOut, std::string& strFailReason, bool lockUnspents, const std::set<int>& setSubtractFeeFromOutputs, CCoinControl coinControl) @@ -2886,25 +3024,9 @@ bool CWallet::CreateTransaction(interfaces::Chain::Lock& locked_chain, const std txNew.vin.push_back(CTxIn(coin.outpoint, CScript(), nSequence)); } - if (sign) - { - int nIn = 0; - for (const auto& coin : selected_coins) - { - const CScript& scriptPubKey = coin.txout.scriptPubKey; - SignatureData sigdata; - - std::unique_ptr<SigningProvider> provider = GetSigningProvider(scriptPubKey); - if (!provider || !ProduceSignature(*provider, MutableTransactionSignatureCreator(&txNew, nIn, coin.txout.nValue, SIGHASH_ALL), scriptPubKey, sigdata)) - { - strFailReason = _("Signing transaction failed").translated; - return false; - } else { - UpdateInput(txNew.vin.at(nIn), sigdata); - } - - nIn++; - } + if (sign && !SignTransaction(txNew)) { + strFailReason = _("Signing transaction failed").translated; + return false; } // Return the constructed transaction data. @@ -3112,7 +3234,7 @@ bool CWallet::DelAddressBook(const CTxDestination& address) return WalletBatch(*database).EraseName(EncodeDestination(address)); } -size_t CWallet::KeypoolCountExternalKeys() +size_t CWallet::KeypoolCountExternalKeys() const { AssertLockHeld(cs_wallet); @@ -3177,7 +3299,7 @@ bool CWallet::GetNewChangeDestination(const OutputType type, CTxDestination& des return true; } -int64_t CWallet::GetOldestKeyPoolTime() +int64_t CWallet::GetOldestKeyPoolTime() const { LOCK(cs_wallet); int64_t oldestKey = std::numeric_limits<int64_t>::max(); @@ -3201,7 +3323,7 @@ void CWallet::MarkDestinationsDirty(const std::set<CTxDestination>& destinations } } -std::map<CTxDestination, CAmount> CWallet::GetAddressBalances(interfaces::Chain::Lock& locked_chain) +std::map<CTxDestination, CAmount> CWallet::GetAddressBalances(interfaces::Chain::Lock& locked_chain) const { std::map<CTxDestination, CAmount> balances; @@ -3242,7 +3364,7 @@ std::map<CTxDestination, CAmount> CWallet::GetAddressBalances(interfaces::Chain: return balances; } -std::set< std::set<CTxDestination> > CWallet::GetAddressGroupings() +std::set< std::set<CTxDestination> > CWallet::GetAddressGroupings() const { AssertLockHeld(cs_wallet); std::set< std::set<CTxDestination> > groupings; @@ -4004,7 +4126,7 @@ void CWallet::postInitProcess() chain().requestMempoolTransactions(*this); } -bool CWallet::BackupWallet(const std::string& strDest) +bool CWallet::BackupWallet(const std::string& strDest) const { return database->Backup(strDest); } @@ -4155,6 +4277,17 @@ ScriptPubKeyMan* CWallet::GetScriptPubKeyMan(const OutputType& type, bool intern return it->second; } +std::set<ScriptPubKeyMan*> CWallet::GetScriptPubKeyMans(const CScript& script, SignatureData& sigdata) const +{ + std::set<ScriptPubKeyMan*> spk_mans; + for (const auto& spk_man_pair : m_spk_managers) { + if (spk_man_pair.second->CanProvide(script, sigdata)) { + spk_mans.insert(spk_man_pair.second.get()); + } + } + return spk_mans; +} + ScriptPubKeyMan* CWallet::GetScriptPubKeyMan(const CScript& script) const { SignatureData sigdata; @@ -4174,17 +4307,17 @@ ScriptPubKeyMan* CWallet::GetScriptPubKeyMan(const uint256& id) const return nullptr; } -std::unique_ptr<SigningProvider> CWallet::GetSigningProvider(const CScript& script) const +std::unique_ptr<SigningProvider> CWallet::GetSolvingProvider(const CScript& script) const { SignatureData sigdata; - return GetSigningProvider(script, sigdata); + return GetSolvingProvider(script, sigdata); } -std::unique_ptr<SigningProvider> CWallet::GetSigningProvider(const CScript& script, SignatureData& sigdata) const +std::unique_ptr<SigningProvider> CWallet::GetSolvingProvider(const CScript& script, SignatureData& sigdata) const { for (const auto& spk_man_pair : m_spk_managers) { if (spk_man_pair.second->CanProvide(script, sigdata)) { - return spk_man_pair.second->GetSigningProvider(script); + return spk_man_pair.second->GetSolvingProvider(script); } } return nullptr; diff --git a/src/wallet/wallet.h b/src/wallet/wallet.h index 6c1c3040c2..0c86a0c1e8 100644 --- a/src/wallet/wallet.h +++ b/src/wallet/wallet.h @@ -11,8 +11,10 @@ #include <interfaces/handler.h> #include <outputtype.h> #include <policy/feerate.h> +#include <psbt.h> #include <tinyformat.h> #include <ui_interface.h> +#include <util/message.h> #include <util/strencodings.h> #include <util/system.h> #include <validationinterface.h> @@ -141,7 +143,7 @@ class ReserveDestination { protected: //! The wallet to reserve from - CWallet* const pwallet; + const CWallet* const pwallet; //! The ScriptPubKeyMan to reserve from. Based on type when GetReservedDestination is called ScriptPubKeyMan* m_spk_man{nullptr}; OutputType const type; @@ -832,8 +834,8 @@ public: * Rescan abort properties */ void AbortRescan() { fAbortRescan = true; } - bool IsAbortingRescan() { return fAbortRescan; } - bool IsScanning() { return fScanningWallet; } + bool IsAbortingRescan() const { return fAbortRescan; } + bool IsScanning() const { return fScanningWallet; } int64_t ScanningDuration() const { return fScanningWallet ? GetTimeMillis() - m_scanning_start : 0; } double ScanningProgress() const { return fScanningWallet ? (double) m_scanning_progress : 0; } @@ -916,7 +918,30 @@ public: * calling CreateTransaction(); */ bool FundTransaction(CMutableTransaction& tx, CAmount& nFeeRet, int& nChangePosInOut, std::string& strFailReason, bool lockUnspents, const std::set<int>& setSubtractFeeFromOutputs, CCoinControl); - bool SignTransaction(CMutableTransaction& tx) EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); + // Fetch the inputs and sign with SIGHASH_ALL. + bool SignTransaction(CMutableTransaction& tx) const EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); + // Sign the tx given the input coins and sighash. + bool SignTransaction(CMutableTransaction& tx, const std::map<COutPoint, Coin>& coins, int sighash, std::map<int, std::string>& input_errors) const; + SigningResult SignMessage(const std::string& message, const PKHash& pkhash, std::string& str_sig) const; + + /** + * Fills out a PSBT with information from the wallet. Fills in UTXOs if we have + * them. Tries to sign if sign=true. Sets `complete` if the PSBT is now complete + * (i.e. has all required signatures or signature-parts, and is ready to + * finalize.) Sets `error` and returns false if something goes wrong. + * + * @param[in] psbtx PartiallySignedTransaction to fill in + * @param[out] complete indicates whether the PSBT is now complete + * @param[in] sighash_type the sighash type to use when signing (if PSBT does not specify) + * @param[in] sign whether to sign or not + * @param[in] bip32derivs whether to fill in bip32 derivation information if available + * return error + */ + TransactionError FillPSBT(PartiallySignedTransaction& psbtx, + bool& complete, + int sighash_type = 1 /* SIGHASH_ALL */, + bool sign = true, + bool bip32derivs = true) const; /** * Create a new transaction paying the recipients with a set of coins @@ -968,13 +993,13 @@ public: /** Absolute maximum transaction fee (in satoshis) used by default for the wallet */ CAmount m_default_max_tx_fee{DEFAULT_TRANSACTION_MAXFEE}; - size_t KeypoolCountExternalKeys() EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); + size_t KeypoolCountExternalKeys() const EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); bool TopUpKeyPool(unsigned int kpSize = 0); - int64_t GetOldestKeyPoolTime(); + int64_t GetOldestKeyPoolTime() const; - std::set<std::set<CTxDestination>> GetAddressGroupings() EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); - std::map<CTxDestination, CAmount> GetAddressBalances(interfaces::Chain::Lock& locked_chain); + std::set<std::set<CTxDestination>> GetAddressGroupings() const EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); + std::map<CTxDestination, CAmount> GetAddressBalances(interfaces::Chain::Lock& locked_chain) const; std::set<CTxDestination> GetLabelAddresses(const std::string& label) const; @@ -1027,7 +1052,7 @@ public: bool SetMaxVersion(int nVersion); //! get the current wallet format (the oldest client version guaranteed to understand this wallet) - int GetVersion() { LOCK(cs_wallet); return nWalletVersion; } + int GetVersion() const { LOCK(cs_wallet); return nWalletVersion; } //! Get wallet transactions that conflict with given transaction (spend same outputs) std::set<uint256> GetConflicts(const uint256& txid) const EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); @@ -1098,13 +1123,13 @@ public: */ void postInitProcess(); - bool BackupWallet(const std::string& strDest); + bool BackupWallet(const std::string& strDest) const; /* Returns true if HD is enabled */ bool IsHDEnabled() const; /* Returns true if the wallet can give out new addresses. This means it has keys in the keypool or can generate new keys */ - bool CanGetAddresses(bool internal = false); + bool CanGetAddresses(bool internal = false) const; /** * Blocks until the wallet state is up-to-date to /at least/ the current @@ -1112,7 +1137,7 @@ public: * Obviously holding cs_main/cs_wallet when going into this call may cause * deadlock */ - void BlockUntilSyncedToCurrentChain() LOCKS_EXCLUDED(cs_main, cs_wallet); + void BlockUntilSyncedToCurrentChain() const LOCKS_EXCLUDED(cs_main, cs_wallet); /** set a single wallet flag */ void SetWalletFlag(uint64_t flags); @@ -1153,9 +1178,12 @@ public: //! Get the ScriptPubKeyMan by id ScriptPubKeyMan* GetScriptPubKeyMan(const uint256& id) const; + //! Get all of the ScriptPubKeyMans for a script given additional information in sigdata (populated by e.g. a psbt) + std::set<ScriptPubKeyMan*> GetScriptPubKeyMans(const CScript& script, SignatureData& sigdata) const; + //! Get the SigningProvider for a script - std::unique_ptr<SigningProvider> GetSigningProvider(const CScript& script) const; - std::unique_ptr<SigningProvider> GetSigningProvider(const CScript& script, SignatureData& sigdata) const; + std::unique_ptr<SigningProvider> GetSolvingProvider(const CScript& script) const; + std::unique_ptr<SigningProvider> GetSolvingProvider(const CScript& script, SignatureData& sigdata) const; //! Get the LegacyScriptPubKeyMan which is used for all types, internal, and external. LegacyScriptPubKeyMan* GetLegacyScriptPubKeyMan() const; diff --git a/test/README.md b/test/README.md index c3e4ae9ad2..e1dab92a06 100644 --- a/test/README.md +++ b/test/README.md @@ -145,7 +145,7 @@ levels using the logger included in the test_framework, e.g. `test_framework.log` and no logs are output to the console. - when run directly, *all* logs are written to `test_framework.log` and INFO level and above are output to the console. -- when run on Travis, no logs are output to the console. However, if a test +- when run by [our CI (Continuous Integration)](/ci/README.md), no logs are output to the console. However, if a test fails, the `test_framework.log` and bitcoind `debug.log`s will all be dumped to the console to help troubleshooting. diff --git a/test/functional/README.md b/test/functional/README.md index 77a9ce9acb..6582c1cbcd 100644 --- a/test/functional/README.md +++ b/test/functional/README.md @@ -51,10 +51,13 @@ don't have test cases for. #### General test-writing advice +- Instead of inline comments or no test documentation at all, log the comments to the test log, e.g. + `self.log.info('Create enough transactions to fill a block')`. Logs make the test code easier to read and the test + logic easier [to debug](/test/README.md#test-logging). - Set `self.num_nodes` to the minimum number of nodes necessary for the test. Having additional unrequired nodes adds to the execution time of the test as well as memory/CPU/disk requirements (which is important when running tests in - parallel or on Travis). + parallel). - Avoid stop-starting the nodes multiple times during the test if possible. A stop-start takes several seconds, so doing it several times blows up the runtime of the test. diff --git a/test/functional/feature_cltv.py b/test/functional/feature_cltv.py index e2b347f925..073ed8d7c7 100755 --- a/test/functional/feature_cltv.py +++ b/test/functional/feature_cltv.py @@ -60,7 +60,7 @@ class BIP65Test(BitcoinTestFramework): '-acceptnonstdtxn=1', # cltv_invalidate is nonstandard ]] self.setup_clean_chain = True - self.rpc_timeout = 120 + self.rpc_timeout = 480 def skip_test_if_missing_module(self): self.skip_if_no_wallet() diff --git a/test/functional/feature_segwit.py b/test/functional/feature_segwit.py index 82c7e55245..909a43c8d9 100755 --- a/test/functional/feature_segwit.py +++ b/test/functional/feature_segwit.py @@ -72,6 +72,7 @@ class SegWitTest(BitcoinTestFramework): "-addresstype=legacy", ], ] + self.rpc_timeout = 120 def skip_test_if_missing_module(self): self.skip_if_no_wallet() diff --git a/test/functional/rpc_getdescriptorinfo.py b/test/functional/rpc_getdescriptorinfo.py new file mode 100755 index 0000000000..977dc805ef --- /dev/null +++ b/test/functional/rpc_getdescriptorinfo.py @@ -0,0 +1,65 @@ +#!/usr/bin/env python3 +# Copyright (c) 2019 The Bitcoin Core developers +# Distributed under the MIT software license, see the accompanying +# file COPYING or http://www.opensource.org/licenses/mit-license.php. +"""Test getdescriptorinfo RPC. +""" + +from test_framework.test_framework import BitcoinTestFramework +from test_framework.descriptors import descsum_create +from test_framework.util import ( + assert_equal, + assert_raises_rpc_error, +) + + +class DescriptorTest(BitcoinTestFramework): + def set_test_params(self): + self.num_nodes = 1 + self.extra_args = [["-disablewallet"]] + + def test_desc(self, desc, isrange, issolvable, hasprivatekeys): + info = self.nodes[0].getdescriptorinfo(desc) + assert_equal(info, self.nodes[0].getdescriptorinfo(descsum_create(desc))) + assert_equal(info['descriptor'], descsum_create(desc)) + assert_equal(info['isrange'], isrange) + assert_equal(info['issolvable'], issolvable) + assert_equal(info['hasprivatekeys'], hasprivatekeys) + + def run_test(self): + assert_raises_rpc_error(-1, 'getdescriptorinfo', self.nodes[0].getdescriptorinfo) + assert_raises_rpc_error(-3, 'Expected type string', self.nodes[0].getdescriptorinfo, 1) + assert_raises_rpc_error(-5, 'is not a valid descriptor function', self.nodes[0].getdescriptorinfo, '') + + # P2PK output with the specified public key. + self.test_desc('pk(0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798)', isrange=False, issolvable=True, hasprivatekeys=False) + # P2PKH output with the specified public key. + self.test_desc('pkh(02c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5)', isrange=False, issolvable=True, hasprivatekeys=False) + # P2WPKH output with the specified public key. + self.test_desc('wpkh(02f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9)', isrange=False, issolvable=True, hasprivatekeys=False) + # P2SH-P2WPKH output with the specified public key. + self.test_desc('sh(wpkh(03fff97bd5755eeea420453a14355235d382f6472f8568a18b2f057a1460297556))', isrange=False, issolvable=True, hasprivatekeys=False) + # Any P2PK, P2PKH, P2WPKH, or P2SH-P2WPKH output with the specified public key. + self.test_desc('combo(0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798)', isrange=False, issolvable=True, hasprivatekeys=False) + # An (overly complicated) P2SH-P2WSH-P2PKH output with the specified public key. + self.test_desc('sh(wsh(pkh(02e493dbf1c10d80f3581e4904930b1404cc6c13900ee0758474fa94abe8c4cd13)))', isrange=False, issolvable=True, hasprivatekeys=False) + # A bare *1-of-2* multisig output with keys in the specified order. + self.test_desc('multi(1,022f8bde4d1a07209355b4a7250a5c5128e88b84bddc619ab7cba8d569b240efe4,025cbdf0646e5db4eaa398f365f2ea7a0e3d419b7e0330e39ce92bddedcac4f9bc)', isrange=False, issolvable=True, hasprivatekeys=False) + # A P2SH *2-of-2* multisig output with keys in the specified order. + self.test_desc('sh(multi(2,022f01e5e15cca351daff3843fb70f3c2f0a1bdd05e5af888a67784ef3e10a2a01,03acd484e2f0c7f65309ad178a9f559abde09796974c57e714c35f110dfc27ccbe))', isrange=False, issolvable=True, hasprivatekeys=False) + # A P2WSH *2-of-3* multisig output with keys in the specified order. + self.test_desc('wsh(multi(2,03a0434d9e47f3c86235477c7b1ae6ae5d3442d49b1943c2b752a68e2a47e247c7,03774ae7f858a9411e5ef4246b70c65aac5649980be5c17891bbec17895da008cb,03d01115d548e7561b15c38f004d734633687cf4419620095bc5b0f47070afe85a))', isrange=False, issolvable=True, hasprivatekeys=False) + # A P2SH-P2WSH *1-of-3* multisig output with keys in the specified order. + self.test_desc('sh(wsh(multi(1,03f28773c2d975288bc7d1d205c3748651b075fbc6610e58cddeeddf8f19405aa8,03499fdf9e895e719cfd64e67f07d38e3226aa7b63678949e6e49b241a60e823e4,02d7924d4f7d43ea965a465ae3095ff41131e5946f3c85f79e44adbcf8e27e080e)))', isrange=False, issolvable=True, hasprivatekeys=False) + # A P2PK output with the public key of the specified xpub. + self.test_desc('pk(tpubD6NzVbkrYhZ4WaWSyoBvQwbpLkojyoTZPRsgXELWz3Popb3qkjcJyJUGLnL4qHHoQvao8ESaAstxYSnhyswJ76uZPStJRJCTKvosUCJZL5B)', isrange=False, issolvable=True, hasprivatekeys=False) + # A P2PKH output with child key *1'/2* of the specified xpub. + self.test_desc("pkh(tpubD6NzVbkrYhZ4WaWSyoBvQwbpLkojyoTZPRsgXELWz3Popb3qkjcJyJUGLnL4qHHoQvao8ESaAstxYSnhyswJ76uZPStJRJCTKvosUCJZL5B/1'/2)", isrange=False, issolvable=True, hasprivatekeys=False) + # A set of P2PKH outputs, but additionally specifies that the specified xpub is a child of a master with fingerprint `d34db33f`, and derived using path `44'/0'/0'`. + self.test_desc("pkh([d34db33f/44'/0'/0']tpubD6NzVbkrYhZ4WaWSyoBvQwbpLkojyoTZPRsgXELWz3Popb3qkjcJyJUGLnL4qHHoQvao8ESaAstxYSnhyswJ76uZPStJRJCTKvosUCJZL5B/1/*)", isrange=True, issolvable=True, hasprivatekeys=False) + # A set of *1-of-2* P2WSH multisig outputs where the first multisig key is the *1/0/`i`* child of the first specified xpub and the second multisig key is the *0/0/`i`* child of the second specified xpub, and `i` is any number in a configurable range (`0-1000` by default). + self.test_desc("wsh(multi(1,tpubD6NzVbkrYhZ4WaWSyoBvQwbpLkojyoTZPRsgXELWz3Popb3qkjcJyJUGLnL4qHHoQvao8ESaAstxYSnhyswJ76uZPStJRJCTKvosUCJZL5B/1/0/*,tpubD6NzVbkrYhZ4WaWSyoBvQwbpLkojyoTZPRsgXELWz3Popb3qkjcJyJUGLnL4qHHoQvao8ESaAstxYSnhyswJ76uZPStJRJCTKvosUCJZL5B/0/0/*))", isrange=True, issolvable=True, hasprivatekeys=False) + + +if __name__ == '__main__': + DescriptorTest().main() diff --git a/test/functional/test_runner.py b/test/functional/test_runner.py index 2036d20852..7edb36c151 100755 --- a/test/functional/test_runner.py +++ b/test/functional/test_runner.py @@ -217,6 +217,7 @@ BASE_SCRIPTS = [ 'feature_config_args.py', 'rpc_getaddressinfo_labels_purpose_deprecation.py', 'rpc_getaddressinfo_label_deprecation.py', + 'rpc_getdescriptorinfo.py', 'rpc_help.py', 'feature_help.py', 'feature_shutdown.py', diff --git a/test/functional/wallet_bumpfee.py b/test/functional/wallet_bumpfee.py index 4eb0d19a4f..336e246e33 100755 --- a/test/functional/wallet_bumpfee.py +++ b/test/functional/wallet_bumpfee.py @@ -71,28 +71,29 @@ class BumpFeeTest(BitcoinTestFramework): test_simple_bumpfee_succeeds(self, "default", rbf_node, peer_node, dest_address) test_simple_bumpfee_succeeds(self, "fee_rate", rbf_node, peer_node, dest_address) test_feerate_args(self, rbf_node, peer_node, dest_address) - test_segwit_bumpfee_succeeds(rbf_node, dest_address) - test_nonrbf_bumpfee_fails(peer_node, dest_address) - test_notmine_bumpfee_fails(rbf_node, peer_node, dest_address) - test_bumpfee_with_descendant_fails(rbf_node, rbf_node_address, dest_address) - test_small_output_fails(rbf_node, dest_address) - test_dust_to_fee(rbf_node, dest_address) - test_settxfee(rbf_node, dest_address) + test_segwit_bumpfee_succeeds(self, rbf_node, dest_address) + test_nonrbf_bumpfee_fails(self, peer_node, dest_address) + test_notmine_bumpfee_fails(self, rbf_node, peer_node, dest_address) + test_bumpfee_with_descendant_fails(self, rbf_node, rbf_node_address, dest_address) + test_small_output_fails(self, rbf_node, dest_address) + test_dust_to_fee(self, rbf_node, dest_address) + test_settxfee(self, rbf_node, dest_address) test_watchonly_psbt(self, peer_node, rbf_node, dest_address) - test_rebumping(rbf_node, dest_address) - test_rebumping_not_replaceable(rbf_node, dest_address) - test_unconfirmed_not_spendable(rbf_node, rbf_node_address) - test_bumpfee_metadata(rbf_node, dest_address) - test_locked_wallet_fails(rbf_node, dest_address) - test_change_script_match(rbf_node, dest_address) + test_rebumping(self, rbf_node, dest_address) + test_rebumping_not_replaceable(self, rbf_node, dest_address) + test_unconfirmed_not_spendable(self, rbf_node, rbf_node_address) + test_bumpfee_metadata(self, rbf_node, dest_address) + test_locked_wallet_fails(self, rbf_node, dest_address) + test_change_script_match(self, rbf_node, dest_address) test_maxtxfee_fails(self, rbf_node, dest_address) # These tests wipe out a number of utxos that are expected in other tests - test_small_output_with_feerate_succeeds(rbf_node, dest_address) - test_no_more_inputs_fails(rbf_node, dest_address) + test_small_output_with_feerate_succeeds(self, rbf_node, dest_address) + test_no_more_inputs_fails(self, rbf_node, dest_address) self.log.info("Success") def test_simple_bumpfee_succeeds(self, mode, rbf_node, peer_node, dest_address): + self.log.info('Test simple bumpfee') rbfid = spend_one_input(rbf_node, dest_address) rbftx = rbf_node.gettransaction(rbfid) self.sync_mempools((rbf_node, peer_node)) @@ -119,6 +120,7 @@ def test_simple_bumpfee_succeeds(self, mode, rbf_node, peer_node, dest_address): assert_equal(bumpedwtx["replaces_txid"], rbfid) def test_feerate_args(self, rbf_node, peer_node, dest_address): + self.log.info('Test feerate args') rbfid = spend_one_input(rbf_node, dest_address) self.sync_mempools((rbf_node, peer_node)) assert rbfid in rbf_node.getrawmempool() and rbfid in peer_node.getrawmempool() @@ -135,7 +137,8 @@ def test_feerate_args(self, rbf_node, peer_node, dest_address): assert_raises_rpc_error(-4, "is too high (cannot be higher than", rbf_node.bumpfee, rbfid, {"fee_rate":1}) -def test_segwit_bumpfee_succeeds(rbf_node, dest_address): +def test_segwit_bumpfee_succeeds(self, rbf_node, dest_address): + self.log.info('Test that segwit-sourcing bumpfee works') # Create a transaction with segwit output, then create an RBF transaction # which spends it, and make sure bumpfee can be called on it. @@ -165,14 +168,14 @@ def test_segwit_bumpfee_succeeds(rbf_node, dest_address): assert rbfid not in rbf_node.getrawmempool() -def test_nonrbf_bumpfee_fails(peer_node, dest_address): - # cannot replace a non RBF transaction (from node which did not enable RBF) +def test_nonrbf_bumpfee_fails(self, peer_node, dest_address): + self.log.info('Test that we cannot replace a non RBF transaction') not_rbfid = peer_node.sendtoaddress(dest_address, Decimal("0.00090000")) assert_raises_rpc_error(-4, "not BIP 125 replaceable", peer_node.bumpfee, not_rbfid) -def test_notmine_bumpfee_fails(rbf_node, peer_node, dest_address): - # cannot bump fee unless the tx has only inputs that we own. +def test_notmine_bumpfee_fails(self, rbf_node, peer_node, dest_address): + self.log.info('Test that it cannot bump fee if non-owned inputs are included') # here, the rbftx has a peer_node coin and then adds a rbf_node input # Note that this test depends upon the RPC code checking input ownership prior to change outputs # (since it can't use fundrawtransaction, it lacks a proper change output) @@ -192,8 +195,8 @@ def test_notmine_bumpfee_fails(rbf_node, peer_node, dest_address): rbf_node.bumpfee, rbfid) -def test_bumpfee_with_descendant_fails(rbf_node, rbf_node_address, dest_address): - # cannot bump fee if the transaction has a descendant +def test_bumpfee_with_descendant_fails(self, rbf_node, rbf_node_address, dest_address): + self.log.info('Test that fee cannot be bumped when it has descendant') # parent is send-to-self, so we don't have to check which output is change when creating the child tx parent_id = spend_one_input(rbf_node, rbf_node_address) tx = rbf_node.createrawtransaction([{"txid": parent_id, "vout": 0}], {dest_address: 0.00020000}) @@ -201,7 +204,8 @@ def test_bumpfee_with_descendant_fails(rbf_node, rbf_node_address, dest_address) rbf_node.sendrawtransaction(tx["hex"]) assert_raises_rpc_error(-8, "Transaction has descendants in the wallet", rbf_node.bumpfee, parent_id) -def test_small_output_fails(rbf_node, dest_address): +def test_small_output_fails(self, rbf_node, dest_address): + self.log.info('Test totalFee bump with small output fails') # cannot bump fee with a too-small output rbfid = spend_one_input(rbf_node, dest_address) rbf_node.bumpfee(rbfid, {"totalFee": 50000}) @@ -209,7 +213,8 @@ def test_small_output_fails(rbf_node, dest_address): rbfid = spend_one_input(rbf_node, dest_address) assert_raises_rpc_error(-4, "Change output is too small", rbf_node.bumpfee, rbfid, {"totalFee": 50001}) -def test_small_output_with_feerate_succeeds(rbf_node, dest_address): +def test_small_output_with_feerate_succeeds(self, rbf_node, dest_address): + self.log.info('Testing small output with feerate bump succeeds') # Make sure additional inputs exist rbf_node.generatetoaddress(101, rbf_node.getnewaddress()) @@ -217,9 +222,9 @@ def test_small_output_with_feerate_succeeds(rbf_node, dest_address): input_list = rbf_node.getrawtransaction(rbfid, 1)["vin"] assert_equal(len(input_list), 1) original_txin = input_list[0] - # Keep bumping until we out-spend change output + self.log.info('Keep bumping until transaction fee out-spends non-destination value') tx_fee = 0 - while tx_fee < Decimal("0.0005"): + while True: input_list = rbf_node.getrawtransaction(rbfid, 1)["vin"] new_item = list(input_list)[0] assert_equal(len(input_list), 1) @@ -231,7 +236,11 @@ def test_small_output_with_feerate_succeeds(rbf_node, dest_address): assert rbfid not in raw_pool assert rbfid_new in raw_pool rbfid = rbfid_new - tx_fee = rbfid_new_details["origfee"] + tx_fee = rbfid_new_details["fee"] + + # Total value from input not going to destination + if tx_fee > Decimal('0.00050000'): + break # input(s) have been added final_input_list = rbf_node.getrawtransaction(rbfid, 1)["vin"] @@ -244,8 +253,8 @@ def test_small_output_with_feerate_succeeds(rbf_node, dest_address): rbf_node.generatetoaddress(1, rbf_node.getnewaddress()) assert_equal(rbf_node.gettransaction(rbfid)["confirmations"], 1) -def test_dust_to_fee(rbf_node, dest_address): - # check that if output is reduced to dust, it will be converted to fee +def test_dust_to_fee(self, rbf_node, dest_address): + self.log.info('Test that bumped output that is dust is dropped to fee') # the bumped tx sets fee=49,900, but it converts to 50,000 rbfid = spend_one_input(rbf_node, dest_address) fulltx = rbf_node.getrawtransaction(rbfid, 1) @@ -257,7 +266,8 @@ def test_dust_to_fee(rbf_node, dest_address): assert_equal(len(full_bumped_tx["vout"]), 1) # change output is eliminated -def test_settxfee(rbf_node, dest_address): +def test_settxfee(self, rbf_node, dest_address): + self.log.info('Test settxfee') assert_raises_rpc_error(-8, "txfee cannot be less than min relay tx fee", rbf_node.settxfee, Decimal('0.000005')) assert_raises_rpc_error(-8, "txfee cannot be less than wallet min fee", rbf_node.settxfee, Decimal('0.000015')) # check that bumpfee reacts correctly to the use of settxfee (paytxfee) @@ -272,17 +282,19 @@ def test_settxfee(rbf_node, dest_address): rbf_node.settxfee(Decimal("0.00000000")) # unset paytxfee -def test_maxtxfee_fails(test, rbf_node, dest_address): +def test_maxtxfee_fails(self, rbf_node, dest_address): + self.log.info('Test that bumpfee fails when it hits -matxfee') # size of bumped transaction (p2wpkh, 1 input, 2 outputs): 141 vbytes # expected bumping feerate of 20 sats/vbyte => 141*20 sats = 0.00002820 btc - test.restart_node(1, ['-maxtxfee=0.000025'] + test.extra_args[1]) + self.restart_node(1, ['-maxtxfee=0.000025'] + self.extra_args[1]) rbf_node.walletpassphrase(WALLET_PASSPHRASE, WALLET_PASSPHRASE_TIMEOUT) rbfid = spend_one_input(rbf_node, dest_address) assert_raises_rpc_error(-4, "Unable to create transaction: Fee exceeds maximum configured by -maxtxfee", rbf_node.bumpfee, rbfid) - test.restart_node(1, test.extra_args[1]) + self.restart_node(1, self.extra_args[1]) rbf_node.walletpassphrase(WALLET_PASSPHRASE, WALLET_PASSPHRASE_TIMEOUT) -def test_watchonly_psbt(test, peer_node, rbf_node, dest_address): +def test_watchonly_psbt(self, peer_node, rbf_node, dest_address): + self.log.info('Test that PSBT is returned for bumpfee in watchonly wallets') priv_rec_desc = "wpkh([00000001/84'/1'/0']tprv8ZgxMBicQKsPd7Uf69XL1XwhmjHopUGep8GuEiJDZmbQz6o58LninorQAfcKZWARbtRtfnLcJ5MQ2AtHcQJCCRUcMRvmDUjyEmNUWwx8UbK/0/*)#rweraev0" pub_rec_desc = rbf_node.getdescriptorinfo(priv_rec_desc)["descriptor"] priv_change_desc = "wpkh([00000001/84'/1'/0']tprv8ZgxMBicQKsPd7Uf69XL1XwhmjHopUGep8GuEiJDZmbQz6o58LninorQAfcKZWARbtRtfnLcJ5MQ2AtHcQJCCRUcMRvmDUjyEmNUWwx8UbK/1/*)#j6uzqvuh" @@ -334,7 +346,7 @@ def test_watchonly_psbt(test, peer_node, rbf_node, dest_address): funding_address2 = watcher.getnewaddress(address_type='bech32') peer_node.sendmany("", {funding_address1: 0.001, funding_address2: 0.001}) peer_node.generate(1) - test.sync_all() + self.sync_all() # Create single-input PSBT for transaction to be bumped psbt = watcher.walletcreatefundedpsbt([], {dest_address:0.0005}, 0, {"feeRate": 0.00001}, True)['psbt'] @@ -363,24 +375,24 @@ def test_watchonly_psbt(test, peer_node, rbf_node, dest_address): rbf_node.unloadwallet("watcher") rbf_node.unloadwallet("signer") -def test_rebumping(rbf_node, dest_address): - # check that re-bumping the original tx fails, but bumping the bumper succeeds +def test_rebumping(self, rbf_node, dest_address): + self.log.info('Test that re-bumping the original tx fails, but bumping successor works') rbfid = spend_one_input(rbf_node, dest_address) bumped = rbf_node.bumpfee(rbfid, {"totalFee": 2000}) assert_raises_rpc_error(-4, "already bumped", rbf_node.bumpfee, rbfid, {"totalFee": 3000}) rbf_node.bumpfee(bumped["txid"], {"totalFee": 3000}) -def test_rebumping_not_replaceable(rbf_node, dest_address): - # check that re-bumping a non-replaceable bump tx fails +def test_rebumping_not_replaceable(self, rbf_node, dest_address): + self.log.info('Test that re-bumping non-replaceable fails') rbfid = spend_one_input(rbf_node, dest_address) bumped = rbf_node.bumpfee(rbfid, {"totalFee": 10000, "replaceable": False}) assert_raises_rpc_error(-4, "Transaction is not BIP 125 replaceable", rbf_node.bumpfee, bumped["txid"], {"totalFee": 20000}) -def test_unconfirmed_not_spendable(rbf_node, rbf_node_address): - # check that unconfirmed outputs from bumped transactions are not spendable +def test_unconfirmed_not_spendable(self, rbf_node, rbf_node_address): + self.log.info('Test that unconfirmed outputs from bumped txns are not spendable') rbfid = spend_one_input(rbf_node, rbf_node_address) rbftx = rbf_node.gettransaction(rbfid)["hex"] assert rbfid in rbf_node.getrawmempool() @@ -418,7 +430,8 @@ def test_unconfirmed_not_spendable(rbf_node, rbf_node_address): if t["txid"] == rbfid and t["address"] == rbf_node_address and t["spendable"]), 1) -def test_bumpfee_metadata(rbf_node, dest_address): +def test_bumpfee_metadata(self, rbf_node, dest_address): + self.log.info('Test that bumped txn metadata persists to new txn record') assert(rbf_node.getbalance() < 49) rbf_node.generatetoaddress(101, rbf_node.getnewaddress()) rbfid = rbf_node.sendtoaddress(dest_address, 49, "comment value", "to value") @@ -428,15 +441,17 @@ def test_bumpfee_metadata(rbf_node, dest_address): assert_equal(bumped_wtx["to"], "to value") -def test_locked_wallet_fails(rbf_node, dest_address): +def test_locked_wallet_fails(self, rbf_node, dest_address): + self.log.info('Test that locked wallet cannot bump txn') rbfid = spend_one_input(rbf_node, dest_address) rbf_node.walletlock() assert_raises_rpc_error(-13, "Please enter the wallet passphrase with walletpassphrase first.", rbf_node.bumpfee, rbfid) rbf_node.walletpassphrase(WALLET_PASSPHRASE, WALLET_PASSPHRASE_TIMEOUT) -def test_change_script_match(rbf_node, dest_address): - """Test that the same change addresses is used for the replacement transaction when possible.""" +def test_change_script_match(self, rbf_node, dest_address): + self.log.info('Test that the same change addresses is used for the replacement transaction when possible.') + def get_change_address(tx): tx_details = rbf_node.getrawtransaction(tx, 1) txout_addresses = [txout['scriptPubKey']['addresses'][0] for txout in tx_details["vout"]] @@ -480,7 +495,8 @@ def submit_block_with_tx(node, tx): node.submitblock(block.serialize().hex()) return block -def test_no_more_inputs_fails(rbf_node, dest_address): +def test_no_more_inputs_fails(self, rbf_node, dest_address): + self.log.info('Test that bumpfee fails when there are no available confirmed outputs') # feerate rbf requires confirmed outputs when change output doesn't exist or is insufficient rbf_node.generatetoaddress(1, dest_address) # spend all funds, no change output diff --git a/test/functional/wallet_createwallet.py b/test/functional/wallet_createwallet.py index 048b3127ff..dabd78f66c 100755 --- a/test/functional/wallet_createwallet.py +++ b/test/functional/wallet_createwallet.py @@ -79,7 +79,7 @@ class CreateWalletTest(BitcoinTestFramework): assert_raises_rpc_error(-4, "Error: This wallet has no available keys", w4.getnewaddress) assert_raises_rpc_error(-4, "Error: This wallet has no available keys", w4.getrawchangeaddress) # Now set a seed and it should work. Wallet should also be encrypted - w4.walletpassphrase('pass', 2) + w4.walletpassphrase('pass', 60) w4.sethdseed() w4.getnewaddress() w4.getrawchangeaddress() diff --git a/test/functional/wallet_listsinceblock.py b/test/functional/wallet_listsinceblock.py index 6f248c9bd3..229eda9806 100755 --- a/test/functional/wallet_listsinceblock.py +++ b/test/functional/wallet_listsinceblock.py @@ -2,7 +2,7 @@ # Copyright (c) 2017-2019 The Bitcoin Core developers # Distributed under the MIT software license, see the accompanying # file COPYING or http://www.opensource.org/licenses/mit-license.php. -"""Test the listsincelast RPC.""" +"""Test the listsinceblock RPC.""" from test_framework.test_framework import BitcoinTestFramework from test_framework.messages import BIP125_SEQUENCE_NUMBER @@ -38,6 +38,7 @@ class ListSinceBlockTest(BitcoinTestFramework): self.double_spends_filtered() def test_no_blockhash(self): + self.log.info("Test no blockhash") txid = self.nodes[2].sendtoaddress(self.nodes[0].getnewaddress(), 1) blockhash, = self.nodes[2].generate(1) blockheight = self.nodes[2].getblockheader(blockhash)['height'] @@ -63,6 +64,7 @@ class ListSinceBlockTest(BitcoinTestFramework): "transactions": txs}) def test_invalid_blockhash(self): + self.log.info("Test invalid blockhash") assert_raises_rpc_error(-5, "Block not found", self.nodes[0].listsinceblock, "42759cde25462784395a337460bde75f58e73d3f08bd31fdc3507cbac856a2c4") assert_raises_rpc_error(-5, "Block not found", self.nodes[0].listsinceblock, @@ -100,6 +102,7 @@ class ListSinceBlockTest(BitcoinTestFramework): This test only checks that [tx0] is present. ''' + self.log.info("Test reorg") # Split network into two self.split_network() @@ -110,7 +113,7 @@ class ListSinceBlockTest(BitcoinTestFramework): # generate on both sides lastblockhash = self.nodes[1].generate(6)[5] self.nodes[2].generate(7) - self.log.info('lastblockhash=%s' % (lastblockhash)) + self.log.debug('lastblockhash={}'.format(lastblockhash)) self.sync_all(self.nodes[:2]) self.sync_all(self.nodes[2:]) @@ -155,6 +158,7 @@ class ListSinceBlockTest(BitcoinTestFramework): until the fork point, and to include all transactions that relate to the node wallet. ''' + self.log.info("Test double spend") self.sync_all() @@ -234,6 +238,7 @@ class ListSinceBlockTest(BitcoinTestFramework): 3. It is listed with a confirmation count of 2 (bb3, bb4), not 3 (aa1, aa2, aa3). ''' + self.log.info("Test double send") self.sync_all() @@ -302,6 +307,7 @@ class ListSinceBlockTest(BitcoinTestFramework): `listsinceblock` was returning conflicted transactions even if they occurred before the specified cutoff blockhash ''' + self.log.info("Test spends filtered") spending_node = self.nodes[2] dest_address = spending_node.getnewaddress() diff --git a/test/fuzz/test_runner.py b/test/fuzz/test_runner.py index 5df2789638..520a2b5a95 100755 --- a/test/fuzz/test_runner.py +++ b/test/fuzz/test_runner.py @@ -12,46 +12,6 @@ import sys import subprocess import logging -# Fuzzers known to lack a seed corpus in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus -FUZZERS_MISSING_CORPORA = [ - "addr_info_deserialize", - "asmap", - "base_encode_decode", - "block", - "block_file_info_deserialize", - "block_filter_deserialize", - "block_header_and_short_txids_deserialize", - "bloom_filter", - "decode_tx", - "fee_rate_deserialize", - "flat_file_pos_deserialize", - "float", - "hex", - "integer", - "key", - "key_origin_info_deserialize", - "merkle_block_deserialize", - "out_point_deserialize", - "p2p_transport_deserializer", - "parse_hd_keypath", - "parse_numbers", - "parse_script", - "parse_univalue", - "partial_merkle_tree_deserialize", - "partially_signed_transaction_deserialize", - "prefilled_transaction_deserialize", - "psbt_input_deserialize", - "psbt_output_deserialize", - "pub_key_deserialize", - "rolling_bloom_filter", - "script_deserialize", - "strprintf", - "sub_net_deserialize", - "tx_in", - "tx_in_deserialize", - "tx_out", -] - def main(): parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter) @@ -130,6 +90,20 @@ def main(): logging.info("{} of {} detected fuzz target(s) selected: {}".format(len(test_list_selection), len(test_list_all), " ".join(test_list_selection))) + test_list_seedless = [] + for t in test_list_selection: + corpus_path = os.path.join(args.seed_dir, t) + if not os.path.exists(corpus_path) or len(os.listdir(corpus_path)) == 0: + test_list_seedless.append(t) + test_list_seedless.sort() + if test_list_seedless: + logging.info( + "Fuzzing harnesses lacking a seed corpus: {}".format( + " ".join(test_list_seedless) + ) + ) + logging.info("Please consider adding a fuzz seed corpus at https://github.com/bitcoin-core/qa-assets") + try: help_output = subprocess.run( args=[ @@ -160,8 +134,7 @@ def main(): def run_once(*, corpus, test_list, build_dir, export_coverage, use_valgrind): for t in test_list: corpus_path = os.path.join(corpus, t) - if t in FUZZERS_MISSING_CORPORA: - os.makedirs(corpus_path, exist_ok=True) + os.makedirs(corpus_path, exist_ok=True) args = [ os.path.join(build_dir, 'src', 'test', 'fuzz', t), '-runs=1', diff --git a/test/lint/extended-lint-cppcheck.sh b/test/lint/extended-lint-cppcheck.sh index 4d6c70660f..ae18d74ebf 100755 --- a/test/lint/extended-lint-cppcheck.sh +++ b/test/lint/extended-lint-cppcheck.sh @@ -66,7 +66,7 @@ function join_array { ENABLED_CHECKS_REGEXP=$(join_array "|" "${ENABLED_CHECKS[@]}") IGNORED_WARNINGS_REGEXP=$(join_array "|" "${IGNORED_WARNINGS[@]}") WARNINGS=$(git ls-files -- "*.cpp" "*.h" ":(exclude)src/leveldb/" ":(exclude)src/crc32c/" ":(exclude)src/secp256k1/" ":(exclude)src/univalue/" | \ - xargs cppcheck --enable=all -j "$(getconf _NPROCESSORS_ONLN)" --language=c++ --std=c++11 --template=gcc -D__cplusplus -DCLIENT_VERSION_BUILD -DCLIENT_VERSION_IS_RELEASE -DCLIENT_VERSION_MAJOR -DCLIENT_VERSION_MINOR -DCLIENT_VERSION_REVISION -DCOPYRIGHT_YEAR -DDEBUG -DHAVE_WORKING_BOOST_SLEEP_FOR -I src/ -q 2>&1 | sort -u | \ + xargs cppcheck --enable=all -j "$(getconf _NPROCESSORS_ONLN)" --language=c++ --std=c++11 --template=gcc -D__cplusplus -DCLIENT_VERSION_BUILD -DCLIENT_VERSION_IS_RELEASE -DCLIENT_VERSION_MAJOR -DCLIENT_VERSION_MINOR -DCLIENT_VERSION_REVISION -DCOPYRIGHT_YEAR -DDEBUG -I src/ -q 2>&1 | sort -u | \ grep -E "${ENABLED_CHECKS_REGEXP}" | \ grep -vE "${IGNORED_WARNINGS_REGEXP}") if [[ ${WARNINGS} != "" ]]; then diff --git a/test/lint/lint-includes.sh b/test/lint/lint-includes.sh index ced2fd2bb6..1cece6a525 100755 --- a/test/lint/lint-includes.sh +++ b/test/lint/lint-includes.sh @@ -53,7 +53,6 @@ EXPECTED_BOOST_INCLUDES=( boost/algorithm/string/classification.hpp boost/algorithm/string/replace.hpp boost/algorithm/string/split.hpp - boost/chrono/chrono.hpp boost/date_time/posix_time/posix_time.hpp boost/filesystem.hpp boost/filesystem/fstream.hpp diff --git a/test/lint/lint-locale-dependence.sh b/test/lint/lint-locale-dependence.sh index 35e58c2df6..0cb38b6fdb 100755 --- a/test/lint/lint-locale-dependence.sh +++ b/test/lint/lint-locale-dependence.sh @@ -22,6 +22,7 @@ KNOWN_VIOLATIONS=( "src/test/blockchain_tests.cpp.*std::to_string" "src/test/dbwrapper_tests.cpp:.*snprintf" "src/test/denialofservice_tests.cpp.*std::to_string" + "src/test/fuzz/locale.cpp" "src/test/fuzz/parse_numbers.cpp:.*atoi" "src/test/key_tests.cpp.*std::to_string" "src/test/net_tests.cpp.*std::to_string" |